T. Jarrett, IPAC
(980708)
On the night of 980725 a series of 10 scans (all of the same area of the sky) were acquired under sequentially changing focus setting. The purpose of these tests was to impart a non-symmetric distortion to the PSF -- starting from vertically flattened profile to symmetric profile (i.e., in focus), to a horizontally flattened profile. The PSF can be viewed here: 2MASS North Focus Test Results for 980625 UT along with some of the basic characterization parameters.
To properly assess the performance of extended source processor, it is necessary to have several galaxies of differing flux and orientation in the field of study. The chosen field contains several Hercules cluster galaxies (about 20 or so) in one-degree (length) scans. This memo outlines the GALWORKS results for these scans.
Note: previous GALWORKS focus tests can be found here:
970724 Focus Tests:
Extended Source Detection
. Final conclusion from this work:
It is clear that the reliability of the extended source
database will be compromised by image data with a PSF
distortion less than 0.85, as measured by the 2nd moment
X-Y ratio. The results also suggest that the
reliability is poor for distortions greater than
1.3 or so, as measured by the 2nd moment
X-Y ratio. The focus tests are not sufficient to address
the question of galaxy repeatibility or completeness.
.
Results
Each extended source candidate was visually inspected and compared with the DSS counterpart to classify the object (e.g., galaxy, star, double star, artifact, etc). At the sensitivity limit, most of the candidates cannot be classified and are left as "unknown" and no further tracking is made on these objects.
The completeness and reliability can is given below in the
table of results for extended source candidates and in the
following plots. The mag limits
have been restricted to the lev-1 targets, K < 13.5, J < 15.0.
Extended Source Candidates: C & R
sc# focus Jrat Hrat Krat Jba Hba Kba J- J+ ng nb R H- H+ ng nb R K- K+ ng nb R
021 1033 0.621 0.733 0.679 0.570 0.650 0.400 9.00 15.00 16 3 .84 9.00 14.00 15 1 .94 9.00 13.50 14 2 .88
022 1033 0.668 0.660 0.664 0.670 0.600 0.550 9.00 15.00 15 5 .75 9.00 14.00 15 3 .83 9.00 13.50 14 1 .93
023 1013 0.723 0.775 0.760 0.650 0.720 0.720 9.00 15.00 17 2 .89 9.00 14.00 15 0 1.00 9.00 13.50 14 0 1.00
024 993 0.768 0.849 0.843 0.750 0.720 0.800 9.00 15.00 16 1 .94 9.00 14.00 15 0 1.00 9.00 13.50 14 0 1.00
025 973 0.881 0.943 0.984 0.900 0.900 0.920 9.00 15.00 17 0 1.00 9.00 14.00 16 1 .94 9.00 13.50 14 0 1.00
026 953 0.977 1.050 1.114 0.820 1.000 0.870 9.00 15.00 16 1 .94 9.00 14.00 15 1 .94 9.00 13.50 14 1 .93
027 933 1.100 1.158 1.221 0.900 0.720 0.750 9.00 15.00 17 1 .94 9.00 14.00 15 1 0.94 9.00 13.50 14 1 0.90
028 913 1.219 1.247 1.351 0.770 0.750 0.650 9.00 15.00 16 3 .84 9.00 14.00 15 1 .94 9.00 13.50 13 1 .91
029 893 1.317 1.304 1.500 0.770 0.620 0.550 9.00 15.00 16 2 .89 9.00 14.00 14 2 .88 9.00 13.50 14 2 .88
030 873 1.386 1.463 1.634 0.600 0.520 0.400 9.00 15.00 16 2 .89 9.00 14.00 16 2 .89 9.00 13.50 13 1 0.93
Table notes: ng == number of real galaxies, nb == number of false
detections (stars), and R == reliability. Scan 026 has a nasty
meteor streak (which triggered many false detections; here they
are ignored). Scan 029 has a meteor streak.
Bottom line: The second moment ratio should be limited to 0.85 to 1.0 for optimum results (i.e., at all three bands simultaneously). For more generous (but increasing unreliability) limits, second moment ratio should be limited to 0.80 to 1.1. We the galaxy lollipop guild prefer the former, but can live with the latter.
Some notes flying about. -tj
Roc Cutri wrote:
>
> I'm a little puzzled why the acceptable limits are asymmetric (i.e. 0.85-1.0
> rather than 0.85-1.15). Is this a result of the image moment ratio's
> sensitivity to position angle? What is the acceptable range expressed
> in terms of the aspect ratio?
>
--- reply from tj --
OK i have added the axis ratio to the table and made a plot. As noted to in the doc, the upper limit for the axis ratio should be set at <1.1, which is not symmetric with the lower limit. The axis ratio plots really shows this effect: reliability is double- valued for a given axis ratio! It depends on whether the PSF is vertical/flat or horizontal/flat. Why this is I don't know. Speculation: low number stats are fooling us. there are 17 objects, of which 1 or 2 are false. we simply could be toggling +-1 due to stats (noise differences).
other spec: we know the PSF shape due to the optics is tilted toward p.a. -45deg (0 = north), this is the direction of the low focus settings where the R is bad. somehow this triggers false things (only +- 2 srcs, keep in mind -- this is not a flood; see the table in the doc);
The former implies that we have nothing to worry about. go with 0.8 to 1.2 or so for mom ratio. The latter implies that we do worry, and we must set tight limits for the upper limit.
Time to raise the dreaded idea: perform 10X6 more focus test scans with 6 degree scan lengths of Herc. Why? number stats. We need a lot of galaxies (hundreds) to do this right.
Feasible? probably not. what then? continue with our current limits and see if we can generate a high R (99%) cat this spring?
Moot point idea: since Rae and Co will not accept our tighter mom rat limits (just guesssing here, but this is a no brainer, duh!) EVEN IF THE SPEC is compromised, we might as well forge ahead and see what falls out with big BIG numbers during our catalog generation and validation procedure. (note: we could always limit the ext. src cat to scans in which we meet our mom rat limits; this is a compromise I spose).
--- from schneider ---
Hi Tom,
I'm puzzled in looking over your most recent memo why the reliability
depends on the orientation of the PSF flattening. It seems like a
ratio of 1.1 ought to be no worse than 1/1.1. Is there something about
the algorithm or the galaxies in the Hercules test field that make this
result asymmetric?
---Steve
p.s. Thought of one more question--are the number counts of detections
and false sources based on results from all three channels, or have you
considered each separately? I just want to make sure I understand why
the number counts differ between channels.
---- reply from jarrett ---
number counts and reliab is treated for each band separately.
however, note that since J,H or K can "pull" a detection
(only one band is required to place a src in the ext src db),
the number counts at the faint end reflect this action
(e.g., the K counts extend well beyond 14 mag because
J is pulling them along).
for catalog generation, we will probably want to apply a 2-band
rule (at least) to improve reliability. use OBDT, we are effectively
applying a 3-band rule. it is not clear yet whether OBDTs are the
way to go to get 99%.
in the coming weeks I will begin to address C&r for the catalog
generation (i'm waiting for all of the RTBs to be reduced with
2MAPPS v2.1) -- we will have plenty of statistics to nail it down.
-tj
--- note from TChester ---
good analysis. i basically agree with your conclusions. here's some
other thoughts:
1. what software was used to compute the second moment ratios? the one
with the bug or the one sans bug? this may cause some of the observed
asymmetry. if we had a circularly symmetric psf, the bug wouldn't
affect axial ratios, but since we don't, it probably does.
2. note that the plot of galaxy reliability vs. focus on tj's page
looks just like the shape vs. focus on roc's page, but inverted with
focus direction. i actually can't figure out a good reason quickly why
these two should be related, so perhaps it is also related to the bug.
3. we are certainly dealing with small number stats, so i would
conclude only that the best focus for j is 960 +-20, and for h and k,
960 +-40, consistent with roc's results.
4. for a noticeable, significant, 5% total unrealibility extra, i would
conclude that j axial ratio must be 0.8 to 1.1, h 0.72 to 1.1?, and k
0.7 to 1.05. so there is little doubt that we cannot tolerate axial
ratios above 1.1, and hence i would recommend changing that threshold
immediately in the q.a., and looking over past results to see if some
scans should be downgraded. it clearly is not symmetric as we
originally thought. so i agree with tj that at a minimum, 0.8 to 1.1
should be the new limits, once we make sure we are dealing with same
version of software producing those numbers.
