MAPCOR H and K Band Glint Problems

T. Evans and R. Tam - IPAC

This problem has since been fixed and all affected scans re-processed correctly.

Between 7/30/98 and 8/27/98, an incorrect namelist was used for MAPCOR for northern operations processing. The error stemmed from the fact that a new version of MAPCOR , version 3.6, was delivered and put into the ops pipeline, but the namelist was not placed in the proper delivery area at the proper time. The only change in the namelist was a change in format, not in parameter values. (The format change was required to accommodate the southern processing.) However, this format change caused the new version of MAPCOR to incorrectly read the namelist values for some of the H and K band glint parameters. (Please see the MAPCOR SDS (PostScript file) for full explanations of the parameters.) The incorrect parameters affect the 1st H band glint and both K band glints (there are 2 glints in each band in the north).

The glint_delm parameter for the 1st H band glint became 7.7 mag, instead of the correct value of 6.6 mag. This caused a 1.1 mag shift in the "allowable mag window" -- the range of magnitude within which a source must fall to be considered a glint. The glint_delm value is added to the parent source's mag to get the center of this window.
The glint_h parameter for the 1st H band glint was shifted by 0.5 camera pix, or 1 arcsec, which caused the predicted y position for that glint to be shifted by the same amount (1").
The glint_sigm parameter for both K band glints became 1.6 mag instead of 3.0 mag. This parameter is the half-width of the allowable mag window.

After the namelist error was discovered, it was immediately corrected and the "sampler release" night of 971116n was re-processed. Raymond and I then compared the processing results of the "old run" and "new run" for that night. We've been working mainly with the mapcor output (before BANDMERGE runs), so the following analysis treats each band's results independently . After Raymond ran catcor to match all sources in the old and new runs within 2", I looked at all of the non-matched sources -- sources that found no match in the other run's output file -- and found that all of them were either faint (m > 16) or were diffraction spike sources, so the old and new runs matched up very well. We've found that all of the changes in purge flags between the runs were as follows:

53 H sources and 75 K sources are "new glints" -- marked as glints in the new run but not in the old. These are the sources that were incorrectly unmarked because of the namelist problem. Raymond found that only 3 of these H and K sources belong to the same bandmerged source. The results exactly follow what we expect from the incorrect parameters (see also the summary table below):

15 H glints were previously missed because of the 1.1 mag shift in the mag "window"; 38 H glints were previously missed because of the 1" shift in the predicted y-position.

26 of the total 53 H "new glints" are also other kinds of artifacts, 24 of which would therefore not have appeared in a catalog made from the old run. The other 2 are new-glint+confusion-level-1 sources, one of which is also a contaminated-by-diffraction source. (Confusion level 1 and contaminated-by-diffraction sources are considered to be "artifact-affected" sources, meaning they are likely to be real but their measurements are contaminated by artifacts from bright stars or by mild confusion. Since they are likely to be real, for our purposes herein they would appear in a catalog.)
So in the H band there are (53-24=) 29 new glints that would have appeared in an old catalog, but not in the new. (There is a total of 408649 H band sources in that night; 29/408649 = 0.0071%.)

All 75 K sources were previously missed because of the too-narrow mag window (1.6 mag half-width instead of 3.0 mag).

12 of these sources are also other kinds of artifacts, 5 of which would therefore not have appeared in a catalog made from the old run. The other 7 are also confusion-level-1 sources and/or contaminated-by-diffraction sources.
So in the K band there are (75-12=) 70 new glints that would have appeared in an old catalog, but not in the new. (There is a total of 461573 K band sources in that night; 70/461573 = 0.0152%.)

**Summary Table**
	Number of new glints (1)	Number of new glints only (2)	both new glints and "artifact-affected" (3)	number of new artifacts (4)	number of old artifacts (5)	total number of sources in band (6)	percentage of new artifacts (7)
H band	53	27	2	29	24	408649	0.0071%
K band	75	63	7	70	5	461573	0.0152%

col. 1 -- the number of new glints found in the new run.
col. 2 -- the total number of new glints not already classified as any 
          other artifacts.
col. 3 -- the number of sources which are both new glints and are, in 
          both runs, "artifact-affected".  "Artifact-affected" sources
          are sources that are likely to be real, but are affected by 
          artifacts such as bright star halos or diffraction spikes.
col. 4 -- number of new glint artifacts that WOULD have appeared in the 
          old catalog, but will not appear in the new catalog 
          (also col. 2 + col. 3)
col. 5 -- number of new glint artifacts that would NOT have appeared in 
          either the old or new catalogs because they were already  
          classified as other artifacts (also col. 1 - col. 4)
col. 6 -- the total number of sources extracted in the band during the 
          entire night.
col. 7 -- the total number of new artifacts from col. 4 divided by the 
          total number of sources in the band, which gives the percentage 
          of new artifacts over the night.

19 H sources and 1 K source are "new not-glints" -- marked as glints in the old run but not in the new. These are the sources that were incorrectly marked because of the namelist problem -- the reverse of 1) above. These results also exactly follow what we expect from the incorrect parameters, and do not indicate any problems in the reliability of the correct parameters.

11 of these H sources are diffraction spike (not real) sources in both runs, of which 2 are also confusion level 2 sources; in any case they would not be in the catalog. Of the remaining 8 sources, 7 are very faint (m>16) and could be real but I can't really tell, and 1 is not too faint (m ~ 15) and looks like a real source.
The single K source is also marked as a confusion level 2 source; in any case it would not be in the catalog.

All other purge flag differences between the old and new runs -- 2 J band sources, 17 H band sources, and 1 K band source -- are all attributable to:

Changes in mag which make a source toggle between diffraction flag values 5 (contaminated-by-diffraction) and 6 (part-of-diffraction-spike); some of these mag changes seem to come because PROPHOT is now nulling out sources with background problems (near bright stars?)
The mess inside bright stars causes small changes in extraction parameters; thus some matched sources toggled between different flag values, different r1/r2 sources are matched between old and new runs, etc.
The 13 H band sources in scan 102 are artifacts of a bright source right around RA=106.028, Dec=+20.57 that is badly saturated in r1 and not extracted properly in any band (besides perhaps k). Thus its artifacts are not properly marked in either the old or new runs. This is one for db_mapcor...

As a check, a random set of coadd images containing the "new glints" were examined to determine whether the new flag values were correct. The coadds showed no anomalous results when a short-form file was overlaid on the image and the new glint was subsequently marked. Every new glint coincided with one artifact from the .short file (as to be expected.) Some of the new glints were found in the mess inside saturated stars, but many were associated with unsaturated (and less messy) stars (see the example images below).

As an aside, for the 2nd K band glint, the fraction of real sources called glints which are found in the outer 1.4 mag of the 3.0 mag "allowable window" half-width can be roughly estimated. This is possible because no other band has a glint at its position, while the 1st glints in all three bands lie at approximately the same position relative to the parent source. While examining 12 "new" 2nd K band glints in the images, I found that 6 of the sources were 3-band extractions, which are thus marked as a glint only in the K band. These sources are most likely real sources, though many are quite faint and less reliable in any case; but of course they are all contaminated in the K band by the glint.

The following are sample images of the two overlays over an image, with the new glint marked in red: