The following figure, constructed largely for my convenience, outlines the various hardware breaks used in v2 processing compared to the breaks I found were warranted in this analysis.
The shared breaks are essentially the seasonal shutdowns (including the Christmas '97 break). Two potential extra breaks in '97 are found in this analysis (though they could be collapsed together at the cost of ~30 pixels at H) and one additional period noted during v2 (period e/7) where a vertical column was bad at H read1. Breaks drawn in grey denote hardware periods needed for darks and/or flats but not for masks (nothing changed in the canonicals).
The bottom row shows a period of time during which a bad K read1 column was
masked in the canonicals (outer dashed bracket) compared to the period where
that column shows very high dispersions in the night-to-night read1 K dark
standard deviations. The period from 981119-990427 is when the masked column
is OK in the darks deviations but the adjacent column degrades to a lesser
degree (this was never masked during production). Using the darks analysis
as a guide, we may be able to recover ~9 months of read1 data for that column.
In considering the possibility of combining the previous canonical masks with the lists of bad pixels found in the current analysis, the following table summarizes the number of bad read2 and read1 pixels in the existing v2 canonicals (first two columns in each band) with the number of additional pixels that would be masked by including the masks from this analysis (second two columns in each band). Period numbers refer to the canonical v2 periods from the figure above.
J H K
period r2 r1 r2+ r1+ r2 r1 r2+ r1+ r2 r1 r2+ r1+
----- --------------- --------------- ---------------
0 24 75 1 1 174 332 1 1 124 395 17 1
1 22 75 2 1 250 332 30 6 133 395 25 1
2-4 18 75 6 1 272 332 8 6 93 395 39 2
5-6 18 75 9 2 310 308 27 25 93 395 110 3
7 18 75 9 2 310 552 27 24 93 395 110 3
8-9 18 75 9 2 310 308 27 25 93 395 110 3
10 16 16 2 1 108 108 3 1 119 119 13 1
11 16 16 2 1 108 108 3 1 119 119 11 2
To investigate the impact we have run the new combined masks through v3 testing for the RTB nights 971117n (period 1) and 990523n (period 9) which are the hardest hit with extra masked pixels as seen in the table above.
Fortunately there appears to be no sigificant hit on coverage due to the additional masking. The following figure demonstrate this for the H & K bands on these nights (J has very few masked pixels).
The first column shows the coverage map stretched from 0 to 4 coverages (white is 4 or more). The second shows the difference in coverage between the v2 masks and the new combined masks. Black is a change in coverage by -1 (a loss), white is +1 (a gain) and grey is no change. Nothing drops by 2 or more coverages in these test scans.
971117n H Mask (new)
971117n H Mask Differences
971117n K Mask (new)
971117n K Mask Differences
990523n H Mask (new)
990523n H Mask Differences
990523n K Mask (new)
990523n K Mask Differences
|
Unfortunately there appears to be no real historical record of which pixels were turned off for what reasons during v2 processing, making the canonicals the most complete record of things we knew weren't working well. If we were to drop them entirely and try to recreate the information from existing records we run the risk of getting hit again by "known" problems for which we didn't have clear records. Restoring the K read1 column for 981119-990720 may be the only safe recovery from the original masks.
The read1 coverage maps for the same scans have been produced in the pipeline and compared with the read2 maps. There is qualitatively little difference between the read2 and read1 coverages since most of the pixels masked in one are masked in the other. In particular, no additional low-coverage read1 holes are noted beyond the problem spots already noted in read2 (except for the bad K read1 column in 971117n which was already known to be a problem).
It appears that the worst-case extra masked pixels has relatively little impact on low coverage areas. The wost H holes seen in the figures above were also present in v2 processing (ergo we survived them once), and K maintains high coverage across the field.
Actual impact of additional masking on data quality has yet to be evaluated.