For version 3 processing we wish to use the full knowledge of the survey mask history to derive overall canonical masks that can be applied across longer ranges of nights. In addition to providing better uniformity of processing, this gives us the opportunity to catch bad pixels that are intermittant, and therefore difficult to identify in a shorter set of masks even though they may be causing real if infrequent problems over a wider range of dates.
Our approach to analyze the masks has been to combine the entire set of nightly read2-read1 and read1 masks and look at the global properties. Consistently bad pixels were isolated by counting the total number of times each pixel was flagged as bad. We then plot up the time history of all "bad" pixels, seeing if there are correlated times for them being bad or good, leading us to determine the necessary breaks in the mask history.
The mask histories for H & Ks are more involved, showing significant changes during each summer shutdown. Moreover, the entire H array was replaced during the summer '99 shutdown, introducing a disjoint set of bad pixels (but greatly reducing the total number).
The plots in the first two columns of the table below show the time behavior as a function of frame number; white points indicate a pixel that has been flagged as masked. The frame number axis is effectively time. Blocks of pixels are seen to toggle on/off at the shutdowns. Note that these figures exclude all pixels within 4 pixels of the edge; typically 1-3 rows and/or columns are also masked due to vignetting, but these effects are considered separately from masks due to electronically bad pixels.
Pixel Mask Time Histories/Cumulative Plots
| Ctr Pixels read2 |
Ctr Pixels read1 |
All Pixels read2 |
All Pixels read1 |
Cumul read2 |
|
| J | * | * | * | * | * |
| H | * | * | * | * | * |
| Ks | * | * | * | * | * |
The northern K array suffered from a period where the read1 levels in a column fluctuated in level introducing false read1 detections (but cancelling out in read2-read1). This column was not caught in the nightly masks and therefore is not represented in this analysis. However, it is obvious in the canonical darks analysis (in the standard deviation images) and will be factored in at a later stage in mask production.
Given the time histories of the masks we need to decide several things:
The first of these questions depends on the degree to which we want to struggle to recover a small number of pixels during intervals before they have gone bad. Northern J band is simple and well-behaved; there are only a handfull of bad pixels, and the only significant change occurs during the summer '99 shutdown.
H & Ks are more complex, showing significant changes in the bad pixel characteristics at every summer shutdown (roughly around frames 60, 230, 425, & 625. In addition, the bad pixels also show jumps at a number of other times, generally associated with hardware maintenance.
The question of how many times a pixel must get flagged before it is considered to be bad for the entire period is tricky. If pixels are masked too aggressively we run into the problem of introducing many low coverage spots in the maps. But if the pixel triage is too leniant then a number of intermittentantly bad pixels may make it into the survey.
The latter consideration is particularly important since the primary way bad pixels are identified is through the dispersions in the nightly darks. A pixel that is unstable for only short periods (ones like this have been found during various analyses) may only occasionally happen to be causing problems during darks aquisition, and darks are only collected irregularly. It would make some sense to adopt a policy that if a pixel shows much of any tendency to be bad it should be masked out for the entire hardware period.
The cumulative plots in the table above help visualize the impact of masking thresholds. The Y-axes show the number of pixels that would be dropped if one cut at the threshold on the X-axis. J band is an easy choice; even cutting pixels with 3-4 bad apparitions will hardly affect the total. The H band cumulative plot shows there is relatively little difference between cutting at 150 bad apparitions and 8-10 (the total number of bad pixels is nonetheless quite high and at this point, even a few extra bad pixels can lead to singletons and holes. Ks is a more difficult judgement call since the cumulative distribution is more steeply inclined, and masking more aggressively could result in a large number of dropped pixels.
Note that these cumulative distributions do cover the entire observing period, and given the big changes noted at H & Ks at the summer shutdowns, they should tighten up a bit when plotted for shorter periods. Also, these numbers exclude the vignetted pixels along the frame edges, so the total number of blanked pixels is substantially higher, typically by 500 or more pixels in all bands.
