This is a "work-in-progress" document on the pixel saturation thresholds for 2MASS. Some earlier background information can be found here.
| 1. | Preliminary South Camera Data |
| 2. | Summary of Available Camera Data |
| 3. | X & Y Saturation Variables |
Here is some preliminary information on the statistics of the saturation behavior of individual pixels in the south cameras. To suppress noise, for convenience a quadratic fit to each pixel has been used as a surrogate for the data, and compared to a linear fit similar to the pixel responsivity actually used to flatten the frames. As will be seen below, because there is small but significant curvature characteristic of the array response, it turns out that the quadratic model is sufficiently better than the linear one that a large fraction, though not all, of the true residual can be absorbed into it in the region before saturation becomes too extreme.
The criterion for satisfactory accuracy that has been adopted is that the relative error be < 1% of the data. Previously it has been applied to the array median residual for each frame in a flat series, to obtain a single threshold for the entire array. Here the same 1% criterion has been applied to each pixel (subject to the validity of the quadratic approximation), and used to determine an array of thresholds for each.
READ2 DU versus the array median of (READ1-DARK):
Figure 1: J Camera Saturation Curve
Each dot is one frame. Note the hard saturation around 56000 DU and the slight negative curvature in the middle. The smooth dashed and dotted lines are overlaid overall linear and quadratic fits, respectively, to the frame median data in the 15000-47000 DU range.
Figure 2 is a histogram of thresholds for the south J camera array determined by criterion that
where q is the quadratic surrogate for the data, and l is a linear model similar to that actually used in 2MASS analysis. Masked pixels have been ignored, but a total of 59 other pixels that have (+) curvature have also been thrown out. Half of these were in a 3-pixel border of the array.
A histogram of the quadratic co-efficients (ie, "A", in A*x^2 + B*x + C) appears as Figure 3. A few are positive, and some more are so near zero that they give an unreasonably high threshold. However, there is a fairly well-defined and narrow distribution, about 3 sigma below zero (thus, downward curvature).
A histogram of RMS residuals over all pixels and all frames, using the individual linear fit (in the 15000-47000 fitted range) to each, appears as Figure 4.
A similar histogram of RMS residuals over pixels and frames, using the individual quadratic fit to each, is in Figure 5.
The linear RMS residual is about three times the quadratic one over the same range, and the distribution of quadratic residuals is also a little narrower than the linear.
Parallel information for the south H and K arrays will be available shortly, although there are some gaps in the available K flats sequence that may affect the results.
Next step beyond will be to redo the analysis with separate fit ranges for the linear and quadratic fits. The linear fits will be done starting with a lower threshold, and the quadratic ones in a higher range (eg., 35000-55000 DU) where curvature is strong.
Figure 6: H Camera Saturation Curve
The threshold histogram for H seems too good to be true, but a careful look at the saturation curve (Fig. 6, above) indicates that the 1% criterion applied to the median data actually is 52000 to 53000, as indicated.
Here is a collection of figures showing saturation information for all six cameras. Notes on the figures are at the end.
NOTES ON THE FIGURES:
Saturation Curve: Summarizes the data on which all the other figures are based. The x-axis is (READ1-DARK1) and should be a measure of illumination. The y-axis is the raw READ2 in DN, the rationale being that that is physically what characterizes the saturation. Other possibilites such as (READ2-DARK2), (READ2-DARK1) & (READ2-READ1-DARK) also seem reasonable to consider. The data are the frame median, and are shown as small dots if there are a lot of data and +'s if there are only a few. The dashed line is an unweighted straight line fit to the data in a restricted range where it seems reasonably linear; the lower fit limit is 8K to 15K, and the upper limit is 45K to 49K. The dotted line is a similar quadratic fit to the same data over the same range.
Linear Offset Histogram:
Besides the fits to the frame median data shown in the saturation curve,
both linear and quadratic fits have been performed to each unmasked pixel.
This histogram (NB log scale) is of the constant in the linear fits.
The messy dots scattered about are the masked data; they are excluded from
the statistics for median, mean, and sigma shown.
For both the linear and quadratic offsets, there is frequently double or
even quadruple structure apparently due to variations within the array
quads.
These can be seen more clearly in the images, below.
Linear Offset Image:
An image of the data in the histogram.
The stretch is 0-15K, the range of the corresponding histogram.
The images have just been added recently, and masked data are not
guaranteed to be excluded.
Colors are in spectral order, with black below blue and white above red.
Differences among the quads is typically obvious, as is structure within
the quad, such as vertical striping from odd-even effects, etc.
Linear Slope Histogram:
Slopes of the linear pixels fits.
Nominally the ratio of READ2 and READ1 accumulation times, about 26,
but some significant variations are present.
Dots are masked data; they are excluded from
the statistics for median, mean, and sigma.
Linear Slope Image:
Fitted slopes of individual pixels in the array, from the histogram above.
Slope images are typically spatially fairly smooth, at least on this scale.
Stretch is 10. to 35., the range of the histogram.
Color mapping within the stretch is as described for the linear offset image.
Quadratic Offset Histogram:
The constant term in the quadratic fits, for each pixel.
Information is the same as for the linear offset histogram described above.
Typically similar in form, except for a general shift to lower values.
Quadratic Offset Image:
The constant term in the quadratic fits, for each pixel.
Similar to the
linear offset image, above, except typically shifted towards the blue.
Stretch, color mapping, etc, are the same.
Quadratic Slope Histogram:
Histogram of the linear coefficient in the quadratic fits to each pixel.
Like the linear fit slopes, these are roughly 25, but shifted generally higher
due to the overall curvature, which is downward.
Information is the same as for the linear slope histogram above.
Quadratic Slope Image:
As for the linear fits,
slope images are spatially fairly smooth.
Details are as described above for the linear slope image.
Quadratic Coefficient Histogram:
Negative for nearly all pixels, at least if the data available
covers the range from below 20K up to the saturation region.
Typical value is around -0.001, but there is often considerable variation.
Quadratic Coefficient Image:
Typically spatially smooth.
Spots, probably due to the recent passage of bright stars, can be seen
in some of these images.
In this case, the stretch is from -0.003 to +0.001.
Threshold Histogram:
Histogram of the threshold for each unmasked pixel, defined as the DN
where the linear fit is more than 1% above the quadratic, as described
above.
In general one likes to see reasonable agreement between the median of this
distribution and the value derived from the linear percentile plot,
where the difference between the median data and the linear fit is 1%.
It can happen that some pixels are unreasonably high in this histogram.
A bad example is given by the north H camera on 970608n.
It is proposed to set the thresholds for such pixels to the median
threshold as previously determined.
Thresholds for "reasonable" have been tentatively picked based on
examination of the
hard saturation level of the median curves and of the percentile plots.
A further sanity check is given by the quadratic percentile plots, which
ought to look considerably better than the linear ones if the whole method
is working properly.
Note that this threshold histogram is fairly sensitive to
data range available in a given set of flats, and to the fit limits
chosen.
Further work on this issue is needed, such as
decoupling the linear and quadratic fit limits, which are currently the same.
Threshold Image:
This image has been processed so that masked pixels should be black.
Stretch is 25K-65K, so 45K is green.
The remaining figures are the same as the ones in the earlier theshold study of 1998, except for the addition of images for the fit residuals.
Some question has come up now and then about what are the best variables to use in the saturation analysis. The saturation analysis program has been revised since the previous (22 May) entry to reflect my most recent understanding of the subject, which is discussed here.
