(Note that all the analysis results shown here are based on just a sample of the frames in the flat scans, up to about 20% of the total in the most densely populated regions of the saturation curve.)
This is a curve of READ2 ADU versus the array median of (READ1-DARK):
Figure 1: J Camera Saturation Curve
Note the saturation around 52200 ADU, the turn-up at the low end, and the slight negative curvature in the middle.
The following are for a linear LSQ fit to each unmasked pixel, with the data used in the fit restricted to the range read2 ADU = 13825-43076. This is the range where the saturation curve is reasonably linear, avoiding the low-end turn up evident in the curve above.
This is the RMS residual for the model. Over the 13825-43076 interval, the median residual is about 170 READ2 ADU.
Figure 2: Histogram of RMS Residuals
The following is a histogram of the residuals for the array at just one ADU value, 43076, which turns out to be roughly the limit, based on a 1% max contribution to the relative error, for a single threshold for the entire array.
Figure 3: Residual Histogram at ADU = 43076
The median residual is -411, or about a 1% error. (Note that, based on protocamera results, 43000 had been used previously. At least for 6/8/97 for the J-camera, this still appears to be a reasonable choice.)
The following summarizes the residuals for the linear model. Each point shows the median, +/- 10%, and +/- 1% tails for a histogram such as the one above. Input data points are indicated by +'s on the median curve, and the subset of the data (from the linear region of the saturation curve) actually used in the fit, by the heavy segment overlaid on the zero line.
Figure 4: Summary of residual distribution vs. READ2 ADU, percent
The peaks in the preceding graph around 15000 and 18000 correspond to the glitches in the READ1 array sigma values noted previously. Note that the overall shape of the residuals from the linear model would be expected to be quadratic, as observed, if the higher order terms are not too large.
Again, based on a 1% criterion for the percent deviation of the median residual, it appears that a single-array threshold of 43000 is satisfactory for the J-camera on 6/8/97.
Because the array saturation curves show a curvature noticeable to the eye even in their most linear part, a set of quadratic fits was also performed, corresponding to the linear model above. It was originally thought that this model would be helpful in studying any higher-order non-linearity that comes in as hard saturation is reached.
Fitting to the same range as above, the quadratic co-efficients A2 are shown below:
Figure 5: Histogram of quadratic co-efficient, A2.
Note that practically all pixels share the negative curvature obvious by eye, and that aside from a slight asymmetry towards more negative values (larger curvatures) the histogram indicates that the distribution is well-behaved.
This is the RMS residual for the quadratic model. Over the 13825-43076 interval, the median residual is about 90 READ2 ADU.
Figure 6: Histogram of RMS Residuals
Evidently the quadratic model does fit the pixel saturation data much better than a linear one. This suggests that using a quadratic model may be helpful in diagnosing subtle array problems and characteristics which might be missed in the nearly two times greater RMS noise of the linear model.
Histogram of the residuals for the array at ADU = 43076.
Figure 7: Residual Histogram at ADU = 43076
The median is -71. Besides being much narrower and centered closer to zero, the distribution lacks the skew seen in the corresponding linear histogram. This is presumably due to to the skew being absorbed into the A2 co-efficients, which have removed the curvature effect so that it is not seen in the raw residuals.
Summary of the residuals for the quadratic model analogous to that for the linear model above.
Figure 8: Summary of residual distribution vs. READ2 ADU, percent
Here we expect the overall shape of the residual curve to be a (negative) cubic, especially considering the known saturation and turn up at low ADU, and again assuming the higher-order terms are negligible; and this is indeed what we observe.
Three things are worthy of note: first, the x_sig glitches noted in the linear summary are much more prominent, underlining the usefulness of the quadratic model for diagnostic purposes. In fact, it is apparent that the peak around 32000 ADU is likely to be anomalous, and a re-examination of the data for x_sigma shows that it is essentially like the peaks at 15000 and 18000. (Note that the widths of the peaks is similar in time, though not in READ2 ADU.) There is a data gap between about 20000 and 25000, so the character of the anomaly around 25000 is unknown. Second, the median residual does not reach -1% at the upper end until 49000 ADU. This probably means that the saturation threshold (in view of the nice behavior of the quadratic fits) at 43000 is essentially due to the departure of the linear model from the quadratic, not due to the sudden appearance of higher order terms. Third, the median residual below 49000 ADU is much less for the quadratic model than for the linear, so it seems reasonable (in view of the somewhat arbitrary nature of the 1% limit on relative error) to use the quadratic as a surrogate for the expectation value of the data in computing individual pixel thresholds for the array. That is, I propose to obtain such thresholds by the condition, for pixel i, that as a function of ADU,
1% < [lm(i) - qm(i)]/qm(i)
where qm(i) is the quadratic model for the ith pixel, and lm(i) is the linear model.
The figures below present parallel information for the H-camera.
Figure 9: H Camera Saturation Curve
Saturation is around 55300 ADU. Other features are similar to J.
For the H-band camera the fits were to the range 17525-47860 ADU in READ2.
Figure 10: Histogram of RMS Residuals.
The median is about 128 ADU. The additional structure seen (compared to the J-band, Figure 2) is probably due to a systematic difference between array quads. This is yet to be investigated.
The following is a histogram of the residuals for the array at READ2 = 47860 ADU, which is roughly the limit for the H camera, based on the 1% max contribution to the relative error.
Figure 11: Residual Histogram at ADU = 47860.
Mean residual here is -608, around 1.5%. Note the additional structure compared to the J camera, as expected from Figure 11.
The following summarizes the residuals for the linear model from 17525-47860.
Figure 12: Summary of residual distribution vs. READ2 ADU, percent.
Based on protocamera results, a threshold of 43000 had been used previously. For 6/8/97 for the H-camera, this appears to be somewhat more conservative than necessary. Based on this plot, an H-camera threshold of 46000 seems appropriate.
Fitting to the same range as above, the quadratic co-efficients A2 are shown below:
Figure 13: Histogram of Quadratic Co-efficient, A2.
Again we see the same kind of structure, as foretold by Figures 11 and 12. A new feature is that a group comprising a small fraction of the pixels (perhaps 0.3%) seem to have no or even positive curvature.
Figure 14: Histogram of RMS Residuals.
Again the RMS residual for the quadratic model is extremely clean. The median is 78 ADU. It appears that the structure seen in the linear model can be mostly understood as differences in curvature for individual pixels.
Figure 15: Residual Histogram at ADU = 47860.
Mean residual is -69, and histogram is very clean, though not so pretty as for J band camera.
Summary of the residuals for the quadratic model from 17525-47860, analogous to that for the linear model above.
Figure 16: Summary of residual distribution vs. READ2 ADU, percent.
Comments parallel Figure 8, above.
The K-band camera does not saturate during the flats taken on 6/8/97. Data from 980106n has now been analyzed to obtain the thresholds. Here is the response curve:
Figure 17: K Camera Saturation Curve
Other figures parallel those above for J and H:
Figure 18: Histogram of RMS Residuals
Figure 19: TBS. Residual Histogram near upper threshold.
Figure 20: Summary of residual distribution vs. READ2 ADU, percent
From this we conclude that 46000 is a good threshold for the K-camera in these data.
Figure 21: Histogram of quadratic co-efficient, A2.
Figure 22: Histogram of RMS Residuals
Figure 23: TBS. Residual Histogram near upper threshold.
Figure 24: Summary of residual distribution vs. READ2 ADU, percent
The major excursions in the summary plots, especially evident in the quadratic summary plot around 14500, 27000, and 45000, again seem to correspond to obvious glitches in the x_sigma values described previously. (Recall that the abscissa x in the array saturation plot is the array median of READ1-DARK [masked]; x_sig is the RMS of x. Similarly the ordinate y is the array median of READ2 [masked], and y_sig its RMS over camera array. The cause of the glitches seems to be substantial segments of rows of pixels, along the quad boundaries, which vary by roughly 2000 ADU from time to time.) The following plot shows this for the K camera on this date:
Figure 25: RMS of array values during morning flat sequence, versus array median.
Note that the two panels do not correspond vertically; one must refer to the saturation curve (Figure 17) to link up the peaks in one with those in the other. However, based on this plot, both the linear and quadratic fits for K have excluded data points taken during the obvious large peaks in x_sigma. Eventually improved analysis of the J and H cameras will give similar plots for them, and allow the saturation model for J and H to be improved. However, at least for these K data, repeating the analysis with and without the omission of the peaks in x_sigma affected the threshold value by only a few 100's, a negligible amount.
See later write-up on this topic.
