Initial Analysis (K. Xu)

These plots here show the effect of applying the seeing correction to the list of stars which Kevin used in his analysis. Points are colored red without correction and blue with correction, and the magnitudes plotted are the difference between v3 and literature mag versus literature mag. Stars plotted beyond mag 5 can be ignored since they are not saturated (they are included here only because they were listed in Kevin's table). Any decrease in scatter is not very obvious, but some of the outliers are brought in.

Effect of seeing correction, Kevin's 254 stars, J
Effect of seeing correction, Kevin's 254 stars, H
Effect of seeing correction, Kevin's 254 stars, K

Analysis of Colors from Working Database

Rd_flg	Number
113	3826
123	0
131	6075
132	0
133	7526
213	2
223	0
231	0
232	0
233	0
311	1386
312	0
313	556
321	0
322	0
323	0
331	1883
332	0
333	22330

In other words, there are 22330 stars R1-saturated in all three bands, 9965 in two bands and 11289 in one band, for a total of 43584 stars.

The following plot flashes between non-corrected and corrected color-mag and color-color diagrams for the three band saturated stars. It is readily apparent that the seeing correction increases the scatter in the color-color plot.
Comparison of corrected and non-corrected colors

RMS values in H-K were measured in two 0.05 mag wide bins centered on J-H=0.5 and 0.74 (see also K. Marsh's page, section 3.2, which uses these same measures):

	Uncorrected	Seeing Corrected
RMS(H-K) at J-H~0.5	0.092928	0.110140
RMS(H-K) at J-H~0.74	0.111334	0.159364

Separating the data into seeing shape bins failed to indicate any trends associated with bad seeing It was thought that the seeing correction might work best for the worst seeing and just increase the scatter in good seeing, but this was not the case. A marginal improvement in the RMS was seen in one bin, while for all the others the scatter increased by 0.02 mag.

What did appear to make a significant difference for the better was using the correction on stars that were R1 saturated in only one or two bands. This behavior is consistent with what was noted by the QA reviewers (see colormag and QA review for 981023s). Read-1 stars saturated at H (green points) often appeared displaced on the colormag plots, and in the case of 980123s, Kevin showed that applying the seeing correction (without the cutoff values mentioned in his report, however) gave these stars more realistic colors. The following plots show the RMS values as calculated above, plotted against seeing shape for all one or two band R1-saturated stars (the seeing corrections were applied only to the saturated bands). Again, red points are not seeing corrected, while blue ones are.

Plot for J-H~0.5
Plot for J-H~0.74
(these are postscript at present; gif files will take their place eventually).

Here are animated Hess plots of these stars. The improvement in the RMS is clear. Roc had the following to say about this issue:

"I think that this is finally starting to make some sense!

Imagine if the slope of the seeing correction for saturated R1 stars was the same in all three bands. Then stars which were saturated in 3 bands would have nearly the same corrections applied in each band (differences due only to the slightly different shape value in each band). The expected change in colors with the correction is minimal since each band changed similar amounts. Admittedly, this doesn't explain why the RMS at fixed color ranged goes up, but it does help understand why the colors would not be significantly affected.

The fact that the RMS are significantly improved for stars which are saturated in <3 bands argues strongly that the seeing corrections **should** be applied for any saturated band. It indicates that there is a serious bias between non-corrected saturated R1 and non-saturated R1 (or R2 for that matter) in band seeing. As you point out, that is exactly what the QA reviewers saw!"

So what is the correlation of the seeing correction between the bands? This plot, analogous to a color-color diagram, shows the color of the seeing correction for 3-band saturated stars. The sign on this plot indicates how the correction changes the color of the star. The points are coded as follows:
black: jsh < 1.0
blue : 1.0 <= jsh < 1.1 (overlaps most of the black points on the plot)
green: 1.1 <= jsh < 1.2
red : jsh >= 1.2
One notices a trend immediately: the worse the seeing, the more 'colored' the correction. Worse seeing pulls the correction color into redder H-K, while keeping J-H unchanged. From the way the signs of the correction work out, this means that K needs relatively less correction in bad seeing than do J and H. Moreover, for a majority of the stars, the seeing correction does not significantly change the color. Median values of the black and blue points (96.5% of the data) are (-0.015,-0.002) and (0.015,-0.04) respectively. Since the seeing correction does not significantly alter the color, plots based on color are of rather limited use as a diagnostic tool for evaluating whether or not to apply the correction.

Analysis of cal star repeats

Magnitude Estimates vs Shape for Calibration Field Sources Saturated in Read1

Please note that for the fainter sources, both saturated and non-saturated measurements are included. The non-saturated ones form the clear lines of zero slope and little scatter (especially clear in the 2nd plot). Further examination of these plots shows a number of important points:
1. Residual slope is frequently seen, especially at the faint end in J, indicating that the magnitudes are generally undercorrected.
2. Even though there is some residual slope, nothing is made worse, and there are clear improvements in many cases.
3. A few cases of over correction are seen (most clearly at H; see the 3rd to last plot), but given the scatter in the mags, these seem minor.
4. A bias of 0.2 mag between saturated and non-saturated J magnitudes is clearly shown in a few plots, in the sense that the saturated mags are too faint. If these mags really are too faint, this would lead to an insufficient seeing correction. It is interesting to note that it is exactly those plots that show the bias that also show the most obvious residual slopes.

Can we do better?

Follow-up 4/24/02

Star	RMS	RMS(seeing corrected)
J
1	0.192	0.150
2	0.085	0.078
3	0.123	0.106
4	0.259	0.149
5	0.280	0.158
6	0.157	0.141
7	0.000	0.000
8	0.255	0.160
9	0.000	0.000
H
1	0.167	0.104
2	0.166	0.092
3	0.161	0.100
4	0.221	0.115
5	0.216	0.121
6	0.158	0.125
7	0.151	0.113
8	0.219	0.129
9	0.171	0.052
K
1	0.171	0.128
2	0.143	0.095
3	0.132	0.093
4	0.257	0.174
5	0.198	0.133
6	0.138	0.104
7	0.124	0.109
8	0.234	0.170
9	0.000	0.000

Concerning the bias noted in point 4 above, there was some discussion between Gene and I regarding Kevin's equations being formulated with respect to a seeing shape of 1.0. However, as I am looking at his analysis again, this doesn't make sense to me. When Kevin derived the seeing correction, the data he fit included some points with seeing shapes less than 1.0, so there doesn't appear to be anything analogous to a "zero point" at 1.0.

Analyses During June 2002

More analysis of cal star repeats

Analysis of GKM Giants

B-J versus shape
B-H versus shape
B-K versus shape

Stars have been binned into whole mag ranges with the following color codes: black (non-saturated), red: mag <2, green: mag 2-3, blue: mag 3-4, magenta: mag 4+. Straight lines were fit for all of these categories (unfortunately teplot cannot color them), the goal being to determine the slope in mag vs. shape and also the intercepts, the shape values where the lines cross zero color excess. Since the slopes appeared to be similar in all mag bins, an average slope (designated b below) was determined at each band, and the intercepts at zero color excess were derived by refitting the saturated mags with these average slopes. The correction (cs) to be subtracted from the saturated mags was expressed as:

cs = b * (ssh - a)

where

a = c1 + c2*m + c3*m*m [for m>1.5]
a = c1 + c2*1.5 + c3*1.5*1.5 [for m<1.5]
b = average slope of color excess vs. seeing shape from all mag bins
c1,c2,c3 = parameters of 2nd order fit to cs=0 versus magnitude
m = magnitude
ssh = seeing shape

Here is a table of the parameters involved. Noted below is the fact that these values are only valid for data taken from the south.

Correction Parameters for Southern Camera Only
Jump to BEST paremeters

Because the average mag in the brightest bin is about 1.5, the 2nd order fit is not known to be valid at brighter mags. The correction at brighter mags is therefore set equal to that at mag 1.5 at the same shape. This action prevents ridiculous results at the extreme bright end of the mag range. At the faint end, stars just fall out of saturation, so the behavior of the 2nd order fit at fainter mags is not an issue.

In some intermediate trials, I repeatedly determined the intercepts relative to the fits made to the non-saturated stars, which in all cases were close to, but not exactly zero. None of these resulted in good K0 color-mag diagrams. The shifts, particularly in J-H were always too much by a few tenths of a mag. Another trial was done only using stars at galactic latitude > |30|, again with bad results. However as soon as I calculated intercepts at a color excess of zero, effectively assuming that Roc's derived colors were the truth as opposed to the colors of a subset of just-undersaturated stars, the color-mag diagrams became much more reasonable.

[Color-magnitude plots are available in a section near the bottom of the page]

This is clearly the best algorithm I have developed for calculating the seeing correction, but it is not without faults. Subtle mag-dependent shifts still exist, and if one can believe the small number statistics of the very brightest stars, a wiggle in the J-H and J-K colors is still present. Still, the size of these effects is small when considering the size of the error bars these stars have.

A more serious problem came to light when applying the correction to the cal star repeats: the terms used in calculating the correction are only good for southern hemisphere data! I hadn't checked it, but only about 1% of the giants in Roc's lists were observed from the north. I had also seen that there was a suggestive difference in slopes between north and south in the cal star repeats, but I chose to ignore it. Therefore, the task must be repeated for northern data in order to derive a procedure for correcting data from that hemisphere.

Since no equivalent of the Michigan Spectral Atlas exists for the northern sky, the analysis for the northern camera was more limited in terms of data, being based on a search for GKM giants with SIMBAD. A similar procedure as outlined above was used, and the following parameters were derived:

Correction Parameters for Northern Camera Only
Jump to BEST paremeters

The one difference here was that I used the average slopes determined from the cal star repeats, since they appeared to be much better determined than those shown by the giant star plots.

Plots of LogN-LogS

Plot 1
Plot 2

Rae began to generate similar plots of number counts before and after the seeing correction. Here are the plots using the parameters from the tables of parameters shown above.
South J number counts vs. mag
South H number counts vs. mag
South K number counts vs. mag
North J number counts vs. mag
North H number counts vs. mag
North K number counts vs. mag
In these plots, the uncorrected data show marked dropoffs around where read-1 photometry becomes saturated. The corrected data for J and K shows clear improvement with less severe or even no dropoffs. The H plots, however are not improved.

Could the situation at H be improved by using different parameters? I noticed that the derivation of the southern slope parameter at H was being biased toward that of the brightest magnitude bin (b=-2.59 including all mag bins, and -3.03 if the brightest bin is omitted). H is the only band where this effect was seen; the slopes at J and K show less scatter.

New H Correction Parameters for Southern Camera
but see discussion below.
Jump to BEST paremeters

The northern data is just harder to work with, as noted already. The slopes of delta mag versus seeing shape show much more scatter. At first, I derived a new average slope by omitting the brightest mag bin from the giant star plot and including the slopes derived from the cal star repeats. The slope parameter b changed from -2.51 to -2.08, which as it turned out moved it in the wrong direction.

New (bad) H Correction Parameters for Northern Camera
Jump to BEST paremeters

Since this plot is clearly much worse than before, I had another look at the scatter plots and decided that the most accurately determined slope came from two cal star repeats that showed the largest range in seeing shape. From these, the b parameter became -2.75.

New (good) H Correction Parameters for Northern Camera
but see discussion below.
Jump to BEST paremeters

The dropoff is still there, but it is less than before.

The fact that the number counts of bright stars are improved by adjusting the photometry according to the seeing correction is further confirmation of the analysis presented here.

On Dec 9, an error in the determination of the new H parameters was discovered after colog-mag plots were made. The parameters immediately above drive the colors involving H far away from what they should be. This effect is a result of a faulty derivation which modified the slope parameter but did not correctly recalculate the fit involving c1,c2,c3. A correct derivation was done which preserves the fairly good color-mag plots already obtained, but with the modified slopes. For the sake of clarity, the full set of new, best parameters is presented here.

New, Best Correction Parameters for Southern Camera

New, Best H Correction Parameters for Northern Camera

The attached plot shows the distribution of J, H and Ks magnitude differences
(Cohen-Walker - 2MASS) versus CW magnitude for the 602 CW4
calibration stars kindly provided by Martin.  The CW magnitudes are
synthetic photometry derived by integrating the CW4 spectral templates
for these stars over the 2MASS bandpasses.
Note that the J-band templates cut-off at ~1.2 microns, so the
CW J magnitudes are lower limits to the true irradiance.  As expected,
the 2MASS magnitudes are systemmatically brighter than the CW mags.
The J-plot is included only for informational purposes.

The trimmed average and RMS of the magnitude difference distributions
are given in each panel of the plot.  There are a couple of outliers
in each band that fall outside the range of the plots.
Virtually all of the CW calibrators are saturated in 2MASS R1 exposures,
although there are a few non-saturated R1 sources, and even
one non-saturated R2 source.

None of the saturated R1 sources has had seeing corrections applied
to the 2MASS photometry.  Nonetheless, the RMS of the distributions
are all less than the typical saturated R1 magnitude uncertainty
quoted for the 2MASS sources.  Thus, the 2MASS uncertainty
estimates for saturated R1 sources are conservative.  There are
a few outliers that have underestimated uncertainties.

There is no evidence for any systemmatic magnitude-dependent bias
between the CW synthetic photometry and 2MASS photometry.

Roc