Analysis of Scoring Changes between Version 2 and Version 3 Processing
The qasit file generated on 010815 contains all of the nights (except for
970615n) included in the v3 OPS test. Using this qasit file, scans whose
scores changed between v2 and v3 processing have been analyzed to discover
why the scores are different.
In total there are 3262 scans in this sit file, of which the vast majority
(85.5%) had no score changes between v2 and v3 processing. A total of 464
scans had changes in scoring, and
this histogram shows the their distribution versus amplitude of the change.
Those with improved scores outnumber those with degraded scores by a factor
of 2.5.
Listed below are the causes for each of the scoring changes and
what their extrapolations suggest for the rest of the data (the full v3
complement of 70744 scans) --
Photometric dispersion downgrade changed
148 scans improved
0 scans degraded
Only good news here.
Photometric overlap downgrade changed
91 scans improved
7 scans degraded
This is mostly good news. The degraded scans were all from a single sci
block on 980815s. These all fell from Q=10 in v2 to Q=7 in v3. Thus, new
overlap downgrades are not expected to degrade an appreciable number of
scans below catalog quality during v3.
Photometric interval downgrade changed (too few cals in the photometric interval)
80 scans improved
0 scans degraded
Only good news here.
Cloud downgrade changed (cloud suspected before and not there, or newly found)
2 scans improved
8 scans degraded (all to Q=1)
Three of the degraded scans are from the first night of the survey
(970607n), and the other degraded scans came from a single sci block on
980322s (the 3rd good night of southern OPS). Both of these night were run
in v2 before many of our diagnostic cloud-catchers were in place. We conclude
that these numbers are not representative of the rest of the survey, and
hence newly discovered clouds should affect only a small number of scans.
Sensitivity scoring changed
5 scans improved
11 scans degraded
The eleven degraded scans had delta Q = -2 or -3. For only one of these
did the new Q fall below 5. This implies that ~22 more scans in the rest of
the sky will fall below catalog quality during v3 because of sensitivity
toggling.
Airglow scoring changed
0 scans improved
7 scans degraded
Since this downgrade is an automatic Q=10 or Q=1, it is not possible to
get improved scans here since those aren't run in v3. The 7 degraded scans
suggest that ~152 scans total will be dedgraded in v3 to Q=1 because of
airglow.
Aspect ratios toggled over downgrade threshold
0 scans improved
46 scans degraded
Of the 46 degraded scans, 45 came from the first week of northern OPS data.
The other one came from another early night, 970712n. Again, these numbers are
not representative of the rest of the survey, so aspect ratio downgrades are
not expected to downgrade many other scans during v3.
Untracked seeing toggled over threshold
0 scans improved
2 scans degraded
Both scans (of course) dropped to Q=1. Extrapolation suggests that ~43
such scans could be so effected in v3.
Maximum seeing shape toggled over threshold
0 scans improved
2 scans degraded
Both scans (of course) dropped to Q=1. Extrapolation suggests that ~43
such scans could be so effected in v3.
Non-uniform v2 scoring (scoring rules changed after night was run)
3 scans improved (all by delta Q=+2)
51 scans degraded (all by delta Q=-1)
None of these previously ok scans dropped below catalog quality, so this
effect is not likely to produce many problems in the rest of v3.
The Bottom Line
It is expected that ~260 scans of the 70744 run in v3 will fall below
catalog quality (Q<5) due to sensitivity, airglow, untracked seeing, or
max seeing shape downgrades. The other causes for downgrades listed above
are believed to affect only isolated cases -- clouds have degraded 8 early
scans below catalog quality, and 46 scans (all from early northern nights)
have been similarly downgraded due to poor aspect ratios
D. Kirkpatrick - IPAC
Updated 17 August 2001