> 
>         1. Inject Calmon Zero points (note: these zero points come
>         from the V2 pipeline, and hence are subject to change
>         during the cat generation process; the changes, however,
>         are likely to be insignificantly small)
> 
> Weren't there occasional problems with this still in V2? If so, is
> there some way for us to identify and fix those in advance of using
> the numbers for V3?

Good point.  I'll await what Roc has to say.
As long as the corrections stay within 10% of the
true values, I think we are ok.

> 
>         2. Improved Large Galaxy Processing. Modification
>         allowed processing of all large galaxies, including M31.
>         More modifications/details given below. The down side to
>         this improvement" are the inevitable addition of bogus
>         sources induced by these bright extended objects. This
>         chaff will need to be carefully removed during the cat
>         generation process.
> 
> I thought you were referring to processing of areas that were formerly
> blanked out based on cataloged objects, but I see that's under #5. Can
> you describe what specific changes you made here?

Yes, do not mix-up the two mods.  The large-galaxy
processing mods are: (1) enlarge working area (template from which
I process each indiv object, (2) apply differnt background removal
and (3) enlarge the postage stamp images.  Lots of minor mods
to make things work robustly (e.g.., nearby star subtraction
is tricky because the large galaxy is full of structure that
are resolved as point sources.

Now, I think much of this is a moot point.  Big galaxies require 
offline special treatment.  That is the subject of my AAS
poster.  GALWORKS is attempting to get some basic numbers and
shove out a postage stamp that may or may not be helpful (depends
on size and prox to scan edge).  I think of this as a more
detail "placeholder".  *The mags are surely compromised by the
background subtraction, scan edge and foreground contamination.*
V3 just does a better job than V2.

> 
>         4. Coversion to SNR thresholds.
> 
> Can you say more precisely what the thresholds are (and how SNR
> is calculated at low b's)? I'm guessing there may be some "edges"
> where processing changes at certain latitudes, and that we might
> want to look for "jumps" across those edges (or reprocess scans
> near the edges using the alternative methods to make sure there
> are no large changes in the processing).

Good question.  I'll just take a little walk into the code, back
in a bit ...

Ok, the SNR is computed from the 7" aperture photometry,
where SNR = 1.0857 / dmag7
(dmag7 is the mag uncertainty in the mag7 measurement)

The SNR *limit* (threshold) is a function of the confusion noise:
        SNR limit = 5.0 * 10**[(cnoise-0.5)/2.5 
where cnoise is the expected confusion noise "mag"
for a given K-band stellar number density (K < 14.0 mag).
For low density fields, cnoise = 0.0 mag, and the SNR limit
is ~3.2.  For a moderate density, say 3.5,  the cnoise is
0.5 mag and the SNR limit is 5.0.  For high density, say 4.0,
the cnoise is 1.2 mag and the SNR limit is 10.0. 
The SNR limit cannot exceed 25.

The only jump that you would see would be the coadd area constraint
itself.  The density (and hence cnoise) are computed per coadd.
So the SNR limit can "jump" between coadds.

I've invented this kind of SNR formula to aide with reliability in
the confused regions.  The SNR itself does not fully do the job
of throttling away stars.  Raising the bar was the only way to 
eliminate stars while retaining high SNR galaxies.

Now, to go deeper in the g-plane, all we need to do is adjust the
function.  The result will be more sources and slower runtime.
Both I have tried to minimize (note: galworks V3 already runs
significantly
slower than the V2 version).

Still, the question remains to whether I have introduced a completeness
bias (as a function of stellar number density) that is bad news for
uniform surveys.  Chester is probably the best person to contemplate
this.
*ATTENTION TOM CHESTER*

> 
>         5. Remove large galaxy bias. That is, we process large
>         galaxies without any bias as to its nature or even apriori
>         existence (i.e., process blind).
> 
>         6. Improve background determination for scans with large
>         galaxies.
> 
> Again, can you describe what you specifically changed here?

I still use the NED seed catalog to help with background removal.
We need to know if a NGC/Messier is nearby.  But GALWORKS V3 will
process
the object in the normal fashion, *including* elimination of the
object if it fails various tests (e.g., SNR, Gscore, etc).

Improved background removal is this:  if a large galaxy (D>5') is
nearby, I do not apply the nominal background removal.  Instead I apply
a 
1st order (linear) fit across the scan (inscan), in which
I have ignored coadds with the big galaxy.  I also avoid bright stars.

So, the upshot is I apply a background removal that is geared to large
scales, many coadds in size per parameter of the fit.  Hence, we
completely
lose the ability to subtract airglow and resolved background gradients.
A bitter pill indeed.  That is why I think the mags for large galaxies
are compromised.  Better to do these extraordinary (in size) objects
by hand.


> 
>         7. Enlarge the GALWORKS working "area" for large
>         galaxies. That is, process galaxies with larger working
>         areas, thus capturing more of their flux. The postage
>         stamps are now allowed to grow as large as 5' (and hence,
>         more disk storage is needed).
> 
> What is the algorithm for deciding when the image is larger enough?

The initial aperture photom mag.  The brighter the source,
the bigger the area -- this is the working area.
For known big galaxies, I use the maximum working area size
(120 arcsec radius).  Note that this method is not so great
for LSB galaxies that are not catalogued in NED.
I do some tweeks now that I have an interative system
(where I compute 3-sigma radii and moment ratios) in which I make
two passes at the working area (a big runtime draine I might add).

> 
>         8. Improved Star Subtraction. Modifications included
>         improvements to both stellar "subtraction" and 2-D ellipse
>         fitting. The basic algorithmic improvements are described
>         here. The primary driver is to improve photometry for
>         galaxies seen through the Galactic plane (and hence
>         confused by foreground stars). Highlights of the new
>         method:
>            For the first "guess" at the 2-D elliptical shape of the
>            galaxy, use the rotating 2nd moment algorithm
>            (described in the SIS /2massc/doc/design/sis/see03b).
> 
>            Before star subtraction, use the first guess shape of
>            the galaxy and subtract this from the image (i.e,
>            remove the galaxy). The galaxy removal is carried out
>            by using the firs-guess axis ratio and position angle.
>            For each elliptical annulus, compute the lower quartile
>            of the pixel distribution. The lower quartile is robust to
>            star contamination. Then subtract the LQ value from
>            each annulus pixel. The G-subtraction is carried out
>            until the point at which the LQ is deep in the noise.
> 
>            With the galaxy removed, examine the neighboring
>            stars. Subtract the star using the PSF "shape" ("sh") to
>            determine its size or extent and the local background
>            median. Subtract by using the local annulus median for
>            each star: (a) determine size of star, (b) determine
>            annulus size, (c) compute median, (4) using median,
>            determine how much of the star should be removed
>            from the non-galaxy subtracted image. In this way the
>            star is removed without losing any of the galaxy. The
>            danger is removing stars close to the galaxy nucleus.
>            Here the galaxy removal step is not as accurate (due to
>            our sampling and pixelization) and the local annulus
>            median is not well determined. Hence, for sources near
>            the nucleus, do not subtract the star, instead mask it.
>            The masked pixels are later recovered with isophotal
>            substitution.
> 
> What happens when there's a star close enough to the center of the
> galaxy to be confused with the nucleus? What happens when the galaxy
> is quite irregular?

Good question.  The star is incorporated into the galaxy. Eaten alive.
Also, the star *might* be the center of the "galaxy" -- at least
GALWORKS thinks the star is the nucleus.  This happens quite frequently,
a bright star near the nucleus becomes the nominal center of the
galaxy.

Irregulars and LSBs may not have well defined cores.  GALWORKS looks for
the brightest core and starts from there.  Works sometimes.
Sometimes GALWORKS peaks up on a star, captures the galaxy flux (or
most of it) and processes as usual.  You end up with a irregular
galaxy that is off-center.  The mags are not going to be right.

This is a nasty scenerio.  There is no robust way to improve it.
These guys require post-processing massaging.  I've been working
hard on just this sort of thing -- processing of postage stamp images.
I did this for the 6DF people.

> 
>            With stars removed, determine the true isophotal shape
>            (2-D ellipse) of the galaxy.
> 
>            This process is iterated twice to robustly determine the
>            ellipse shape.
> 
>            The modfication includes adding three new "planes" to
>            the postage stamp images.
>            Current planes: 1 = J, 2 = H, 3 = K.
>            New Planes: 4,5 6 = J, H, K with stars subtracted or
>            removed
> 
>         9. Re-built OBDT using updated training sets.
> 
>         10. Injected the OBDT classifier into GALWORKS V3.
>         GALWORKS now computes the E and G-scores on the fly.
>         It rejects sources whose E and G scores exceed a value of
>         1.75 (i.e., rather generous, erring on the side of
>         completeness, but still eliminating unwanted stars).
> 
>         11. Added Huchra star-galaxy classifier to the OBDT; see
>         Getting Rid of Stars
> 
> This is a fairly traditional kind of cut to use, but have enough bright
> galaxies been tested at this point to know that it won't do something
> nasty to them on occasion?

The HUCHRA stuff was added into the N-Dim OBDT, so it is absorbed and
cannot do too much damage or fully save the day.

I did a full battery of tests on the OBDT, with the intention of
improving C & R.  The tests are detailed in the noted memo.
Big galaxies survive, as far as I know.

> 
>         12. Constructed a better surface brightness metric
>         (half-light measures via the total mag extrapolation)
> 
>         13. Improved the Kron and Total photometry; see the
>         memo 2MASS Photometric Repeatibility
> 
>         14. Improved star subtraction for edge-on galaxies. The
>         improvement is to minimize subtraction of "sources" on the
>         major-axis of inclined galaxies (i.e., avoid subtracting
>         pieces of the galaxy from itself -- this is a major headache
>         for V2).
> 
>         15. Improved the postage stamp FITS cube images. They
>         are now *six* planes deep, with the first three planes the
>         nominal set (J, H, Ks), and the additional three planes the
>         same except with stars subtracted. These allow the user to
>         perform direct photometric measurements on the postage
>         stamps without the deleterious stellar contamination. The
>         FITS header has also been update (and improved). Here is
>         an example of a new postage stamp FITS header.
> 
> Between this and the larger maximum size, are we getting into trouble
> with disk space for the postage stamps?

This is TBD.  My initial estimates said that we are OK.
But this still makes me nervous.  I need some better numbers.
The RTBs will nail the issue.  Roc ain't gonna be pleased if
a TB of stamps are overflowing the archive disks.

> 
> I think your list of Useful Tests also looks very good. Which of these
> have already been carried out?

I'm now doing point-to-point mag comparisons (V2 vs. V3) and have
alreadly identified a nasty bug with CALMON.  I'll write this up
as I go ... next week probably.

Cong has done some stuff with "asymmetric" objects.
But other than that, we have done very little with the RTBs.
Note that I have been testing all year (and last year) using
my own set of RTBs.
  
I'll be cranking up the work in the next two weeks.
I'll keep a log or something that will help you track my progress.
Roc is going to supply some help.
-tj