I. Data Processing Overview

Data processing for 2MASS was conducted using the highly automated 2MASS Production Processing System (2MAPPS) which converted each night's raw imaging data from each telescope into calibrated images and extracted source lists. Processing was done in two stages. Preliminary processing was carried out while the survey was ongoing using a version of 2MAPPS that was allowed to evolve and improve continuously as our knowledge of the telescopes, cameras, detectors, atmosphere and the near infrared sky improved. The 2MASS Sampler, First and Second Incremental Data Releases were derived from the results of the preliminary data processing. A second, complete processing of all survey data following the completion of the survey observations was designed to reduce all survey data with a single, stable software platform, 2MAPPS v3.0. 2MAPPS v3.0 incorporated lessons learned during the preliminary processing and feedback provided by users of the Incremental Data Release products. It also made use of reference datasets not available earlier, such as the Tycho 2 astrometric reference catalog, and information gathered from analysis of the complete survey data set. 2MAPPS v3.0 also contained numerous upgrades that improved the general quality and yield of data from the survey. Finally, 2MAPPS v3.0 included more extensive Quality Assurance tied explicitly to metrics derived from scientific investigations of the data.

II. Upgrades in 2MAPPS v3.0

2MAPPS v3.0 included all of the modifications to the major subsystems proposed during the June 2001 External Review Board Meeting, except for one - implementation of active deblending in the profile-fit photometry routines. Numerous other modifications were made to fix minor bugs, improve runtime efficiency, increase data yield, and improve quality assessment of the end products. The list below provides a summary of all top level improvements that were integrated into 2MAPPS v3.0. The Detailed Documentation links following most of the items connect to detailed on-line descriptions of each of the upgrades.

• Instrumental Calibration



1. Use of 2-dimensional saturation map for each array. A constant saturation threshold value for all pixels in each array was used during preliminary processing. (Detailed Documentation)
2. Best time-Dependent Canonical Dark Images and Flat-field Images generated using all survey data. Flat-field images incorporate corrections for cross-scan biases observed in point source photometry. (Detailed Documentation - Darks) ( Detailed Documentation - Flats)



• Atlas Images



1. Meteor trail detection and blanking for Atlas Images implemented. ( Detailed Documentation)
2. Improved frame background "tilting" in Atlas Image construction to deal with high-order spatial structure in frame backgrounds. Detailed Documentation)
3. Atlas Image backgrounds provide better represenation of true Poisson noise for setting detection thresholds. Result is that Atlas Image backgrounds may not match smoothly in-scan. (Detailed Documentation)
4. Single pixel coverage allowed into Atlas Images to minimize "holes"



• Point Source Characterization



1. Improved noise estimator for establishing source detection thresholds. Less sensitive to low frequency background structure.
2. Saturated R1 star detection, centroiding and brightness estimation implemented using 1-dimensional radial profile fitting. (Detailed Documentation)
3. Improved set of point-spread-functions and variance maps developed for all arrays and all hardware periods. Results are better uncertainty estimation vs. brightness and chi-squared statistical distributions. (Detailed Documentation)
4. Used seeing-dependent look-up table values for aperture photometry curve-of-growth and normalization of R1/R2 photometry. This minimized errors due to small number statistics in empirical derivation on scan-by-scan basis. (Detailed Documentation)



• Extended Source Characterization



1. Higher-order background fits used for extended sources measurements to mitigate H-band airglow structure
2. Training set for decision tree used for extended source classification expanded using data from full survey.
3. Improved robustness of photometry in crowded regions by using an iterative fit/subtraction of foreground stars. (Detailed Documentation)
4. Galaxy completeness improved in high source density regions by ~0.5 mag through better multiple star discrimination. (Detailed Documentation)
5. Specialized processing for Large Galaxies. (Detailed Documentation)
6. Extended source coverage maps generated. (Detailed Documentation)



• Astrometry



1. Higher density Tycho 2 Catalog used a primary astrometric reference catalog. (Detailed Documentation)
2. Measure and momitor focal plan distortion and included in source astrometric reconstruction. (Detailed Documentation)
3. Used position/uncertainty correction tables derived from preliminary processing to propagate astrometric solutions across scan boundaries. This improves performance in regions with low astrometric reference and 2MASS star densities. (Detailed Documentation)



• Artifact Identification



1. Improved identification of artifacts from bright star artifact enabled by saturated star brightness estimation and updated geometric algorithm parameter derivation. Detailed Documentation)



• Photometric Calibration



1. Derived improved secondary standard list compiled from full survey. Best possible internal consistency and rejection of variables. (Detailed Documentation)
2. Time variable atmospheric extinction model used (varies by month) derived from global calibration solutions from entire survey. (Deatiled Documentation)
3. Employed higher order fits to nightly zero point offset measurements. Piece-wise fit used in the J-band, linear fits used for the H and K_s bands). These fits produce lowest overall residuals for calibration stars.
   (Detailed Documentation)



• Minor Planet Identification



1. Newest ephemerides used for asteroid and comet identification.
2. Planetary satellites identified. (Detailed Documentation)



• Quality Assurance



1. Greatly expanded automated quality review of nightly processing. Introduction of QA diagnostics based on scientific metrics. (Detailed Documentation)

III. Active Deblending

Active deblending refers to the process of fitting two or more point-response-function components to a single detection if the chi-squared goodness-of-fit for a single component is above some threshold. Although the testing and analysis of the active deblending software showed that it worked well under most conditions, the Science Team opted to not implement it in final processing. We were led to this decision primarily because the slight astigmatism in the 2MASS optical system produced point-spread-functions that varied slightly across the focal plane. The degree of variation was exaggerated when the telescope was out of focus, and this could have led to preferentially deblending single objects at the edges of the scans. Small focus errors were not uncommon early during nights on nights with large diurnal temperature differences and rapid cooling after sunset when the temperature drop was faster than the response of the telescope's autofocus routines.

IV. Final Processing Summary

Final 2MASS Survey processing using 2MAPPS v3.0 began on 2 August 2001 and was completed on 6 February 2002. All survey data acquired under photometric conditions were included in the final processing. This included scans of 70,712 Tiles taken on 1415 survey nights (691 northern and 750 southern) spanning 7 June 1997 UT to 15 February 2001 UT.

Final processing generated Working Source Databases containing data for 1,314,981,867 Point and 2,590,500 Extended source extractions, and an Atlas of 4,879,128 calibrated FITS images, from which the All-Sky Release Products were drawn.

Last Updated: 31 January 2003
R. Cutri - IPAC