Editorial note by Tom Chester, 29 May 1997: This is a 0th draft task list and schedule, which has not been reviewed within ipac yet. It is presented here solely to give a brief overview of some of the tasks that need to be done before version 2.0 of 2MAPPS can be completed. There are definitely items missing below, and the critical tasks of careful validation and parameter tuning will undoubtedly require significant effort not shown here. Since this has just been put together, we have not had a chance to review the consistency of the entire plan, so all dates must be viewed as preliminary. tchester DRAFT ** DRAFT ** DRAFT ** DRAFT ** 2MAPPS Task List and Schedule DRAFT ** DRAFT ** DRAFT ** DRAFT ** 1 June, 1997 (Note: Most subsystem work will include parameter tuning based on further testing with real three-channel camera data, although this is not specifically listed in many cases herein.) 1. TAPELOAD 1.1 DLTLOAD & COPYCAT 1.1.1) COMPRESSION: Processing of compressed data is coded (except for the c program differential addition) but not unit tested. Need time testing and developing. Estimated Completion Date: June 15, 1997 1.1.2) MULTI-NIGHT STACKING: Seems to work, but without compression. Could use more testing. Estimated Completion Date: June 15, 1997 1.1.3) QUALITY AND HISTORY FILES: Coded, unit tested but needs to be system tested and accepted. Estimated Completion Date: July 1, 1997 1.1.4) SYSTEM INTEGRATION: This software has only run stand-alone. Needs to be delivered, integrated and system tested. May need some enhancements. Estimated Completion Date: Jul1, 1997 1.1.5) ERROR PROCESSING: All the various error paths need to be further tested and soft shut down verified. Additional error path may be needed. Estimated Completion Date: July 1, 1997 1.2 RDFRAME 1.2.1) Saturation representation is still a single number for all pixels; a determination of a separate threshold for each pixel and installation of saturation-threshold images in RDFRAME seems desirable. Estimated Completion Date: TBD 1.2.2) Replace all mixed-case keywords in FITS headers with upper case only. There are just under 90 keyword lines in the FITS header. This must be coordinated with other subsystems. Estimated Completion Date: TBD 1.3 OBSERVATORY/IPAC I/F Document 1.3.1) Probably several weeks of effort. There are many changes, some of a clerical nature others requiring additional text or explanatory remarks. Multi-night stacking as well as other changes had a significant impact. Estimated Completion Date: August 15, 1997 2. DARKS 2.1) Newly computed darks and responsivities from evening and morning scans (when available) are tested for consistency, with the results determining whether the new images may be used; similar tests are done between the new and the canonical images. Currently, the decision whether to use a new image is done separately for each band and read type. It may be desirable to install an all-or-nothing requirement. A Science Team decision is required. Estimated Completion Date: September 1, 1997 3. PIXPHOT/PICMAN 3.1) Solo blanking knocks holes in bright/extended sources; analysis is in progress. Estimated Completion Date: TBD 3.2) Solo blanking thresholds need to be optimized. Estimated Completion Date: TBD 3.3) FIND FWHM dependence on seeing is not yet automated. Analysis of optimum FWHM vs. seeing is not yet complete. Detection thresholds are preliminary. Estimated Completion Date: TBD 3.4) Cross-scan error monitoring code is not yet implemented. Estimated Completion Date: TBD 3.5) Blanking of internal reflections is not implemented. Estimated Completion Date: TBD 3.6) No focal plane distortion model has been incorporated. Estimated Completion Date: TBD 3.7) Further code optimization and clean up are needed. Estimated Completion Date: TBD 4. DFLAT 4.1) Currently there is no strategy developed for diagnosing and handling flattening errors caused by confusion noise (e.g., in the galactic plane), although there are references to TBD processing for such cases in the SDS; the expanded stack size implemented in version 1.8 may eliminate the need for this kind of processing. A Science Team decision whether the current processing is adequate is needed. Estimated Completion Date: September 1, 1997 5. FREXAS 5.1) Develop production PSF model and zero total offsets between FREXAS source centroids and PROPHOT source centroids. This will have to be done using final PSFs for FREXAS and PROPHOT. Estimated Completion Date: TBD 5.2) Develop production position error model, including Read1/Read2-Read1 correlated errors vs. seeing. Estimated Completion Date: TBD 5.3) Special treatment for bright/saturated sources: (a.) possibly modified centroid estimation; (b.) modified error model. Estimated Completion Date: TBD 5.4) Improve background estimation. Estimated Completion Date: TBD 5.5) Speed ups: (a.) skip Read1 extraction for frames with no bright Read2-Read1 sources; (b.) replace ASCII intermediate file with binary format. Estimated Completion Date: TBD 6. MAPCOR 6.1) Tune parameters. Estimated Completion Date: September 5, 1997 6.2) Add output for QUALITY subsystem. Estimated Completion Date: June 13, 1997 6.3) Determine how to handle the "lines" and ghosts at +/- 128 pixels for very bright stars (only a few sources are possibly created/contaminated by these artifacts). Estimated Completion Date: June 27, 1997 7. PROPHOT (Note: all estimated completion dates are dependent upon receipt of observatory data suitable for certain testing, e.g., spanning a sufficient range in seeing, etc.) 7.1) Deblending algorithm: Various approaches to deblending were considered, but the simplest seems to work well enough, while keeping the additional run-time to a modest level. Under good seeing, the PIXPHOT detector can find distinct pairs down to roughly 2 camera pixels (4"). The current deblend algorithm, working on simulated data (noise but no chip defects), can separate unresolved pairs down to approximately 1.1 pixels (2.2"). Photometric accuracy seems to suffer considerably when deblending closer than about 1.5 pixels (3"). Putting simulated pairs down in "real" (1995) data (bad pixels, imperfect dithering, etc.) increases the minimum possible separation to around 1.5 pixels (3"). Since the choice of where to put the secondary source requires good sampling of the double, proper dithering is probably the most important variable in real data. Thus, a 3" separation may be close to the practical limit of the algorithm. Tests with real survey-camera data (with reasonable dithering) will help with setting parameters needed to deblend properly, but probably won't change the general conclusions stated above. Simulated data (from real PSFs with various profile widths) will be used to set a crude seeing cutoff in deblending. One remaining test should be performed to see if triple sources can be deblended. The deblending algorithm has been tested with simulated data (using "real" PROPHOT PSFs from May 1995) and seems to be robust given the well-understood data. A small amount of development may be necessary, but most of the time needed will be in the parameterization of deblending, and other algorithms within PROPHOT. Using a test bed of data with the proper range in seeing, and a significant range in density, it should be possible to have PROPHOT pipeline-ready in a month to six weeks. A significant part of this time is required because of the PSFMAN commissioning. Estimated Completion Date: August 1, 1997 7.2) Further Documentation: As more is learned about deblending, additions will be made to the PROPHOT SDS. Additional information on the calculation of errors, especially the position errors, is being included. Estimated Completion Date: August 1, 1997 7.3) Deblending Parameter Tuning: a.) Reduced-ChiSq cutoffs for double (triple?) fits; b.) Stellar density cutoff for deblending; c.) Magnitude limit for deblending; d.) Seeing variation and deblending, cutoff or variable parameter if necessary. Estimated Completion Date: August 1, 1997 7.4) PSF Parameter Tuning: a.) Check that the stepping between neighboring PSFs (slight change in seeing) does not introduce significant photometric biases. b.) Using the series of scans taken through a series of different focus setttings, confirm that the preliminary finding of PSF uniformity in the survey camera is not affected by drifts infocus. Estimated Completion Date: September 1, 1997 7.5) PSFMAN PSF Grid: a.) How many PSFs should cover an adequate range in seeing? Actual PSFs may only be extracted and placed on a much coarser grid, and interpolation used to fill in grid positions.How coarse, and how important is the interpolation kernel? b.) The jump from one to the next PSF should not introduce significant photometric error; compare photometric errors introduced by interpolation and by "picket" PSFs. c.) It may be possible to not react to apparent seeing variations keeping the PSF, and thereby the photometry, more stable. Estimated Completion Date: September 1, 1997 7.6) PSF Monitor: a.) What parameters (shape, etc.) should be used when comparing new empirical PSFs to those in the grid; b.) How much significantly deviant data should be accumulated before a new set of PSFs should be made? If all PSFs should be changed at one time, how long should "new" PSFs beaccumulated and compiled before replacing the "old" ones? Estimated Completion Date: September 15, 1997 7.7) PSFMAN Parameter Tuning: a.) Number of primary (empirical) PSFs in grid, number of interpolated PSFs between two primary PSFs; b.) Weighting parameter used when combining individual extracted PSFs into a single empirical PSF used in the grid; c.) PSF Monitor: Set thresholds for parameters measuring the differences in the PSFs extracted for the current night compared to the fiducial set. Estimated Completion Date: September 15, 1997 8. POSMAN 8.1 PFPREP 8.1.1) Compute and output QA/Summary data. Estimated Completion Date: July 7, 1997 8.2 POSFRM 8.2.1) Replace work-around for a bug in the way aveusp applies the overall fit to the three bands, the linear transformation from U-scan to N-scan is currently based on J-scan extractions only. Estimated Completion Date: September 15, 1997 8.2.2) Investigate the Posfrm solution's sensitivity (<1/2 arcsec) to the initial position estimate. Until this can be resolved, a double pass through PosFrm appears to cure the problem. Estimated Completion Date: September 22, 1997 8.2.3) Investigate position repeatability (sigma and/or mean) problems between some adjacent scans and not others. Estimated Completion Date: October 20, 1997 8.2.4) Resolve problems due to proper motions and parallax in the Tycho positions being turned off because of the very large sigmas on these parameters. There is presently no uncertainty growth added to account for the fact that the proper motions are not included. There is also no code to eliminate stars with large known parallax values. Estimated Completion Date: October 6, 1997 8.2.5) Compute and output QA/Summary data. Estimated Completion Date: July 7, 1997 8.2.6) Add code to model the changes in atmospheric refraction and distortion over each frame. Estimated Completion Date: August 4, 1997 8.2.7) Design and code for the position uncertainty modeling of reconstructed frame positions. Estimated Completion Date: September 1, 1997 8.2.8) Finish rewriting routines borrowed from SlvAll for maximum robustness and maintainability. Estimated Completion Date: December 15, 1997 8.2.9) Verify model and tune parameters for band-to-band relations using real 3-band data. Estimated Completion Date: June 7, 1997 8.2.10) Do a major rewrite of the PosMan SDS. Estimated Completion Date: December 15, 1997 8.2.11) Install a two-tier pattern matcher capability. Estimated Completion Date: September 8, 1997 8.3 PFPOST 8.3.1) Replace approximate method being used to determine magnitudes in exr1. Estimated Completion Date: July 7, 1997 8.3.2) Compute and output QA/Summary data. Estimated Completion Date: July 7, 1997 8.4 POSPTS 8.4.1) Add code to do a linear fit of U-scan to J2000 using point sources from BandMerge. Estimated Completion Date: November 3, 1997 8.4.2) Add code to do associations of the final point sources from BandMerge with the USNO-A and to compare positions with the PosFrm point source reconstruction (pfmp) for QA. Estimated Completion Date: November 3, 1997 8.4.3) Design and code the position uncertainty modeling of the final reconstructed (J2000) point source positions. Estimated Completion Date: November 17, 1997 8.4.4) Compute and output QA/Summary data. Estimated Completion Date: November 24, 1997 9. GALWORKS (Note: the order in which these tasks are performed depends upon the order in which various types of analyses are done, which depends on the kinds of data received from the observatory and the order in which they are processed at IPAC.) 9.1) Modify GALWORKS to handle "special" catalogs, process objects from said catalog, write results to file(s) and stamp images; tag with special flag value. Estimated Completion Date: September 1, 1997 9.2) Read various flags generated by MAPCOR and others that tag artifacts and highly improbably real objects (e.g., persistence ghosts). Estimated Completion Date: September 1, 1997 9.3) Algorithm 2 has not yet been fully tested on 3-channel data; design is still under construction; work in progress. Estimated Completion Date: September 1, 1997 9.4) Algorithm 1 "deblend algorithm" yet to be fully tested. Estimated Completion Date: September 1, 1997 9.5) GALWORKS requires run time tests and possible design changes to speed things up (that is, jettison some nifty but not crucial things GALWORKS does). Estimated Completion Date: September 1, 1997 9.6) Include additional ellipse-fitting values for one to two additional isophotes. Estimated Completion Date: September 1, 1997 10. CALMON 10.1) Compute extinction slope from initial data taken from the northern telescope site (do the same for the southern site once we have data). Estimated Completion Date: June 6, 1997 10.2) Modify CALMON to incorporate calibration acceptance. Estimated Completion Date: June 13, 1997 10.3) Build secondary standards file, compute statistics, and track over mission. Remove non-acceptable stars and add to standards file. Estimated Completion Date: June 27, 1997 10.4) Compute extinction slope from nightly data and maintain running average; track trend vs. time. Estimated Completion Date: July 11, 1997 10.5) Write SDS Estimated Completion Date: August 1, 1997 10.6) Measure and monitor color term Estimated Completion Date: July 25, 1997 10.7) Polish or add logic for more sophisticated zero point determination (unknown requirements; 4-hour average, another fit as needed). Estimated Completion Date: TBD 10.8) Output information directly to QUALITY. Estimated Completion Date: TBD 10.9) Tie measurements between hemispheres and around sky? Estimated Completion Date: TBD 11. QUALITY 11.1) CONMAN function: in per-scan statistics table add: a.) size of overlap area b.) USNOA catalog match statistics c.) mean & rms. color information? Estimated Completion Date: TBD 11.2) Nightly QUALITY analysis: a.) gather quality output from various subsystems. b.) compile nightly quick-look tables from subsystem input, one line per scan: 1). One table per night for multi-band parameters, e.g., airmass. 2). One table per band-per-night for band-dependent parameters. c.) compile nightly quick-look table for each subsystem? d.) determine an overall quality value per scan. e.) automate per/scan plots of background vs. frame and delta mag. vs. UT. f.) create and automate plots of 1). Seeing vs. UT 2). Array temperatures (J,H,Ks) vs. UT 3). Telescope temperature/focus vs. UT 4). Unique survey area covered. g.) compare actual scan coordinates to the nominal tile coordinates. Estimated Completion Date: TBD 11.3) General QUALITY analysis: implement web interface for quality checking Estimated Completion Date: TBD 12. DBMAN 12.1) Add code to put "auxiliary files" (everything besides point and extended source files) into the data base. Estimated Completion Date: August 8, 1997 12.2) Change table and index creation routines to read "data dictionaries". Estimated Completion Date: July 11, 1997 12.3) Finish converting SIS files into "data dictionary" files. Estimated Completion Date: June 13, 1997 12.4) Develop program and procedures for updating data statistics needed by access programs. Estimated Completion Date: November 21, 1997 12.5) Perform extensive timing tests. Estimated Completion Date: December 12, 1997 12.6) Integrate DBMAN programs and subroutines into (web?) interface. Estimated Completion Date: January 23, 1998 13. MPCAT 13.1) The asteroid-position code and 2MASS association code remains undelivered; the driver that prepares input for the asteroid-position code and runs it and the association program is ready for test but has not been delivered. Estimated Completion Date: August 1, 1997