2MAPPS v3.0 Possible Improvements 6 March 2001 ------------------------------------------------------------------------------ EXEC - Capture and archive versions for all subsystems. RDFRAME - Develop saturation maps for all arrays to flag saturated pixels. Currently using single saturation value for each array now. DARKS - Generate and use best available canonical darks and flats from long periods. What is optimal period? PIXCAL - Better saturated R1 star centroiding and brightness estimates. DONE - better centroiding. - Reduce R1 chaffe detected around bright stars, if possible. Goal is to yield one unambiguous R1 detection for each bright star to be passed to MAPCOR. - Change sky annulus used in FREXAS aperture photometry (currently 24"-30") to be consistent with PROPHOT aperture photometry annulus (14"-20"). PIXPHOT - DONE - Noise estimator that is less sensitive to low frequency background noise due to airglow. This has the effect of scaling back detection sensitivity in v2.x for periods of strong BG variations due to airglow. - Meteor detection and blanking in frames. - Reduce R2 chaffe detected around bright stars, if possible. - Improve tiltfix to accomodate higher order structure in northern J and H-band arrays. - Incorporate distortion map in Atlas Image generation. WILL NOT BE DONE. - Create Atlas Image coverage map for each band. Disk space is an issue. Kopan and Jarrett discussing methods for highly compressed versions carried along with images. - Atlas Images as data cubes? - Allow N=1 pixels into Atlas Images. Currently blank any pixel that had less than 2 frame coverage, leaving hole in resulting image. - Do not adjust Atlas Image backgrounds to insure smooth in-scan overlaps. This means that Image backgrounds will give more accurate measurements of true Poisson noise. PROPHOT - Generate new, high SNR production PSFs to thoroughly populate all seeing bins for all hardware periods. - DONE: PSF's built from many scans. Production PSF generation software operational. Requires characterizing detector gain as a function of time. - Examine possibility of extra dimensions in PSF tables: o Image elongation o Variation across focal plane (including elongation) o Must be able to diagnose based on scan-averaged values (i.e. cannot measure in-situ because of data and runtime limitations), and characterize completely as a function of temperature, focus (or mis-focus), seeing, telescope position, etc. WILL NOT BE DONE. - Use a priori knowledge of nightly zero points - Initiate active deblending in PROPHOT. Better logical handling in passive and active deblending. Must build in proper source density, and SNR thresholds to avoid deblending everything in the galactic plane and deblending every galaxy and nebula. - Mag 99.999 problem in high density regions. - Correct error that causes chi-squared's to fall below 1.0 at high SNR. Requires better error model to account for both pixelation and poisson noise dominated regimes. - Take into account confusion noise component in the total noise model. This may be causing elevated chi-squared values at faint magnitudes in the current version. - Make sure proper detector gain value is being used. Time-dependent? - Incorporate distortion model. - Use time-dependent detector gain values during PSF generation. MAPCOR - MAPCOR "mystery error" must be fixed. - Radial distance dependent diffraction spike widths - Very bright star and off-scan bright star treatment. Pull JCAT' into pipeline. May be mitigated by better bright star position and brightness estimation in FREXAS. - Measure and remove any R1/R2 position offset. - Toggle to look-up table for normalizations in very dense fields. Consider using lookup table values for all scans. Calibrate with preliminary processing results. - Toggle to look-up table when curve-of-growth and psf-fit normalizations solutions deviate from canonical values by some amount. Use survey-average calues of normalizations as a function of seeing. Perhaps use only look-up table values as a function of seeing? - Better bright star artifact parameters in high density regions. Less conservative very bright star artifact parameters in general. - Use multi-bandedness or other parameters (i.e. optical match) in discriminating artifacts from real objects. - Create point source masks from MAPCOR algorithms. Made difficult because of mag dependence. Magnitude dependent masks? BANDMERGE - Incorporate distortion model to correct cross-scan if distortion not incorporated earlier. - Use position uncertainties supplied by POSFRM (assuming those are improved). - Identify and handle active (if implemented) and passive deblends that are inconsistent between bands. POSMAN - Done: Incorporate higher density reference catalog (Tycho 2). - Prior to running v3.0, martinize positions/uncertainties from previous pipeline processing to build correction tables. Use Tycho-2 and band-merged distortion model in this step. - Use position/uncertainty correction tables from step 1 with the CORRESPONDING previously-generated fpos files to create super fpos files with improved positions and realistic frame-variant uncertainties. - Modify PFPrep and PosFrm to: a) Take the super fpos files as a given and generate auxiliary files to match. b) Bypass code not needed in this mode in order to save cpu time. - Modify PosPts to change IR/opt match vector position angle to be degrees East of North. (currently modified in catalog reformatting) - Coordinate distortion and possibly differential refraction modeling with other pipeline subsystems. - Drop back to single pass through PosFrm during running of the 2MAPPS v3.0 pipeline: - Eliminate wide-separation optical association searching. - Include a distortion correction monitor to check if there are unexplained variations. GALWORKS - Utilize prophot chi-square information in target selection. Use possible multiple star information from PROPHOT deblending to help discriminate multiples from resolved sources. - Provide multiple star feedback to point source lists. - Create extended source bright star mask images. - Use LCSB processed sources in Catalog Generation. Galactic plane? - Higher-order background fits to mitigate H-band airglow structure (decide what spatial scales willing to sacrifice for completeness). - Use Atlas Image coverage maps (see PIXCAL). Generate additional coverage maps that reflect blanking of artifacts done in GALWORKS. - Improved known galaxy seed list. Coordinates especially critical. - Update decision trees with full initial processing results. - Use a priori knowledge of nightly zero points - Improve robustness of photometry in crowded regions by using an iterative fit/subtraction of foreground stars. - Improve galaxy completeness in high density regions by ~0.5 mag through better multiple star discrimination. CALMON - Improved Secondary Standard List from global analysis (numbers, photometric, positions). - Incorporate time-variable extinction coefficients (i.e. seasonal)i, if indicated by global analysis - Apply cal-observation (set of 6 cal scans) averaged zero point to cal scan data. Currently deriving zero point per scan and applying to the scan. - Determine if hemispheric color-corrections are necessary, and implement if so. - Possibly include single cal scan rejection within cal sets when photometry is abberrant. - Incorporate new nightly zero point offset fits, as recommended by analysis. Fits to include constant, linear, quadratic and piecewise. MPCAT - Open position match radius slightly to accomodate martinized position updates. - Possibly use larger match radius for comets to accomodate poorer predicted positions. - List ephemerides file version in MPCAT output file (*.mpasn). - Correct error that caused bright asteroids to be missed in data from last quarter of 1998. DBMAN - Load asteroid data into DB, and set asteroid flags in OPS DB. - Load source lists, scan and coadd info tables into the DB directly out of pipeline. Minimize dwell time of table on disk to improve efficiency. - Do all table preparation **before** DBMAN. DBMAN should do only DB table loading. Review current DBMAN to pass back all data juggling, derivation into relevant subsystems. - Revise interfaces to handle new pipeline output and load new columns, as necessary. QA - Improve efficiency of QA turnaround. Simple repetition of v2.0 QA is not adequate - much too slow. Must reduce dwell time of data on disk after processing to maximize throughput. Steps to improve effiency will include: o Triage survey data for reprocessing, using initial processing quality results. However, enough margin should be left in triage to accomodate data that will high higher score due to new scoring algorithms and new software. Triage must also make sure that enough multiply observed tiles are processed so that the best observation can be selected. o Better detection and identification of very bright stars in pipeline. o Use results of preliminary processing as often as possible. This would include meteor and artifact results, bright star id's, untracked seeing, interesting object identification. CATALOG GENERATION TASKS Tile Selection: - Refine algorithm for selecting between multiple observations of the same Tile. Incorporate Tile overlap information as well as quality. Nightly photometric quality will not be a factor in selecting between different obscervations of a given tile. Source Selection: - Lower SNR thresholds? OVERLAPS - Parallelize operation to speed runtime. - Identify any remaining "bookkeeping" issues. - Preserve completeness and reliability data from tile overlaps in initial processing. Identify bright, non-confirming sources and report in initial processing. - Generate an N/M statistic for overlap processing. DB_MAPCOR - (See MAPCOR) - Improve bright star seed-list for very bright star and off-scan star processing. Use 2MASS "truth" to validate all candidates. POSMAN - Implement positional "martinizing". Updated positions must be propogated through optical source matching, asteroid/comet matching, extended sources, images. Proposal is to do this in the pipeline, using corrected FPOS files generated before the start of final uniform processing. CATFORM - For XSC: Set aspect ratio values ?_ba=1.0 and phi=0.0 if r3sig < 7.0 Set radius values (j_r_e, r_fe, r_fc) to null if < 0. For PSC: Set default mags to 99.999 for R1 saturation Incporporate MP_FLG data from revised database. Do not pass chi-squared values for R1 bands. Set bl_flg = 1 for any aperture photometry, including rd_flg=4 PHOTOMETRY - Derive and implement global uniformity corrections for point and extended source photometry based on global chi-squared minimization analyses. Issue is if this is possible given known cross-scan bias in PSF-fit photometry due to PSF variations. MISC - Correct extended source contamination flag in point source catalog.