From hlm@ventusMon Aug 9 17:17:54 1999 Date: Wed, 23 Jun 1999 17:01:01 -0700 (PDT) From: Howard McCallon To: Roc Cutri Cc: Howard McCallon , Sherry Wheelock , Tracey Evans Subject: Steps to Position Uncertainty Updates Roc- Despite the fact that we've already done a major release, the steps required to update the position uncertainties for the upcoming release will be somewhat different than what was done previously. Efforts for the last release were dominated by the fact that there was not enough time to pull all the overlap match differences from the database. A mix where 2/3 of the data was taken from the pipeline overlap files, which did not retain position uncertainties, was required to meet the schedule. Time pressure also dictated that date-and-time-of-processing identifiers could not be included as confirming verification that the data used for the variance factor computation came from the same processing sessions as the uncertainties being updated. Attached is a list of the steps needed to support the position updates for the Fall Release. Regards, Howard %%%%%%%%%%%%%% STEPS TO POSITION UNCERTAINTY UPDATES FOR FALL RELEASE ------------------------------------------------------ 1) Generate a SCANS file of all scans currently in the database (tmass) including the following information for each scan: DatScn = date/hem/scan numbers (yymmddhsss) NTile = tile number DTPpf = date-and-time-of-processing ID (yymmddhhmm) RelFlg = release flag (1=>in release; 2=>borders release) Already have a file from Sherry which provides date/hem/scan and tile number information. The date-and-time-of-processing IDs will have to be pulled from yymmddh.SMRYpm files and matched up with what's in the database using other parameters from the SMRYpm files already maintained in the database. Ask Tracey to backfill the DTPpf values in the database using the SCANS file. 2) Obtain file of date/hem/scan numbers to be included in the Fall Release and set RelFlg=1 where appropriate in the SCANS file. 3) Walk through the list of scans for release and build new list of possible overlap scans using PosOLT routine. Set RelFlg=2 where appropriate in the SCANS file. These will be referred to as the auxiliary scans. 4) Run Tracey's ucin.hlm.csh script to extract original positions and uncertainties from database and write to "ucin" ascii files for later update. It is important that these are indeed the ORIGINAL uncertainties, rather than those updated for the Spring Release. It might be best to simply use the "ucin" generated for the Spring Release for those scans. Could be a little tricky, since the "ucin" files are by observation night. 5) Use gen_dbol.csh script to execute runtile (using tmass) against list of auxiliary scans. This will generate tttttt-tttttt (matched source differences) and tttttt-ACTRES (matched ACT differences) files. 6) Receive tttttt-tttttt and tttttt-ACTRES files for release scans generated by Sherry's comprehensive overlap extraction run against the tmasss database. Sherry will pass these along in observation-night packets since it will take many weeks to run the whole release. Will compress and then backup these files to tape as insurance. Will periodically loop through steps 6 to 8 as new data becomes available. 7) Use gen_olst.csh script to execute db2ols which will use tttttt-tttttt files to generate tttttt-tttttt.olst (overlap segment stats) and tttttt-tttttt.mchl (segment tagged overlap matches) files. 8) Use gen_acst.csh script to execute db2acs which will use tttttt-ACTRES files to generate tttttt-ACTRES.acst (ACT segment stats) and tttttt-ACTRES.mchl (segment tagged ACT matches) files. 9) Cat all files generated in steps 7 and 8 to generate master files and then sort on tile numbers and eliminate dups in each of the four master files. 10) Iteratively solve for variance factors with the VFIter program, using the olst, ACTRES.mchl and ACTRES.acst master files from step 9 as input and write to vfacx (x-scan variance factor) and vfacy (in-scan variance factor) files. 11) Verify variance factors using the CkChi2 program to generate chi-squares for individual overlap matches (from mchl master file) both before and after variance factor corrections are applied. Identify scans without ACT reconstructions and break out results for comparison with overall stats. 12) Use ChgPUC program to read in "ucin" ascii files with the original uncertainties and apply the variance factors to output updated uncertainties in the "ucout" files. 13) After the "ucout" files have been up-loaded back into the database, re-extract the tttttt-tttttt and tttttt-ACTRES files for selected scans to verify the new uncertainties made it in properly. Need to select some scans which were in the Spring Release and some which were not.