Overview of 2MASS QA Utilities

Updated for Pipeline 3

I've written a number of scripts to help streamline my QA review process; since they may be of use to others I will keep info on them updated on this page. Many are designed to be used on production nights that are on disk.

Most of these scripts will give basic usage info if called without any arguments. All can be found in /home/hurt/bin (easiest way to access them is just to add /home/hurt/bin to your path).

Required Environment Variables

Several of the scripts require knowledge of your preferred text editor and where you wish to keep QA-related scratch files on your local disk. An example set from a .cshrc is shown below:

setenv EDITOR nc               # preferred editor
setenv coadds_host "vorlon"    # home unix machine name
setenv coadds_rmtdir "/scr/2mass/qaimages"  # preferred scratch dir
setenv coadds_extd "yes"       # gives extended source markers on coadds

It is also recommended you set the following in your .cshrc so others in the 2MASS group can edit or delete your QA files:

umask 2

Basic Review Tools

qaprod

This simple script prints a machine-by-machine summary of currently online production nights. It simply reformats output from Ron's where2 command.

qaready

This simple script merely lists all the .tmpl files found in Davy's ~davy/QAreview/ directory. This generally corresponds to the list of nights that are ready to be reviewed.

qalog

This script opens up all the basic files needed at the start of a QA review: the template *.QAreview file, the observer's log, the version 2 .QAreview, and Roc's nightly review files. The logs will be displayed using the editor specified in the EDITOR system environment variable (this is usually defined in one's .cshrc). The script is "smart" with respect to north/south hemispheres and the current status of the *.QAreview file (found in Davy's ~davy/QAreview/ directory):

if only a .QAreview file exists, then it is opened
if only a .QAreview.tmpl file exists, it is renamed to .QAreview and opened
if both files exist, both are opened (the reviewer should eventually delete the .tmpl file)

Useage:

qalog YYMMDDH qalog YYMMDDH log [only open observer's log and Roc review file] qalog YYMMDDH rev [only open QA review files]

qastart

Executed from the datequal or night root directory, qastart displays the most commonly accessed postscript files and summarizes useful info on calmon overrides. It also copies to the user's local directory the nightly qameteor and qabkg image directories.

Useage:

qastart [from datequal or night] qastart bkg [copies bkg plots] qastart info [only prints summary information] qastart help [prints command summary]

run_scansci

Executed from the datequal or night root directory, run_scansci will rerun key QA scripts and update the QA web pages with current values of reviewer overrides. Typically this should follow any editing of the .qagrade or .qacat files.

Useage:

run_scansci [from datequal or night]

Display/Analysis Tools

qaseeing/qaseetrack [display seeing plots]

Executed from the datequal or night root directories, these are shorthand scripts for calling Tom Jarrett's seetracker plots. The only argument you need to supply is the scan number.

Useage:

qaseeing NNN [from datequal or night]

qaseetrack NNN [from datequal or night]

qamagndec [display magnitude overlaps plots]

This opens up in ghostview one or more magndec plots, if present, that cover overlaps of specified scans. It can be used with the 'all' argument to display, one after another, every magndec plot for the night (each new plot will open after the previous plot is closed).

Useage:

qamagndec NNN(1) NNN(2) ... [from datequal or night]

qamagndec all [from datequal or night]

qadpos [display position overlaps plots]

This opens up in ghostview one or more dec.dpos plots, if present, that cover overlaps of specified scans. As with qamagndec it can be used with the 'all' argument to display, one after another, every dec.dpos plot for the night (each new plot will open after the previous plot is closed).

Useage:

qadpos NNN(1) NNN(2) ... [from datequal or night]

qadpos all [from datequal or night]

pltbkg [display night background summary plot]

This plots the scan-by-scan background summary for the night, leaving you in sm.

Useage:

pltbkg [from datequal or night]

bkgflash/bkg [display scan background plots; semi-obsolete]

These scripts show the backgrounds for the night/scan, leaving you in sm.

For pipeline 3, bkg plots will be pregenerated and viewable from within Netscape on your local disk partition.

Useage:

pltbkg [from datequal or night]

bkgflash [from datequal or night]

bkg NNN [from datequal or night]

coadds/ccoadds/ccoadds_multi [display coadd images]

These scripts allow easy viewing of coadds for single bands and 3-color composites.

Useage:

coadds k*.fits [from sNNN/image dir]

ccoadds *009.fits [from sNNN/image dir]

ccoadds_multi NNN(1) NNN(2) ... [from night root only]

Full documentation for these is available elsewhere.

overlays/coverlays [display coadd images w/ point source overlays]

These scripts overlay point sources on the specified coadd; overlays handles a single greyscale band while coverlays makes a 3-band RGB image.

Useage:

overlays ji0130009 [from sNNN/image dir]

coverlays *i*009.fits [from sNNN/image dir]

Special Purpose Tools

These accomplish special-purpose tasks I've dealt with on occasion. They are not necessarily supported tools but I include them here for reference.

qatile

This lists relevant lines from a pared-down V2 .sit file indicating all instances of a given tile number. This is useful for determining whether newly-downgraded tiles will have other, better apparitions. The key column to check is "Qn", the final V2 score.

Note that this will generally only work on a production machine as it must access the updatesit file on /2massd. Possibly adapted from a routine from Brant.

Useage:

qatile TILENUM [from anywhere on a prod machine]

qaphotmatch

This creates tables and plotfiles for making high SNR overlap photometry plots for two adjacent scans. Possibly adapted from a routine from Brant.

Useage:

qaphotmatch SSS TTT j|h|k [from datequal]

qabgsum

This extracts background summary info from an input filelist. Example for using John Fowler's stf utility.

Useage:

qabgsum filelist [from datequal]

QA Web Status Scripts

Status Update Scripts

qastatusnew YYMMDDH [YYMMDDH [...]]

used primarily by: DMT's

Sets up new entries in the database for online nights that have made it through OPS. New nights can be set to status "online" (the default) or "hold" (in case there was a production problem and it isn't ready for anormal review).

qasignup

used primarily by: Reviewers

Gives a list of nights ready for reveiw and signs a reviewer up for it.

qaend YYMMDDH

used primarily by: Reviewers

This is run at the end of the review process (i.e. the complement to qastart) to copy the QA review to the web server and to change its status to "submitted" and ready for "cleanup". It also allows you to assume responsibility for a night originally claimed by someone else. You may change any/all/none of these states as it asks for confirmation for each step. It can also be run multiple times as needed.

qaapproved YYMMDDH [YYMMDDH [...]]

used primarily by: Reviewers

This updates the status of a night to "approved" once UMASS sends the OK.

qacleanup YYMMDDH [YYMMDDH [...]]

used primarily by: DMT's

Changes the status of entries marked ready for "cleanup" after review to "archive", indicating they are now on the V3ARC archives.

qastatus YYMMDDH [YYMMDDH [...]]

used primarily by: DMT's/Davy/Robert

This is a general purpose editing tool that allows any status state to be changed to any acceptable value. It also allows for the addition/removal of notes.

qastatusweb

used primarily by: troubleshooting purposes only

This regenerates the web status board in case something gets confused during one of the other scripts. No data is changed; this routine recompiles the HTML pages based on the current database. It is called internally by each of the other scripts.

Outdated Pipeline 2 Tools

qadark [Pipeline 2 only]

Executed from the datequal directory, qadark extracts useful info about flats and darks from the fort.44 file.

Useage:

qadark [in datequal dir]

qatr/qatrans [Pipeline 2 only]

This opens up 4 files useful when submitting transaction files: the QA review for the night, the .qua and .QAtrans transaction files, and a template note for the DMT's. The files will be displayed using the editor specified in the EDITOR system environment variable. Calling qatr first also fills in the grades in the .QAtrans file.

Useage:

qatrans 990103s

Last updated: 9/20/01 by R. Hurt