RULES FOR QUALITY SCORING
OF 2MASS SCANS
L. Hurt, J. D. Kirkpatrick (IPAC/Caltech), December 2000
These represent the current grading rules used for nightly QA evaluations.
Older grading rules are archived
Survey Scan Grading
Each scan is scored using a base quality number multiplied by the minimum
of a series of quality factors as detailed below.
The "new" strategy of calibration is for observing a single calibration
block at 1 hour intervals.
score = (base) * min(fct1,
fct2, fct3, fct4, fct5)
fct1 ranges from 0.0 to 1.0; all others range from 0.1 to 1.0
score will always be >= 1 unless the photometric quality factor fct1 =
an additional fct7 is
carried along with the score to flag untracked seeing but does not affect
scoring (see below)
For optimally dithered scans (after 970606n), base = 10.
For scans not optimally dithered (on or before 970606n), base = 5.
A photometric quality factor applies to each photometric interval during
a night. It roughly corresponds to a probability in meeting Level 1 specs.
A photometric interval is defined as 2 or more cal sets spanning photometric
survey sets. The standard photometric grading strategy follows the form:
fct1 = (mult1)
* (mult2) * (mult4)
fct1 is always rounded up to 0.1 unless the scan is non-photometric.
a separate factor no longer implemented in QA but appears in archival rules.
A survey block is considered to be non-photometric
under any of the following conditions:
It is not bounded on either side by calibration blocks
passing CALMON with a photometric solution.
Scatter in overlaps summary exceeds 0.1 mag in any
Reasonable evidence for clouds is found in any of the scans in the block.
mult1/Number of Cal Blocks
Downgraded if number of cal sets in block < 5.
# of cal sets
* A solitary 2-cal interval is allowed to pass
only if it is photometrically perfect in its cals' dispersions and overlaps
and if its zero-point appears to be consistent.
**A factor of 0.8 applies for a 2-cal photometric
interval that is spanned by a photometric solution spanning 3 or more cal
sets; see below. Such a 2-cal subset of a longer interval is not subject
to the stringent photometric requirements of a solitary 2-cal interval.
Downgraded if dispersion in photometry for field
stars> 0.04 mag for one or more cal sets.
mult2 = 2 - ((worst
Downgrade used is the worst for any cal set in a
given photometric interval.
Downgrade is waived if it occurs only for a high
airmass (> ~1.5) cal set in an otherwise photometrically stable night.
It may also be waived if it applies to only a single band of a single cal
set which still has a zero point highly consistent with the other zero
points that night.
mult4/Scatter in Overlaps
Downgraded if peak-peak spread of photometric repeatablility
in overlaps region > 0.05 mag.
The worst observed overlaps downgrade within a single
survey set applies to the entire survey set (sometimes a survey set may
have two or more disjoint overlaps intervals or include unoverlapped scans).
|< 0.05 mag
|< 0.075 mag
|< 0.1 mag
|>= 0.1 mag
Downgrades for overlaps can be waived under the following conditions:
Overlaps are based on very few stars due to low source densities (poor
Dec overlap is limited to only a small portion of the scan (as with disjoint
pole scans or N/S overlaps).
Overlap statistics are strongly biased by stars with poorly measured photometry
(such as stars falling in bright star haloes or ones that are close doubles).
Peak-to-peak scatters that are a reflection only of the PSF cross-scan
bias residual and not of the transmission of the atmosphere (as verified
by lack of evidence of clouds in the background and/or magndec plots);
this condition has been sometimes called "K-only clouds".
Clouds & Photometric Downgrades
If any evidence for clouds exist within a given survey
set, then the photometric quality for that entire set will be set to fct1
= 0.0. Conditions indicating clouds include:
Large jumps in the overlaps summary plot
Swings in the magndec plots of > 0.2 mag across a
K-bright jumps in the background frame median plots
(can be J-bright under a full moon).
High numbers of K jump counter counts
Scans with no overlaps information that show
suspicious background frame median plots should be considered to be non-photometric
(guilty until proven innocent).
Note that under special circumstances a cloud
event may be determined to be localized to a single scan, and only that
scan is deemed non-photometric. Such an event is sometimes called a "con
trail" and characteristically shows up on an otherwise perfectly clear
night with no hint of clouds in either the observer's log or weather imagery.
The cloud signature is very brief in the scan, suggestive of a very isolated
Clouds & Contiguous Photometric Intervals
Sometimes a valid photometric solution will span
one or more non-photometric survey sets . In this case the photometric
solution spanning this survey block should be allowed to stand as long
as it appears to be a stable across the non-photometric period.
The survey scans to either side of this break
will be graded as if that were an actual photometric break, with mult1
calculated from the number of cal sets that span the photometric interval
(however a 2-cal interval will be downgraded only to 0.8, not 0.3). Any
mult2 photmetric dispersion downgrades will apply to all intervals covered
by the photometric solution, however, since it reflects uncertainty in
the overall zero-point solution.
If the zero-point solution is not consistent across
the interval, CALMON will be rerun after "breaking" the interval
into photometric sub intervals by altering the .lgo file and setting the
A sensitivity quality factor is computed from a convolution
of the seeing shape and background level (PSP or point-source probability),
and corresponds roughly to the probability that a scan will meet the Level
1 specs for sensitivity. The grades are dependent upon hemisphere and date
(reflecting differing sensitivities of detectors).
||North K PSP (&
H before 990701)
|North H PSP
|South H PSP
||South K PSP
*Note that current Northern H PSP downgrades are
only allowed to go down to 0.3, not 0.1, so this last condition does not
Scans with poor seeing shapes are downgraded to fct3
= 0.1. Any of the following conditions warrants a downgrade:
A downgrade override is also warranted if there is
any indication that the seetracker score is in error and does not properly
detect a known problem (i.e. many false extended sources seen across a
Max J seeing shape > 1.3
Average J seeing shape > 1.25
Note that in a related downgrade, a scan with
a 2nd image moment < 0.81 receives a maximum final quality grade of
Scans with extensive runs of untracked seeing are
downgraded to fct4
= 0.1. The following conditions warrants a downgrade:
All untracked seeing scans are inspected to verify
the downgrade is warranted and not caused by a galaxy cluster.
Untracked seeing extent (any band) > 900"
Scans with strong residual structure in the coadds
induced by OH airglow variations are downgraded to fct5
= 0.1. The following condition warrants a downgrade:
All airglow scans are inspected to verify the downgrade
is warranted . The cnoise4 statistic is the difference between the measured
coadd background noise (after modelling large-scale gradients & structure)
and the theoretical noise expected from the overall background level. Reviewers
should note that this theoretical noise level varies with time and may
need to be updated periodically by Tom Jarrett.
H cnoise4 > 4.5 & log(max source density) <
Individual scans should also be set to quality 0
for any of the following reasons:
Single-frame electronics glitches (so far seen only
in north H band) should not receive grades higher than 4 (however, there
is currently no quality factor associated with this downgrade, but such
cases will also be flagged with Bla0 = 1 in the SIT file).
Bugs crawling on the window
Bright airplane passing through field
Transient illumination problems that generate significant
structure in coadds (e.g. strong Dome jumps)
Transient electronics glitches or power failures
Image blurring due to earthquakes
Scans that fail due to unrecoverable problems in
the data (e.g. observing too close to moon)
Grades of scans may also be overridden on recommendation
by UMASS (e.g. to force a single scan with a high grade to match the low
grades of the rest of the block).
fct7/Untracked Seeing Eyeball Estimate
This final QA factor is carried along with the database
transaction and SIT files for the night as a record of untracked seeing
problems that have caused false extended source detections, even if the
incident is not bad enough to warrant a downgrade. Reviewers inspect every
scan with marginally high seetracker scores to look for obvious signs of
point sources misidentified as extended. If such an incident is noted in
at least two stars then the fct7
is set to 0.0 for the scan (otherwise it is 1.0). The default is for fct7
to be set equal to fct4.
is not used for grading scans but may be a useful flag in selecting candidates
Calibration Scan Grading
Calibration scans receive grades of either 0 or 10. Conditions sufficient
to fail a single calibration scan (grade 0) are:
Conversely, any cal block that technically passes as photometric, even
if it bounds non-photometric survey data or is not part of a solution spanning
multiple blocks, should receive a grade of 10 for inclusion in the calibration
working database. This is a change from previous rules in which cal blocks
that bounded non-photometric data were themselves considered to be non-photometric.
the scan failed processing at any stage
scan is in a block that CALMON does not pass as photometric
there is any evidence for clouds during the cal block, even if it technically
Default Grading and Overrides
Much of the grading is computed automatically by
the SCANSCI subsystem. This includes generation of the template QAreview
and tile/database transaction files. If a default grade needs to be overridden
the reviewer should enter that override into the YYMMDDH.qagrade file (found
in the datequal directory) and use the /home/hurt/bin/run_scansci script
to update the grading.
When overrides have been implemented they will
show up on the second set of quality factors found on the "Scan Grading"
section of the QA Report web pages.
A number of scans have been observed that do not
overlap any adjacent tiles on the same night. To find potential photometric
problems that would otherwise have been noticible in these overlaps, comparisons
have been run between them and tiles from other nights to look for any
photometric biases or evidence for clouds.
Scans in which clouds have been found are given
retroactive downgrades to quality = 3, and they are also flagged with Bla1
= 1 in the SIT file.
Other scans have been found in which a clear,
constant photometric bias is seen with respect to tiles on other nights.
This indicates an incorrect zero-point determination, likely the result
of weather-induced problems in these or nearby cal scans. If the bias is
deemed sufficiently large to be a problem (~> 0.15 mag), the affected scans
are flagged with Bla2 = 1 in the SIT file and their final quality scores are
downgraded to 5 if they were higher than 5.
Last Updated 10/01/01 by R. Hurt