24um SWIRE Mosaicking: Pre-processing and Mosaicking
          ====================================================

                           F. Masci  7-22-05


The purpose of this document is twofold: first, it outlines a recipe
for preparing 24um BCD products from the Spitzer Science Archive for
SWIRE mosaicking, and second, it outlines the mosaicking procedure
itself, using MOPEX. Note that all BCDs from all SWIRE fields have
been preprocessed (background subtracted and latent corrected where
applicable). Thus, you may never have to perform these steps unless 
significantly enhanced 24um BCDs become available in future. 
I doubt this since the current BCDs are the best we will ever have. 


Generic Directory Structure for Processing
------------------------------------------

It's best I first tell you where everything is.

usagi% pwd
/swire/sci4/fmasci

usagi% ls
MOSAICS/        <-- location of final field mosaics organized by date
MOPEX/          <-- MOPEX package installation with namelists, cals etc..
SWIRE_CDFS/     <-- contains:
                     ./BackgndSubtract/
                     ./bcdimgs/
                     ./maskimgs/
                     ./bcdimgs_BackgndSub/
                     ImageList.txt
                     MaskList.txt 
SWIRE_EN1/      <-- same as above
SWIRE_EN2/      <-- same as above
SWIRE_ES1/      <-- same as above
SWIRE_LOCKMAN/  <-- same as above
SWIRE_XMM/      <-- same as above, except includes ./LatentCorr/

Raw, unprocessed BCDs reside in subdirectories "./bcdimgs/", masks in
"./maskimgs/", background subtracted BCDs in "./bcdimgs_BackgndSub/",
scripts and cdfs for the background subtraction in "./BackgndSubtract/",
and scripts and cdfs for the latent-residual correction in "./LatentCorr/".
Image and mask lists are named ImageList.txt and MaskList.txt respectively. 


BCD pre-processing (pre-mosaicking) steps
-----------------------------------------

There are two optional BCD pre-processing steps:

1.)Removal of background levels (effectively giving a "background-
   matched" or seamless mosaic in the end). This is usually always
   performed on SWIRE BCDs since the primary science goal is
   extragalactic and the background (primarily zodiacal) can be
   discarded.

   This step can be accomplished via a perl script called:
   "bckgndsubtract.pl" which is located in all subdirectories
   /swire/sci4/fmasci/SWIRE_*/BackgndSubtract/ (see above directory
   structure). I usually execute this in the "BackgndSubtract/"
   subdirectory after setting the environment and ensuring the
   namelists are in their correct location. See the script
   tutorial by executing it with no command-line arguments.
   Here's a snippet of the tutorial:

   ~~~~~~~~~~~~~~~~~
   Command-line usage (ALL INPUTS ARE _REQUIRED_):
      
   bckgndsubtract.pl -i <FITSImageList>   (List filename containing BCDs)
                     -o <OuputDirectory>  (Directory of outputs)
                     -m <Method? 1 or 2>  (1 = Median filter with window
                                               sizes in $SIRTF_CDF/med.nl;
                                           2 = Single image median with
                                               parameters in $SIRTF_CDF/qa.nl)
   ~~~~~~~~~~~~~~~~~
   
   Here's an example command-line for execution under the directory:
   /swire/sci4/fmasci/SWIRE_CDFS/BackgndSubtract/
   
   source mopex.csh 
   
   bckgndsubtract.pl -i ImageList.txt -o ../bcdimgs_BackgndSub -m 2
   
   The background-subtracted BCDs will appear in ../bcdimgs_BackgndSub/   
   

2.)Removal of dark-latent temporal artifacts. This step has only been
   performed on the XMM field where the presence of a well known star,
   "Mira", left a trail of dark latent residuals in about half of the BCDs.
   Blame the schedulers!
   
   This was accomplished by executing the perl script:
   "MakeSelfCalFlatsAndFlatten.pl" under the directory:
   /swire/sci4/fmasci/SWIRE_XMM/LatentCorr
   
   See the script tutorial by executing it with no command-line
   arguments. Here's a snippet of the tutorial:

   ~~~~~~~~~~~~~~~~~   
   Usage (command-line inputs):   
   MakeSelfCalFlatsAndFlatten.pl
           -i <InputFITSImageList>            (Required)
           -r <Reject(BrightSource)ImageList> (Optional)
           -n <NumImagesPerFlatfieldStack>    (Required)
           -w <WantFlatsCreated>              (0=No; 1=Yes; Default=0)
           -c <WantSelfCalFlat>               (0=No; 1=Yes; Default=0)
           -f <OutFlatDir>                    (Required)
           -o <OutFlattenedImgDir>            (Required only if "-c 1")
   ~~~~~~~~~~~~~~~~~	   
           
   Here's an example command-line for execution under the above directory:
  
   MakeSelfCalFlatsAndFlatten.pl -i ImageList.txt -r BrightImgList.txt
           -n 100 -w 1 -c 1 -f flatdir -o ../bcdimgs_flattened
   
   This creates flats from time-ordered sequences of 100 images from a list
   of images in "ImageList.txt", with bright source images (containing the
   offending star) listed in "BrightImgList.txt" and then self-flattens
   accordingly. Flattened (latent residual-corrected) images are written
   to the directory "flattenedBCDsDir/" and intermediate flats are written
   to "flatdir/".
   
   A note on the order of background subtraction versus latent correction
   (or self-flattening): I performed the self-flattening first, and then
   the background subtraction, otherwise, your background estimates would
   be biased by the presence of the latent residuals. You better get rid of
   these first.   


Mosaicking
----------

The Spitzer MOPEX software is used to create the mosaics. For all SWIRE
fields (except maybe EN2), you will need a machine with at least 3GB
of RAM. There are not many machines in greater IPAC with this amount of
memory. One of them is "sscsci100" with 4GB RAM. The downside is that this
machine is not behind the SWIRE firewall. So, what I do is the painful chore
of ftp'ing all BCDs to a local disk that can be seen from sscsci100, e.g.,
I pick a disk like /ssc/sci/scr/ which can hold up to 1TB of data.

An example MOPEX directory structure I use is under:
usagi% pwd
/swire/sci4/fmasci/MOPEX/mosaicker/

usagi% ls
FIFs/              MaskList.txt       cdf/
ImageList.txt      README*            mopex/
cal/               

The optimized Control Data File (CDF) for 24um SWIRE mosaicking is
./cdf/mosaic_mips24.nl_swire

Here are the steps for a "successful" mosaicking run:

1.)Ensure you have the correct ImageList.txt and MaskList.txt under the
   the above directory for the field to be mosaicked.

2.)If you want to create the mosaic using a specific Fiducial Image Frame
   (FIF), then in ./cdf/mosaic_mips24.nl_swire, perform the following:
   set "run_fiducial_image_frame = 0"; set "FIF_FILE_NAME = your FIF.tbl".
   Some example FIFs are under ./FIFs/ (see above directory listing).
   
3.)Still under /swire/sci4/fmasci/MOPEX/mosaicker/,

   source ./mopex/mopex.csh
   
4.)Execute the mosaicker and sit back:

   mosaic.pl -n mosaic_mips24.nl_swire

Outputs will be written to the directory specified by the "OUTPUT_DIR"
CDF variable, which in this case is set to "output_mosaic". Also, you
may have noticed that we make no use of BCD uncertainty images. The simple
answer is that there's just not enough memory and disk space to simultaneously
handle uncertainty images too. This is okay for mips24 processing since the
image noise is very uniform amongst all BCDs of a field. Also, there are
options in the CDF to create outlier (rmask) mosaics, but beware, their
creation is memory intensive and I have only managed to create one for the
EN2 field.

Versions of all the 24um mosaics created until now are under:
usagi% pwd
/swire/sci4/fmasci/MOSAICS

usagi% ls
CDFS/    ELAIS_N1/    ELAIS_N2/    ELAIS_S2/    LOCKMAN/    XMM/

Good Luck!

____________________________________________________________________________

Frank Masci (fmasci@ipac.caltech.edu; fmasci@caltech.edu; fjmasci@yahoo.com)
Copyright (C) 2005 California Institute of Technology