Reduction Process FOR CTIO 0.9 meter

There are eight steps to the reduction process. The first consists of preliminary set-up tasks. The other seven are often called OTZDFQI: Overscan, Trim, Zero, Dark, Flats, Fringing, and Illumination.

Preliminary Tasks
-->COPY THE UNPROCESSED FILES FOR EACH NIGHT TO A SPECIAL DIRECTORY BEFORE YOU DO ANYTHING. Always have a Raw directory in case something goes wrong and you need to start over. This is very important.

--->We first need to change the files into iraf readable files.
1. Make two lists of files: "all.list" will be a list of file names with the .fit extension. "all2.list" will be an identical list with the .fits extension.
2. Change file types (enter in iraf): cl> rfits @all.list 0 @all2.list datatype=u
3. Delete the .fits files: cl> imdel @all.list
4. Rename the .fit files as .fits files (this doesn't change the files' intrinsic type): cl> imrename @all2.list @all.list

--->Now, we need to add some information to the header of the images, using an IDL code called mkheaderv2.
1. Copy mkheaderv2 to working directory.
2. Run in IDL. The program will prompt you for the observatory, the night's log file name, and observers' names. The output is called header.bin.
3. Pipe header.bin into iraf: cl> cl < header.bin
4. Check status of images (enter in iraf): cl> ccdlist @all.list. If you've done everything correctly, next to each filename it should say [ushort] (NOT [short]...if it only says [short], something is wrong), and the type of file ("object" for object frames, "zero" for biases, "dark" for darks, and "flat" for flats).
-->Remove bad images from the processing directory: images with low flux counts, high seeing, etc.
Summary of commands: (cl> refers to IRAF terminal)
      >ls *.fits > all.list
      >ls *.fits > all2.list
      cl> rfits @all.list 0 @all2.list datatype=u
      cl> del @all.list
      cl> imrename @all2.list @all.list
      >cp ~/programs/mkheaderv2.pro .
      idl> mkheaderv2
            enter observatory: either WIRO, BOK, CTIO09m
            'monthdaydayyear'.log
            observer names
      cl> cl < header.bin
      cl> ccdlist @all.list   (check [ushort],filter,objname)

Overscan, Trim, Zero, Dark, and Flat (OTZDF)
***Correcting for OTZF can be done using quadproc (in the quadred package).
1. Combine bias images using qzerocombine. Notable parameters:
      input = b*.fit
      output = Zero
      combine = median
      reject = avsigclip
      ccdtype = (blank)
      process, delete, clobber = no

2. Process flats, darks, and objects (each separately) using quadproc. Notable parameters:
      images = t*.fit or a*.fit or d*.fits
      ccdtype = (blank)
      fixpix = no
      oversca, trim, zerocor = yes
      darkcor, flatcor, illumco, etc. = no
      xskip1,xskip2,xtrim1,xtrim2,ytrim1,ytrim2 = INDEF
      zero = Zero.fits
***The overscan and trim sections are listed in the headers, but I have listed them here as well:
WIRO:
bias: [2100:2200,1:2048]
trim: [55:2100,1:2048]

BOK:
OLD (before Sep/Oct 2010)
BIASSEC = '[2033:2052,1:2032]' / Bias section
TRIMSEC = '[1:2032,1:2032]' / Trim section
NEW (After Sep/Oct 2010)
BIASSEC = '[2017:2036,1:2048]' / Bias section
TRIMSEC = '[1:2016,1:2048]' / Trim section

CTIO09m:
BSEC11 = '[1055:1108,1:1023]' / Bias section definition for Amp11
TSEC11 = '[21:1044,1:1023]' / Trim section definition for Amp11
BSEC12 = '[1109:1162,1:1023]' / Bias section definition for Amp12
TSEC12 = '[1173:2196,1:1023]' / Trim section definition for Amp12
BSEC21 = '[1055:1108,1024:2046]' / Bias section definition for Amp21
TSEC21 = '[21:1044,1024:2046]' / Trim section definition for Amp21
BSEC22 = '[1109:1162,1024:2046]' / Bias section definition for Amp22
TSEC22 = '[1173:2196,1024:2046]' / Trim section definition for Amp22

3.Combine darks using qdarkcombine. MAKE SURE TO COMBINE SEPARATE EXPOSURE TIMES SEPARATELY.
Sort and make lists of darks and images:
      >ls a*.fit > object300.list
      >ls d*.fit > dark300.list
***Remove the non-300 second dark images from dark300.list and the non-300 second object images from dark300.list. Do this for all exposure times which you want to dark correct.
Notable parameters:
      input = @dark300.list
      output = Dark300
      combine = average
      reject = minmax
      ccdtype = (blank)
      scale = none
      process, delete, clobber = no

4. Combine flats using flatcombine. MAKE SURE TO COMBINE SEPARATE FILTERS SEPARATELY.
Sort and make lists of images:
      >ls a*.fit > objectR.list
      >ls t*.fit > flatR.list
***Remove the non-R band flat images from flatR.list and the non-R band object images from objectR.list. Do this for all filters.
Notable parameters:
      input = @flatR.list
      output = FlatR
      combine = median
      reject = avsigclip
      ccdtype = (blank)
      subsets = yes
      process, delete, clobber = no
      scale = mode

5.Correct objects for darks using quadproc. MAKE SURE TO CORRECT SEPARATE EXPTIMES SEPARATELY. Notable parameters:
      images = @object300.list
      ccdtype = (blank)
      darkcor = yes; everything else in that section = no
      dark = Dark300
***Repeat for all exposure times.

6.Correct objects for flats using quadproc. MAKE SURE TO CORRECT SEPARATE FILTERS SEPARATELY. Notable parameters:
      images = @objectR.list
      ccdtype = (blank)
      flatcor = yes; everything else in that section = no
      flat = FlatR
***Repeat for all filters.

Fringing and Illumination (QI)
***MAKE PREFRINGING DIRECTORY***
Remember this section only applies if your images have fringing or needs illumination correction. Also, the fringe pattern will change if you have switched from filter to filter multiple times throughout the night. The filter can be placed in a slightly different positions, so if you come back to a filter later in the night it may not be in the exact same postion as it was previously. Therefore, it is best to lump all observations of a particular filter into one portion of the night without switching the filters.
List of filters that need illumination and fringe correction:
7598:
7661:
8132:
8199: fringe and Illumination correction
8615: fringe and Illumination correction
8685: fringe and Illumination correction
9155: fringe and Illumination correction
9233: fringe and Illumination correction
U:
B: Niether
V: Niether
R: Illumination correction
I: fringe and Illumination correction
Summary of commands: (cl> refers to iraf terminal)
      >cp object300.list object_good.list
      cl> imstat @object_good.list
      cl> disp "highest count".fit
      cl> imstat @object_good.list low=0 up="saturation count" > count
      >sort -n count +2 > object_sort.list
      >n object_sort.list (erase all but image name and sorted counts)
      >cp object_sort.list imarith
      cl> cl < imarith
      cl> epar imcom
           Input: @fringe.list
           output=FringeFILTER_1
           combine=average
           lthresh=0
           hthresh=50-100 above sat of highest count fringe image
           nlow=0
           nhigh=2
           nkeep=1
      cl> disp FringeFILTER_1.fits
      *****cl> epar ccdp (set all to no)*****
      cl> mkfringecor FringeFILTER_1 out=FringeFILTER_1_cor
      cl> mkskycor FringeFILTER_1 output=illumcorFILTER_1
      cl> imhist FringeFILTER_1_cor z1=-# z2=# (close to 10^3)
      cl> epar imcom:
           Input: FringeFILTER_1_cor
           output=FringeFILTER_1_cor2
           combine=average
           lthresh=z1
           hthresh=z2
           nlow=0
           nhigh=0
           nkeep=1
      cl> epar ccdp
           Input: @group.list
           output= Leave blank
           ccdtype =object
           fringec=yes ;illuco=yes
           all others in that section = no
           fringe= FringeFILTER_1_cor2 ;illum=illumcorFILTER_1
      cl> prevu @group.list
      >cp object_good.list rename.list
      >n add a001"_monthdaydayyear".fit to rename.list
      cl> imcopy @object_good.list @rename.list
1. Make a "good" list without the calibration images. Imstat the "good" list, display image with highest counts, redo imstat with lower=0 and upper=just below "saturation limit" in image with highest counts. Sort this list by counts.
---> There are two different ways to proceed from here and it depends on whether the night you are reducing has 2 or less filters or 3 or more filters. When reducing a night with 2 or less filters that have fringing it becomes very easy and can usually get rid of the diffraction effect with only 2 sets of fringes; the other becomes more difficult and tedius. Keep this in mind when observering.
   2 OR LESS FILTERS
2.For each filter take the 3 highest count images, all three of which have to be different fields; also try not to use fields with large saturated stars:any stars larger than 1/8th the size of the entire image. These stars have residual effects on the final corrected images. Put these three in one list (fringelist), and the rest of the group in another list (grouplist).
3.Use the iraf command 'imarith' to normalize each images' counts and bring them up to the counts of the highest count image. This alows all of the images to be corrected virtually at once. Make a text file called "imarith" by copying "object_sort.list" to "imarith". The syntax is :
imarith a001.fit * ("highest count"/"a001 count") a001_temp.fit
cl> cl< imarith
4. Use imstat to find the "saturation limit" of the highest-count image in fringelist. Epar into imcombine.
Input: @FRINGELIST
output=FringeFILTER_1
combine=average
reject=minmax
lthresh=0
hthresh=50-125 above "saturation limit" of highest-count image in fringe list, (for low counts, around 200-400, add 50 or 75 or so; for higher counts add 100 or more)
nlow=0
nhigh=2
nkeep=1
Run imcombine.
5. Make sure you go into ccdproc and set all corrections to no.(second section) The next step somehow uses ccdproc and may accidently correct for fringing, dark, flat, etc. This has a noticeable effect on your images. Take a look at fringeFILTER_1. If it looks ok (i.e., with no noticable very bright or very dark spots, and no noticable traces of stars), do this: "mkfringecor FringeFILTER_1 output=FringeFILTER_1_cor." NOTE: If you get an error, it's probably because a parameter in ccdproc is set to "yes." Epar into ccdproc and make sure every parameter in the second section is set to "no." Mkfringecor centers the counts of your fringe image around 0, so the bright fringes are positive and dark fringes are negative (or vice versa).
6. Now the corrected fringe image must be trimmed so only the fringe pattern remains; no remnants of stars or anything else. To do this, use imhist and set z1 and z2 so each cuts the curve off slightly lower than 10^3 counts. If done correctly, the absolute values of z1 and z2 should be very close, i.e. within 2 or 3. Remember these values.
7. To trim our corrected fringe, we can be sneaky and use imcombine on only one image, namely the corrected fringe. So, epar into imcombine.
Input: FringeFILTER_1_cor
output=FringeFILTER_1_cor2
combine=average
lthresh=z1
hthresh=z2
nlow=0
nhigh=0
nkeep=1
Run imcombine.
8. Take a look at the new corrected image. If it looks ok, by the same standards as above (it should, if the above image was ok), you can now correct the group list for fringing. Epar into ccdproc.
Input=@GROUPLIST
Output: leave output blank (i.e., this will overwrite the images in grouplist)
ccdtype=object
fringec=yes
all others in that section=no
fringe=FringeFILTER_1_cor2
Run ccdproc
9. Finally, illumination correction. We'll use mkskycor to create an illumination image, i.e. an image that contains only the underlying illumination differences in all the images. Do this: "mkskycor FringeFILTER_1 output=illumcorFILTER_1" Now epar into ccdproc. Everything remains the same except: illumco=yes and fringec=no, and illum=illumcorFILTER_1 Run ccdproc. 10. Use prevu to peruse grouplist. Repeat above steps on any images in which fringing still exists, and on other groups. You'll have to play around with which images, when combined, will result in successful fringing correction on which images.
11. Undo your imarith on the newly corrected images.
12. Make copies of the processed images with a different name to be moved to the photometry directory. The nomenclature will be similar to the names of the log files: a001_jul228292008.fit. Now move all of the images to the corresponding field directory inside the photometry directory.
   3 OR MORE FILTERS
2. Divide this sorted list by counts, keeping roughly similar counts together. Keep in mind that now the fringing pattern usually changes over the course of the night so you may need group images according to time of night for each filter as well.
3. follow the steps above.