diff options
Diffstat (limited to 'docs/INTRO_TO_CalFUSEv3.2.3')
| -rw-r--r-- | docs/INTRO_TO_CalFUSEv3.2.3 | 385 |
1 files changed, 0 insertions, 385 deletions
diff --git a/docs/INTRO_TO_CalFUSEv3.2.3 b/docs/INTRO_TO_CalFUSEv3.2.3 deleted file mode 100644 index 0408121..0000000 --- a/docs/INTRO_TO_CalFUSEv3.2.3 +++ /dev/null @@ -1,385 +0,0 @@ -# Documentation for CalFUSE, the FUSE Calibration Pipeline -# -# Introduction to CalFUSE v3.2 -# -# Created 02/03/2004 -# Van Dixon -# -# History: -# 03/05/2004 Update for v3.0.5. -# Change name of POTHOLE column to QUALITY. -# 04/01/2004 Update for v3.0.6. -# Pipeline now runs on Linux and works on HIST data. -# Default output binning reduced to 0.013 A. -# 05/05/2004 Update for v3.0.7. -# Fixed bug in initial spectral-centroiding routine. -# Extraction routine now uses the same algorithm to -# compute the spectral, weights, and bkgd arrays. -# Modified boxcar extraction to improve background -# estimate. -# 08/20/2004 Update for v3.0.7 release. -# Pipeline now runs on Mac OS X 10.2 and higher. -# Screening now preceeds aperture centroiding. -# Raised minimum pulse height to 2 for all segments. -# 01/21/2005 Add a note about reading the new extracted -# spectral files into IDL. -# 04/19/2005 Update for v3.1.0 release. -# 06/06/2005 Update for v3.1.1 release. -# 06/10/2005 Clarify discussion of spectral file format. -# 06/15/2005 Update for v3.1.2 release. -# 06/24/2005 Add instructions for generating new jitter files. -# 07/12/2005 Modify instructions for generating new jitter files. -# 07/21/2005 Add information about post-processing tools. -# 09/07/2005 Update for v3.1.3 release. -# 12/02/2005 Update for v3.1.5 release. -# 12/03/2005 New changes to cf_mirror_motion and cf_grating_motion -# 01/24/2006 Update for v3.1.6 release. -# 02/02/2006 Update for v3.1.7. -# 05/24/2006 Update for v3.1.8. -# 11/08/2006 Update for v3.2.0. -# 01/15/2007 Move FIFO-overflow correction to cf_convert_to_farf. -# 08/24/2007 Update for v3.2.1. -# 02/15/2008 Update for v3.2.2. -# 12/05/2008 Update for v3.2.3. -# - - -I. Introduction - - CalFUSE v3.2.3 is the latest version of the FUSE data-reduction - pipeline. This package includes a new set of calibration files and - support for Linux 64-bit and Solaris 10 machines. - - For complete details about the program, see the paper, - "CalFUSE v3: A Data-Reduction Pipeline for the Far Ultraviolet - Spectroscopic Explorer" by Dixon et al. (2007) - (http://adsabs.harvard.edu/abs/2007PASP..119..527D). It describes - the CalFUSE pipeline, the instrument calibrations upon which it is - based, and the format of the resulting calibrated data files. - - -II. Recent Improvements - - New in CalFUSE v3.1.5 - - Bad-Pixel Correction: A new routine, cf_screen_bad_pixels, flags - photon events falling in known bad-pixel regions. - - Histogram Mode: For data obtained in HIST mode, - cf_count_rate_y_distort must use a single count-rate value for - the entire exposure. To better estimate it, we replace the - time-dependent FEC-rate array with its weighted mean value. - Rather than using different HIST_PAD values for each aperture, - cf_extraction_limits now pads apertures by 8 pixels (top and - bottom). - - Jitter Correction: To improve the jitter-correction algorithm, we - removed the data-screening functions from cf_satellite_jitter and - moved them to a new routine, cf_screen_jitter. - - New in CalFUSE v3.1.6 - - Burst Correction: Because cf_screen_bursts attempts to exclude - airglow photons from the calculated background, these photons must - be flagged before the routine is called. To this end, - cf_screen_airglow is now the first of the screening routines. - - Background Models: The model background consists of a - spatially-uniform component and a scattered-light image. As a - result of changes introduced in CalFUSE v3.1.3, the scattered-light - component was over-estimated, because the data were summed over a - larger region of the detector than the scattered-light image. Now, - the same limits are applied to both data and models. - - New in CalFUSE v3.1.7 - - Spectral Centroids: We tightened the limits over which - cf_find_spectra computes the spectral centroid to prevent a bright - star in a nearby aperture or the enhanced background near the - detector edge from pulling the extraction window off center. In - both cf_compute_y_centroid and cf_find_spectra, we returned the - parameter-file keywords EMAX_LIF and EMAX_SIC to their original - behavior: if the centroid differs from the default value by more - than this limit, the default value is used. - - New in CalFUSE v3.1.8 - - Astigmatism and Dispersion: The routine - cf_astigmatism_and_dispersion was divided into two programs, - cf_astigmatism and cf_dispersion. - - Fewer Warnings: The pipeline no longer issues WARNING messages - when a raw or intermediate data file is empty, when the jitter file - is missing, when the detector voltage is low, or when it cannot - perform optimal extraction. - - Fewer Bad Spectra: The program cf_extract_spectra no longer - discards (that is, sets to zero the flux and error arrays of) - spectra with non-zero values of EXP_STAT. - - New in CalFUSE v3.2.0 - - Time-Dependent X and Y Distortions: It turns out that the X and Y - coordinates computed by the detector electronics for a particular - spectral feature drift slowly with time. The new routine - cf_time_xy_distort corrects for this effect. - - Grating Motion: The grating-motion correction has been revised to - better correct the zero point of the wavelength scale. - - Jitter Files: Over the years, FUSE engineers have repeatedly - modified the spacecraft-control software, but the routine that - generates jitter files has not been kept up to date. As a result, - the program was not using all available information to determine - the quality and value of pointing errors. We have modified the - code to make use of this information. As a result, the meaning of - the jitter-file TRKFLG array has changed: values between 1 and 5 - reflect increasing levels of confidence in the quality of reported - pointing errors, a value of 0 indicates that pointing telemetry is - missing, and a value of -1 says that the pointing is unknown but - likely to be bad. Jitter files generated with the new routine - receive a version number >= 3.0. The use of old jitter files with - the new software is likely to yield unsatisfactory results. If you - have the housekeeping files, you can generate jitter files that are - consistent with the new pipeline. To do it, delete (or hide) the - old jitter files, then type - - prompt> cf_jitter P9990101001hskpf.fit P9990101001jitrf.fit - - for each exposure. Run the pipeline as usual. - - Jitter Correction: The jitter routines cf_screen_jitter and - cf_satellite_jitter have been updated to take advantage of the - information present in the new jitter files, and three new keywords - have been added to the parameter file. Times for which the X or Y - pointing errors are greater than DX_MAX or DY_MAX (default is 30") - are flagged as bad. Only pointing errors with associated tracking - flag values >= TRKFLG (default is 3) are considered reliable. - - FIFO Overflow: Before, the screening routines flagged times when - the detector count rate went to zero because of FIFO overflows. - This technique underestimated the actual target flux. Now, we - modify the IDS dead-time correction to account for photon losses - due to FIFO overflows. - - New in CalFUSE v3.2.1 - - Jitter Correction: We have modified the program cf_jitter, which - constructs jitter files, to be more robust when presented with - weird pointing data. - - SAA Contours: The size of the South Atlantic Anomaly varies on - timescales of weeks. If you find that the default contours (in the - file saac004.fit) are too small, causing times with elevated - background rates to be flagged as good, you can switch to the - smaller set of contours in the file saac005.fit by modifying the - file master_calib_file.dat. - - New in CalFUSE v3.2.2 - - New Platforms: CalFUSE now runs on 64-bit machines running Linux - and Sun workstations running Solaris v10. See the file - INSTALLING_CalFUSEv3.2.2 for installation details. - - We have used data from the last few months of the mission to - produce final versions of the PHAH_CAL, STIM_CAL, and PHAX_CAL - files. - - New in CalFUSE v3.2.3 - - The final version of the pipeline includes updates needed for the - final processing of the FUSE archive. First, airglow exposures get - special treatment: data taken close to the earth limb are flagged - but not rejected. Airglow exposures are ignored by cf_combine and - idf_combine unless the -a flag is set. Second, the usual limits on - the spectral centroids are not applied to HIST data, because - spectra from other apertures are not present to confuse the - centroiding algorithm. - - -III. Overview of Pipeline - - The following modules are called by the shell script calfuse.csh: - - cf_ttag_init, cf_hist_init - Converts raw-data file into an IDF containing photon-event - list, GTI's, and timeline table. Histogram data are converted - to a pseudo-time-tag format. - - cf_convert_to_farf - Corrects for detector deadtime. Transforms photon coordinates - into the FARF (flight alignment reference frame). - - cf_screen_photons - Checks data quality. Assigns status flag to each photon. - - cf_remove_motions - Corrects for mirror, grating, FPA, and spacecraft motions. - - cf_assign_wavelength - Applies astigmatism and Doppler corrections. - Assigns wavelength to each photon. - - cf_flux_calibrate - Converts WEIGHT to ERG/CM2 for each photon. - - cf_bad_pixels - Applies image-motion corrections to bad-pixel map. - - cf_extract_spectra - Extracts LiF and SiC spectra only for target aperture. - Wavelength array is user defined; default spacing is 0.013 A. - - -IV. Installing and Running the Pipeline - - CalFUSE is now available for Macs running OS X 10.2 and higher, - 64-bit Linux machines, and Sun workstations running Solaris 10. - - Please see the document INSTALLING_CalFUSEv3.2.2 for instructions. - - For both TTAG and HIST data, the command is - - prompt> calfuse P99901010011attagfraw.fit - prompt> calfuse P99901010011ahistfraw.fit - - The pipeline expects to find all data files (*raw.fit, *jitrf.fit, - *hskpf.fit) in the current directory. It operates on one detector - segment at a time. - - CalFUSE v3 produces the following output files: an intermediate - data file (IDF), discussed below; a bad-pixel map (BPM), with a - format similar to the IDF; two extracted spectral files, one for - each of the LiF and SiC channels; and (if IDL is installed on your - machine) a pair of GIF or JPEG files, one an image of the detector - and the other a count-rate plot. - - -V. The Intermediate Data File (IDF) - - The photon-list files consist of three FITS binary tables. The - first contains the photon events themselves. The pipeline does not - discard any events, but uses a series of flags to indicate whether - particular photons violate pulse-height limits, limb-angle - constraints, etc. The IDL tool cf_edit (discussed below) allows - users to modify these flags and combine IDF files from multiple - exposures. - - The second extension to the data files is a list of good-time - intervals (GTI's). They are not used by the extraction routine, - but may prove helpful to pipeline users. - - The third extension is called a timeline table. For each second - during the exposure, it lists a dozen parameters, including count - rate, day-night status, and detector voltage. The pipeline uses - this list to set the status flags for each photon. - - Note: because of the new file format employed by IDF files, - extensions 1 and 3 must be read using the /fscale keyword. - - idl> a=mrdfits('P99901010011attagfidf.fit',1,/fscale) - idl> help,a,/str - - Elements of individual arrays must be addressed using the syntax - - idl> print,a.time[3:30] -- not a[3:30].time - - For details about the format of the IDF, please see the document - - [calfuse_directory]/v3.1/src/README_FILES/IDF_Format.txt - - -VI. Manipulating the IDF with cf_edit - - CF_EDIT is an Interactive Data Language (IDL) visualization tool - for the examination and modification of FUSE IDF files containing - flux- and wavelength-calibrated photon-event lists. Users without - IDL licenses may use the IDL "Virtual Machine" version of the - tool for free. Complete instructions for installing and using - cf_edit are available at this URL: - - http://fuse.pha.jhu.edu/analysis/fuse_idl_tools.html - - More information on the IDL Virtual Machine is available here: - - http://www.rsinc.com/idlvm/index.asp - - -VII. The Output Spectral File - - The format of the output spectral files is somewhat changed from - earlier versions of CalFUSE. - - float WAVE Wavelength (Angstroms) - float FLUX Flux (erg/cm2/s/A) - float ERROR Gaussian errors (erg/cm2/s/A) - int COUNTS Raw counts in extraction window - float WEIGHTS Raw counts corrected for deadtime, etc. - float BKGD Estimated background in extraction window - short QUALITY Percentage of window used for extraction - - Note that the QUALITY array no longer lists the number of Y pixels - that contribute to each output X pixel. Instead, it is the fraction - of the extraction window considered to contain valid data, expressed - as a percentage (0 is all bad; 100 is all good). - - Do-it-yourself-ers can use the following recipe to generate a - flux-calibrated spectrum: - - TARGET_COUNTS = WEIGHTS - BKGD - TARGET_FLUX = TARGET_COUNTS * HC / LAMBDA / AEFF / EXPTIME / WPC - - where AEFF = effective area in cm^2 and WPC = size of output pixel in - Angstroms. - -VIII. Additional Files Available from MAST - - CalFUSE produces eight extracted spectral (*fcal.fit) files for - each exposure. We combine them into a set of three - observation-level files for submission to the MAST archive. - Depending on the target and the questions that you are trying to - answer, you may find that these files are of sufficient fidelity - for scientific investigation. Here's a brief description of their - contents: - - ALL: For each channel (LiF 1A, SiC 1A, etc.), we combine data from - all exposures in the observation into a single spectrum. If the - individual spectra are bright enough, we cross correlate and shift - before combining them. If the spectra are too faint, we combine the - individual IDF files and extract a single spectrum to optimize the - background model. The combined spectrum for each channel is stored - in a separate extension. - - Note: The cataloging software used by MAST requires the presence of - an ALL file for each exposure, not just for the entire observation. - We now generate such a file, but it contains no data, only a FITS - file header. The observation-level ALL files discussed above can be - identified by the string "00000all" in their names. - - ANO (all, night-only): Same format as the ALL files, but using only - data obtained during the night-time portion of each exposure. These - files are generated only for TTAG data, and only if EXPNIGHT > 0. - - NVO (National Virtual Observatory): Contains a single spectrum - spanning the entire FUSE wavelength range. The spectrum is - assembled by cutting and pasting segments from the most sensitive - channel at each wavelength. Segments are shifted to match LiF 1A - between 1045 and 1070 E. Columns are WAVE, FLUX, and ERROR and are - stored in a single binary table extension. - -IX. Data-Analysis Tools - - A variety of IDL routines to display and manipulate FUSE data are - available from the FUSE IDL Tools Reference Page: - - http://fuse.pha.jhu.edu/analysis/fuse_idl_tools.html - - Data-analysis tools distributed with the CalFUSE pipeline are - reviewed in the document FUSE Tools in C: - - http://fuse.pha.jhu.edu/analysis/fuse_toolbox.html - - A subset of these programs, designed specifically for the - manipulation of IDF files, is described in the IDF Cookbook: - - http://fuse.pha.jhu.edu/analysis/idfcook/index.html |