diff options
Diffstat (limited to 'noao/imred/echelle/doc/dofoe.hlp')
| -rw-r--r-- | noao/imred/echelle/doc/dofoe.hlp | 1155 |
1 files changed, 1155 insertions, 0 deletions
diff --git a/noao/imred/echelle/doc/dofoe.hlp b/noao/imred/echelle/doc/dofoe.hlp new file mode 100644 index 00000000..6dfab76a --- /dev/null +++ b/noao/imred/echelle/doc/dofoe.hlp @@ -0,0 +1,1155 @@ +.help dofoe Feb93 noao.imred.echelle +.ih +NAME +dofoe -- Fiber Optic Echelle (FOE) data reduction task +.ih +USAGE +dofoe objects +.ih +SUMMARY +The \fBdofoe\fR reduction task is specialized for scattered light +subtraction, extraction, flat fielding, and wavelength calibration of Fiber +Optic Echelle (FOE) spectra. There may be one fiber or two fibers where +the second fiber is illuminated by an arc calibration during arc and object +exposures and a flat field during flat field exposures. It is a command +language script which collects and combines the functions and parameters of +many general purpose tasks to provide a single complete data reduction +path. The task provides a degree of guidance, automation, and record +keeping necessary when dealing with the complexities of reducing this type +of data. +.ih +PARAMETERS +.ls objects +List of object spectra to be processed. Previously processed spectra are +ignored unless the \fIredo\fR flag is set or the \fIupdate\fR flag is set and +dependent calibration data has changed. Extracted spectra are ignored. +.le +.ls apref = "" +Aperture reference spectrum. This spectrum is used to define the basic +extraction apertures and is typically a flat field spectrum. +.le +.ls flat = "" (optional) +Flat field spectrum. If specified the one dimensional flat field spectrum +is extracted and used to make flat field calibrations. +.le +.ls arcs = "" (at least one if dispersion correcting) +List of arc spectra. The first arc in the list is used to create a +dispersion solution interactively. All other arc spectra will be +automatically reidentified. +.le +.ls arctable = "" (optional) (refspectra) +Table defining arc spectra to be assigned to object spectra (see +\fBrefspectra\fR). If not specified an assignment based on a header +parameter, \fIparams.sort\fR, such as the observation time is made. +.le + +.ls readnoise = "0." (apsum) +Read out noise in photons. This parameter defines the minimum noise +sigma. It is defined in terms of photons (or electrons) and scales +to the data values through the gain parameter. A image header keyword +(case insensitive) may be specified to get the value from the image. +.le +.ls gain = "1." (apsum) +Detector gain or conversion factor between photons/electrons and +data values. It is specified as the number of photons per data value. +A image header keyword (case insensitive) may be specified to get the value +from the image. +.le +.ls datamax = INDEF (apsum.saturation) +The maximum data value which is not a cosmic ray. +When cleaning cosmic rays and/or using variance weighted extraction +very strong cosmic rays (pixel values much larger than the data) can +cause these operations to behave poorly. If a value other than INDEF +is specified then all data pixels in excess of this value will be +excluded and the algorithms will yield improved results. +This applies only to the object spectra and not the flat field or +arc spectra. For more +on this see the discussion of the saturation parameter in the +\fBapextract\fR package. +.le +.ls norders = 12 (apfind) +Number of orders to be found. This number is used during the automatic +definition of the apertures from the aperture reference spectrum. Note +that when there is a second fiber for simultaneous arcs the specified +number will be automatically doubled for finding both sets of orders. +So in either case specify only the number of orders from a single fiber. +The interactive review of the aperture assignments allows verification +and adjustments to the automatic aperture definitions. +.le +.ls width = 4. (apedit) +Approximate base full width of the fiber profiles. This parameter is used +for the profile centering algorithm. +.le +.ls arcaps = "2x2" +When there is only a single fiber set this parameter to "". When there is +a second fiber used to create simultaneous arcs during the object exposures +this parameter specifies a list of aperture numbers for the arc fibers. +Since the object and arc fiber orders are paired the default setting +expects the even number apertures to be the are apertures. This should be +checked interactively. +.le + +.ls fitflat = yes (flat1d) +Fit and divide the extracted flat field orders by a smooth function +in order to normalize the wavelength response? If not done the flat field +spectral shape (which includes the blaze function) will be divided +out of the object spectra, thus altering the object data values. +If done only the small scale response variations are included in the +flat field and the object spectra will retain their observed flux +levels and blaze function. +.le +.ls background = "none" (apsum, apscatter) +Type of background light subtraction. The choices are "none" for no +background subtraction, "scattered" for a global scattered light +subtraction, "average" to average the background within background regions, +"median" to use the median in background regions, "minimum" to use the +minimum in background regions, or "fit" to fit across the dispersion using +the background within background regions. The scattered light option fits +and subtracts a smooth global background and modifies the input images. +This is a slow operation and so is NOT performed in quicklook mode. The +other background options are local to each aperture at each point along the +dispersion. The "fit" option uses additional fitting parameters from +\fBparams\fR and the "scattered" option uses parameters from \fBapscat1\fR +and \fBapscat2\fR. +.le +.ls clean = yes (apsum) +Detect and correct for bad pixels during extraction? This is the same +as the clean option in the \fBapextract\fR package. If yes this also +implies variance weighted extraction and requires reasonably good values +for the readout noise and gain. In addition the datamax parameters +can be useful. +.le +.ls dispcor = yes +Dispersion correct spectra? Depending on the \fIparams.linearize\fR +parameter this may either resample the spectra or insert a dispersion +function in the image header. +.le +.ls redo = no +Redo operations previously done? If no then previously processed spectra +in the objects list will not be processed (unless they need to be updated). +.le +.ls update = no +Update processing of previously processed spectra if aperture, flat +field, or dispersion reference definitions are changed? +.le +.ls batch = no +Process spectra as a background or batch job. +.le +.ls listonly = no +List processing steps but don't process? +.le + +.ls params = "" (pset) +Name of parameter set containing additional processing parameters. The +default is parameter set \fBparams\fR. The parameter set may be examined +and modified in the usual ways (typically with "epar params" or ":e params" +from the parameter editor). Note that using a different parameter file +is not allowed. The parameters are described below. +.le + +.ce +-- PACKAGE PARAMETERS + +Package parameters are those which generally apply to all task in the +package. This is also true of \fBdofoe\fR. +.ls observatory = "observatory" +Observatory at which the spectra were obtained if not specified in the +image header by the keyword OBSERVAT. For FOE data the image headers +identify the observatory as "kpno" so this parameter is not used. +For data from other observatories this parameter may be used +as describe in \fBobservatory\fR. +.le +.ls interp = "poly5" (nearest|linear|poly3|poly5|spline3|sinc) +Spectrum interpolation type used when spectra are resampled. The choices are: + +.nf + nearest - nearest neighbor + linear - linear + poly3 - 3rd order polynomial + poly5 - 5th order polynomial + spline3 - cubic spline + sinc - sinc function +.fi +.le +.ls dispaxis = 2 +Default dispersion axis. The dispersion axis is 1 for dispersion +running along image lines and 2 for dispersion running along image +columns. If the image header parameter DISPAXIS is defined it has +precedence over this parameter. +.le +.ls database = "database" +Database (directory) used for storing aperture and dispersion information. +.le +.ls verbose = no +Print verbose information available with various tasks. +.le +.ls logfile = "logfile", plotfile = "" +Text and plot log files. If a filename is not specified then no log is +kept. The plot file contains IRAF graphics metacode which may be examined +in various ways such as with \fBgkimosaic\fR. +.le +.ls records = "" +Dummy parameter to be ignored. +.le +.ls version = "ECHELLE: ..." +Version of the package. +.le + +.ce +PARAMS PARAMETERS + +The following parameters are part of the \fBparams\fR parameter set and +define various algorithm parameters for \fBdofoe\fR. + +.ce +-- GENERAL PARAMETERS -- +.ls line = INDEF, nsum = 10 +The dispersion line (line or column perpendicular to the dispersion +axis) and number of adjacent lines (half before and half after unless +at the end of the image) used in finding, recentering, resizing, +editing, and tracing operations. A line of INDEF selects the middle of the +image along the dispersion axis. +.le +.ls extras = no (apsum) +Include extra information in the output spectra? When cleaning or using +variance weighting the cleaned and weighted spectra are recorded in the +first 2D plane of a 3D image, the raw, simple sum spectra are recorded in +the second plane, and the estimated sigmas are recorded in the third plane. +.le + +.ce +-- DEFAULT APERTURE LIMITS -- +.ls lower = -3., upper = 3. (apdefault) +Default lower and upper aperture limits relative to the aperture center. +These limits are used when the apertures are first found and may be +resized automatically or interactively. +.le + +.ce +-- AUTOMATIC APERTURE RESIZING PARAMETERS -- +.ls ylevel = 0.05 (apresize) +Data level at which to set aperture limits during automatic resizing. +It is a fraction of the peak relative to a local background. +.le + +.ce +-- TRACE PARAMETERS -- +.ls t_step = 10 (aptrace) +Step along the dispersion axis between determination of the spectrum +positions. Note the \fInsum\fR parameter is also used to enhance the +signal-to-noise at each step. +.le +.ls t_function = "spline3", t_order = 2 (aptrace) +Default trace fitting function and order. The fitting function types are +"chebyshev" polynomial, "legendre" polynomial, "spline1" linear spline, and +"spline3" cubic spline. The order refers to the number of +terms in the polynomial functions or the number of spline pieces in the spline +functions. +.le +.ls t_niterate = 1, t_low = 3., t_high = 3. (aptrace) +Default number of rejection iterations and rejection sigma thresholds. +.le + +.ce +-- DEFAULT BACKGROUND PARAMETERS -- +.ls buffer = 1. (apscatter) +Buffer distance from the edge of any aperture for data to be included +in the scattered light determination. This parameter may be modified +interactively. +.le +.ls apscat1 = "", apscat2 = "" (apscatter) +Parameter sets for the fitting functions across and along the dispersion. +These parameters are those used by \fBicfit\fR. These parameters are +usually set interactively. +.le +.ls b_function = "legendre", b_order = 1 (apsum) +Default background fitting function and order. The fitting function types are +"chebyshev" polynomial, "legendre" polynomial, "spline1" linear spline, and +"spline3" cubic spline. The order refers to the number of +terms in the polynomial functions or the number of spline pieces in the spline +functions. +.le +.ls b_naverage = -100 (apsum) +Default number of points to average or median. Positive numbers +average that number of sequential points to form a fitting point. +Negative numbers median that number, in absolute value, of sequential +points. A value of 1 does no averaging and each data point is used in the +fit. +.le +.ls b_niterate = 0 (apsum) +Default number of rejection iterations. If greater than zero the fit is +used to detect deviant fitting points and reject them before repeating the +fit. The number of iterations of this process is given by this parameter. +.le +.ls b_low_reject = 3., b_high_reject = 3. (apsum) +Default background lower and upper rejection sigmas. If greater than zero +points deviating from the fit below and above the fit by more than this +number of times the sigma of the residuals are rejected before refitting. +.le +.ls b_smooth = 10 (apsum) +Box car smoothing length for background when using background +subtraction. Since the background noise is often the limiting factor +for good extraction one may box car smooth the background to improve the +statistics. +.le + + +.ce +-- APERTURE EXTRACTION PARAMETERS -- +.ls weights = "none" (apsum) +Type of extraction weighting. Note that if the \fIclean\fR parameter is +set then the weights used are "variance" regardless of the weights +specified by this parameter. The choices are: +.ls "none" +The pixels are summed without weights except for partial pixels at the +ends. +.le +.ls "variance" +The extraction is weighted by the variance based on the data values +and a poisson/ccd model using the \fIgain\fR and \fIreadnoise\fR +parameters. +.le +.le +.ls pfit = "fit1d" (apsum) (fit1d|fit2d) +Profile fitting algorithm for cleaning and variance weighted extractions. +The default is generally appropriate for FOE data but users +may try the other algorithm. See \fBapprofiles\fR for further information. +.le +.ls lsigma = 3., usigma = 3. (apsum) +Lower and upper rejection thresholds, given as a number of times the +estimated sigma of a pixel, for cleaning. +.le + +.ce +-- FLAT FIELD FUNCTION FITTING PARAMETERS -- +.ls f_interactive = no (fit1d) +Fit the one dimensional flat field order spectra interactively? +This is used if \fIfitflat\fR is set and a two dimensional flat field +spectrum is specified. +.le +.ls f_function = "spline3", f_order = 20 (fit1d) +Function and order used to fit the composite one dimensional flat field +spectrum. The functions are "legendre", "chebyshev", "spline1", and +"spline3". The spline functions are linear and cubic splines with the +order specifying the number of pieces. +.le + +.ce +-- ARC DISPERSION FUNCTION PARAMETERS -- +.ls threshold = 10. (identify/reidentify) +In order for a feature center to be determined the range of pixel intensities +around the feature must exceed this threshold. +.le +.ls coordlist = "linelist$thar.dat" (ecidentify) +Arc line list consisting of an ordered list of wavelengths. +Some standard line lists are available in the directory "linelist$". +.le +.ls match = 1. (ecidentify) +The maximum difference for a match between the dispersion function computed +value and a wavelength in the coordinate list. +.le +.ls fwidth = 4. (ecidentify) +Approximate full base width (in pixels) of arc lines. +.le +.ls cradius = 4. (reidentify) +Radius from previous position to reidentify arc line. +.le +.ls i_function = "chebyshev", i_xorder = 3, i_yorder = 3 (ecidentify) +The default function, function order for the pixel position dependence, and +function order for the aperture number dependence to be fit to the arc +wavelengths. The functions choices are "chebyshev" or "legendre". +.le +.ls i_niterate = 3, i_low = 3.0, i_high = 3.0 (ecidentify) +Number of rejection iterations and sigma thresholds for rejecting arc +lines from the dispersion function fits. +.le +.ls refit = yes (ecreidentify) +Refit the dispersion function? If yes and there is more than 1 line +and a dispersion function was defined in the arc reference then a new +dispersion function of the same type as in the reference image is fit +using the new pixel positions. Otherwise only a zero point shift is +determined for the revised fitted coordinates without changing the +form of the dispersion function. +.le + +.ce +-- AUTOMATIC ARC ASSIGNMENT PARAMETERS -- +.ls select = "interp" (refspectra) +Selection method for assigning wavelength calibration spectra. +Note that an arc assignment table may be used to override the selection +method and explicitly assign arc spectra to object spectra. +The automatic selection methods are: +.ls average +Average two reference spectra without regard to any sort parameter. +If only one reference spectrum is specified then it is assigned with a +warning. If more than two reference spectra are specified then only the +first two are used and a warning is given. +This option is used to assign two reference spectra, with equal weights, +independent of any sorting parameter. +.le +.ls following +Select the nearest following spectrum in the reference list based on the +sorting parameter. If there is no following spectrum use the nearest preceding +spectrum. +.le +.ls interp +Interpolate between the preceding and following spectra in the reference +list based on the sorting parameter. If there is no preceding and following +spectrum use the nearest spectrum. The interpolation is weighted by the +relative distances of the sorting parameter. +.le +.ls match +Match each input spectrum with the reference spectrum list in order. +This overrides the reference aperture check. +.le +.ls nearest +Select the nearest spectrum in the reference list based on the sorting +parameter. +.le +.ls preceding +Select the nearest preceding spectrum in the reference list based on the +sorting parameter. If there is no preceding spectrum use the nearest following +spectrum. +.le +.le +.ls sort = "jd", group = "ljd" (refspectra) +Image header keywords to be used as the sorting parameter for selection +based on order and to group spectra. +A null string, "", or the word "none" may be use to disable the sorting +or grouping parameters. +The sorting parameter +must be numeric but otherwise may be anything. The grouping parameter +may be a string or number and must simply be the same for all spectra within +the same group (say a single night). +Common sorting parameters are times or positions. +In \fBdofoe\fR the Julian date (JD) and the local Julian day number (LJD) +at the middle of the exposure are automatically computed from the universal +time at the beginning of the exposure and the exposure time. Also the +parameter UTMIDDLE is computed. +.le +.ls time = no, timewrap = 17. (refspectra) +Is the sorting parameter a 24 hour time? If so then the time origin +for the sorting is specified by the timewrap parameter. This time +should precede the first observation and follow the last observation +in a 24 hour cycle. +.le + +.ce +-- DISPERSION CORRECTION PARAMETERS -- +.ls linearize = yes (dispcor) +Interpolate the spectra to a linear dispersion sampling? If yes the +spectra will be interpolated to a linear or log linear sampling +If no the nonlinear dispersion function(s) from the dispersion function +database are assigned to the input image world coordinate system +and the spectral data are not interpolated. +.le +.ls log = no (dispcor) +Use linear logarithmic wavelength coordinates? Linear logarithmic +wavelength coordinates have wavelength intervals which are constant +in the logarithm of the wavelength. +.le +.ls flux = yes (dispcor) +Conserve the total flux during interpolation? If \fIno\fR the output +spectrum is interpolated from the input spectrum at each output +wavelength coordinate. If \fIyes\fR the input spectrum is integrated +over the extent of each output pixel. This is slower than +simple interpolation. +.le +.ih +ENVIRONMENT PARAMETERS +The environment parameter \fIimtype\fR is used to determine the extension +of the images to be processed and created. This allows use with any +supported image extension. For STF images the extension has to be exact; +for example "d1h". +.ih +DESCRIPTION +The \fBdofoe\fR reduction task is specialized for scattered light +subtraction, extraction, flat fielding, and wavelength calibration of Fiber +Optic Echelle (FOE) spectra. There may be one fiber or two fibers where +the second fiber is illuminated by an arc calibration during arc and object +exposures and a flat field during flat field exposures. When there is +just one fiber the parameter \fIarcaps\fR is set to "" and when there are +two fibers the parameter is used to select which of the defined +apertures are the orders from the simultaneous arc fiber. + +This task is a command language script which collects and combines the +functions and parameters of many general purpose tasks to provide a single +complete data reduction path. The task provides a degree of guidance, +automation, and record keeping necessary when dealing with the complexities +of reducing this type of data. + +The general organization of the task is to do the interactive setup steps +first using representative calibration data and then perform the majority +of the reductions automatically, possibly as a background process, with +reference to the setup data. In addition, the task determines which setup +and processing operations have been completed in previous executions of the +task and, contingent on the \fIredo\fR and \fIupdate\fR options, skip or +repeat some or all the steps. + +The description is divided into a quick usage outline followed by details +of the parameters and algorithms. The usage outline is provided as a +checklist and a refresher for those familiar with this task and the +component tasks. It presents only the default or recommended usage. Since +\fBdofoe\fR combines many separate, general purpose tasks the description +given here refers to these tasks and leaves some of the details to their +help documentation. + +\fBUsage Outline\fR + +.ls 6 [1] +The images must first be processed with \fBccdproc\fR for overscan, +bias, and dark corrections. +.le +.ls [2] +Set the \fBdofoe\fR parameters with \fBeparam\fR. Specify the object +images to be processed, the flat field image as the aperture reference and +the flat field, and one or more arc images. If there are many +object or arc spectra per setup you might want to prepare "@ files". +Verify and set the format parameters, particularly the number of orders to be +extracted and processed. The processing parameters are set +for simple extraction and dispersion correction but dispersion correction +can be turned off for quicklook or background subtraction and cleaning +may be added. +.le +.ls [3] +Run the task. This may be repeated multiple times with different +observations and the task will generally only do the setup steps +once and only process new images. Queries presented during the +execution for various interactive operations may be answered with +"yes", "no", "YES", or "NO". The lower case responses apply just +to that query while the upper case responses apply to all further +such queries during the execution and no further queries of that +type will be made. +.le +.ls [4] +The apertures are defined using the specified aperture reference image +which is usually a flat field in which both the object and arc fibers are +illuminated. The specified number of orders are found automatically and +sequential apertures assigned. The resize option sets the aperture size to +the widths of the profiles at a fixed fraction of the peak height. +.le +.ls [5] +The automatic order identification and aperture assignment is based on peak +height and may be incorrect. The interactive aperture editor is entered +with a plot of the apertures. When there is a second simultaneous arc +fiber it is essential that the object and arc +fiber orders are properly paired with the arc fibers having even aperture +numbers and the object fibers having odd aperture numbers. It is also +required that no orders be skipped in the region of interest. Missing +orders are added with the 'm' key. Once all orders have been marked the +aperture numbers are resequenced with 'o'. If local background subtraction +is selected the background regions should be checked with the 'b' key. +Preceding this with the 'a' key allows any changes to the background +regions to be applied to all orders. To exit type 'q'. +.le +.ls [6] +The order positions at a series of points along the dispersion are measured +and a function is fit to these positions. This may be done interactively to +adjust the fitting parameters. Not all orders need be examined and the "NO" +response will quit the interactive fitting. To exit the interactive +fitting type 'q'. +.le +.ls [7] +If flat fielding is to be done the flat field spectra are extracted. A +smooth function is fit to each flat field spectrum to remove the large +scale spectral signature. The final response spectra are normalized to a +unit mean over all fibers. +.le +.ls [8] +If scattered light subtraction is selected the scattered light parameters +are set using the aperture reference image and the task \fBapscatter\fR. +The purpose of this is to interactively define the aperture buffer distance +for the scattered light and the cross and parallel dispersion fitting +parameters. The fitting parameters are taken from and recorded in the +parameter sets \fBapscat1\fR and \fBapscat2\fR. All other scattered light +subtractions are done noninteractively with these parameters. Note that +the scattered light correction modifies the input images. +.le +.ls [9] +If dispersion correction is selected the first arc in the arc list is +extracted. One fiber is used to identify the arc lines and define the +dispersion function using the task \fBecidentify\fR. Identify a few arc +lines in a few orders with 'm' and 'k' or 'o', use the 'l' line list +identification command to automatically add additional lines and fit the +dispersion function. Check the quality of the dispersion function fit +with 'f'. When satisfied exit with 'q'. +.le +.ls [10] +If there is a second fiber the dispersion function is automatically +determined using the task \fBecreidentify\fR. +.le +.ls [11] +The arc reference spectrum is dispersion corrected. +If the spectra are resampled to a linear dispersion system +(which will be the same for all spectra) the dispersion parameters +determined from the dispersion solution are printed. +.le +.ls [12] +The object spectra are now automatically background subtracted (an +alternative to scattered light subtraction), extracted, flat fielded, +and dispersion corrected. Any new dispersion function reference arcs +assigned to the object images are automatically extracted and +dispersion functions determined. A zero point wavelength correction +is computed from the simultaneous arc fiber spectrum and applied to +the object spectrum if orders from the second fiber have been identified +with the \fIarcaps\fR parameter. +.le +.ls [13] +The final spectra will have the same name as the original 2D images +with a ".ec" extension added. +.le + +\fBSpectra and Data Files\fR + +The basic input consists of single or dual fiber FOE object and calibration +spectra stored as IRAF images. The \fIarcaps\fR parameter is used to +discriminate between the two cases. The type of image format is defined by +the environment parameter \fIimtype\fR. Only images with that extension +will be processed and created. The raw CCD images must be processed to +remove overscan, bias, and dark count effects. This is generally done +using the \fBccdred\fR package. Flat fielding is generally not done at +this stage but as part of \fBdofoe\fR. The calibration spectra are flat +field observations in all fibers, comparison arc lamp spectra in all +fibers, and, for dual fiber model, arc spectra in one fiber while the +second fiber observes the object. If for some reason the flat field or +calibration arc spectra have separate exposures for the two fibers the +separate exposures may simply be added. + +The assignment of arc calibration exposures to object exposures is +generally done by selecting the nearest in time and interpolating. +However, the optional \fIarc assignment table\fR may be used to explicitly +assign arc images to specific objects. The format of this file is +described in the task \fBrefspectra\fR. + +The final reduced spectra are recorded in two or three dimensional IRAF +images. The images have the same name as the original images with an added +".ec" extension. Each line in the reduced image is a one dimensional +spectrum (an echelle order) with associated aperture and wavelength +information. When the \fIextras\fR parameter is set the lines in the +third dimension contain additional information (see +\fBapsum\fR for further details). These spectral formats are accepted by the +one dimensional spectroscopy tasks such as the plotting tasks \fBsplot\fR +and \fBspecplot\fR. The special task \fBscopy\fR may be used to extract +specific apertures or to change format to individual one dimensional +images. The task \fBscombine\fR is used to combine or merge orders into +a single spectrum. + +\fBPackage Parameters\fR + +The \fBechelle\fR package parameters set parameters affecting all the tasks +in the package. Some of the parameters are not applicable to the +\fBdofoe\fR task. The observatory parameter is only required for data +without an OBSERVAT header parameter (currently included in NOAO data). +The spectrum interpolation type might be changed to "sinc" but with the +cautions given in \fBonedspec.package\fR. The dispersion axis parameter is +only needed if a DISPAXIS image header parameter is not defined. The other +parameters define the standard I/O functions. The verbose parameter +selects whether to print everything which goes into the log file on the +terminal. It is useful for monitoring what the \fBdofoe\fR task does. The +log and plot files are useful for keeping a record of the processing. A +log file is highly recommended. A plot file provides a record of +apertures, traces, and extracted spectra but can become quite large. +The plotfile is most conveniently viewed and printed with \fBgkimosaic\fR. + +\fBProcessing Parameters\fR + +The input images are specified by image lists. The lists may be +a list of explicit, comma separate image names, @ files, or image +templates using pattern matching against file names in the directory. +The aperture reference spectrum is used to find the orders and trace +them. Thus, this requires an image with good signal in both fibers +which usually means a flat field spectrum. It is recommended that +flat field correction be done using one dimensional extracted spectra +rather than as two dimensional images. This is done if a flat field +spectrum is specified. The arc assignment table is used to specifically +assign arc spectra to particular object spectra and the format +of the file is described in \fBrefspectra\fR. + +The detector read out noise and gain are used for cleaning and variance +(optimal) extraction. The dispersion axis defines the wavelength direction +of spectra in the image if not defined in the image header by the keyword +DISPAXIS. The width parameter (in pixels) is used for the profile +centering algorithm (\fBcenter1d\fR). + +The number of orders selects the number of orders for a single +fiber and "pairs" of object and arc +fiber profiles for dual fibers. The number specified will be +automatically found based on the strongest peaks. +In the dual fiber case it is important that both elements of a pair be found, +so no orders be skipped, and the aperture numbers must be sequential with +arc profiles having even aperture numbers and object profiles having +odd numbers in the region of interest, the automatic identification is +just a starting point for the interactive review. The even/odd +relationship between object and arc profiles is set by the \fIarcaps\fR +parameter and so may be reversed if desired. + +The next set of parameters select the processing steps and options. The +flat fitting option allows fitting and removing the overall shape of the +flat field spectra while preserving the pixel-to-pixel response +corrections. This is useful for maintaining the approximate object count +levels, including the blaze function, and not introducing the reciprocal of +the flat field spectrum into the object spectra. If not selected the flat +field will remove the blaze function from the observations and introduce +some wavelength dependence from the flat field lamp spectrum. + +The \fIbackground\fR option selects the type of correction for background or +scattered light. If the type is "scattered" a global scattered light is +fit to the data between the apertures and subtracted from the images. +\fINote that the input images are modified by this operation\fR. This +option is slow. Alternatively, a local background may be subtracted using +background regions defined for each aperture. The data in the regions may +be averaged, medianed, or the minimum value used. Another choice is to fit +the data in the background regions by a function and interpolate to the +object aperture. + +The \fIclean\fR option invokes a profile fitting and deviant point rejection +algorithm as well as a variance weighting of points in the aperture. These +options require knowing the effective (i.e. accounting for any image +combining) read out noise and gain. For a discussion of cleaning and +variance weighted extraction see \fBapvariance\fR and \fBapprofiles\fR. + +The dispersion correction option selects whether to extract arc spectra, +determine a dispersion function, assign them to the object spectra, and, +possibly, resample the spectra to a linear (or log-linear) wavelength +scale. + +Generally once a spectrum has been processed it will not be reprocessed if +specified as an input spectrum. However, changes to the underlying +calibration data can cause such spectra to be reprocessed if the +\fIupdate\fR flag is set. The changes which will cause an update are a new +reference image, new flat field, adding the scattered light option, and a +new arc reference image. If all input spectra are to be processed +regardless of previous processing the \fIredo\fR flag may be used. Note +that reprocessing clobbers the previously processed output spectra. + +The \fIbatch\fR processing option allows object spectra to be processed as +a background or batch job. The \fIlistonly\fR option prints a summary of +the processing steps which will be performed on the input spectra without +actually doing anything. This is useful for verifying which spectra will +be affected if the input list contains previously processed spectra. The +listing does not include any arc spectra which may be extracted to +dispersion calibrate an object spectrum. + +The last parameter (excluding the task mode parameter) points to another +parameter set for the algorithm parameters. The way \fBdofoe\fR works +this may not have any value and the parameter set \fBparams\fR is always +used. The algorithm parameters are discussed further in the next section. + +\fBAlgorithms and Algorithm Parameters\fR + +This section summarizes the various algorithms used by the \fBdofoe\fR +task and the parameters which control and modify the algorithms. The +algorithm parameters available to the user are collected in the parameter +set \fBparams\fR. These parameters are taken from the various general +purpose tasks used by the \fBdofoe\fR processing task. Additional +information about these parameters and algorithms may be found in the help +for the actual task executed. These tasks are identified in the parameter +section listing in parenthesis. The aim of this parameter set organization +is to collect all the algorithm parameters in one place separate from the +processing parameters and include only those which are relevant for +FOE data. The parameter values can be changed from the +defaults by using the parameter editor, +.nf + + cl> epar params + +.fi +or simple typing \fIparams\fR. The parameter editor can also be +entered when editing the \fBdofoe\fR parameters by typing \fI:e +params\fR or simply \fI:e\fR if positioned at the \fIparams\fR +parameter. + +\fBAperture Definitions\fR + +The first operation is to define the extraction apertures, which include the +aperture width, background regions, and position dependence with +wavelength, for the object and arc orders of interest. This is done +on a reference spectrum which is usually a flat field taken through +all fibers. Other spectra will inherit the reference apertures and +apply a correction for any shift of the orders across the dispersion. +The reference apertures are defined only once unless the \fIredo\fR +option is set. + +The selected number of orders are found automatically by selecting the +highest peaks in a cut across the dispersion. Note that the specified +number of orders is multiplied by two in defining the apertures when +there is a second fiber. Apertures +are assigned with a limits set by the \fIlower\fR and +\fIupper\fR parameter and numbered sequentially. A query is then +given allowing the aperture limits to be "resized" based on the profile +itself (see \fBapresize\fR). + +A cut across the orders is then shown with the apertures marked and +an interactive aperture editing mode is entered (see \fBapedit\fR). +For \fBdofoe\fR the aperture identifications and numbering is particularly +critical. When there is a single fiber the aperture numbers must +be sequential with the order numbers. If an order is skipped then the +aperture number must also be skipped. + +For dual fibers all "pairs" of object and arc orders in the region of +interest must be defined without skipping any orders. The orders must +also be numbered sequentially (though the direction does not matter) +so that the arc apertures are either all even or all odd as defined +by the \fIarcaps\fR parameter (the default is even numbers for the +arc apertures). The 'o' key will provide the necessary reordering. + +If local background subtraction is used the background regions should +also be checked with the 'b' key. Typically one adjusts all +the background regions at the same time by selecting all apertures with +the 'a' key first. To exit the background and aperture editing steps type +'q'. + +Next the positions of the orders at various points along the dispersion are +measured and "trace functions" are fit. The user is asked whether to fit +each trace function interactively. This is selected to adjust the fitting +parameters such as function type and order. When interactively fitting a +query is given for each aperture. After the first aperture one may skip +reviewing the other traces by responding with "NO". Queries made by +\fBdofoe\fR generally may be answered with either lower case "yes" or "no" +or with upper case "YES" or "NO". The upper case responses apply to all +further queries and so are used to eliminate further queries of that kind. + +The above steps are all performed using tasks from the \fBapextract\fR +package and parameters from the \fBparams\fR parameters. As a quick +summary, the dispersion direction of the spectra are determined from the +package \fBdispaxis\fR parameter if not defined in the image header. The +default line or column for finding the orders and the number of image lines +or columns to sum are set by the \fIline\fR and \fInsum\fR parameters. A +line of INDEF (the default) selects the middle of the image. The automatic +finding algorithm is described for the task \fBapfind\fR and basically +finds the strongest peaks. The resizing is described in the task +\fBapresize\fR and the parameters used are also described there and +identified in the PARAMETERS section. The tracing is done as described in +\fBaptrace\fR and consists of stepping along the image using the specified +\fIt_step\fR parameter. The function fitting uses the \fBicfit\fR commands +with the other parameters from the tracing section. + +\fBBackground or Scattered Light Subtraction\fR + +In addition to not subtracting any background scattered light there are two +approaches to subtracting this light. The first is to determine a smooth +global scattered light component. The second is to subtract a locally +determined background at each point along the dispersion and for each +aperture. Note that background subtraction is only done for object images +and not for arc images. + +The global scattered light fitting and subtraction is done with the task +\fBapscatter\fR. The function fitting parameters are set interactively +using the aperture reference spectrum. All other subtractions are done +noninteractively with the same set of parameters. The scattered light is +subtracted from the input images, thus modifying them, and one might wish +to first make backups of the original images. + +The scattered light is measured between the apertures using a specified +buffer distance from the aperture edges. The scattered light pixels are +fit by a series of one dimensional functions across the dispersion. The +independent fits are then smoothed along the dispersion by again fitting +low order functions. These fits then define the smooth scattered light +surface to be subtracted from the image. The fitting parameters are +defined and recorded in the two parameter sets \fIapscat1\fR and +\fIapscat2\fR. The scattered light algorithm is described more fully in +\fBapscatter\fR. This algorithm is relatively slow. + +Local background subtraction is done during extraction based on background +regions and parameters defined by the default background parameters or +changed during interactive review of the apertures. The background +subtraction options are to subtract the average, median, or minimum of the +pixels in the background regions, or to fit a function and subtract the +function from under the extracted object pixels. The background regions +are specified in pixels from the aperture center and follow changes in +center of the spectrum along the dispersion. The syntax is colon separated +ranges with multiple ranges separated by a comma or space. The background +fitting uses the \fBicfit\fR routines which include medians, iterative +rejection of deviant points, and a choice of function types and orders. +Note that it is important to use a method which rejects cosmic rays such as +using either medians over all the background regions (\fIbackground\fR = +"median") or median samples during fitting (\fIb_naverage\fR < -1). +The background smoothing parameter \fIb_smooth\fR is may be used +to provide some additional local smoothing of the background light. +The background subtraction algorithm and options are described in greater +detail in \fBapsum\fR and \fBapbackground\fR. + +\fBExtraction\fR + +The actual extraction of the spectra is done by summing across the fixed +width apertures at each point along the dispersion. The default is to +simply sum the pixels using partial pixels at the ends. There is an +option to weight the sum based on a Poisson noise model using the +\fIreadnoise\fR and \fIgain\fR detector parameters. Note that if the +\fIclean\fR option is selected the variance weighted extraction is used +regardless of the \fIweights\fR parameter. The sigma threshold for +cleaning are also set in the \fBparams\fR parameters. + +The cleaning and variance weighting options require knowing the effective +(i.e. accounting for any image combining) read out noise and gain. These +numbers need to be adjusted if the image has been processed such that the +intensity scale has a different origin (such as a scattered light +subtraction) or scaling (such as caused by unnormalized flat fielding). +These options also require using background subtraction if the profile does +not go to zero. For optimal extraction and cleaning to work it is +recommended that any scattered light be accounted for by local background +subtraction rather than with the scattered light subtraction and the +\fIfitflat\fR option be used. The \fIb_smooth\fR parameter is also +appropriate in this application and improves the optimal extraction results +by reducing noise in the background signal. For further discussion of +cleaning and variance weighted extraction see \fBapvariance\fR and +\fBapprofiles\fR as well as \fBapsum\fR. + +\fBFlat Field Correction\fR + +Flat field corrections may be made during the basic CCD processing; i.e. +direct division by the two dimensional flat field observation. In that +case do not specify a flat field spectrum; use the null string "". The +\fBdofoe\fR task provides an alternative flat field response correction +based on division of the extracted object spectra by the extracted flat field +spectra. A discussion of the theory and merits of flat fielding directly +verses using the extracted spectra will not be made here. The +\fBdofoe\fR flat fielding algorithm is the \fIrecommended\fR method for +flat fielding since it works well and is not subject to the many problems +involved in two dimensional flat fielding. + +The first step is extraction of the flat field spectrum, if one is specified, +using the reference apertures. Only one flat field is allowed so if +multiple flat fields are required the data must be reduced in groups. When +the \fIfitflat\fR option is selected (the default) the extracted flat field +spectra are fit by smooth functions and the ratio of the flat field spectra +to the smooth functions define the response spectra. The default fitting +function and order are given by the parameters \fIf_function\fR and +\fIf_order\fR. If the parameter \fIf_interactive\fR is "yes" then the +fitting is done interactively using the \fBfit1d\fR task which uses the +\fBicfit\fR interactive fitting commands. + +If the \fIfitflat\fR option is not selected the extracted and globally +normalized flat field spectra are directly divided in the object spectra. +This removes the blaze function, thus altering the data counts, and +introduces the reciprocal of the flat field spectrum in the object +spectra. + +The final step is to normalize the flat field spectra by the mean counts over +all the fibers. This normalization step is simply to preserve the average +counts of the extracted object and arc spectra after division by the +response spectra. + +\fBDispersion Correction\fR + +If dispersion correction is not selected, \fIdispcor\fR=no, then the object +spectra are simply extracted. If it is selected the arc spectra are used +to dispersion calibrate the object spectra. There are three steps involved; +determining the dispersion functions relating pixel position to wavelength, +assigning the appropriate dispersion function to a particular observation, +and either storing the nonlinear +dispersion function in the image headers or resampling the spectra to +evenly spaced pixels in wavelength. When there are two fibers there is +also a step of applying a zero point correction to the object fiber based +on the arc fiber. + +The first arc spectrum in the arc list is used to define the reference +dispersion solution. It is extracted using the reference aperture +definitions. Note extractions of arc spectra are not background or +scattered light subtracted. The interactive task \fBecidentify\fR is used +to define the dispersion function in one fiber. The idea is to mark some +lines in a few orders whose wavelengths are known (with the line list used +to supply additional lines after the first few identifications define the +approximate wavelengths) and to fit a function giving the wavelength from +the aperture number and pixel position. The dispersion function for the +second fiber, if one is present, is then determined automatically by +reference to the first fiber using the task \fBecreidentify\fR. + +The arc dispersion function parameters are for \fBecidentify\fR and it's +related partner \fBecreidentify\fR. The parameters define a line list for +use in automatically assigning wavelengths to arc lines, a centering width +(which should match the line widths at the base of the lines), the +dispersion function type and orders, parameters to exclude bad lines from +function fits, and defining whether to refit the dispersion function as +opposed to simply determining a zero point shift. The defaults should +generally be adequate and the dispersion function fitting parameters may be +altered interactively. One should consult the help for the two tasks for +additional details of these parameters and the interactive operation of +\fBecidentify\fR. + +Once the reference dispersion functions are defined other arc spectra are +extracted as they are assign to the object spectra. The assignment of +arcs is done either explicitly with an arc assignment table (parameter +\fIarctable\fR) or based on a header parameter such as a time. +The assignments are made by the task \fBrefspectra\fR. When two arcs are +assigned to an object spectrum an interpolation is done between the two +dispersion functions. This makes an approximate correction for steady +drifts in the dispersion. + +When a second arc fiber monitors any zero point shifts in the dispersion +functions it is probably only necessary to have one or two arc spectra, one +at the beginning and/or one at the end of the night. + +The tasks \fBsetjd\fR and \fBsetairmass\fR are automatically run on all +spectra. This computes and adds the header parameters for the Julian date +(JD), the local Julian day number (LJD), the universal time (UTMIDDLE), and +the air mass at the middle of the exposure. The default arc assignment is +to use the Julian date grouped by the local Julian day number. The +grouping allows multiple nights of data to be correctly assigned at the +same time. + +Defining the dispersion function for a new arc extraction is done with +the task \fBecreidentify\fR. This is done noninteractively with log +information recorded about the line reidentifications and the fit. + +When there are two fibers there are two full dispersion function from the +single or pair of arc spectra, one for the object fiber and one for the arc +fiber. When an object spectrum is extracted so is the simultaneous arc +spectrum. A zero point shift of the arc spectrum relative to the +dispersion solution of the dual arc observation is computed using +\fBecreidentify\fR (\fIrefit\fR=no). This zero point shift is assumed to +be the same for the object fiber and it is added to the dispersion function +of the dual arc observation for the object fiber. Note that this does not +assume that the object and arc fiber dispersion functions are the same or +have the same wavelength origin, but only that the same shift in wavelength +zero point applies to both fibers. Once the dispersion function correction +is determined from the extracted arc fiber spectrum it is deleted leaving +only the object spectrum. + +The last step of dispersion correction is setting the dispersion +of the object spectrum. There are two choices here. +If the \fIlinearize\fR parameter is not set the nonlinear dispersion +function is stored in the image header. Other IRAF tasks interpret +this information when dispersion coordinates are needed for plotting +or analysis. This has the advantage of not requiring the spectra +to be interpolated and the disadvantage that the dispersion +information is only understood by IRAF tasks and cannot be readily +exported to other analysis software. + +If the \fIlinearize\fR parameter is set then the spectra are resampled to a +linear dispersion relation either in wavelength or the log of the +wavelength. For echelle spectra each order is linearized independently so +that the wavelength interval per pixel is different in different orders. +This preserves most of the resolution and avoids over or under sampling of +the highest or lowest dispersion orders. The wavelength limits are +taken from the limits determined from the arc reference spectrum and +the number of pixels is the same as the original images. The dispersion +per pixel is then derived from these constraints. + +The linearization algorithm parameters allow selecting the interpolation +function type, whether to conserve flux per pixel by integrating across the +extent of the final pixel, and whether to linearize to equal linear or +logarithmic intervals. The latter may be appropriate for radial velocity +studies. The default is to use a fifth order polynomial for interpolation, +to conserve flux, and to not use logarithmic wavelength bins. These +parameters are described fully in the help for the task \fBdispcor\fR which +performs the correction. +.ih +EXAMPLES +1. The following example uses artificial data and may be executed +at the terminal (with IRAF V2.10). This is also the sequence performed +by the test procedure "demos dofoe". Because the images are small the +dispersion solution is somewhat simplistic. + +.nf +ec> demos mkdofoe +Creating image demoobj ... +Creating image demoflat ... +Creating image demoarc ... +ec> echelle.verbose = yes +ec> dofoe demoobj apref=demoflat flat=demoflat arcs=demoarc \ +>>> norders=3 width=5. +Set reference apertures for demoflat +Searching aperture database ... +Finding apertures ... +Mar 4 9:39: FIND - 6 apertures found for demoflat +Resize apertures for demoflat? (yes): +Resizing apertures ... +Mar 4 9:39: RESIZE - 6 apertures resized for demoflat +<Review aperture assignments. Exit with 'q'> +Fit traced positions for demoflat interactively? (yes): +Tracing apertures ... +Fit curve to aperture 1 of demoflat interactively (yes): +<Review trace and fit. Exit with 'q'> +Fit curve to aperture 2 of demoflat interactively (yes): N +Mar 4 9:39: TRACE - 6 apertures traced in demoflat. +Mar 4 9:39: DATABASE - 6 apertures for demoflat written to database +Create response function demoflatnorm.ec +Extract flat field demoflat +Searching aperture database ... +Mar 4 9:39: DATABASE - 6 apertures read for demoflat from database +Extracting apertures ... +Mar 4 9:39: EXTRACT - Aperture 1 from demoflat --> demoflat.ec +Mar 4 9:39: EXTRACT - Aperture 2 from demoflat --> demoflat.ec +Mar 4 9:39: EXTRACT - Aperture 3 from demoflat --> demoflat.ec +Mar 4 9:39: EXTRACT - Aperture 4 from demoflat --> demoflat.ec +Mar 4 9:39: EXTRACT - Aperture 5 from demoflat --> demoflat.ec +Mar 4 9:40: EXTRACT - Aperture 6 from demoflat --> demoflat.ec +Fit and ratio flat field demoflat +Create the normalized response demoflatnorm.ec +demoflatnorm.ec -> demoflatnorm.ec using bzero: 0. and bscale: 1. + mean: 1. median: 0.9990048 mode: 0.9876572 + upper: INDEF lower: INDEF +Extract arc reference image demoarc +Mar 4 9:40: DATABASE - 6 apertures read for demoflat from database +Mar 4 9:40: DATABASE - 6 apertures for demoarc written to database +Mar 4 9:40: EXTRACT - Aperture 1 from demoarc --> demoarc.ec +Mar 4 9:40: EXTRACT - Aperture 2 from demoarc --> demoarc.ec +Mar 4 9:40: EXTRACT - Aperture 3 from demoarc --> demoarc.ec +Mar 4 9:40: EXTRACT - Aperture 4 from demoarc --> demoarc.ec +Mar 4 9:40: EXTRACT - Aperture 5 from demoarc --> demoarc.ec +Mar 4 9:40: EXTRACT - Aperture 6 from demoarc --> demoarc.ec +Determine dispersion solution for demoarc +<Mark lines with 'm' and change orders with 'k' +<'m' line at pixel 78 and assign 4965. +<'k' to order 2 +<'m' line at pixel 78 and assign 5009 +<'m' line at pixel 78 and assign 5020 +<'k' to order 3 +<'m' line at pixel 78 and assign 5049.8 +<'m' line at pixel 78 and assign 5050.8 +<'m' line at pixel 78 and assign 5055.3 +<'m' line at pixel 78 and assign 5062 +<'m' line at pixel 78 and assign 5064.9 +<'f' to fit +<'q' to quit fit and 'q' to quit ECIDENTIFY + +ECREIDENTIFY: NOAO/IRAF V2.10BETA valdes@puppis Wed 09:54:16 04-Mar-92 + Reference image = demoarc.ec, Refit = yes + Image Found Fit Pix Shift User Shift Z Shift RMS + d...ec 8/8 8/8 1.48 7.06 2.11E-5 0.00879 +d...ec: ap = 1, w1 = 4959.1, w2 = 4978.5, dw = 0.076, nw = 256 +d...ec: ap = 2, w1 = 5003.4, w2 = 5022.1, dw = 0.073, nw = 256 +d...ec: ap = 3, w1 = 5049.0, w2 = 5067.0, dw = 0.070, nw = 256 +Extract object spectrum demoobj +Searching aperture database ... +Mar 4 9:54: DATABASE - 6 apertures read for demoflat from database +Recentering apertures ... +Mar 4 9:54: RECENTER - 6 apertures shifted by -0.03 for demoobj. +Mar 4 9:54: DATABASE - 6 apertures for demoobj written to database +Extracting apertures ... +Mar 4 9:54: EXTRACT - Aperture 1 from demoobj --> demoobj.ec +Mar 4 9:54: EXTRACT - Aperture 2 from demoobj --> demoobj.ec +Mar 4 9:54: EXTRACT - Aperture 3 from demoobj --> demoobj.ec +Mar 4 9:54: EXTRACT - Aperture 4 from demoobj --> demoobj.ec +Mar 4 9:54: EXTRACT - Aperture 5 from demoobj --> demoobj.ec +Mar 4 9:54: EXTRACT - Aperture 6 from demoobj --> demoobj.ec +Assign arc spectra for demoobj +[demoobj] refspec1='demoarc' +Reidentify arc fibers in demoobj with respect to demoarc + +ECREIDENTIFY: NOAO/IRAF V2.10BETA valdes@puppis Wed 09:54:28 04-Mar-92 + Reference image = demoarcarc.ec, Refit = no + Image Found Fit Pix Shift User Shift Z Shift RMS + d...ec 8/8 8/8 0.119 0.566 1.69E-6 0.00834 +Dispersion correct demoobj +d...ec.imh: ap = 1, w1 = 4959.1, w2 = 4978.5, dw = 0.076, nw = 256 +d...ec.imh: ap = 2, w1 = 5003.4, w2 = 5022.1, dw = 0.073, nw = 256 +d...ec.imh: ap = 3, w1 = 5049.0, w2 = 5067.0, dw = 0.070, nw = 256 +.fi +.ih +REVISIONS +.ls DOFOE V2.10.3 +The image format type to be +processed is selected with the \fIimtype\fR environment parameter. The +dispersion axis parameter is now a package parameter. Images will only +be processed if the have the CCDPROC keyword. A \fIdatamax\fR parameter +has been added to help improve cosmic ray rejection. A scattered +light subtraction processing option has been added. +.le +.ih +SEE ALSO +apedit, apfind, approfiles, aprecenter, apresize, apsum, aptrace, apvariance, +ccdred, center1d, dispcor, fit1d, icfit, ecidentify, observatory, +onedspec.package, refspectra, ecreidentify, setairmass, setjd +.endhelp |