path: root/noao/imred/kpnocoude/doc/do3fiber.hlp

.help do3fiber Feb93 noao.imred.kpnocoude
.ih
NAME
do3fiber -- Three fiber data reduction task
.ih
USAGE
do3fiber objects
.ih
SUMMARY
The \fBdo3fiber\fR reduction task is specialized for scattered light
subtraction, extraction, flat
fielding, and wavelength calibration of multifiber data in which some
fibers are used to take object spectra and other fibers are used to
take simultaneous arc spectra.  A three fiber instrument of this
type (one object and two arc fibers) is available at the KPNO coude feed.
The default parameters are set for this configuration.
If there are a large number of fibers and fiber throughput and sky
fiber subtraction is needed the \fBdofiber\fR task should be used.

The \fBdo3fiber\fR task is a command language script which collects
and combines the functions and parameters of many general purpose tasks to
provide a single complete data reduction path.  The task provides a degree
of guidance, automation, and record keeping necessary when dealing with
this type of multifiber data.
.ih
PARAMETERS
.ls objects
List of object spectra to be processed.  Previously processed spectra are
ignored unless the \fIredo\fR flag is set or the \fIupdate\fR flag is set and
dependent calibration data has changed.  Extracted spectra are ignored.
.le
.ls apref = ""
Aperture reference spectrum.  This spectrum is used to define the basic
extraction apertures and is typically a flat field spectrum.
.le
.ls flat = "" (optional)
Flat field spectrum.  If specified the one dimensional flat field spectra
are extracted and used to make flat field corrections.
.le
.ls arcs = "" (at least one if dispersion correcting)
List of primary, all fiber arc spectra.  These spectra are used to define
the dispersion functions for each fiber apart from a possible zero point
correction made with simultaneous arc calibration fibers in the object
spectra.  One fiber from the first spectrum is used to mark lines and set
the dispersion function interactively and dispersion functions for all
other fibers and arc spectra are derived from it.
.le
.ls arctable = "" (optional) (refspectra)
Table defining arc spectra to be assigned to object
spectra (see \fBrefspectra\fR).  If not specified an assignment based
on a header parameter, \fIparams.sort\fR, such as the observation time is made.
.le

.ls readnoise = "RDNOISE" (apsum)
Read out noise in photons.  This parameter defines the minimum noise
sigma.  It is defined in terms of photons (or electrons) and scales
to the data values through the gain parameter.  A image header keyword
(case insensitive) may be specified to get the value from the image.
.le
.ls gain = "GAIN" (apsum)
Detector gain or conversion factor between photons/electrons and
data values.  It is specified as the number of photons per data value.
A image header keyword (case insensitive) may be specified to get the value
from the image.
.le
.ls datamax = INDEF (apsum.saturation)
The maximum data value which is not a cosmic ray.
When cleaning cosmic rays and/or using variance weighted extraction
very strong cosmic rays (pixel values much larger than the data) can
cause these operations to behave poorly.  If a value other than INDEF
is specified then all data pixels in excess of this value will be
excluded and the algorithms will yield improved results.
This applies only to the object spectra and not the flat field or arc
spectra.  For more
on this see the discussion of the saturation parameter in the
\fBapextract\fR package.
.le
.ls fibers = 3 (apfind)
Number of fibers.  This number is used during the automatic definition of
the apertures from the aperture reference spectrum.
.le
.ls width = 6. (apedit)
Approximate base full width of the fiber profiles.  This parameter is used
for the profile centering algorithm.
.le
.ls crval = INDEF, cdelt = INDEF (autoidentify)
These parameters specify an approximate central wavelength and dispersion.
They may be specified as numerical values, INDEF, or image header keyword
names whose values are to be used.  If one or both of these parameters are
specified as INDEF the search for a solution will be slower and more likely
to fail.
.le
.ls objaps = "2", arcaps = "1,3"
List of object and arc aperture numbers.  These are used to
identify arc apertures for wavelength calibration and object apertures
for the final results.
.le

.ls scattered = no (apscatter)
Smooth and subtracted scattered light from the object and flat field
images.  This operation consists of fitting independent smooth functions
across the dispersion using data outside the fiber apertures and then
smoothing the individual fits along the dispersion.  The initial
flat field, or if none is given the aperture reference image, are
done interactively to allow setting the fitting parameters.  All
subsequent subtractions use the same fitting parameters.
.le
.ls fitflat = yes (flat1d)
Fit the composite flat field spectrum by a smooth function and divide each
flat field spectrum by this function?  This operation removes the average
spectral signature of the flat field lamp from the sensitivity correction to
avoid modifying the object fluxes.
.le
.ls recenter = yes (aprecenter)
Recenter reference apertures for each object spectrum?
.le
.ls edit = no (apedit)
Review aperture definitions for each object spectrum?  Note that this does
not apply to the initial reference aperture which always allows
interactive review of the aperture definitions.
.le
.ls clean = no (apsum)
Detect and correct for bad pixels during extraction?  This is the same
as the clean option in the \fBapextract\fR package.  If yes this also
implies variance weighted extraction and requires reasonably good values
for the readout noise and gain.  In addition the datamax parameters
can be useful.
.le
.ls dispcor = yes
Dispersion correct spectra?  Depending on the \fIparams.linearize\fR
parameter this may either resample the spectra or insert a dispersion
function in the image header.
.le
.ls splot = yes
Plot the final spectra with the task \fBsplot\fR?
.le
.ls redo = no
Redo operations previously done?  If no then previously processed spectra
in the objects list will not be processed (unless they need to be updated).
.le
.ls update = yes
Update processing of previously processed spectra if aperture, flat
field, or dispersion reference definitions are changed?
.le
.ls batch = no
Process spectra as a background or batch job provided there are no interactive
options (\fIedit\fR and \fIsplot\fR) selected.
.le
.ls listonly = no
List processing steps but don't process?
.le

.ls params = "" (pset)
Name of parameter set containing additional processing parameters.  The
default is parameter set \fBparams\fR.  The parameter set may be examined
and modified in the usual ways (typically with "epar params" or ":e params"
from the parameter editor).  Note that using a different parameter file
is not allowed.  The parameters are described below.
.le

.ce
-- PACKAGE PARAMETERS

Package parameters are those which generally apply to all task in the
package.  This is also true of \fBdo3fiber\fR.
.ls observatory = "observatory"
Observatory at which the spectra were obtained if not specified in the
image header by the keyword OBSERVAT.  For NOAO data the image headers
identify the observatory as "kpno" or "ctio" so this parameter is not used.
For data from other observatories this parameter may be used
as describe in \fBobservatory\fR.
.le
.ls interp = "poly5" (nearest|linear|poly3|poly5|spline3|sinc)
Spectrum interpolation type used when spectra are resampled.  The choices are:

.nf
	nearest - nearest neighbor
	 linear - linear
	  poly3 - 3rd order polynomial
	  poly5 - 5th order polynomial
	spline3 - cubic spline
	   sinc - sinc function
.fi
.le
.ls dispaxis = 2
Default dispersion axis.  The dispersion axis is 1 for dispersion
running along image lines and 2 for dispersion running along image
columns.  If the image header parameter DISPAXIS is defined it has
precedence over this parameter.
.le
.ls database = "database"
Database (directory) used for storing aperture and dispersion information.
.le
.ls verbose = no
Print verbose information available with various tasks.
.le
.ls logfile = "logfile", plotfile = ""
Text and plot log files.  If a filename is not specified then no log is
kept.  The plot file contains IRAF graphics metacode which may be examined
in various ways such as with \fBgkimosaic\fR.
.le
.ls records = ""
Dummy parameter to be ignored.
.le
.ls version = "KPNOCOUDE: ..."
Version of the package.
.le

.ce
PARAMS PARAMETERS

The following parameters are part of the \fBparams\fR parameter set and
define various algorithm parameters for \fBdo3fiber\fR.

.ce
--  GENERAL PARAMETERS --
.ls line = INDEF, nsum = 10
The dispersion line (line or column perpendicular to the dispersion
axis) and number of adjacent lines (half before and half after unless
at the end of the image) used in finding, recentering, resizing,
editing, and tracing operations.  A line of INDEF selects the middle of the
image along the dispersion axis.
.le
.ls extras = no (apsum)
Include extra information in the output spectra?  When cleaning or using
variance weighting the cleaned and weighted spectra are recorded in the
first 2D plane of a 3D image, the raw, simple sum spectra are recorded in
the second plane, and the estimated sigmas are recorded in the third plane.
.le

.ce
-- DEFAULT APERTURE LIMITS --
.ls lower = -3., upper = 3. (apdefault)
Default lower and upper aperture limits relative to the aperture center.
These limits are used when the apertures are first found and may be
resized automatically or interactively.
.le

.ce
-- AUTOMATIC APERTURE RESIZING PARAMETERS --
.ls ylevel = 0.05 (apresize)
Data level at which to set aperture limits during automatic resizing.
It is a fraction of the peak relative to a local background.
.le

.ce
-- TRACE PARAMETERS --
.ls t_step = 10 (aptrace)
Step along the dispersion axis between determination of the spectrum
positions.  Note the \fInsum\fR parameter is also used to enhance the
signal-to-noise at each step.
.le
.ls t_function = "spline3", t_order = 2 (aptrace)
Default trace fitting function and order.  The fitting function types are
"chebyshev" polynomial, "legendre" polynomial, "spline1" linear spline, and
"spline3" cubic spline.  The order refers to the number of
terms in the polynomial functions or the number of spline pieces in the spline
functions.
.le
.ls t_niterate = 1, t_low = 3., t_high = 3. (aptrace)
Default number of rejection iterations and rejection sigma thresholds.
.le

.ce
-- SCATTERED LIGHT PARAMETERS --
.ls buffer = 1. (apscatter)
Buffer distance from the aperture edges to be excluded in selecting the
scattered light pixels to be used.
.le
.ls apscat1 = "" (apscatter)
Fitting parameters across the dispersion.  This references an additional
set of parameters for the ICFIT package.  The default is the "apscat1"
parameter set.
.le
.ls apscat2 = "" (apscatter)
Fitting parameters along the dispersion.  This references an additional
set of parameters for the ICFIT package.  The default is the "apscat2"
parameter set.
.le

.ce
-- APERTURE EXTRACTION PARAMETERS --
.ls weights = "none" (apsum)
Type of extraction weighting.  Note that if the \fIclean\fR parameter is
set then the weights used are "variance" regardless of the weights
specified by this parameter.  The choices are:
.ls "none"
The pixels are summed without weights except for partial pixels at the
ends.
.le
.ls "variance"
The extraction is weighted by the variance based on the data values
and a poisson/ccd model using the \fIgain\fR and \fIreadnoise\fR
parameters.
.le
.le
.ls pfit = "fit1d" (apsum) (fit1d|fit2d)
Profile fitting algorithm for cleaning and variance weighted extractions.
The default is generally appropriate for most data but users
may try the other algorithm.  See \fBapprofiles\fR for further information.
.le
.ls lsigma = 3., usigma = 3. (apsum)
Lower and upper rejection thresholds, given as a number of times the
estimated sigma of a pixel, for cleaning.
.le
.ls nsubaps = 1 (apsum)
During extraction it is possible to equally divide the apertures into
this number of subapertures.
.le

.ce
-- FLAT FIELD FUNCTION FITTING PARAMETERS --
.ls f_interactive = yes (fit1d)
Fit the composite one dimensional flat field spectrum interactively?
This is used if \fIfitflat\fR is set and a two dimensional flat field
spectrum is specified.
.le
.ls f_function = "spline3", f_order = 20 (fit1d)
Function and order used to fit the composite one dimensional flat field
spectrum.  The functions are "legendre", "chebyshev", "spline1", and
"spline3".  The spline functions are linear and cubic splines with the
order specifying the number of pieces.
.le

.ce
-- ARC DISPERSION FUNCTION PARAMETERS --
.ls threshold = 10. (autoidentify/identify/reidentify)
In order for a feature center to be determined the range of pixel intensities
around the feature must exceed this threshold.
.le
.ls coordlist = "linelists$idhenear.dat" (autoidentify/identify)
Arc line list consisting of an ordered list of wavelengths.
Some standard line lists are available in the directory "linelists$".
.le
.ls match = -3. (autoidentify/identify)
The maximum difference for a match between the dispersion function prediction
value and a wavelength in the coordinate list.
.le
.ls fwidth = 3.5 (autoidentify/identify)
Approximate full base width (in pixels) of arc lines.
.le
.ls cradius = 4. (reidentify)
Radius from previous position to reidentify arc line.
.le
.ls i_function = "legendre", i_order = 3 (autoidentify/identify)
The default function and order to be fit to the arc wavelengths as a
function of the pixel coordinate.  The functions choices are "chebyshev",
"legendre", "spline1", or "spline3".
.le
.ls i_niterate = 3, i_low = 3.0, i_high = 3.0 (autoidentify/identify)
Number of rejection iterations and sigma thresholds for rejecting arc
lines from the dispersion function fits.
.le
.ls refit = yes (reidentify)
Refit the dispersion function?  If yes and there is more than 1 line
and a dispersion function was defined in the arc reference then a new
dispersion function of the same type as in the reference image is fit
using the new pixel positions.  Otherwise only a zero point shift is
determined for the revised fitted coordinates without changing the
form of the dispersion function.
.le
.ls addfeatures = no (reidentify)
Add new features from a line list during each reidentification?
This option can be used to compensate for lost features from the
reference solution.  Care should be exercised that misidentified features
are not introduced.
.le

.ce
-- AUTOMATIC ARC ASSIGNMENT PARAMETERS --
.ls select = "interp" (refspectra)
Selection method for assigning wavelength calibration spectra.
Note that an arc assignment table may be used to override the selection
method and explicitly assign arc spectra to object spectra.
The automatic selection methods are:
.ls average
Average two reference spectra without regard to any sort parameter.
If only one reference spectrum is specified then it is assigned with a
warning.  If more than two reference spectra are specified then only the
first two are used and a warning is given.
This option is used to assign two reference spectra, with equal weights,
independent of any sorting parameter.
.le
.ls following
Select the nearest following spectrum in the reference list based on the
sorting parameter.  If there is no following spectrum use the nearest preceding
spectrum.
.le
.ls interp
Interpolate between the preceding and following spectra in the reference
list based on the sorting parameter.  If there is no preceding and following
spectrum use the nearest spectrum.  The interpolation is weighted by the
relative distances of the sorting parameter.
.le
.ls match
Match each input spectrum with the reference spectrum list in order.
This overrides the reference aperture check.
.le
.ls nearest
Select the nearest spectrum in the reference list based on the sorting
parameter.
.le
.ls preceding
Select the nearest preceding spectrum in the reference list based on the
sorting parameter.  If there is no preceding spectrum use the nearest following
spectrum.
.le
.le
.ls sort = "jd", group = "ljd" (refspectra)
Image header keywords to be used as the sorting parameter for selection
based on order and to group spectra.
A null string, "", or the word "none" may be use to disable the sorting
or grouping parameters.
The sorting parameter
must be numeric but otherwise may be anything.  The grouping parameter
may be a string or number and must simply be the same for all spectra within
the same group (say a single night).
Common sorting parameters are times or positions.
In \fBdo3fiber\fR the Julian date (JD) and the local Julian day number (LJD)
at the middle of the exposure are automatically computed from the universal
time at the beginning of the exposure and the exposure time.  Also the
parameter UTMIDDLE is computed.
.le
.ls time = no, timewrap = 17. (refspectra)
Is the sorting parameter a 24 hour time?  If so then the time origin
for the sorting is specified by the timewrap parameter.  This time
should precede the first observation and follow the last observation
in a 24 hour cycle.
.le

.ce
-- DISPERSION  CORRECTION PARAMETERS --
.ls linearize = yes (dispcor)
Interpolate the spectra to a linear dispersion sampling?  If yes the
spectra will be interpolated to a linear or log linear sampling
If no the nonlinear dispersion function(s) from the dispersion function
database are assigned to the input image world coordinate system
and the spectral data are not interpolated.
.le
.ls log = no (dispcor)
Use linear logarithmic wavelength coordinates?  Linear logarithmic
wavelength coordinates have wavelength intervals which are constant
in the logarithm of the wavelength.
.le
.ls flux = yes (dispcor)
Conserve the total flux during interpolation?  If \fIno\fR the output
spectrum is interpolated from the input spectrum at each output
wavelength coordinate.  If \fIyes\fR the input spectrum is integrated
over the extent of each output pixel.  This is slower than
simple interpolation.
.le
.ih
ENVIRONMENT PARAMETERS
The environment parameter \fIimtype\fR is used to determine the extension
of the images to be processed and created.  This allows use with any
supported image extension.  For STF images the extension has to be exact;
for example "d1h".
.ih
DESCRIPTION
The \fBdo3fiber\fR reduction task is specialized for scattered light
subtraction, extraction, flat
fielding, and wavelength calibration of multifiber data in which some
fibers are used to take object spectra and other fibers are used to
take simultaneous arc spectra.  A three fiber instrument of this
type (one object and two arc fibers) is available at the KPNO coude feed.
The default parameters are set for this configuration.
If there are a large number of fibers and fiber throughput and sky
fiber subtraction is needed the \fBdofiber\fR task should be used.

The \fBdo3fiber\fR task is a command language script which collects
and combines the functions and parameters of many general purpose tasks to
provide a single complete data reduction path.  The task provides a degree
of guidance, automation, and record keeping necessary when dealing with
this type of multifiber data.

The general organization of the task is to do the interactive setup steps
first using representative calibration data and then perform the majority
of the reductions automatically, and possibly as a background process, with
reference to the setup data.  In addition, the task determines which setup
and processing operations have been completed in previous executions of the
task and, contingent on the \fIredo\fR and \fIupdate\fR options, skip or
repeat some or all the steps.

The description is divided into a quick usage outline followed by details
of the parameters and algorithms.  The usage outline is provided as a
checklist and a refresher for those familiar with this task and the
component tasks.  It presents only the default or recommended usage.  Since
\fBdo3fiber\fR combines many separate, general purpose tasks the
description given here refers to these tasks and leaves some of the details
to their help documentation.

\fBUsage Outline\fR

.ls 6 [1]
The images are first processed with \fBccdproc\fR for overscan,
bias, and dark corrections.
The \fBdo3fibers\fR task will abort if the image header keyword CCDPROC,
which is added by \fBccdproc\fR, is missing.  If the data processed outside
of the IRAF \fBccdred\fR package then a dummy CCDPROC keyword should be
added to the image headers; say with \fBhedit\fR.
.le
.ls [2]
Set the \fBdo3fiber\fR parameters with \fBeparam\fR.  Specify the object
images to be processed, the flat field image as the aperture reference and
the flat field, and one or more arc images.  If there are many
object or arc spectra per setup you might want to prepare "@ files".
.le
.ls [3]
Run the task.  This may be repeated multiple times with different
observations and the task will generally only do the setup steps
once and only process new images.  Queries presented during the
execution for various interactive operations may be answered with
"yes", "no", "YES", or "NO".  The lower case responses apply just
to that query while the upper case responses apply to all further
such queries during the execution and no further queries of that
type will be made.
.le
.ls [4]
The apertures are defined using the specified aperture reference image
which is usually a flat field in which both the object and arc fibers are
illuminated.  The specified number of fibers are found automatically and
sequential apertures assigned.
.le
.ls [5]
A query is given allowing the apertures to be interactively reviewed.
In this mode one may adjust the aperture widths as desired either
explicitly (:lower and :upper), with the cursor ('l' and 'u'), at a
particular flux level ('y'), or with an automatic algorithm ('z')
as described by \fBapresize\fR.  To exit type 'q'.
.le
.ls [6]
The fiber positions at a series of points along the dispersion are measured
and a function is fit to these positions.  This may be done interactively to
adjust the fitting parameters.  Not all fibers need be examined and the "NO"
response will quit the interactive fitting.  To exit the interactive
fitting type 'q'.
.le
.ls [7]
If scattered light subtraction is to be done the flat field image is
used to define the scattered light fitting parameters interactively.
If one is not specified then the aperture reference image is used for
this purpose.

There are two queries for the interactive fitting.  A graph of the
data between the defined reference apertures separated by a specified
buffer distance is first shown.  The function order and type may be
adjusted.  After quiting with 'q' the user has the option of changing
the buffer value and returning to the fitting, changing the image line
or column to check if the fit parameters are satisfactory at other points,
or to quit and accept the fit parameters.  After fitting all points
across the dispersion another graph showing the scattered light from
the individual fits is shown and the smoothing parameters along the
dispersion may be adjusted.  Upon quiting with 'q' you have the option
of checking other cuts parallel to the dispersion or quiting and finishing
the scattered light function smoothing and subtraction.

If there is a throughput image then this is corrected for scattered light
noninteractively using the previous fitting parameters.
.le
.ls [8]
If flat fielding is to be done the flat field spectra are extracted.  The
average spectrum over all fibers is determined and a function is fit
interactively (exit with 'q').  This function is generally of sufficiently
high order that the overall shape is well fit.  This function is then used
to normalize the individual flat field spectra.
The final response spectra are normalized to a unit
mean over all fibers.
.le
.ls [9]
If dispersion correction is selected the first arc in the arc list is
extracted.  The middle fiber is used to identify the arc lines and define
the dispersion function using the task \fBautoidentify\fR.  The
\fIcrval\fR and \fIcdelt\fR parameters are used in the automatic
identification.  Whether or not the automatic identification is
successful you will be shown the result of the arc line identification.
If the automatic identification is not successful identify a few arc
lines with 'm' and use the 'l' line list identification command to
automatically add additional lines and fit the dispersion function.  Check
the quality of the dispersion function fit with 'f'.  When satisfied exit
with 'q'.
.le
.ls [10]
The remaining fibers are automatically reidentified.  You have the option
to review the line identifications and dispersion function for each fiber
and interactively add or delete arc lines and change fitting parameters.
This can be done selectively, such as when the reported RMS increases
significantly.
.le
.ls [11]
If the spectra are to be resampled to a linear dispersion system
(which will be the same for all spectra) default dispersion parameters
are printed and you are allowed to adjust these as desired.
.le
.ls [12]
The object spectra are now automatically scattered light subtracted,
 extracted, flat fielded, and dispersion corrected.
The reference apertures are first assigned
to the object spectra.  If the \fIrecenter\fR option is set the apertures
will have a shift applied based on recentering the fiber profiles.
If the \fIedit\fR option is set you may review and modify
the aperture definitions interactively.  Any new
arcs assigned to the object images are automatically extracted and
dispersion functions determined.  A zero point wavelength correction
is computed from the arc fiber spectra and applied to the object spectrum.
.le
.ls [13]
The option to examine the final spectra with \fBsplot\fR may be given.
To exit type 'q'.
.le
.ls [14]
If scattered light is subtracted from the input data a copy of the
original image is made by appending "noscat" to the image name.
If the data are reprocessed with the \fIredo\fR flag the original
image will be used again to allow modification of the scattered
light parameters.

The final spectra will have the same name as the original 2D images
with a ".ms" extension added.
.le

\fBSpectra and Data Files\fR

The basic input consists of multifiber object and calibration spectra
stored as IRAF images.  The type of image format is defined by the
environment parameter \fIimtype\fR.  Only images with that extension will
be processed and created.
There are two types of calibration images.  These
are flat fields and comparison lamp arc spectra.  The raw CCD images must
be processed to remove overscan, bias, and dark count effects.  This is
generally done using the \fBccdred\fR package.
The \fBdo3fiber\fR task will abort if the image header keyword CCDPROC,
which is added by \fBccdproc\fR, is missing.  If the data processed outside
of the IRAF \fBccdred\fR package then a dummy CCDPROC keyword should be
added to the image headers; say with \fBhedit\fR.
Flat fielding is generally
not done at this stage but as part of \fBdo3fiber\fR.  If for some reason
the flat field or calibration arc spectra have separate exposures through
different fibers they may be simply added.

The assignment of arc calibration exposures to object exposures is
generally done by selecting the nearest in time and interpolating.
However, the optional \fIarc assignment table\fR may be used to explicitly
assign arc images to specific objects.  The format of this file is
described in the task \fBrefspectra\fR.

The final reduced spectra are recorded in one, two or three dimensional IRAF
images.  The images have the same name as the original images with an added
".ms" extension.  Each line in the reduced image is a one dimensional
spectrum with associated aperture, wavelength, and identification
information.  With a single object spectrum the image will be one dimensional
and with multiple object spectra the image will be two dimensional.
When the \fIextras\fR parameter is set the images will be three
dimensional (regardless of the number of apertures) and the lines in the
third dimension contain additional information (see
\fBapsum\fR for further details).  These spectral formats are accepted by the
one dimensional spectroscopy tasks such as the plotting tasks \fBsplot\fR
and \fBspecplot\fR.

\fBPackage Parameters\fR

The \fBkpnocoude\fR package parameters set parameters affecting all the tasks
in the package.  Some of the parameters are not applicable to the
\fBdo3fiber\fR task.  The observatory parameter is only required for data
without an OBSERVAT header parameter (currently included in NOAO data).
The spectrum interpolation type might be changed to "sinc" but with the
cautions given in \fBonedspec.package\fR.  The dispersion axis parameter is
only needed if a DISPAXIS image header parameter is not defined.  The other
parameters define the standard I/O functions.  The verbose parameter
selects whether to print everything which goes into the log file on the
terminal.  It is useful for monitoring what the \fBdo3fiber\fR task does.  The
log and plot files are useful for keeping a record of the processing.  A
log file is highly recommended.  A plot file provides a record of
apertures, traces, and extracted spectra but can become quite large.
The plotfile is most conveniently viewed and printed with \fBgkimosaic\fR.

\fBProcessing Parameters\fR

The input images are specified by image lists.  The lists may be
a list of explicit, comma separate image names, @ files, or image
templates using pattern matching against file names in the directory.
The aperture reference spectrum is used to find the spectrum profiles and trace
them.  Thus, this requires an image with good signal in all fibers
which usually means a flat field spectrum.  It is recommended that
flat field correction be done using one dimensional extracted spectra
rather than as two dimensional images.  This is done if a flat field
spectrum is specified.  The arc assignment table is used to specifically
assign arc spectra to particular object spectra and the format
of the file is described in \fBrefspectra\fR.

The detector read out noise and gain are used for cleaning and variance
(optimal) extraction.  The dispersion axis defines the wavelength direction
of spectra in the image if not defined in the image header by the keyword
DISPAXIS.  The width parameter (in pixels) is used for the profile finding and
centering algorithm (\fBcenter1d\fR).

The number of fibers is fairly obvious.  It is the number of
fibers, including the arc fibers, to be automatically found and
assigned apertures.  The apertures are assigned aperture
numbers sequentially.  The object and arc fibers are identified
by these aperture numbers as specified by the \fIobjaps\fR and
\fIarcaps\fR parameters.  The defaults are for the case of three
fibers in the sequence arc fiber, object fiber, and arc fiber.

The approximate central wavelength and dispersion are used for the
automatic identification of the arc reference.  They may be specified
as image header keywords or values.  The INDEF values search the
entire range of the coordinate reference file but the automatic
line identification algorithm works much better and faster if
approximate values are given.

The next set of parameters select the processing steps and options.  The
scattered light option allows fitting and subtracting a scattered light
surface from the input object and flat field.  If there is significant
scattered light which is not subtracted the fiber throughput correction
will not be accurate.  The
flat fitting option allows fitting and removing the overall shape of the
flat field spectra while preserving the pixel-to-pixel response
corrections.  This is useful for maintaining the approximate object count
levels and not introducing the reciprocal of the flat field spectrum into
the object spectra.

The apertures defined for the aperture reference image are assigned to
each image.  For the object images the apertures may be shifted across
the dispersion by recentering the strongest profiles and averaging
the individual shifts to form a single shift for all apertures.  This
corrects for shifts in the detector during the observations.  The
\fIrecenter\fR parameter selects whether to apply this shift or not.

The \fIedit\fR option allows you to be queried to review the apertures
assigned to each object image.  If selected and the query answered
affirmatively the apertures may be interactively shifted and resized.  The
query may also be answered with "NO" to turn off this option during
processing.  Note that the initial aperture definitions for the aperture
reference image always allows editing.

The \fIclean\fR option invokes a profile fitting and deviant
point rejection algorithm as well as a variance weighting of points in the
aperture.  These options require knowing the effective (i.e. accounting for
any image combining) read out noise and gain.  For a discussion of cleaning
and variance weighted extraction see \fBapvariance\fR and
\fBapprofiles\fR.

The dispersion correction option selects whether to extract arc spectra,
determine dispersion functions, assign them to the object spectra, and,
possibly, resample the spectra to a linear (or log-linear) wavelength
scale.

The \fIsplot\fR option allows a query (which may be answered with "YES"
or "NO" to eliminate the query) and then plotting of the final object
spectra if answered affirmatively.  The plotting is done with the
task \fBsplot\fR.

Generally once a spectrum has been processed it will not be reprocessed if
specified as an input spectrum.  However, changes to the underlying
calibration data can cause such spectra to be reprocessed if the
\fIupdate\fR flag is set.  The changes which will cause an update are a new
reference image, new flat field, and a new arc reference image.  If all
input spectra are to be processed regardless of previous processing the
\fIredo\fR flag may be used.  Note that reprocessing clobbers the
previously processed output spectra.

The \fIbatch\fR processing option allows object spectra to be processed as
a background or batch job.  This will only occur if the aperture editing
and final spectrum plotting have been turned off, either with the task
option parameter or by answering "NO" to the queries.  The \fIlistonly\fR
option prints a summary of the processing steps which will be performed on
the input spectra without actually doing anything.  This is useful for
verifying which spectra will be affected if the input list contains
previously processed spectra.  The listing does not include any arc spectra
which may be extracted to dispersion calibrate an object spectrum.

The last parameter (excluding the task mode parameter) points to another
parameter set for the algorithm parameters.  The way \fBdo3fiber\fR works
this may not have any value and the parameter set \fBparams\fR is always
used.  The algorithm parameters are discussed further in the next section.

\fBAlgorithms and Algorithm Parameters\fR

This section summarizes the various algorithms used by the \fBdo3fiber\fR
task and the parameters which control and modify the algorithms.  The
algorithm parameters available to the user are collected in the parameter
set \fBparams\fR.  These parameters are taken from the various general
purpose tasks used by the \fBdo3fiber\fR processing task.  Additional
information about these parameters and algorithms may be found in the help
for the actual task executed.  These tasks are identified in the parameter
section listing in parenthesis.  The aim of this parameter set organization
is to collect all the algorithm parameters in one place separate from the
processing parameters and include only those which are relevant for
this type of data.  The parameter values can be changed from the
defaults by using the parameter editor,
.nf

	cl> epar params

.fi
or simple typing \fIparams\fR.  The parameter editor can also be
entered when editing the \fBdo3fiber\fR parameters by typing \fI:e
params\fR or simply \fI:e\fR if positioned at the \fIparams\fR
parameter.

\fBAperture Definitions\fR

The first operation is to define the extraction apertures, which include
the aperture width and position dependence with wavelength, for the object
and arc fibers.  This is done on a reference spectrum which is usually a
flat field taken through both fibers.  Other spectra will inherit the
reference apertures and may apply a correction for any shift of the orders
across the dispersion.  The reference apertures are defined only once
unless the \fIredo\fR option is set.

The selected number of fibers are found automatically by selecting the
highest peaks in a cut across the dispersion.  Apertures are assigned with
a limits set by the \fIlower\fR and \fIupper\fR parameter and numbered
sequentially.  A query is then given allowing the apertures to be reviewed
interactively.  If answered affirmatively a cut across the orders is shown
with the apertures marked and an interactive aperture editing mode is
entered (see \fBapedit\fR).  The main thing to be concerned about is that
the aperture numbers agree with the \fIobjaps\fR and \fIarcaps\fR
definitions.  The aperture numbers may be changed with the 'i' or 'o'
keys.  The apertures may also be resized from the default limits.
To exit the background and aperture editing steps type 'q'.

Next the positions of the fiber profiles at various points along the
dispersion are measured and a "trace function" is fit.  The user is asked
whether to fit the trace function interactively.  This is selected to
adjust the fitting parameters such as function type and order.  When
interactively fitting a query is given for each aperture.  After the first
aperture one may skip reviewing the other traces by responding with "NO".
Queries made by \fBdo3fiber\fR generally may be answered with either lower
case "yes" or "no" or with upper case "YES" or "NO".  The upper case
responses apply to all further queries and so are used to eliminate further
queries of that kind.

The above steps are all performed using tasks from the \fBapextract\fR
package and parameters from the \fBparams\fR parameters.  As a quick
summary, the dispersion direction of the spectra are determined from the
package \fBdispaxis\fR parameter if not defined in the image header.  The default
line or column for finding the orders and the number of image lines or
columns to sum are set by the \fIline\fR and \fInsum\fR parameters.  A line
of INDEF (the default) selects the middle of the image.  The automatic
finding algorithm is described for the task \fBapfind\fR and basically
finds the strongest peaks.  The tracing is done as described in
\fBaptrace\fR and consists of stepping along the image using the specified
\fIt_step\fR parameter.  The function fitting uses the \fBicfit\fR commands
with the other parameters from the tracing section.

\fBExtraction\fR

The actual extraction of the spectra is done by summing across the fixed
width apertures at each point along the dispersion.  The default is to
simply sum the pixels using partial pixels at the ends.  There is an
option to weight the sum based on a Poisson noise model using the
\fIreadnoise\fR and \fIgain\fR detector parameters.  Note that if the
\fIclean\fR option is selected the variance weighted extraction is used
regardless of the \fIweights\fR parameter.  The sigma thresholds for
cleaning are also set in the \fBparams\fR parameters.

The cleaning and variance weighting options require knowing the effective
(i.e. accounting for any image combining) read out noise and gain.  These
numbers need to be adjusted if the image has been processed such that the
intensity scale has a different origin (such as a background light
subtraction) or scaling (such as caused by unnormalized flat fielding).
For optimal extraction and cleaning to work it is recommended that
a \fIdatamax\fR value be determined for the data and the
\fIfitflat\fR option be used.  For further discussion of cleaning and
variance weighted extraction see \fBapvariance\fR and \fBapprofiles\fR as
well as  \fBapsum\fR.

\fBScattered Light Subtraction\fR

Scattered light may be subtracted from the input two dimensional image as
the first step.  This is done using the algorithm described in
\fBapscatter\fR.  This can be important if there is significant scattered
light since the flat field/throughput correction will otherwise be
incorrect.  The algorithm consists of fitting a function to the data
outside the defined apertures by a specified \fIbuffer\fR at each line or
column across the dispersion.  The function fitting parameters are the same
at each line.  Because the fitted functions are independent at each line or
column a second set of one dimensional functions are fit parallel to the
dispersion using the evaluated fit values from the cross-dispersion step.
This produces a smooth scattered light surface which is finally subtracted
from the input image.  Again the function fitting parameters are the
same at each line or column though they may be different than the parameters
used to fit across the dispersion.

The first time the task is run with a particular flat field (or aperture
reference image if no flat field is used) the scattered light fitting
parameters are set interactively using that image.  The interactive step
selects a particular line or column upon which the fitting is done
interactively with the \fBicfit\fR commands.  A query is first issued
which allows skipping this interactive stage.  Note that the interactive
fitting is only for defining the fitting functions and orders.  When
the graphical \fBicfit\fR fitting is exited (with 'q') there is a second prompt
allowing you to change the buffer distance (in the first cross-dispersion
stage) from the apertures, change the line/column, or finally quit.

The initial fitting parameters and the final set parameters are recorded
in the \fBapscat1\fR and \fBapscat2\fR hidden parameter sets.  These
parameters are then used automatically for every subsequent image
which is scattered light corrected.

The scattered light subtraction modifies the input 2D images.  To preserve
the original data a copy of the original image is made with the same
root name and the word "noscat" appended.  The scattered light subtracted
images will have the header keyword "APSCATTE" which is how the task
avoids repeating the scattered light subtraction during any reprocessing.
However if the \fIredo\fR option is selected the scattered light subtraction
will also be redone by first restoring the "noscat" images to the original
input names.

\fBFlat Field Correction\fR

Flat field corrections may be made during the basic CCD processing; i.e.
direct division by the two dimensional flat field observation.  In that
case do not specify a flat field spectrum; use the null string "".  The
\fBdo3fiber\fR task provides an alternative flat field response correction
based on division of the extracted object spectra by the extracted flat field
spectra.  A discussion of the theory and merits of flat fielding directly
verses using the extracted spectra will not be made here.  The
\fBdo3fiber\fR flat fielding algorithm is the \fIrecommended\fR method for
flat fielding since it works well and is not subject to the many problems
involved in two dimensional flat fielding.

The first step is extraction of the flat field spectrum, if specified,
using the reference apertures.  Only one flat field is allowed so if
multiple flat fields are required the data must be reduced in groups.
If the \fIfitflat\fR
option is selected (the default) the extracted flat field spectra are
averaged together and a smooth function is fit.  The default fitting
function and order are given by the parameters \fIf_function\fR and
\fIf_order\fR.  If the parameter \fIf_interactive\fR is "yes" then the
fitting is done interactively using the \fBfit1d\fR task which uses the
\fBicfit\fR interactive fitting commands.

The fitted function is divided into the individual flat field spectra to
remove the basic shape of the spectrum while maintaining the relative
individual pixel responses and any fiber to fiber differences.  This step
avoids introducing the flat field spectrum shape into the object spectra
and closely preserves the object counts.

The final step is to normalize the flat field spectra by the mean counts over
all the fibers.  This normalization step is simply to preserve the average
counts of the extracted object and arc spectra after division by the
response spectra.

\fBDispersion Correction\fR

If dispersion correction is not selected, \fIdispcor\fR=no, then the object
spectra are simply extracted.  If it is selected the arc spectra are used
to dispersion calibrate the object spectra.  There are four steps involved;
determining the dispersion functions relating pixel position to wavelength,
assigning the appropriate dispersion functions to a particular observation,
determining a zero point wavelength shift from the arc fibers to be applied
to the object fiber dispersion functions, and either storing the nonlinear
dispersion function in the image headers or resampling the spectra to
evenly spaced pixels in wavelength.

The first arc spectrum in the arc list is used to define the reference
dispersion solution.  It is extracted using the reference aperture
definitions.  The interactive task \fBautoidentify\fR is used to
automatically define the dispersion function in one fiber.  Whether or not
it is successful the user is presented with the interactive identification
graph.  The automatic identifications can be reviewed and a new solution or
corrections to the automatic solution may be performed.  The dispersion
functions for the other fibers are then determined automatically by
reference to the first fiber using the task \fBreidentify\fR.  Except in
batch mode a query is given allowing the reidentified arc spectra to be
examined interactively with \fBidentify\fR.  This would normally be done
only if the information about the reidentification printed on the terminal
indicates a problem such as a large increase in the RMS.  This query may be
eliminated in the usual way.

The set of arc dispersion function parameters are from \fBautoidentify\fR and
\fBreidentify\fR.  The parameters define a line list for use in
automatically assigning wavelengths to arc lines, a parameter controlling
the width of the centering window (which should match the base line
widths), the dispersion function type and order, parameters to exclude bad
lines from function fits, and parameters defining whether to refit the
dispersion function, as opposed to simply determining a zero point shift,
and the addition of new lines from the line list when reidentifying
additional arc spectra.  The defaults should generally be adequate and the
dispersion function fitting parameters may be altered interactively.  One
should consult the help for the two tasks for additional details of these
parameters and the operation of \fBautoidentify\fR.

If resampling of the spectra is selected by the parameter \fIlinearize\fR
all the arc dispersion functions are combined to provide a default
starting and ending wavelength and dispersion with the same number of
pixels is determined and the user is queried for any changes.  This
linear dispersion system will be applied to all spectra so that all
the final processed object spectra will have the same dispersion
sampling.

Once the reference dispersion functions are defined other arc spectra are
extracted as they are assign to the object spectra.  The assignment of
arcs is done either explicitly with an arc assignment table (parameter
\fIarctable\fR) or based on a header parameter such as a time.
The assignments are made by the task \fBrefspectra\fR.  When two arcs are
assigned to an object spectrum an interpolation is done between the two
dispersion functions.  This makes an approximate correction for steady
drifts in the dispersion.  Because the arc fibers monitor any zero point
shifts in the dispersion functions, due to translation and rotation of the
detector, it is probably only necessary to have one or two arc spectra, one
at the beginning and/or one at the end of the night.

The tasks \fBsetjd\fR and \fBsetairmass\fR are automatically run on all
spectra.  This computes and adds the header parameters for the Julian date
(JD), the local Julian day number (LJD), the universal time (UTMIDDLE), and
the air mass at the middle of the exposure.  The default arc assignment is
to use the Julian date grouped by the local Julian day number.  The
grouping allows multiple nights of data to be correctly assigned at the
same time.

When the object spectra are extracted so are the simultaneous arc spectra.
A zero point shift of the arc spectra relative to the dispersion solutions
of an assigned full arc observation is computed using \fBreidentify\fR.
The zero point shifts from the arc fibers are then
interpolated across the detector based on the positions of the arc
apertures to the positions of the object apertures.  A linear interpolation
is used which accounts for a rotation of the detector as well as a
translation along the dispersion.  The interpolated zero point wavelength
shifts are then added to the dispersion functions from the full arc
observation for the object fibers.  Note that this does not assume that the
object and arc fiber dispersion functions are the same or have the same
wavelength origin, but only that the interpolated shifts in wavelength zero
point apply to all fibers.  When there are two assigned full arc spectra
the above steps are done independently and the final pair of zero point
corrected dispersion functions for each object fiber are combined using the
assigned weights.  Once the dispersion function correction is determined
from the extracted arc fiber spectra they are deleted leaving only the
object spectra.

The last step of dispersion correction is setting the dispersion
of the object spectra.  There are two choices here.
If the \fIlinearize\fR parameter is not set the nonlinear dispersion
functions are stored in the image header.  Other IRAF tasks interpret
this information when dispersion coordinates are needed for plotting
or analysis.  This has the advantage of not requiring the spectra
to be interpolated and the disadvantage that the dispersion
information is only understood by IRAF tasks and cannot be readily
exported to other analysis software.

If the \fIlinearize\fR parameter is set then the spectra are resampled to a
linear dispersion relation either in wavelength or the log of the
wavelength.  The linear dispersion parameters are those defined
previously for the arc reference image.

The linearization algorithm  parameters allow selecting the interpolation
function type, whether to conserve flux per pixel by integrating across the
extent of the final pixel, and whether to linearize to equal linear or
logarithmic intervals.  The latter may be appropriate for radial velocity
studies.  The default is to use a fifth order polynomial for interpolation,
to conserve flux, and to not use logarithmic wavelength bins.  These
parameters are described fully in the help for the task \fBdispcor\fR which
performs the correction.
.ih
EXAMPLES
1.  The following example uses artificial data and may be executed
at the terminal (with IRAF V2.10).  This is also the sequence performed
by the test procedure "demos do3fiber".

.nf
kp> demos mkdo3fiber
Creating image demoobj ...
Creating image demoflat ...
Creating image demoarc ...
kp> do3fiber demoobj apref=demoflat flat=demoflat arcs=demoarc \
>>> width=4 edit=yes
Set reference apertures for demoflat
Resize apertures for demoflat?  (yes):
Edit apertures for demoflat?  (yes):
<Exit with 'q'>
Fit traced positions for demoflat interactively?  (yes):
Fit curve to aperture 1 of demoflat interactively  (yes):
<Exit with 'q'>
Fit curve to aperture 2 of demoflat interactively  (yes): N
Create response function demoflatnorm.ms
Extract flat field demoflat
Fit and ratio flat field demoflat
Create the normalized response demoflatnorm.ms
demoflatnorm.ms -> demoflatnorm.ms  using bzero: 0.  and bscale: 1.
    mean: 1.  median: 1.034214  mode: 0.8378798
    upper: INDEF  lower: INDEF
Average aperture response:
1.  0.8394014
2.  1.034403
3.  1.126194
Extract arc reference image demoarc
Determine dispersion solution for demoarc
<Reset default line list with ":coord linelists$idhenear.dat">
<A dispersion solution is found automatically.>
<Examine the fit with 'f'>
<Exit fit with 'q' and exit task with 'q'>

REIDENTIFY: NOAO/IRAF V2.10BETA valdes@puppis Fri 11:04:32 06-Mar-92
  Reference image = demoarc.ms, New image = demoarc.ms, Refit = yes
     Image Data Found    Fit Pix Shift  User Shift  Z Shift     RMS
d...ms - Ap 1   30/30  29/30  -0.00675       -0.04  -6.9E-6   0.252
Fit dispersion function interactively? (no|yes|NO|YES) (yes): n
d...ms - Ap 3   30/30  29/30   -0.0154     -0.0928  -1.4E-5   0.303
Fit dispersion function interactively? (no|yes|NO|YES) (no): y
<Exit with 'q'>
d...ms - Ap 3   30/30  29/30   -0.0154     -0.0928  -1.4E-5   0.303

Dispersion correct demoarc
d...ms: w1 = 5785.86, w2 = 7351.59, dw = 6.14, nw = 256
  Change wavelength coordinate assignments? (yes|no|NO): N
Extract object spectrum demoobj
Edit apertures for demoobj?  (yes): n
Assign arc spectra for demoobj
[demoobj] refspec1='demoarc'
Reidentify arc fibers in demoobj with respect to demoarc

REIDENTIFY: NOAO/IRAF V2.10BETA valdes@puppis Fri 11:04:52 06-Mar-92
  Reference image = demoarc.ms, New image = demoobjarc.ms, Refit = no
  Image Data   Found    Fit Pix Shift  User Shift  Z Shift     RMS
d...ms - Ap 1  27/30  27/27   0.00502      0.0263  3.99E-6   0.175
d...ms - Ap 3  27/30  27/27   8.62E-4       0.006  5.07E-7   0.248
Dispersion correct demoobj
demoobj.ms.imh: REFSHFT1 = 'demoobjarc.ms interp', shift = -0.0050,
rms = 0.00282813 intercept = -0.0118401, slope = 2.70764E-4
d...ms: ap = 2, w1 = 5785.86, w2 = 7351.59, dw = 6.14, nw = 256
demoobj.ms.imh:
Splot spectrum? (no|yes|NO|YES) (yes):
<Exit with 'q'>
.fi
.ih
REVISIONS
.ls DO3FIBER V2.11
The initial arc line identifications is done with the automatic line
identification algorithm.
.le
.ls DO3FIBER V2.10.3
The usual output WCS format is "equispec".  The image format type to be
processed is selected with the \fIimtype\fR environment parameter.  The
dispersion axis parameter is now a package parameter.  Images will only
be processed if the have the CCDPROC keyword.  A \fIdatamax\fR parameter
has been added to help improve cosmic ray rejection.  A scattered
light subtraction processing option has been added.
.le
.ih
SEE ALSO
apedit, apfind, approfiles, aprecenter, apresize, apsum, aptrace, apvariance,
ccdred, center1d, dispcor, fit1d, icfit, identify, observatory,
onedspec.package, refspectra, reidentify, setairmass, setjd
.endhelp