path: root/noao/onedspec/doc/odcombine.hlp

.help odcombine Apr04 onedspec
.ih
NAME
odcombine -- Combine spectra using various algorithms
.ih
USAGE	
odcombine input output
.ih
PARAMETERS
.ls input
List of input images containing spectra to be combined.  The spectra
in the images to be combined are selected with the \fIapertures\fR and
\fIgroup\fR parameters.  Only the primary spectrum is combined and
the associated band spectra are ignored.  This task does not work on
higher dimensional spectra data.  To apply it first use a task to
extract it to 1D spectra.  The simplest method is \fBscopy\fR.
.le
.ls output
List of output images to be created containing the combined spectra.  If
the grouping option is "all" then only one output image is created with the
specified name.  If the grouping option is "images" then there will be one
output image for each input image and the output list must match the input
list in number.  If the grouping option is "apertures" then only one output
root name is specified and there will be one output image for each selected
aperture.  In this case the output images will have a name formed from the
root name and a four digit aperture number extension.  In all cases the
output images contain a single 1D spectrum.  Other tasks, such as
\fBscopy\fR, may be used to pack the spectra into a single file.
.le


There are a number of additional optional output files that may be produced.
The lists are handled in the same was as for the primary output; i.e.
depending on the grouping a single name, root name, or a matching list
is specified.
.ls headers = "" (optional)
Optional output multiextension FITS file(s).  The extensions are dataless
headers from each input image.
.le
.ls bpmasks = "" (optional)
Optional output bad pixel mask(s) with good values of 0 and bad values of
1.  Output pixels are marked as bad when no input pixels contributed to the
output pixel.  The file name is also added to the output image header under
the keyword BPM.
.le
.ls rejmask = "" (optional)
Optional output mask file(s) identifying rejected or excluded pixels.  The
pixel mask is the size of the output image but there is one extra dimension
with length equal to the number of input images.  Each element of the
highest dimension is a mask corresponding to an input image with values of
1 for rejected or excluded pixels and values of 0 for pixels which were
used.  The order of the masks is the order of the input images and image
header keywords, indexed by the pixel coordinate of the highest dimension
identify the input images.  Note that the pixel positions are in the output
pixel coordinate system.
.le
.ls nrejmasks = "" (optional)
Optional output pixel mask(s) giving the number of input pixels rejected or
excluded from the input images.
.le
.ls expmasks = "" (optional)
Optional output exposure mask(s) giving the sum of the exposure values of
the input images with non-zero weights that contributed to that pixel.
Since masks are integer, the exposure values may be scaled to preserve
dynamic range and fractional significance.  The scaling values are given in
the header under the keywords MASKSCAL and MASKZERO.  Exposure values are
computed from the mask values by scale * value + zero where scale is the
value of the MASKSCAL keyword and zero is the value of the MASKZERO
keyword.
.le
.ls sigma = "" (optional)
Optional output sigma image(s).  The sigma is the standard deviation,
corrected for a finite population, of the input pixel values (excluding
rejected pixels) about the output combined pixel values.
.le
.ls logfile = "STDOUT" (optional)
Optional output log file.  If no file is specified then no log information is
produced.  The special filename "STDOUT" prints log information to the
terminal.
.le


.ce
Grouping Parameters
.ls apertures = ""
List of apertures to be selected for combining.  If none is specified
then all apertures are selected.  The syntax is a blank or comma separated
list of aperture numbers or hypen separated aperture ranges.
.le
.ls group = "apertures" (all|images|apertures)
Option for grouping input spectra for combining (after selection by aperture)
from one or more input images.  The options are:
.ls "all"
Combine all spectra from all images in the input list into a single output
spectrum.
.le
.ls "images"
Combine all spectra in each input image into a single spectrum in
separate output images.
.le
.ls "apertures"
Combine all spectra of the same aperture from all input images and put it
into an output image with specified root name and a four digit aperture
number extension.
.le
.le


.ce
Dispersion Matching Parameters
.ls first = no
Use the first input spectrum of each set to be combined to define the
dispersion coordinates for combining and output?  If yes then all other
spectra to be combined will be interpolated to the dispersion of this
spectrum and that dispersion defines the dispersion of the
output spectrum.  If no, then all the spectra are interpolated to a linear
dispersion as determined by the following parameters.  The interpolation
type is set by the package parameter \fIinterp\fR.
.le
.ls w1 = INDEF, w2=INDEF, dw = INDEF, nw = INDEF, log = no
The output linear or log linear wavelength scale if the dispersion of the
first spectrum is not used.  INDEF values are filled in from the maximum
wavelength range and minimum dispersion of the spectra to be combined.  The
parameters are aways specified in linear wavelength even when the log
parameter is set to produce constant pixel increments in the log of the
wavelength.  The dispersion is interpreted in that case as the difference
in the log of the endpoints divided by the number of pixel.
.le


.ce
Combining Parameters
.ls combine = "average" (average|median|sum)
Type of combining operation performed on the final set of pixels (after
offsetting, masking, thresholding, and rejection).  The choices are
"average", "median", or "sum".  The median uses the average of the two central
values when the number of pixels is even.  For the average and sum, the
pixel values are multiplied by the weights (1 if no weighting is used)
and summed.  The average is computed by dividing by the sum of the weights.
If the sum of the weights is zero then the unweighted average is used.
.le
.ls reject = "none" (none|minmax|ccdclip|crreject|sigclip|avsigclip|pclip)
Type of rejection operation performed on the pixels remaining after offsetting,
masking and thresholding.  The algorithms are described in the
help page for \fBimcombine\fR.  The rejection choices are:

.nf
      none - No rejection
    minmax - Reject the nlow and nhigh pixels
   ccdclip - Reject pixels using CCD noise parameters
  crreject - Reject only positive pixels using CCD noise parameters
   sigclip - Reject pixels using a sigma clipping algorithm
 avsigclip - Reject pixels using an averaged sigma clipping algorithm
     pclip - Reject pixels using sigma based on percentiles
.fi

.le
.ls outtype = "real" (none|short|ushort|integer|long|real|double)
Output image pixel datatype.  The pixel datatypes are "double", "real",
"long", "integer", unsigned short "ushort", and "short" with highest
precedence first.  If "none" is specified then the highest precedence
datatype of the input images is used.  When there is a mixture of
short and unsigned short images the highest precedence become integer.
The datatypes may be abbreviated to a single character.
.le
.ls outlimits = ""
Output region limits specified as a pair of whitespace separated pixel
values.
.le


.ce
Masking Parameters
.ls smaskformat = "bpmspectrum" (bpmspectrum|bpmpixel)
When a mask is applied it must be matched to the input spectrum.  If the
value of this parameter is "bpmspectrum" the mask file is assumed to have a
spectral file structure with aperture and dispersion information.  The mask
spectrum is matched to the input spectrum by aperture number and is
rebinned from its dispersion to match the rebinned dispersion of the input
spectrum.  If the value is "bpmpixel" the mask file is assumed to have
minimal header information and the pixel information is matched to the
input image pixels.  This means the mask pixels are extracted from the same
line as the input spectrum and the mask pixels are resampled in the same
way as the input spectrum pixels.
.le
.ls smasktype = "none" (none|goodvalue|badvalue|goodbits|badbit)
Type of pixel masking to use.  If "none" or "" then no pixel masking is
done even if an image has an associated  pixel mask.  The other choices are
to select the value in the pixel mask to be treated as good (goodvalue) or
bad (badvalue) or the bits (specified as a value) to be treated as good
(goodbits) or bad (badbits).  The pixel mask filename is specified by the
image header keyword "BPM".  Note that if the input image contains
multiple spectra then the mask file must also contain at least the
selected apertures if the mask format is "bpmspectrum" or matching
image dimensions if the mask format is "bpmpixel".
.le
.ls maskvalue = 0
Mask value used with the \fImasktype\fR parameter.  If the mask type
selects good or bad bits the value may be specified using IRAF notation
for decimal, octal, or hexadecimal; i.e 12, 14b, 0cx to select bits 3
and 4.
.le
.ls blank = 0.
Output value to be used when there are no pixels.
.le


.ce
Scaling/Weighting Parameters

The following scaling and weighting parameters have the following behavior
and constraints, which are particularly relevant to multispec formats where
multiple spectra are contained in an image with a single image header.
When using image statistics these are calculated from the rebinned spectra
being combined as expected.  When using header keywords the values will be
the same for all spectra from the same input file.

When using a file then the list will be applied repeatedly to each
group being combined.  If the grouping is by aperture then the values will
be matched in the order of the input images.  Note that if an image does
not contain a specified aperture the ordering will be wrong.  If the
grouping is by image then the file will be matched to the spectra in the
order of the apertures in the image.  And if the grouping is "all" then the
list is matched in the order of the images and apertures within the
images with the apertures in an image varying first.

.ls scale = "none" (none|mode|median|mean|exposure|@<file>|!<keyword>)
Multiplicative image scaling to be applied.  The choices are none, multiply
by the reciprocal of the mode, median, or mean of the specified statistics
section, multiply by the reciprocal of the exposure time in the image header,
multiply by the values in a specified file, or multiply by a specified
image header keyword.  When specified in a file the scales must be one per
line in the order of the input images.
.le
.ls zero = "none" (none|mode|median|mean|@<file>|!<keyword>)
Additive zero level image shifts to be applied.  The choices are none, add
the negative of the mode, median, or mean of the specified statistics
section, add the values given in a file, or add the values given by an
image header keyword.  When specified in a file the zero values must be one
per line in the order of the input images.  File or keyword zero offset
values do not allow a correction to the weights.
.le
.ls weight = "none" (none|mode|median|mean|exposure|@<file>|!<keyword>)
Weights to be applied during the final averaging.  The choices are none,
the mode, median, or mean of the specified statistics section, the exposure
time, values given in a file, or values given by an image header keyword.
When specified in a file the weights must be one per line in the order of
the input images and the only adjustment made by the task is for the number of
images previously combined.   In this case the weights should be those
appropriate for the scaled images which would normally be the inverse
of the variance in the scaled image.
.le
.ls statsec = ""
Section of images to use in computing image statistics for scaling and
weighting.  If no section is given then the entire region of the input is
sampled (for efficiency the images are sampled if they are big enough).
When the images are offset relative to each other one can precede the image
section with one of the modifiers "input", "output", "overlap".  The first
interprets the section relative to the input image (which is equivalent to
not specifying a modifier), the second interprets the section relative to
the output image, and the last selects the common overlap and any following
section is ignored.
.le
.ls  expname = ""
Image header keyword to be used with the exposure scaling and weighting
options.  Also if an exposure keyword is specified that keyword will be
added to the output image using a weighted average of the input exposure
values.
.le


.ce
Algorithm Parameters
.ls lthreshold = INDEF, hthreshold = INDEF
Low and high thresholds to be applied to the input pixels.  This is done
before any scaling, rejection, and combining.  If INDEF the thresholds
are not used.
.le
.ls nlow = 1,  nhigh = 1 (minmax)
The number of low and high pixels to be rejected by the "minmax" algorithm.
These numbers are converted to fractions of the total number of input images
so that if no rejections have taken place the specified number of pixels
are rejected while if pixels have been rejected by masking, thresholding,
or nonoverlap, then the fraction of the remaining pixels, truncated
to an integer, is used.
.le
.ls nkeep = 1
The minimum number of pixels to retain or the maximum number to reject
when using the clipping algorithms (ccdclip, crreject, sigclip,
avsigclip, or pclip).  When given as a positive value this is the minimum
number to keep.  When given as a negative value the absolute value is
the maximum number to reject.  The latter is in addition to pixels
missing due to non-overlapping offsets, bad pixel masks, or thresholds.
.le
.ls mclip = yes (ccdclip, crreject, sigclip, avsigcliip)
Use the median as the estimate for the true intensity rather than the
average with high and low values excluded in the "ccdclip", "crreject",
"sigclip", and "avsigclip" algorithms?  The median is a better estimator
in the presence of data which one wants to reject than the average.
However, computing the median is slower than the average.
.le
.ls lsigma = 3., hsigma = 3. (ccdclip, crreject, sigclip, avsigclip, pclip)
Low and high sigma clipping factors for the "ccdclip", "crreject", "sigclip",
"avsigclip", and "pclip" algorithms.  They multiply a "sigma" factor
produced by the algorithm to select a point below and above the average or
median value for rejecting pixels.  The lower sigma is ignored for the
"crreject" algorithm.
.le
.ls rdnoise = "0.", gain = "1.", snoise = "0." (ccdclip, crreject)
CCD readout noise in electrons, gain in electrons/DN, and sensitivity noise
as a fraction.  These parameters are used with the "ccdclip" and "crreject"
algorithms.  The values may be either numeric or an image header keyword
which contains the value.  The noise model for a pixel is:

.nf
    variance in DN = (rdnoise/gain)^2 + DN/gain + (snoise*DN)^2
    variance in e- = (rdnoise)^2 + (gain*DN) + (snoise*(gain*DN))^2
		   = rdnoise^2 + Ne + (snoise * Ne)^2
.fi

where DN is the data number and Ne is the number of electrons.  Sensitivity
noise typically comes from noise introduced during flat fielding.
.le
.ls sigscale = 0.1 (ccdclip, crreject, sigclip, avsigclip)
This parameter determines when poisson corrections are made to the
computation of a sigma for images with different scale factors.  If all
relative scales are within this value of unity and all relative zero level
offsets are within this fraction of the mean then no correction is made.
The idea is that if the images are all similarly though not identically
scaled, the extra computations involved in making poisson corrections for
variations in the sigmas can be skipped.  A value of zero will apply the
corrections except in the case of equal images and a large value can be
used if the sigmas of pixels in the images are independent of scale and
zero level.
.le
.ls pclip = -0.5 (pclip)
Percentile clipping algorithm parameter.  If greater than
one in absolute value then it specifies a number of pixels above or
below the median to use for computing the clipping sigma.  If less
than one in absolute value then it specifies the fraction of the pixels
above or below the median to use.  A positive value selects a point
above the median and a negative value selects a point below the median.
The default of -0.5 selects approximately the quartile point.
See the DESCRIPTION section for further details.
.le
.ls grow = 0.
Radius in pixels for additional pixel to be rejected in an image with a
rejected pixel from one of the rejection algorithms.  This applies only to
pixels rejected by one of the rejection algorithms and not the masked or
threshold rejected pixels.
.le

The following parameters are internal to the task and not user parameters:

.nf
    offsets, masktype, maskvalue
.fi

.ce
Environment Variables

.ls <package>.interp
When the spectra have to be interpolated to a common pixel sampling
the "interp" parameter from the package from which ODCOMBINE is used
will be used.
.le
.ih
DESCRIPTION
\fBOdcombine\fR combines input spectra by interpolating them (if necessary)
to a common dispersion sampling, rejecting pixels exceeding specified low
and high thresholds or identified as bad in a bad pixel mask, scaling them
in various ways, applying a rejection algorithm based on known or empirical
noise statistics, and computing the sum, weighted average, or median of the
remaining pixels.  Note that the "sum" option is the direct summation of
the pixels and does not perform any rejection or scaling of the data
regardless of the parameter settings.

The input spectra are specified using an image list in which each image
may contain multiple spectra.  The set of spectra may be restricted
by the \fIaperture\fR parameter to specific apertures.  The set of input
spectra may then be grouped using the \fIgroup\fR parameter and each
group combined separately into final output spectra.  The grouping
options are to select all the input spectra regardless of the input
image or aperture number, select all spectra of the same aperture,
or select all the spectra from the same input image.

The output consists of one image for each combined group.  The output
images and combined spectra inherit the header parameters from the first
spectrum in the combined group.  There are a number of additional optional
outputs provided.  The optional logfile lists parameters, the spectra
combined for each group, scaling, weights, etc., and the output names.

The spectral combining is done using pixels at common dispersion
coordinates rather than physical or logical pixel coordinates.  If the
spectra to be combined do not have identical dispersion coordinates then
the spectra are interpolated to a common dispersion sampling before
combining.  The interpolation conserves pixel values rather pixel fluxes.
This means that flux calibrated data is treated correctly and that
spectra in counts are not corrected in the interpolation for changes in
pixel widths.  The default interpolation function is a 5th order
polynomial.  The choice of interpolation type is made with the package
parameter "interp".  It may be set to "nearest", "linear", "spline3",
"poly5", or "sinc".  Remember that this applies to all tasks which might
need to interpolate spectra in the \fBonedspec\fR and associated packages.
For a discussion of interpolation types see \fBonedspec\fR.

There are two choices for the common dispersion coordinate sampling. If the
\fIfirst\fR parameter is set then the dispersion sampling of the first
spectrum is used.  If this dispersion is nonlinear then the end points and
number of pixels are preserved and a linear dispersion is applied between
the endpoints.  If the parameter is not set then the user specified linear
or log linear dispersion system is used.  Any combination of starting
wavelength, ending wavelength, wavelength per pixel, and number of output
pixels may be specified.  Unspecified values will default to reasonable
values based on the minimum or maximum wavelengths of all spectra, the
minimum dispersion, and the number of pixels needed to satisfy the other
parameters.  If the parameters overspecify the linear system then the
ending wavelength is adjusted based on the other parameters.  Note that for
a log linear system the wavelengths are still specified in nonlog units and
the dispersion is finally recalculated using the difference of the log
wavelength endpoints divided by the number pixel intervals (the number of
pixels minus one).

This task is layered on top of the \fBimcombine\fR task.  What happens
is that the spectra for each group to be combined is extracted from
the input, resampled to a common dispersion, and the resulting spectra
written to temporary images, one per spectrum.  The temporary images
are written to the current working directory with names begining with
"tmp".  The same is done with any bad pixel masks.  Then the list of
images are combined using the IMCOMBINE algorithms.  When the combining
is completed the temporary images are removed.  If ODCOMBINE aborts
for some reason these file may be left behind and the user may delete
them.  Details of what IMCOMBINE does are presented separate under the
help topic for the IMCOMBINE task.

.ih
EXAMPLES
1.  Combine orders of echelle images.

.nf
	cl> odcombine *.ec *%.ec%% group=images combine=sum
.fi

2.  Combine all spectra using range syntax and scale by the exposure times.

.nf
	cl> names irs 10-42 > irs.dat
	cl> odcombine @irs.dat irscombine group=all scale=exptime
.fi

3.  Combine spectra by apertures using exposure time scaling and weighting.

.nf
	cl> odcombine *.ms comb1d \\
	>>> group=apertures scale=exptime weights=exptime
	cl> scopy comb1d.* comb.ms format="multispec"
	cl> imdel comb1d.*
.fi
.ih
REVISIONS
.ls ODCOMBINE V2.12.3
This is a new version that incorporates most of the features of
IMCOMBINE.

In addition to the many new features, including application of pixel
masks, the following functional differences from the old SCOMBINE
are noted.

.ls 1
The output is always a single spectrum per image.
.le
.ls 2
The "first" option does not allow rebinning to a non-linear dispersion.
Instead, it rebins to the nearest linear dispersion matching the first
spectrum.
.le
.ih
SEE ALSO
imcombine, scombine, scopy, sarith, lscombine
.endhelp