diff options
Diffstat (limited to 'noao/onedspec/doc/odcombine.hlp')
| -rw-r--r-- | noao/onedspec/doc/odcombine.hlp | 480 |
1 files changed, 480 insertions, 0 deletions
diff --git a/noao/onedspec/doc/odcombine.hlp b/noao/onedspec/doc/odcombine.hlp new file mode 100644 index 00000000..11ddffe5 --- /dev/null +++ b/noao/onedspec/doc/odcombine.hlp @@ -0,0 +1,480 @@ +.help odcombine Apr04 onedspec +.ih +NAME +odcombine -- Combine spectra using various algorithms +.ih +USAGE +odcombine input output +.ih +PARAMETERS +.ls input +List of input images containing spectra to be combined. The spectra +in the images to be combined are selected with the \fIapertures\fR and +\fIgroup\fR parameters. Only the primary spectrum is combined and +the associated band spectra are ignored. This task does not work on +higher dimensional spectra data. To apply it first use a task to +extract it to 1D spectra. The simplest method is \fBscopy\fR. +.le +.ls output +List of output images to be created containing the combined spectra. If +the grouping option is "all" then only one output image is created with the +specified name. If the grouping option is "images" then there will be one +output image for each input image and the output list must match the input +list in number. If the grouping option is "apertures" then only one output +root name is specified and there will be one output image for each selected +aperture. In this case the output images will have a name formed from the +root name and a four digit aperture number extension. In all cases the +output images contain a single 1D spectrum. Other tasks, such as +\fBscopy\fR, may be used to pack the spectra into a single file. +.le + + +There are a number of additional optional output files that may be produced. +The lists are handled in the same was as for the primary output; i.e. +depending on the grouping a single name, root name, or a matching list +is specified. +.ls headers = "" (optional) +Optional output multiextension FITS file(s). The extensions are dataless +headers from each input image. +.le +.ls bpmasks = "" (optional) +Optional output bad pixel mask(s) with good values of 0 and bad values of +1. Output pixels are marked as bad when no input pixels contributed to the +output pixel. The file name is also added to the output image header under +the keyword BPM. +.le +.ls rejmask = "" (optional) +Optional output mask file(s) identifying rejected or excluded pixels. The +pixel mask is the size of the output image but there is one extra dimension +with length equal to the number of input images. Each element of the +highest dimension is a mask corresponding to an input image with values of +1 for rejected or excluded pixels and values of 0 for pixels which were +used. The order of the masks is the order of the input images and image +header keywords, indexed by the pixel coordinate of the highest dimension +identify the input images. Note that the pixel positions are in the output +pixel coordinate system. +.le +.ls nrejmasks = "" (optional) +Optional output pixel mask(s) giving the number of input pixels rejected or +excluded from the input images. +.le +.ls expmasks = "" (optional) +Optional output exposure mask(s) giving the sum of the exposure values of +the input images with non-zero weights that contributed to that pixel. +Since masks are integer, the exposure values may be scaled to preserve +dynamic range and fractional significance. The scaling values are given in +the header under the keywords MASKSCAL and MASKZERO. Exposure values are +computed from the mask values by scale * value + zero where scale is the +value of the MASKSCAL keyword and zero is the value of the MASKZERO +keyword. +.le +.ls sigma = "" (optional) +Optional output sigma image(s). The sigma is the standard deviation, +corrected for a finite population, of the input pixel values (excluding +rejected pixels) about the output combined pixel values. +.le +.ls logfile = "STDOUT" (optional) +Optional output log file. If no file is specified then no log information is +produced. The special filename "STDOUT" prints log information to the +terminal. +.le + + +.ce +Grouping Parameters +.ls apertures = "" +List of apertures to be selected for combining. If none is specified +then all apertures are selected. The syntax is a blank or comma separated +list of aperture numbers or hypen separated aperture ranges. +.le +.ls group = "apertures" (all|images|apertures) +Option for grouping input spectra for combining (after selection by aperture) +from one or more input images. The options are: +.ls "all" +Combine all spectra from all images in the input list into a single output +spectrum. +.le +.ls "images" +Combine all spectra in each input image into a single spectrum in +separate output images. +.le +.ls "apertures" +Combine all spectra of the same aperture from all input images and put it +into an output image with specified root name and a four digit aperture +number extension. +.le +.le + + +.ce +Dispersion Matching Parameters +.ls first = no +Use the first input spectrum of each set to be combined to define the +dispersion coordinates for combining and output? If yes then all other +spectra to be combined will be interpolated to the dispersion of this +spectrum and that dispersion defines the dispersion of the +output spectrum. If no, then all the spectra are interpolated to a linear +dispersion as determined by the following parameters. The interpolation +type is set by the package parameter \fIinterp\fR. +.le +.ls w1 = INDEF, w2=INDEF, dw = INDEF, nw = INDEF, log = no +The output linear or log linear wavelength scale if the dispersion of the +first spectrum is not used. INDEF values are filled in from the maximum +wavelength range and minimum dispersion of the spectra to be combined. The +parameters are aways specified in linear wavelength even when the log +parameter is set to produce constant pixel increments in the log of the +wavelength. The dispersion is interpreted in that case as the difference +in the log of the endpoints divided by the number of pixel. +.le + + +.ce +Combining Parameters +.ls combine = "average" (average|median|sum) +Type of combining operation performed on the final set of pixels (after +offsetting, masking, thresholding, and rejection). The choices are +"average", "median", or "sum". The median uses the average of the two central +values when the number of pixels is even. For the average and sum, the +pixel values are multiplied by the weights (1 if no weighting is used) +and summed. The average is computed by dividing by the sum of the weights. +If the sum of the weights is zero then the unweighted average is used. +.le +.ls reject = "none" (none|minmax|ccdclip|crreject|sigclip|avsigclip|pclip) +Type of rejection operation performed on the pixels remaining after offsetting, +masking and thresholding. The algorithms are described in the +help page for \fBimcombine\fR. The rejection choices are: + +.nf + none - No rejection + minmax - Reject the nlow and nhigh pixels + ccdclip - Reject pixels using CCD noise parameters + crreject - Reject only positive pixels using CCD noise parameters + sigclip - Reject pixels using a sigma clipping algorithm + avsigclip - Reject pixels using an averaged sigma clipping algorithm + pclip - Reject pixels using sigma based on percentiles +.fi + +.le +.ls outtype = "real" (none|short|ushort|integer|long|real|double) +Output image pixel datatype. The pixel datatypes are "double", "real", +"long", "integer", unsigned short "ushort", and "short" with highest +precedence first. If "none" is specified then the highest precedence +datatype of the input images is used. When there is a mixture of +short and unsigned short images the highest precedence become integer. +The datatypes may be abbreviated to a single character. +.le +.ls outlimits = "" +Output region limits specified as a pair of whitespace separated pixel +values. +.le + + +.ce +Masking Parameters +.ls smaskformat = "bpmspectrum" (bpmspectrum|bpmpixel) +When a mask is applied it must be matched to the input spectrum. If the +value of this parameter is "bpmspectrum" the mask file is assumed to have a +spectral file structure with aperture and dispersion information. The mask +spectrum is matched to the input spectrum by aperture number and is +rebinned from its dispersion to match the rebinned dispersion of the input +spectrum. If the value is "bpmpixel" the mask file is assumed to have +minimal header information and the pixel information is matched to the +input image pixels. This means the mask pixels are extracted from the same +line as the input spectrum and the mask pixels are resampled in the same +way as the input spectrum pixels. +.le +.ls smasktype = "none" (none|goodvalue|badvalue|goodbits|badbit) +Type of pixel masking to use. If "none" or "" then no pixel masking is +done even if an image has an associated pixel mask. The other choices are +to select the value in the pixel mask to be treated as good (goodvalue) or +bad (badvalue) or the bits (specified as a value) to be treated as good +(goodbits) or bad (badbits). The pixel mask filename is specified by the +image header keyword "BPM". Note that if the input image contains +multiple spectra then the mask file must also contain at least the +selected apertures if the mask format is "bpmspectrum" or matching +image dimensions if the mask format is "bpmpixel". +.le +.ls maskvalue = 0 +Mask value used with the \fImasktype\fR parameter. If the mask type +selects good or bad bits the value may be specified using IRAF notation +for decimal, octal, or hexadecimal; i.e 12, 14b, 0cx to select bits 3 +and 4. +.le +.ls blank = 0. +Output value to be used when there are no pixels. +.le + + +.ce +Scaling/Weighting Parameters + +The following scaling and weighting parameters have the following behavior +and constraints, which are particularly relevant to multispec formats where +multiple spectra are contained in an image with a single image header. +When using image statistics these are calculated from the rebinned spectra +being combined as expected. When using header keywords the values will be +the same for all spectra from the same input file. + +When using a file then the list will be applied repeatedly to each +group being combined. If the grouping is by aperture then the values will +be matched in the order of the input images. Note that if an image does +not contain a specified aperture the ordering will be wrong. If the +grouping is by image then the file will be matched to the spectra in the +order of the apertures in the image. And if the grouping is "all" then the +list is matched in the order of the images and apertures within the +images with the apertures in an image varying first. + +.ls scale = "none" (none|mode|median|mean|exposure|@<file>|!<keyword>) +Multiplicative image scaling to be applied. The choices are none, multiply +by the reciprocal of the mode, median, or mean of the specified statistics +section, multiply by the reciprocal of the exposure time in the image header, +multiply by the values in a specified file, or multiply by a specified +image header keyword. When specified in a file the scales must be one per +line in the order of the input images. +.le +.ls zero = "none" (none|mode|median|mean|@<file>|!<keyword>) +Additive zero level image shifts to be applied. The choices are none, add +the negative of the mode, median, or mean of the specified statistics +section, add the values given in a file, or add the values given by an +image header keyword. When specified in a file the zero values must be one +per line in the order of the input images. File or keyword zero offset +values do not allow a correction to the weights. +.le +.ls weight = "none" (none|mode|median|mean|exposure|@<file>|!<keyword>) +Weights to be applied during the final averaging. The choices are none, +the mode, median, or mean of the specified statistics section, the exposure +time, values given in a file, or values given by an image header keyword. +When specified in a file the weights must be one per line in the order of +the input images and the only adjustment made by the task is for the number of +images previously combined. In this case the weights should be those +appropriate for the scaled images which would normally be the inverse +of the variance in the scaled image. +.le +.ls statsec = "" +Section of images to use in computing image statistics for scaling and +weighting. If no section is given then the entire region of the input is +sampled (for efficiency the images are sampled if they are big enough). +When the images are offset relative to each other one can precede the image +section with one of the modifiers "input", "output", "overlap". The first +interprets the section relative to the input image (which is equivalent to +not specifying a modifier), the second interprets the section relative to +the output image, and the last selects the common overlap and any following +section is ignored. +.le +.ls expname = "" +Image header keyword to be used with the exposure scaling and weighting +options. Also if an exposure keyword is specified that keyword will be +added to the output image using a weighted average of the input exposure +values. +.le + + +.ce +Algorithm Parameters +.ls lthreshold = INDEF, hthreshold = INDEF +Low and high thresholds to be applied to the input pixels. This is done +before any scaling, rejection, and combining. If INDEF the thresholds +are not used. +.le +.ls nlow = 1, nhigh = 1 (minmax) +The number of low and high pixels to be rejected by the "minmax" algorithm. +These numbers are converted to fractions of the total number of input images +so that if no rejections have taken place the specified number of pixels +are rejected while if pixels have been rejected by masking, thresholding, +or nonoverlap, then the fraction of the remaining pixels, truncated +to an integer, is used. +.le +.ls nkeep = 1 +The minimum number of pixels to retain or the maximum number to reject +when using the clipping algorithms (ccdclip, crreject, sigclip, +avsigclip, or pclip). When given as a positive value this is the minimum +number to keep. When given as a negative value the absolute value is +the maximum number to reject. The latter is in addition to pixels +missing due to non-overlapping offsets, bad pixel masks, or thresholds. +.le +.ls mclip = yes (ccdclip, crreject, sigclip, avsigcliip) +Use the median as the estimate for the true intensity rather than the +average with high and low values excluded in the "ccdclip", "crreject", +"sigclip", and "avsigclip" algorithms? The median is a better estimator +in the presence of data which one wants to reject than the average. +However, computing the median is slower than the average. +.le +.ls lsigma = 3., hsigma = 3. (ccdclip, crreject, sigclip, avsigclip, pclip) +Low and high sigma clipping factors for the "ccdclip", "crreject", "sigclip", +"avsigclip", and "pclip" algorithms. They multiply a "sigma" factor +produced by the algorithm to select a point below and above the average or +median value for rejecting pixels. The lower sigma is ignored for the +"crreject" algorithm. +.le +.ls rdnoise = "0.", gain = "1.", snoise = "0." (ccdclip, crreject) +CCD readout noise in electrons, gain in electrons/DN, and sensitivity noise +as a fraction. These parameters are used with the "ccdclip" and "crreject" +algorithms. The values may be either numeric or an image header keyword +which contains the value. The noise model for a pixel is: + +.nf + variance in DN = (rdnoise/gain)^2 + DN/gain + (snoise*DN)^2 + variance in e- = (rdnoise)^2 + (gain*DN) + (snoise*(gain*DN))^2 + = rdnoise^2 + Ne + (snoise * Ne)^2 +.fi + +where DN is the data number and Ne is the number of electrons. Sensitivity +noise typically comes from noise introduced during flat fielding. +.le +.ls sigscale = 0.1 (ccdclip, crreject, sigclip, avsigclip) +This parameter determines when poisson corrections are made to the +computation of a sigma for images with different scale factors. If all +relative scales are within this value of unity and all relative zero level +offsets are within this fraction of the mean then no correction is made. +The idea is that if the images are all similarly though not identically +scaled, the extra computations involved in making poisson corrections for +variations in the sigmas can be skipped. A value of zero will apply the +corrections except in the case of equal images and a large value can be +used if the sigmas of pixels in the images are independent of scale and +zero level. +.le +.ls pclip = -0.5 (pclip) +Percentile clipping algorithm parameter. If greater than +one in absolute value then it specifies a number of pixels above or +below the median to use for computing the clipping sigma. If less +than one in absolute value then it specifies the fraction of the pixels +above or below the median to use. A positive value selects a point +above the median and a negative value selects a point below the median. +The default of -0.5 selects approximately the quartile point. +See the DESCRIPTION section for further details. +.le +.ls grow = 0. +Radius in pixels for additional pixel to be rejected in an image with a +rejected pixel from one of the rejection algorithms. This applies only to +pixels rejected by one of the rejection algorithms and not the masked or +threshold rejected pixels. +.le + +The following parameters are internal to the task and not user parameters: + +.nf + offsets, masktype, maskvalue +.fi + +.ce +Environment Variables + +.ls <package>.interp +When the spectra have to be interpolated to a common pixel sampling +the "interp" parameter from the package from which ODCOMBINE is used +will be used. +.le +.ih +DESCRIPTION +\fBOdcombine\fR combines input spectra by interpolating them (if necessary) +to a common dispersion sampling, rejecting pixels exceeding specified low +and high thresholds or identified as bad in a bad pixel mask, scaling them +in various ways, applying a rejection algorithm based on known or empirical +noise statistics, and computing the sum, weighted average, or median of the +remaining pixels. Note that the "sum" option is the direct summation of +the pixels and does not perform any rejection or scaling of the data +regardless of the parameter settings. + +The input spectra are specified using an image list in which each image +may contain multiple spectra. The set of spectra may be restricted +by the \fIaperture\fR parameter to specific apertures. The set of input +spectra may then be grouped using the \fIgroup\fR parameter and each +group combined separately into final output spectra. The grouping +options are to select all the input spectra regardless of the input +image or aperture number, select all spectra of the same aperture, +or select all the spectra from the same input image. + +The output consists of one image for each combined group. The output +images and combined spectra inherit the header parameters from the first +spectrum in the combined group. There are a number of additional optional +outputs provided. The optional logfile lists parameters, the spectra +combined for each group, scaling, weights, etc., and the output names. + +The spectral combining is done using pixels at common dispersion +coordinates rather than physical or logical pixel coordinates. If the +spectra to be combined do not have identical dispersion coordinates then +the spectra are interpolated to a common dispersion sampling before +combining. The interpolation conserves pixel values rather pixel fluxes. +This means that flux calibrated data is treated correctly and that +spectra in counts are not corrected in the interpolation for changes in +pixel widths. The default interpolation function is a 5th order +polynomial. The choice of interpolation type is made with the package +parameter "interp". It may be set to "nearest", "linear", "spline3", +"poly5", or "sinc". Remember that this applies to all tasks which might +need to interpolate spectra in the \fBonedspec\fR and associated packages. +For a discussion of interpolation types see \fBonedspec\fR. + +There are two choices for the common dispersion coordinate sampling. If the +\fIfirst\fR parameter is set then the dispersion sampling of the first +spectrum is used. If this dispersion is nonlinear then the end points and +number of pixels are preserved and a linear dispersion is applied between +the endpoints. If the parameter is not set then the user specified linear +or log linear dispersion system is used. Any combination of starting +wavelength, ending wavelength, wavelength per pixel, and number of output +pixels may be specified. Unspecified values will default to reasonable +values based on the minimum or maximum wavelengths of all spectra, the +minimum dispersion, and the number of pixels needed to satisfy the other +parameters. If the parameters overspecify the linear system then the +ending wavelength is adjusted based on the other parameters. Note that for +a log linear system the wavelengths are still specified in nonlog units and +the dispersion is finally recalculated using the difference of the log +wavelength endpoints divided by the number pixel intervals (the number of +pixels minus one). + +This task is layered on top of the \fBimcombine\fR task. What happens +is that the spectra for each group to be combined is extracted from +the input, resampled to a common dispersion, and the resulting spectra +written to temporary images, one per spectrum. The temporary images +are written to the current working directory with names begining with +"tmp". The same is done with any bad pixel masks. Then the list of +images are combined using the IMCOMBINE algorithms. When the combining +is completed the temporary images are removed. If ODCOMBINE aborts +for some reason these file may be left behind and the user may delete +them. Details of what IMCOMBINE does are presented separate under the +help topic for the IMCOMBINE task. + +.ih +EXAMPLES +1. Combine orders of echelle images. + +.nf + cl> odcombine *.ec *%.ec%% group=images combine=sum +.fi + +2. Combine all spectra using range syntax and scale by the exposure times. + +.nf + cl> names irs 10-42 > irs.dat + cl> odcombine @irs.dat irscombine group=all scale=exptime +.fi + +3. Combine spectra by apertures using exposure time scaling and weighting. + +.nf + cl> odcombine *.ms comb1d \\ + >>> group=apertures scale=exptime weights=exptime + cl> scopy comb1d.* comb.ms format="multispec" + cl> imdel comb1d.* +.fi +.ih +REVISIONS +.ls ODCOMBINE V2.12.3 +This is a new version that incorporates most of the features of +IMCOMBINE. + +In addition to the many new features, including application of pixel +masks, the following functional differences from the old SCOMBINE +are noted. + +.ls 1 +The output is always a single spectrum per image. +.le +.ls 2 +The "first" option does not allow rebinning to a non-linear dispersion. +Instead, it rebins to the nearest linear dispersion matching the first +spectrum. +.le +.ih +SEE ALSO +imcombine, scombine, scopy, sarith, lscombine +.endhelp |