.nr PS 9 .nr VS 11 .RP .ND .TL NOAO Spectroscopy Packages Revision Summary: IRAF Version 2.10.3 .AU Francisco Valdes .AI IRAF Group - Central Computer Services .K2 P.O. Box 26732, Tucson, Arizona 85726 March 1993 .AB This paper summarizes the changes in Version 3.1 of the IRAF/NOAO spectroscopy packages, \fBonedspec\fR, \fBlongslit\fR, \fBapextract\fR, and those in \fBimred\fR. These changes are part of part of IRAF Version 2.10.3. A list of the revisions is: .nf \(bu A simplified \fIequispec\fR image header format \(bu \fIEquispec\fR format allows a larger number of apertures in an image \(bu Extensions to allow tasks to work on 3D images \(bu New task \fBspecshift\fR for applying a zeropoint dispersion shift \(bu Revised \fBsapertures\fR to edit spectrum coordinate parameters \(bu Revised \fBdispcor\fR to easily apply multiple dispersion corrections \(bu Revised \fBscombine\fR weighting and scaling options \(bu Revised \fBscopy\fR to better handle bands in 3D images \(bu Revised \fBcalibrate, dopcor\fR, and \fBspecshift\fR to work on 2D/3D images \(bu New color graphics capabilities in \fBsplot, specplot, sensfunc\fR, and \fBidentify\fR \(bu All spectral tasks use a common package dispersion axis parameter \(bu A more complete suite of tasks in the \fBlongslit\fR package \(bu A \fIdatamax\fR parameter in the \fBimred\fR reduction scripts for better cleaning \(bu Revised the \fBimred\fR reduction scripts to abort on non-CCD processed data \(bu Revised fiber reduction tasks to include a scattered light subtraction option \(bu Revised \fBdoslit\fR to take the reference arc aperture from the first object \(bu Bug fixes .fi .AE .NH Spectral Image Formats and Dispersion World Coordinate Systems .LP As with the original release of V2.10 IRAF, the primary changes in the NOAO spectroscopy software in V2.10.3 are in the area of spectral image formats and dispersion world coordinate systems (WCS). A great deal was learned from experience with the first release and the changes in this release attempt to address problems encountered by users. The main revisions are: .in +4 .nf \(bu A new WCS format called \fIequispec\fR. \(bu Extensions to allow use of 3D images with arbitrary dispersion axis. \(bu Elimination of limits on the number of apertures in an image under certain conditions. \(bu Improved tools for manipulating the spectral coordinate systems. \(bu Bug fixes and solutions to problems found in the previous release. .fi .in - 4 In the previous version all images with multiple spectra used a coordinate system called \fImultispec\fR. This type of WCS is complex and difficult to manipulate by image header editting tools. Only the case of a single linearized spectrum per image, sometimes called \fIonedspec\fR format, provided a simple header format. However, the \fBapextract\fR package used the \fImultispec\fR format even in the case of extracting a single spectrum so to get to the simple format required use of \fBscopy\fR. .LP In many cases all the spectra in a multispectrum image have the same linear dispersion function. The new \fIequispec\fR format uses a simple linear coordinate system for the entire image. This format is produced by the spectral software whenever possible. In addition to being simple and compatible with the standard FITS coordinate representation, the \fIequispec\fR format also avoids a limitation of the \fImultispec\fR WCS on the number of spectra in a single image. This has specific application to multifiber spectrographs with more than 250 fibers. .LP For multiple spectrum data in which the spectra have differing dispersion functions (such as echelle orders) or when the spectra are not linearized but use nonlinear dispersion functions, the \fImultispec\fR format is still used. It is the most general WCS representation. The difficulties with modifying this coordinate system, \fBhedit\fR cannot be used, are addressed by enhancing the \fBsapertures\fR task and by the new task \fBspecshift\fR which covers the common case of modifying the dispersion zeropoint. .LP A feature of the spectral tasks which operate on one dimensional spectra is that they can operate on two dimensional long slit spectra by specifying a dispersion axis and a summing factor. This feature has been extended to three dimensional spectra such as occur with Fabry-Perot and multichannel radio synthesis instruments. The dispersion axis may be along any axis as specified by the DISPAXIS image header keyword or by the \fIdispaxis\fR package parameter. The summing factor parameter \fInsum\fR is now a string which may have one or two values to allow separate summing factors along two spatial axes. Also, some additional tasks which previously did not support this feature are \fBcalibrate\fR, \fBdopcor\fR, and \fBspecshift\fR. .LP The gory details of the spectral image formats and world coordinate systems are laid out in the new help topic \fIspecwcs\fR (also available in a postscript version in the IRAF network documentation archive as iraf/docs/specwcs.ps.Z). .LP Some of the bug fixes and solutions to problems found in the previous release concerning the image formats and WCS are a problem with the WCS dimensionality (WCSDIM keyword) with 3D images and problems reading various imported nonstandard formats. It is hoped that all such formats, including previous IRAF spectral formats will all be allowed by the software in the latest release. .NH DISPCOR .LP The previous versions of \fBdispcor\fR, the dispersion correction task, was designed to prevent accidental repeated application; it is incorrect to apply the dispersion function from the original data to a linearized spectrum. However, it is valid to determine a new dispersion solution, say from a dispersion calibrated arc, and apply that as a second correction. \fBDispcor\fR would not use a new dispersion function, as specified by the REFSPEC keywords, if the dispersion calibration flag was set. In order to override this the user needed to manually change this flag to indicate the spectrum was uncorrected. The problem was that it was difficult to do this with \fImultispec\fR format spectra because the flag is part of the complex WCS attribute strings. .LP \fBDispcor\fR was revised to use a different logic to prevent accidental recalibration using an unintended dispersion function. The logic is as follows. Previously \fBdispcor\fR would simply change the dispersion calibration flag after correcting a spectrum while leaving the dispersion function reference spectrum keywords alone as a record. The revised \fBdispcor\fR keeps this useful record but moves this it to a new keyword DCLOGn (where n is a sequential integer). Because the REFSPEC keyword is removed after each application of \fBdispcor\fR it now takes an explicit act by the user to assign another dispersion function to a spectrum and so it is not possible to accidentally reapply the same dispersion function twice. Thus this version will apply additional dispersion functions by simply adding new REFSPEC keywords. If they are absent the task resamples the spectra based on the current dispersion relation as was the case before. .LP The new version can also tell whether the data was calibrated by the previous version. In this case the check on the dispersion calibration flag is still used so that during the transition users are still protected against accidentally applying the same reference dispersion function twice. The new task \fBsapertures\fR can now be used to change the dispersion calibration flag to override this checking more easily than was the case previously. .NH New Tasks .LP In this release there is only one completely new task and one task which was significantly redesigned. The new task is \fBspecshift\fR. It is relatively simple, it adds a zero point shift to the dispersion coordinates of spectra. This was the most common request for manipulating the spectral world coordinate system. In this regard there was a common confusion about the distinction between shifting the coordinate system and shifting the pixel data. Generally what people want is to apply a shift such that features in the spectrum move to the desired wavelength. One thought is to apply the tasks \fBimshift\fR or \fBshiftlines\fR. The surprise is that this does not to work. The pixels are actually shifted in the image array, but these tasks also apply the same shift to the coordinate system so that features in the spectrum remain at the same wavelength. What is really required is to leave the pixel data alone and shift only the coordinate system. That is what \fBspecshift\fR does. .LP While one hopefully does not need to directly manipulate the image header keywords describing the coordinate system or other aspects of the spectra, instead using such tasks as \fBspecshift\fR, there always seem to be cases where this is needed or desired. In the V2.10 release of the spectral software this was difficult because the general \fImultispec\fR format was the norm and it has information encoded in the complex WCS attribute strings. As mentioned previously several changes have been made reduce the complexity. Now \fIequispec\fR format will generally be the rule and this format has keywords which are more easily manipulated with \fBhedit\fR and \fBwcsedit\fR. However, the task \fBsapertures\fR was revised to provide an editing cabability specifically for spectral images, in either \fImultispec\fR or \fIequispec\fR format, with options to change various parameters globally or aperture-by-aperture. .NH New Features .LP There were a number of miscellaneous minor revisions and bug fixes. One of the major new capabilities available with V2.10.3 is support for color graphics if the graphics device supports it. \fBXgterm\fR supports color on X-window systems with color monitors. Several of the spectral tasks were modified to use different colors for marks and overplots. These tasks include \fBsplot\fR, \fBspecplot\fR, \fBidentify\fR, and \fBsensfunc\fR. In the case of \fBsensfunc\fR the user controls the various color assignments with a task parameter or \fBgtools\fR colon command while in other cases the next available color is used. .LP There were several changes to \fBscombine\fR equivalent to those in \fBimcombine\fR. The weighting, when selected, was changed from the square root of the exposure time or spectrum statistics to the value with no square root. This corresponds to the more commonly used variance weighting. Other options were added to specify the scaling and weighting factors. These allow specifying an image header keyword or a file containing the scale or weighting factors. A new parameter, "nkeep" has been added to allow controling the maximum number of pixels rejected by the clipping algorithms. Previously it was possible to reject all pixels even when some of the data was good though with a higher scatter than estimated; i.e. all pixels might be greater than 3 sigma from the mean without being cosmic rays or other bad values. Finally a parameter \fIsnoise\fR was added to include a sensitivity or scale noise component to a Poisson noise model. .LP In \fBsplot\fR the 'p' and 'u' keys which assign and modify the dispersion coordinates now include options for applying a zero point shift or a doppler shift in addition to defining an absolute wavelength for a feature or starting and ending wavelengths. There are also bug fixes to the equivalent width calculations, it did not handle flux calibrated data, and the scroll keys '(' and ')'. .LP There were several changes to make it easier to deal with with three dimensional \fImultispec\fR and \fIequispec\fR data; that is the additional data from the "extras" option in the \fBapextract\fR tasks. One was to fix problems associated with an incorrect WCSDIM keyword. This allows use of image sections or \fBimcopy\fR for extracting specific bands and apertures. Another was to add a "bands" parameter in \fBscopy/sarith\fR to allow selection of bands. Also the "onedspec" output format in \fBscopy\fR copies any selected bands to separate one dimensional images. .LP As mentioned earlier, many of the \fBonedspec\fR tasks have been extended to work on 2D and 3D spatial spectra. Some tasks which now have this capability in this version and not the previous one are \fBcalibrate\fR and \fBdopcor\fR. \fBIdentify\fR and \fBredentify\fR were extended to operate on 3D images. .NH LONGSLIT .LP With the applicablity of more \fBonedspec\fR tasks to long slit data the \fBlongslit\fR package was modified to add many new tasks. This required adding additional package parameters. One new task to point out is \fBcalibrate\fR. This task is now the prefered one to use for extinction and flux calibration of long slit spectra rather than the obsolete \fBextinction\fR and \fBfluxcalib\fR. The obsolete tasks are still present in this release. .NH APEXTRACT .LP The \fBapextract\fR package had a few, mostly transparent, changes. In the previous version the output image header format was always \fImultispec\fR even when there was a single spectrum, either because only one aperture was defined or because the output format parameter was "onedspec". In this release the default WCS format is the simpler \fIequispec\fR. .LP In the \fBonedspec\fR and \fBimred\fR spectral reduction packages there is a dispersion axis package parameter which is used to defined the dispersion axis for images without a DISPAXIS keyword. This applies to all tasks. However, the \fBapextract\fR tasks had the dispersion axis defined by their own task parameters resulting in some confusion. To make things consistent the dispersion axis parameter in \fBapextract\fR has been moved from the tasks to a package parameter. Now in the \fBimred\fR spectral reduction packages, there is just one dispaxis parameter in the package parameters which applies to all tasks in those packages, both those from \fBonedspec\fR and those from \fBapextract\fR. .LP Some hidden algorithm parameters were adjusted so that the cleaning and variance weighting options perform better in some problem cases without requiring a great deal of knowledge about things to tweak. .NH IMRED Spectroscopic Reduction Tasks .LP The various spectroscopic reductions tasks, those beginning with "do", have had some minor revisions and enhancements in addition to those which apply to the individual tasks which make up these scripts. In the latter class is the output WCS format is \fBequispec\fR except for the echelle tasks and when dispersion linearization is not done. Related to this is that the multifiber tasks can operate on data with more than 250 fibers which was a limitation of the \fBmultispec\fR format. .LP The dispersion axis parameter in the reduction tasks and in the other tasks in the \fBimred\fR spectroscopy packages, such as the \fBapextract\fR tasks, is now solely a package parameter. .LP All the scripts now check the input spectra for the presence of the CCDPROC keyword and abort if it is not found. This keyword indicates that the data have been processed for basic CCD calibrations, though it does not check the operations themselves. For data reduced using \fBccdproc\fR this keyword will be present. If these tasks are used on data not processed by \fBccdproc\fR then it is a simple matter to add this keyword with \fBhedit\fR. Obviously, the purpose of this change is to avoid inadvertently operating on raw data. .LP All the "do" tasks now have a parameter "datamax". This minimizes the effects of very strong cosmic rays during the extraction of object spectra; it does not apply to flat field or arc spectra. When there is a very large difference between data pixel values and cosmic ray pixel values, especially true for very weak spectra, the cosmic ray cleanning operation does not always work well. If it is possible to specify a threshold value between the maximum real data value and cosmic rays then the cosmic ray cleanning can be significantly improved by immediately rejecting those pixels above the threshold. Of course the user must be careful that real data does not exceed this value since such data will be excluded. .LP The fiber reduction tasks, \fBdoargus, dohydra, dofibers, dofoe\fR, and \fBdo3fiber\fR have a new processing option for subtracting scattered light. This is particularly useful if there is significant scattered light in producing uniform sky spectra for sky subtraction since the fiber throughput calibration does not generally correct for this. .LP The slit reduction task \fBdoslit\fR previously required that the spectrum for the reference arc cover the middle of the input data images. There were cases of instrument configurations where this was not true requiring additional manipulation to use this task. This requirement has been eliminated. Instead when the reference arc needs to be extracted it uses the aperture definition from one of the input object spectra since definition of the object apertures occurs prior to setting up the dispersion calibration.