diff options
Diffstat (limited to 'noao/onedspec/doc/sys/onedproto.ms')
| -rw-r--r-- | noao/onedspec/doc/sys/onedproto.ms | 1673 |
1 files changed, 1673 insertions, 0 deletions
diff --git a/noao/onedspec/doc/sys/onedproto.ms b/noao/onedspec/doc/sys/onedproto.ms new file mode 100644 index 00000000..b1b05201 --- /dev/null +++ b/noao/onedspec/doc/sys/onedproto.ms @@ -0,0 +1,1673 @@ +.RP +.ND +.TL +Some Notes on the ONEDSPEC Package +.AU +G. Jacoby +.AI +.K2 "" "" "*" +June 1985 +.AB +The first phase of the ONEDSPEC prototype package is complete. +Comments and some internal description is presented for each task +in the package. Also presented are some more global descriptions +of strategies used in the package and considerations for future +improvements. +.AE +.SH +1. Why is ONEDSPEC Different? +.PP +This section describes some of the ways in which the ONEDSPEC +package diverges from other IRAF package strategies. +A few of these should someday be modified to more closely +adhere to IRAF conventions, but in other cases, restrictions +or limitations in the IRAF system are revealed. +.sp 1 +.SH +Quantity +.PP +One of the major differences between a two dimensional image processing +package and a one dimensional package is that spectra +frequently congregate in groups of hundreds to thousands while two-dimensional +images live in groups of tens to hundreds. What this means is that spectral +processing must be somewhat more automated and streamlined - the software cannot +rely on user input to provide assistance and it cannot afford +excessive overhead; otherwise a large fraction of the processing time will be +spent where it is least useful. +.PP +To process large volumes of spectra in a reasonably automated fashion, +the software must be smart enough to know what to do with a variety +of similar but different spectra. The way adopted here is to key +off header parameters which define the type of spectrum and the +processing required. In fact, most of the ONEDSPEC package will not +work smoothly without some header parameter information. +.PP +It is also important that each task be self-reliant so that the +overhead of task stop and restart is avoided. For many operations, +the actual computation time is a fraction of a second, yet no +operation in the ONEDSPEC package is faster than one second per spectrum +due to task overhead. If task startup and stop were required for each +spectrum, then the overhead would be much worse. +.PP +So the philosophy is one in which each task uses as much information +as it can reasonably expect from the spectral image header. +Usually this is not more than three or four elements. +The strategy of using header information should not be limited to +ONEDSPEC. Many image processing problems can be automated +to a large degree if header information is used. The success of +the KPNO CCD Mountain reduction system emphasizes this point. +It would seem prudent that other IRAF applications make use of +such information when possible. +[See section 3 for a more detailed discussion of headers.] +.sp 1 +.SH +Spectral Image Names +.PP +One implication of the quantity problem is that it must be easy for the user to +specify the names of large numbers of spectra. The approach taken for ONEDSPEC +was to assign a root name to a group of spectra and then +append an index number of 4 or more digits starting with 0000. +So spectra, by default, have the form root.0000, root.0001, ... +To specify the spectra, the user types only the root name and the range +of indices such as "root" and "0-99,112-113,105-108". +The range decoder accesses the spectral indices in the order given +as opposed to access in ascending order, so that the spectrum root.0112 +will be processed before root.0105 in the example specification above. +Spectra having more general names may be specified using the +standard IRAF filename expansion methods if the +the range specification is given as null. +.PP +The specification of large numbers of images is an area where +most IRAF applications are weak. Resorting to odd combinations +of bracket and backslash characters in filename specifications +is obscure to new users and still fails to +meet the general need. The range specification adopted for ONEDSPEC +comes closer but introduces a fixed image name format. +.sp 1 +.SH +Apertures -- A way to group data +.PP +Many spectrographs generate multiple spectra simultaneously by +placing more than one slit or aperture in the focal plane. +Examples include the IIDS, IRS, and Cryogenic Camera in use +at Kitt Peak. The Echelle may be considered a multi-aperture +instrument for purposes of reductions by associating each order +with an "aperture" number. +.PP +The concept of aperture can be generalized to indicate a set of +spectral data having common group properties such as +wavelength coverage. Most tasks in ONEDSPEC will key off +an aperture number in the image header and treat those +common aperture spectra uniformly. +Defining data groups which are to be processed in this fashion +is a technique not generally exploited by reduction programs. +This is due in part to the problem of image header usage. +.PP +For programming convenience and to avoid an additional level +of indirectness, in ONEDSPEC the aperture number is used directly as an +index in many static arrays. The current implementation has +a declaration for 50 apertures and due to the IIDS/IRS +notation of apertures 0 and 1, the apertures are zero-indexed, contrary +to standard IRAF nomenclature, +from 0-49. It would certainly be better to map the aperture numbers +to the allowable index range, but the added complexity of another +level of indirectness seemed distracting. Actually the mapping +can still be done by the header reader, "load_ids_hdr", and +unmapped by the header writer, "store_keywords". +.sp 1 +.SH +Static versus dynamic arrays +.PP +Although dynamic storage would alleviate some of the memory +requirements in the package, the use of static arrays aids +readability and accounts for only about 10 percent of the total +task memory space. Many of the arrays are arrays of pointers. +For example, in the task BSWITCH, there is an array (called "imnames") +of pointers for the names of spectral images, several for each aperture. +The actual space for the names is dynamically allocated, +so first we allocate an array of pointers for each +aperture: +.sp 1 +.DS + call salloc (imnames[aperture], nr_names, TY_POINT) +.DE +.sp 1 +Then, for each of these pointers, space must be allocated for the +character arrays: +.sp 1 +.DS + do i = 1, nr_names + call salloc (Memp[imnames[aperture]+i-1], SZ_LINE, TY_CHAR) +.DE +.sp 1 +Later to access the character strings, a name is specified as: +.sp 1 +.DS + Memc[Memp[imnames[aperture]+nr_of_this_spectrum-1]] +.DE +.sp 1 +If the "imnames" array was also dynamically allocated, the +above access would be even less readable. +If memory requirements become a serious problem, then these ONEDSPEC +tasks should be modified. +.sp 1 +.SH +Output image names +.PP +To retain the consistent usage of root names and ranges, output +spectra also have the form root.nnnn. For user convenience, +the current output root name and next suffix are maintained as +package parameters onedspec.output and onedspec.next_rec. +The latter parameter is automatically updated each time a +new spectrum is written. This is done by the individual tasks +which directly access this package parameter. +.PP +There is an interesting side effect when using indirect parameters +(e.g. )onedspec.output) for input. In the local task parameter +file, the mode of the parameter must be declared hidden. So when the user +does an "lpar task", those parameters appear to be unnecessary +(that is, they are enclosed in parenthesis). When run, +prompts appear because the parameter is an automatic mode +parameter in the package parameter file. +If run as a background task, this is more annoying. +Unfortunately, any other choice of parameter modes produces +less desirable actions. +.sp 1 +.SH +ONEDUTIL +.PP +As the number of tasks in ONEDSPEC started growing, the +need for a subdivision of the package became clear. +The first cut was made at the utility level, and a number +of task names (not necessarily physical tasks) were +moved out into the ONEDUTIL submenu. In the future, +additional tasks will eventually require another subpackage. +.PP +Actually, many of the tasks in ONEDUTIL may be more at home +in some other package, but a conscious effort was made to +avoid contaminating other IRAF packages with tasks written for +the ONEDSPEC project. If all the following tasks are relocated, +then the need for ONEDUTIL is reduced. +.PP +Two of the entries in ONEDUTIL may be considered as more appropriate +to DATAIO - RIDSMTN and WIDSTAPE. In fact RIDSMTN can +replace the version currently in DATAIO. WIDSTAPE may replace the +DATAIO task WIDSOUT if the usage of header parameters does not +present a problem. +.PP +The task MKSPEC may be a candidate for the ARTDATA package. +It should be enhanced to include optional noise generation. +Also, it may be appropriate for SINTERP to replace INTERP +in the UTILITY package. +.PP +I suppose one could argue that SPLOT belongs in the PLOT package. +Certainly, the kludge script BPLOT should be replaced by a more +general batch plot utility in PLOT. +Also, the two task names, IDENTIFY and REIDENTIFY are present +in the ONEDSPEC menu for user convenience, but the task declarations +in ONEDSPEC.CL refer to tasks in the LONGSLIT package. +.PP +Because ONEDUTIL is a logical separation of the tasks, not +a complete physical task breakup, there is no subdirectory +for ONEDUTIL as there is in other packages. This is a bit messy +and it may be best to completely disentangle the tasks in the +subpackage into a true package having all the implications. +.LP +.SH +2. Task Information +.PP +There are currently about 30 tasks in the ONEDSPEC package. +These are summarized in the menu listing below and +a brief description of some less obvious aspects of each follows. +.sp 1 +.DS L + ONEDSPEC + + addsets - Add subsets of strings of spectra + batchred - Batch processing of IIDS/IRS spectra + bswitch - Beam-switch strings of spectra to make obj-sky pairs + calibrate - Apply sensitivity correction to spectra + coincor - Correct spectra for photon coincidence + dispcor - Dispersion correct spectra + extinct - Correct data for atmospheric extinction + flatfit - Sum and normalize flat field spectra + flatdiv - Divide spectra by flat field + identify - Identify features in spectrum for dispersion solution + iids - Set reduction parameters for IIDS + irs - Set reduction parameters for IRS + onedutil - Enter ONEDSPEC Utility package + process - A task generated by BATCHRED + reidentify- Automatically identify features in spectra + sensfunc - Create sensitivity function + slist - List spectral header elements + splot - Preliminary spectral plot/analysis + standard - Identify standard stars to be used in sensitivity calc + subsets - Substract pairs in strings of spectra + + ONEDUTIL + + bplot - Batch plots of spectra + coefs - Extract mtn reduced ceofficients from henear scans + combine - Combine spectra having different wavelength ranges + lcalib - List calibration file data + mkspec - Generate an artificial spectrum + names - Generate a list of image names from a string + rebin - Rebin spectra to new dispersion parameters + ridsmtn - Read IIDS/IRS mountain format tapes + sinterp - Interpolate a table of x,y pairs to create a spectrum + widstape - Write Cyber format IDSOUT tapes +.DE +.sp 1 +.SH +ADDSETS +.PP +Spectra for a given object may have been observed through more than +one instrument aperture. For the IIDS and IRS, this is the most common +mode of operation. Both apertures are used to alternately observe +the program objects. +.PP +Each instrument aperture may be considered an +independent instrument having unique calibration properties, and +the observations may then be processed completely independently +until fully calibrated. At that point the data may be combined to +improve signal-to-noise and reduce systematic errors associated +with the alternating observing technique. Because the data are +obtained in pairs for IIDS and IRS (but may be obtained in groups +of larger sizes from other instruments), ADDSETS provides a way +to combine the pairs of observations. +.PP +Each pair in the input string is added to produce a single output +spectrum. Although the word "pair" is used here, the parameter +"subset" defines the number of elements in a "pair" (default=2). +The input string is broken down into groups where each group +consists of the pair of spectra defined in order of the input +list of image names. +.PP +"Add" in ADDSETS means: +.RS +.IP 1. +Average the pairs if the data are calibrated to flux (CA_FLAG=0) +optionally weighted by the integration time. +.IP 2. +Add the pairs if uncalibrated (CA_FLAG=-1). +.RE +.sp 1 +.SH +BATCHRED +.PP +This is a script task which allows spectra from dual aperture instruments +to be processed completely in a batch mode after the initial wavelength +calibration and correction has been performed. The processes which +may be applied and the tasks referenced are: +.RS +.IP 1. +Declaring observations as standard stars for flux calibration (STANDARD). +.IP 2. +Solving for the sensitivity function based on the standard stars (SENSFUNC). +.IP 3. +Generating object minus sky differences and summing individual +observations if several were made (BSWITCH). +.IP 4. +Correcting for atmospheric extinction (BSWITCH). +.IP 5. +Applying the system sensitivity function to generate flux calibrated +data (CALIBRATE). +.IP 6. +Adding pairs of spectra obtained through the dual apertures (ADDSETS). +.RE +Any or all of these operations may be selected through the task +parameters. +.PP +BATCHRED generates a secondary script task called PROCESS.CL +which is a text file containing constructed commands to the +ONEDSPEC package. This file may be edited by the user if an +entry to BATCHRED is incorrect. It may also be saved, or appended +by further executions of BATCHRED. +.PP +BATCHRED also generates a log file of the output generated by the +ONEDSPEC tasks it calls. +.sp 1 +.SH +BSWITCH +.PP +This task combines multiple observations of a single object +or multiple objects taken through a multiaperture instrument. +Object minus sky differences are generated as pairs of +spectra are accumulated, then optionally corrected for +atmospheric extinction, and the differences added together +with optional weighting using counting statistics. +Each instrument aperture is considered an independent +device. +.PP +Despite the apparently simple goal of this task, it is probably +the most complicated in the ONEDSPEC package due to the +bookkeeping load associated with automated handling of large data sets +having a number of properties associated with each spectrum (e.g +object or sky, aperture number, exposure times). +.PP +There are several modes in which BSWITCH can operate. The mode +appropriate to the IIDS and IRS assumes that the spectra +are input in an order such that after 2N (N=number of +instrument apertures) spectra have been +accumulated, an equal number of object and sky spectra have been +encountered in each aperture. +When in this mode, a check is made after 2N spectra +have been processed, and the optional extinction correction is +applied to the differences of the object minus sky, and then +(optionally weighted and) added into an accumulator for the aperture. +.PP +If the IIDS mode is switched off, then no guarantee can be +made that sky and object spectra pair off. If extinction +correction is required, it is performed on each spectrum +as it arrives, including sky spectra if any. The spectra are +then added into separate accumulators for object and sky for +each aperture after optional weighting is applied. +.PP +If after all spectra have been processed, there are no sky +spectra, the object spectrum is written out. If there is no +object spectrum, the sky spectrum is written out after +multiplying by -1. (This allows adding an object later on with +addsets, but the -1 multiply is probably a mistake.) +If at least one of each, object and sky spectra were encountered, +then the difference is computed and written out. Since +all accumulations are performed in count rates and later converted +back to counts, the object and sky spectra may have different +exposure times (non IIDS mode only). +.PP +A statistics file is maintained to provide an indication of the +quality of the individual spectra going into the sum. The +statistics information is maintained internally and only +written out after the sums have been generated. +The basic data in the file is the count rate of the spectrum +having the largest count rate, and the ratios of the count rates from +all other spectra to that one. +.PP +If weighting is selected, the weights are taken as proportional to +the count rate (prior to extinction correction) over a wavelength +delimited region of the spectrum. (Perhaps the weight +should be proportional to counts, not count rate.) +The default wavelength region is the entire spectrum. +If the total count rate is negative, the weight is assigned +a value of 0.0 and will be disregarded in the sum. (The counts +may be negative if the object minus sky difference approaches zero +on a bright and cloudy night.) +.PP +If extinction is selected, an extinction table is read from the +package calibration file. An optional additive term may be applied +as computed by the system sensitivity task SENSFUNC which is placed +in the parameter sensfunc.add_const. A revision to the standard +extinction table (delta extinction as a function of wavelength) +may be read from a text file whose name is specified by the parameter +sensfunc.rev_ext_file. The file format is that of a text file +having pairs of (wavelength, delta extinction) on each line. +[The option to solve for this function in SENSFUNC has not yet been +implemented, but BSWITCH can read the file that would be generated. +Thus, one can experiment with revisions, although this has never been +tested.] BSWITCH will interpolate the values given in the file +so that a course estimate of the revision may be entered, say if the +deltas at U, B, V, R, and I are known. +.PP +BEWARE that the extinction correction is performed assuming the +header parameters used for airmass refer to a "mean" airmass value +for the exposure. In general the header value is wrong! It usually +refers to the beginning, middle, or end of the exposure. I have +never seen a header airmass value which was an equivalent airmass +for the duration of the exposure. This is partly because there is +no way to compute a single effective airmass; it is a function +of wavelength, telescope position as a function of time, and +the extinction function. Fortunately, for most observations +this is not very significant. But anyone taking a one hour exposure near +3500 Angstroms at airmass values greater than 2, should not complain +when the fluxes look a bit odd. +.sp 1 +.SH +CALIBRATE +.PP +Having a system sensitivity function allows the data to be +placed on an absolute flux scale. CALIBRATE performs this +correction using the output sensitivity function from SENSFUNC. Operations are +keyed to the instrument aperture, and a system sensitivity +function is required for each observing aperture, although +this requirement may be overriden. +.PP +A valid exposure time is required (a value of 1.0 should +probably be assumed if not present) to compute the observed +count rate. Input counts are transformed to units of +ergs/cm2/sec/Angstrom (or optionally ergs/cm2/sec/Hz). +CALIBRATE will calibrate two dimensional images as well, applying the +sensitivity function to all image lines. +.PP +The operation is performed on a pixel-by-pixel basis so that +the defined sensitivity function should overlap precisely +with data in terms of wavelength. +.sp 1 +.SH +COINCOR +.PP +This task applies a statistical correction to each pixel +to account for undetected photoevents as a result of +coincidental arrival of photons. This is a detector +specific correcton, although the photoelectric detector +model provides a reasonable correction for many detectors +when a judicious value for the deadtime parameter is chosen. +This model assumes that the correction follows the +typical procedures applied to photoelectric photometer data: +.sp 1 +.DS L + Ic = Io * exp [Io * dt / T] +.DE +.sp 1 +where Ic is the corrected count rate in a pixel, Io is the +observed count rate in that pixel, dt is the detector deadtime, +and T is the observation integration time. +.PP +In addition to the photoelectric model, a more accurate model +is available for the IIDS and is included in COINCOR. This +model is taken from Goad (1979, SPIE Vol 172, 86.) and the correction +is applied as: +.sp 1 +.DS L + Ic = ln [1 - Io * t] / t +.DE +.sp 1 +where t is sweep time between pixel samples (t=1.424 msec). +The IIDS differs from a photomultiplier detector, in that +there is a fixed rate at which each pixel is sampled due to +time required for the dissector to sweep across the image tube +phospher whether a photoevent has occurred in a pixel or not. +The photomultiplier plus discriminator system +assumes that once a photoevent has been recorded, the detector is +dead until a fixed interval has elapsed. +.sp 1 +.SH +DISPCOR +.PP +If a relation is known linking pixel coordinate to user coordinate +(i.e. wavelength as a function of pixel number), then any non-linearities +can be removed by remapping the pixels to a linear wavelength coordinate. +This procedure, dispersion correction, is complicated by the +lack of a wavelength-pixel solution which is derived from data simultaneously +obtained with the object data. Any drifts in the detector then require +an interpolation among solutions for the solution appropriate to +the object observations. Depending on the detector, this interpolation +may be a function of the time of observation, temperature, or some telescope +parameter such as airmass. +When multiple solutions are available, DISPCOR will linearly interpolate +the solution in any available header parameter known to ONEDSPEC (see +section 3). +.PP +Each solution is read from the database file created by the IDENTIFY +task (in TWODSPEC$LONGSLIT), and the image name leading to that solution +is also read from the database file. The image is opened to extract +the header parameter to be used in the above interpolation. +A null name for the interpolation parameter indicates that none +is to be used. In this case, one of the options on the "guide" +parameter should be set to indicate what solution should be used. +The guide may be "precede", "follow", or "nearest" to select +the most appropriate choice for each spectrum. +.PP +If an explicit wavelength solution is to be used, the parameter +"reference" may be used to specify the image name of a comparison +spectrum to be used as the reference for the wavelength solution. +In this case all spectra will be corrected using a single solution - +no flexure correction will be applied. +.PP +If the parameter to be used for interpolation is a "time-like" +variable, such as RA, UT, ST, then the variable is discontinuous +at 24|0 hours. If UT is the chosen parameter (as has been the +case for IIDS and IRS spectra), the discontinuity occurs at +5 PM local Kitt Peak time. A comparison spectrum taken at 4:59PM +(=23:59h UT, =just before dinner), will be treated as an "end of +the night" observation rather than a beginning of the night +observation. To circumvent this error, the parameter, "time_wrap", +can be specified to a time at which a true zero should be assigned. +For UT at Kitt Peak, a choice like 17h UT (=10AM local, =asleep), +is an unlikely hour for nighttime observations to be made. Then for +a given night's observations, 17h UT becomes the new zero point in time. +.PP +Each solution in the database may be any of the forms legal +to IDENTIFY: legendre, chebyshev, spline3, or spline1 - the form +is encoded in the database and will automatically be recalled. +The interpolation in the solution is performed by locating the +pixel location for each required wavelength for the two +solutions bounding each observation and linearly interpolating +for the appropriate pixel location. One cannot simply interpolate +across the coefficients of the solutions to derive a new +single solution because the solutions may have different forms +or orders, so that the coefficients may have quite different +meanings. +.PP +Dispersion correction requires that there be equal intervals +of wavelength between pixels. The wavelength solution +is of a form describing the wavelength for a given pixel location, +not a pixel location for a given wavelength. So the solution +must be inverted. +.PP +The inversion to pixel location for wavelength is done in the +following way: The pixel coordinate in the solution is incremented +until the desired wavelength is bounded. The pixel value for the +desired wavelength is obtained by linearly interpolating across these +two bounding pixel locations. A linear approximation appears to be +very good for typical solutions, providing proper pixel locations to +better than 0.01 pixels. An improvement may be obtained by +increasing the order of the interpolation, but the improvement +is generally not warranted because the wavelength solutions +are rarely known to this accuracy. [Note that the use of real +and not double precision limits the precision of this technique! +For spectra longer than 50,000 pixels, the errors due to +the precision of reals can be serious.] +.PP +Note that a transformation to +a wavelength coordinate which is linear in the logarithm of +wavelength only requires that the inversion occur at wavelengths +selected by equal increments in the logarithm of wavelength. +.PP +During the actual remapping, 5 possible techniques are available. +Actually there are only two techniques: re-interpolation in 4 flavors, +and rebinning by partial pixel summation. The re-interpolation +may be performed with polynomials of order 1 (=linear), 3, or 5, +or by a cubic spline. The 3rd and 5th order polynomials may introduce +some ringing in the wings of strong, sharp, features, but the 5th order +is good at preserving the high frequency component of the data. +The linear and spline interpolators introduce significant smoothing. +The rebinning algorithm offers conservation of flux but also smooths +the data. In fact, rebinning to a course grid offers a good smoothing +algorithm. +.PP +At some future date, it would be a good idea to include a "synch" +function interpolator in the image interpolator package. This would +be a little slower to process, but results in very good frequency +response. +.PP +Other options in DISPCOR include "ids_mode" which forces spectra +from all apertures to a single output mapping (starting wavelength +and pixel-to-pixel increment), and "cols_out" forces the output spectra +to a specified length, zero-filling if necessary. +.PP +DISPCOR will correct two-dimensional data by applying the +remapping to all lines in the image. If the input two-dimensional +spectrum has only one line, the output spectrum will be written as +a one-dimensional spectrum. +.sp 1 +.SH +EXTINCT +.PP +Extinction is currently only available as a script file which drives +BSWITCH. This is possible by suppressing all options: weighting, +ids_mode, statistics file, and setting the subset pair size to the +number of instrument apertures. +.sp 1 +.SH +FLATDIV +.PP +This task divides the specified spectra by their flat field spectra. +This is not much more than an "en mass" spectrum divider, with the +exceptions that the header elements are used to key on the +aperture number so that the appropriate flat field spectrum is used, +and that the header processing flags are checked to prevent +double divisions and subsequently set after the division. Also, +division by zero is guarded by setting any zeroes in the flat field +spectrum to 1.0 prior to the division. +.sp 1 +.SH +FLATFIT +.PP +Pixel-to-pixel variations in the detector response can be removed +by dividing all observations by a flat field spectrum. +Flat field spectra are generally obtained by observing a source +having a continuous energy distribution, such as a tungsten filament +lamp. This is sometimes called a "quartz" lamp when the enclosing +glass bulb is made with quartz rather than silicon. The quartz +enclosure transmits ultraviolet light much better than glass. +.PP +If the color temperature of the source is very low (or very high, though +this is extremely unlikely), then a color term would be introduced +into the data when the flat is divided into the data. +Large scale variations in the system sensitivity also introduce a +color term into the flat - the same variations that are introduced into +any spectrum taken with the system. [Large scale variations are +evaluated by STANDARD and SENSFUNC, and removed by CALIBRATE.] +This is not of any particular importance except that counting +statistics are destroyed by the division. +.PP +To preserve the statistics, many find it desirable to divide by a flat +field spectrum which has been filtered to remove any large scale variations +but in which the pixel-to-pixel variations have been retained. +A filtered flat can be obtained by fitting a low order polynomial +through the spectrum and dividing the spectrum by the polynomial. +The result is a spectrum normalized to 1.0 and having high frequency +variations only. If one does not care to preserve the statistics, +then this procedure is not required. In fact, for certain instruments +(the IRS), the fitting and normalizing procedure is not recommended +because some intermediate order curvature can be introduced. +.PP +The purpose of FLATFIT is to find the combination of parameters +which produces a well flattened flat with a minimum of wiggles. +The usual curve fitting package is used to fit a function (chebyshev, +legendre, spline3, spline1) to the flats. Pixel rejection is +user selectable by a choice of cutoff sigmas, both above and below +the mean, and an optional growing region [A growing region is the number +of pixels on either side of one rejected which will also be rejected - +Growing regions are not recommended for most spectral applications]. +Any number of iterations may be used to further reject discrepant +pixels. The fitting may be performed interactively and controlled by cursor +keystrokes to select the fitting order, and other fit parameters. +.PP +Prior to the fit, the specified spectra are read, optionally corrected +for coincidence losses, and added to accumulators appropriate to +their instrument apertures. Each aperture is treated independently, +except that, the interactive fitting mode may be selected to operate +on the first aperture only, and then apply the same fitting parameters +to all other aperture accumulations. Or the interactive procedure +may be selected to operate on all apertures or none. +.PP +After the fit has been done, the fit is divided into the accumulation +and written as a new spectrum having a specified root name and a trailing +index indicating the aperture. +.sp 1 +.SH +IDENTIFY +.PP +This task (written by Frank Valdes) is used to identify features +in the comparison arcs to be used in the solution for a wavelength calibration. +The solution is performed interactively for at least one spectrum +and then optionally in a batch mode using REIDENTIFY. +IDENTIFY writes to a database file which will contain the solutions +generated from each input comparison spectrum. The database is +later used by DISPCOR to correct spectra according to the solution. +.sp 1 +.SH +IIDS +.PP +This script file initializes several hidden parameters in a +variety of tasks to values appropriate to the IIDS instrument. +There is also a script for the IRS. There should probably be +a script for resetting the parameters to a default instrument. +These parameters are: +.RS +.IP 1. +onedspec.calib_file - the package parameter indicating which file +should be used for standard star calibration data and the atmospheric +extinction table (=onedspec$iids.cl.) +.IP 2. +addsets.subset - the number of instrument apertures (=2). +.IP 3. +bswitch.ids_mode - assume and check for data taken in beam-switched +quadruple mode (=yes). +.IP 4. +coincor.ccmode - coincidence correction model (=iids). +.IP 5. +coincor.deadtime - detector deadtime (=1.424e-3 seconds) +.IP 6. +dispcor.flex_par - the name of the parameter to be used as the +guide to removing flexure during the observations (=ut). +.IP 7. +dispcor.time_wrap - the zero point to be adopted for the +flexure parameter if it is a time-like variable having a discontinuity +at 0/24 hours (=17). +.IP 8. +dispcor.idsmode - should data from all instrument apertures be dispersion +corrected to a uniform wavelength scale? (=yes). +.IP 9. +dispcor.cols_out - the number of columns (row length of the spectrum) +to which the output corrected spectrum should be forced during +mapping (=1024). +.IP 10. +extinct.nr_aps - the number of instrument apertures (=2). +.IP 11. +flatfit.order - the order of the fit to be used when fitting to +the flat field spectra (=6). +.IP 12. +flatfit.coincor - apply coincidence correction to the flat field +spectra during accumulations (=yes). +.IP 13. +flatdiv.coincor - apply coincidence correction to all spectra during +the flat field division process (=yes). +.IP 14. +identify.function - the fitting function to be used during the wavelength +solution process (=chebyshev). +.IP 15. +identify.order - the order of the fit to be used during the wavelength +solution process (=6). +.RE +.sp 1 +.SH +IRS +.PP +This script file initializes several hidden parameters in a +variety of tasks to values appropriate to the IRS instrument. +These parameters are: +.RS +.IP 1. +onedspec.calib_file - the package parameter indicating which file +should be used for standard star calibration data and the atmospheric +extinction table (=onedspec$irs.cl.) +.IP 2. +addsets.subset - the number of instrument apertures (=2). +.IP 3. +bswitch.ids_mode - assume and check for data taken in beam-switched +quadruple mode (=yes). +.IP 4. +coincor.ccmode - coincidence correction model (=iids). +.IP 5. +coincor.deadtime - detector deadtime (=1.424e-3 seconds) +.IP 6. +dispcor.flex_par - the name of the parameter to be used as the +guide to removing flexure during the observations (=ut). +.IP 7. +dispcor.time_wrap - the zero point to be adopted for the +flexure parameter if it is a time-like variable having a discontinuity +at 0/24 hours (=17). +.IP 8. +dispcor.idsmode - should data from all instrument apertures be dispersion +corrected to a uniform wavelength scale? (=yes). +.IP 9. +dispcor.cols_out - the number of columns (row length of the spectrum) +to which the output corrected spectrum should be forced during +mapping (=1024). +.IP 10. +extinct.nr_aps - the number of instrument apertures (=2). +.IP 11. +flatfit.order - the order of the fit to be used when fitting to +the flat field spectra. IRS users have frequently found that +any curvature in the fit introduces wiggles in the resulting +calibrations and a straight divide by the flat normalized to the +mean works best (=1). +.IP 12. +flatfit.coincor - apply coincidence correction to the flat field +spectra during accumulations (=no). +.IP 13. +flatdiv.coincor - apply coincidence correction to all spectra during +the flat field division process (=no). +.IP 14. +identify.function - the fitting function to be used during the wavelength +solution process (=chebyshev). +.IP 15. +identify.order - the order of the fit to be used during the wavelength +solution process. The IRS has strong deviations from linearity +in the dispersion and a fairly high order is required to correct +the dispersion solution (=8). +.RE +.sp 1 +.SH +ONEDUTIL +.PP +This is a group of utility operators for the ONEDSPEC package. They +are documented separately after the ONEDSPEC operators. ONEDUTIL +is a "pseudo-package" - it acts like a package under ONEDSPEC, but +many of its logical tasks are physically a part of ONEDSPEC. This +is done to minimize disk storage requirements, and to logically +separate some of the functions from the main ONEDSPEC menu which +was getting too large to visually handle. +.sp 1 +.SH +PROCESS +.PP +This task generally does not exist until the user executes the +script task BATCHRED which creates PROCESS.CL, a secondary script +file containing a CL command stream to batch process spectra. +The task is defined so that the CL is aware of its potential +existence. It is not declared as a hidden task so that the +user is also aware of its existence and may execute PROCESS +in the foreground or background. +.sp 1 +.SH +REIDENTIFY +.PP +This task (written b Frank Valdes) is intended to be used after +IDENTIFY has been executed. Once a wavelength solution has been +found for one comparison spectrum, it may be used as a starting point +for subsequent spectra having similar wavelength characteristics. +REIDENTIFY provides a batch-like means of performing wavelength solutions +for many spectra. The output solution is directed to a database text file +used by DISPCOR. +.sp 1 +.SH +SENSFUNC +.PP +This task solves for the system sensitivity function across +the wavelength region of the spectra by comparison of observations +of standard stars to their (assumed) known energy distribution. +Each instrument aperture is treated completely independently +with one exception discussed later. SENSFUNC is probably the +largest task in the ONEDSPEC package due to heavy use of +interactive graphics which represents more than half of the +actual coding. +.PP +Input to SENFUNC is the "std" text file produced by STANDARD +containing the ratio of the count rate adjusted for atmospheric extinction +to the flux of the star in ergs/cm2/s/Angstrom. Both the count rates and +fluxes are the average values in the pre-defined bandpasses tabulated +in the calibration file (indicated by the parameter onedspec.calib_file). +.PP +Each entry is the "std" file may have an independent set of wavelength sampling +points. After all entries have been loaded, a table containing all sampled +wavelengths is built (a "composite" wavelength table) and all sensitivity +values are reinterpolated onto this sampling grid. This allows the inclusion +of standards in which the observational samples are not uniform. +.PP +When multiple measurements are available, one of two corrections may +be applied to the data to account for either clouds or an additive extinction +term. The effect of clouds is assumed to be grey. Each contributing +observation is compared to the one producing the highest count rate ratio +at each wavelength sample. The deviation averaged over all wavelengths +for a given observation is derived and added back to +each wavelength sample for that observation. This produces a shift +(in magnitudes) which, on the average across the spectrum, accounts +for an extinction due to clouds. This process is called "fudge" +primarily for historical reasons (from the IPPS, R.I.P.) and also +because there is questionable justification to apply this correction. +One reason is so that one can better assess the errors +in the data after a zero-point correction has been made. +Another is that the sensitivity function is that closest to a cloud-free +sky so that claibrations may approach a true flux system if one +standard was observed during relatively clear conditions. +Alsom there are claims that the "color solution" is improved by "fudging", but +I admit that I don't fully understand this argument. +.PP +[Perhaps it goes as follows: +Although a grey scale correction is applied to each observation, +a color term is introduced in the overall solution. Consider the +case where 5 magnitudes of cloud extinction obscure one standard +relative to another. This star generates a sensitivity curve which +is a factor of 100 smaller. When averaged with the other curve, +any variations are lost, and the net curve will be +very similar to the first curve divided by 2. Now apply a "fudge" +of 5 magnitudes to the second curve. On the average, both curves have +similar amplitudes, so variations in the second now influence the +average. The net curve then has color dependent variations not +in the "un-fudged" net curve. If we assume that the variations in +the individual observations are not systematic, then "fudge" will +improve the net color solution. Amazing, isn't it? +End of hypothesis.] +.PP +The second form of correction is much more justifiable. In ONEDSPEC +it is referred to as a "grey shift" and accounts for possible +changes in the standard atmospheric extinction model due to +a constant offset. SENSFUNC will optionally solve for this constant +provided the observations sample a range of airmass values. +The constant is computed in terms of magnitudes per airmass, so +if the airmass range is small, then a large error is likely. +To solve for this value, a list of pairs of delta magnitude (from the +observation having the greatest sensitivity) as a function of +delta airmass (relative to the same observation) is generated +for all observations. The list is fit using a least squares solution +of the form: +.sp 1 +.DS L + delta_mag = delta_airmass * grey_shift +.DE +.sp 1 +Note that this is a restricted least-squares in the sense that there +is no zero-point term. The standard curve fit package in IRAF +does not support this option and the code to perform this is included +in SENSFUNC. +.PP +Because the atmosphere is likely to be the same one for observations +with each instrument aperture, it is not appropriate to limit +the least-squares solution to the individual apertures, but rather +to combine all the data to improve the solution. This would mean +that the user could not view the effects of applying the grey term +until all apertures had been analyzed. So, although each aperture is +solved independently to derive a preliminary value, a final value is +computed at the end when all data have been reviewed. This is the +one exception to the independent aperture equals independent +instrument philosophy. +.PP +When "fudging" is applied, the sensitivity function that is generated +is altered to account for the shifts to the observations. But when +the "grey shift" is computed, it cannot be directly applied to +the sensitivity function because it must be modified by the +observing airmass for each individual object. So the grey shift +constant is written into the image headers of the generated +sensitivity functions (which are IRAF images), and also placed +into the task parameter "add_const" to be used later by BSWITCH. +.PP +SENSFUNC can be run in an interactive mode to allow editing +of the sensitivity data. There are two phases of interaction: +(1) a review of the individual observations in which every data +element can be considered and edited, and (2) a review of the +composite sensitivity table and the calculated fit to the table. +In the interactive mode, both phases are executed for every instrument +aperture. +.PP +At both phases of the interactive modes there will be a plot of the +error in the input values for each wavelength. This is an RMS +error. [The IPPS plotted standard error which is always a smaller number +and represents the error in the mean; the RMS represents the error +in the sample. I'm not sure which is better to use, but RMS is easier +to understand. RMS is the same as the standard deviation.] +During phase one, the rms is computed as the standard deviation of +the sensitivity in magnitudes; but during phase two, it is computed +as the standard deviation in raw numbers +and then converted to a magnitude equivalent. The latter is more +correct but both converge for small errors. +.PP +There is one option in SENSFUNC which has never been tried and it won't +work - the option to enter a predefined table of sensitivities as +a function of wavelength as a simple text file. This option may +be useful a some time and should probably be fixed. I think the +only problem with it is a lack of consistency in the units. +.PP +An additional option has been requested but it is not clear that it +is a high priority item - the ability to compute the extinction +function. There may be instances when the mean extinction table +is not appropriate, or is not known. If sufficient data are +available (many observations of high precision over a range of airmasses +during a photometric night), then the extinction function is +calculable. Presently SENSFUNC can only compute a constant offset to +the extinction function, but the same algorithm used may be applied +at each wavelength for which observations are made to compute a +correction to an adopted extinction function (which may be zero), +and the correction can then be written out to the revised extinction +table file. This file will then be read by BSWITCH during the +extinction correction process. +So at each wavelength, pairs of delta magnitude as a function of +delta airmass are tabulated and fit as above: +.sp 1 +.DS L + delta_mag[lambda] = delta_airmass * delta_extinction[lambda] +.DE +.sp 1 +Because the data have been heavily subdivided into wavelength bins, +there are only a few measurements available for solving this +least-squares problem and the uncertainties are large unless many +observations have been taken. Experience has shown that at least +7-8 measurements are needed to come close, and 15 measurements are +about the minimum to get a good solution. Unless the data are of +high quality, the uncertainty in the solution is comparable to +the error in assuming a constant offset to the mean extinction function. +Nevertheless, the option should be installed at some time since +some observers do obtain the necessary data. +.sp 1 +.SH +SLIST +.PP +The spectrum specific header elements are listed in either a short +or long form. See the discussion on headers (section 3) for an explanation +of the terms. Values for airmass are printed if present in the header; +otherwise, the value is given as the string "?????" to indicate no +value present (even if one can be calculated from the telescope +pointing information elsewhere in the header). +.PP +The short form header lists only the image name, whether it is +an object or sky observation, the spectrum length, and the title. +.sp 1 +.SH +SPLOT +.PP +This is probably the second largest task in the ONEDSPEC package. It continues +to grow as users provide suggestions for enhancement, although +the growth rate appears to be slowing. SPLOT is an interactive +plot program with spectroscopy in mind, although it can be used +to plot two dimensional images as well. +.PP +SPLOT should still be considered a prototype - many of the algortihms +used in the analysis functions are crude, provided as interim +software to get results from the data until a more elaborate package +is written. It would probably be best to create an analysis specific +package - SPLOT is reasonably general, and to enhance it further +would complicate the keystroke sequences. +.PP +Ideally it should be possible to do anything to a spectrum with +a single keystroke. In reality, several keystrokes are required. +And after 15 or 20 functions have been installed, the keystroke +nomenclature becomes obscure - all the best keys are used up, and +you have to resort to things like '(' which is rather less +mneumonic than a letter. So some of the functionality in SPLOT +has been assigned to the "function" submenu invoked by 'f' and +exited by 'q' keystrokes. These include the arithmetic operators: +add, multiply by a constant, add, subtract, multiply, divide by +a spectrum, and logarithms, square root, inverse, and absolute +value of a spectrum. +.PP +Some of the analysis functions include: equivalent width, line centers, +flux integration under a line, smoothing, spectrum flattening, +and deblending of lines. +.PP +The deblender has serious limitations but handles about half the +cases that IIDS/IRS users are interested in. It fits only +Gaussian models to the blends, and only a single width parameter. +The fit is a non-linear least-squares problem, so starting values +present some difficulties. All starting values are initialized to 1.0 - +this includes the width, relative strengths of the lines, and deviation +from intial marked centers. The iterative solution usually converges +for high signal-to-noise data, but may go astray, resulting in +a numerical abort for noisy data. If this occurs, it is often +possible to find a solution by fitting to a single strong line +to force a better approximation to the starting values, and then refit +the blend of interest. +.PP +The non-linear least-squares routine is one obtained from an industrial +source. The code is very poorly written and in FORTRAN. No one should +attempt to understand it. The basic algorithm is an unconstrained simplex +minization search combined with a parabolic linear least-squares approximation +when in the region of a local minimum. +A test was made comparing this to the algorithm in Bevington, and the +Bevington algorithm appeared less likely to converge on noisy data. +Only one test case was used, so this is hardly a fair benchmark. +.PP +The problem with non-convergence is that a floating point error is +almost surely to arise. This is usually a floating point over/under +flow while computing an exponential (as required for a Gaussian). +In UNIX, there is apparently no easy way to discriminate from +FORTRAN which floating point exception has occurred, and so there +is no easy way to execute a fix up and continue. This is most +unfortunate because the nature of these non-linear techniques is +that given a chance, they will often recover from searching +down the wrong alley. A VMS version of the same routines seems to +survive the worst data because the error recovery is handled +somewhat better. [The VMS version also seems to run much faster, +presumably because the floating point library support is better +optimized.] +.PP +The net result of all this, is that a weird undocumented subroutine +is used which provides no error estimate. The Bevington routines +do provide an error estimate which is why I wanted to use them. +[In fact, there is no way to exactly compute the errors in the +fit of a non-linear least-squares fit. One can however apply +an approximation theory which assumes the hypersurface can be +treated locally as a linear function.] +.PP +There are several methods for computing equivalent widths in SPLOT. +The first method for measuring equivalent width is simply to integrate the +flux above/under a user defined continuum level. Partial pixels +are considered at the marked endpoints. A correction for the pixel size, +in Angstroms, is applied because the units of equivalent width are Angstroms. +You will probably get a different answer when doing equivalent +width measurements in channel mode ('$' keystroke) as compared to +wavelength mode ('p'). +.PP +Centering is performed as a weighted first moment of the region: +.sp 1 +.DS L + int1 = integral [ (I-Ic) * sqrt (I-Ic) * w] + int2 = integral [ (I-Ic) * sqrt (I-Ic) ] + xc = int1 / int2 +.DE +.sp 1 +where I is the intensity at the pixel at wavelength w, and Ic is +the estimated continuum intensity. The square root term provides +the weighting assuming photon statistics [sigma = sqrt(I)], and xc +is the derived center of the region. +.PP +An alternative method for equivalent widths was supplied by Caty +Pilachowski and is described in some detail in the help file for +SPLOT. This method is fast and insensitive to cursor settings, so +the user can really zip through a spectrum quickly. +.PP +Smoothing is performed using a simple boxcar smooth of user specified +size (in pixels). To handle edge effects, the boxcar size is +dynamically reduced as the edge is approached, thereby reducing +the smoothing size in those regions. +.PP +The flattening operator is a preliminary one, written before the +curve fitting package was available in IRAF. This operator +should probably be re-written to include the interactive +style used in FLATFIT. Currently the flattening is done +using classic polynomial least-squares with pixel rejection +chosen to preferentially reject absorption lines and strong +emission lines. The rejection process is repeated through +a number of iterations specifiable as a hidden parameter to SPLOT. +This is poorly done - the order of the fit and the number of +iterations should be controllable while in SPLOT. However, +experimentation has shown that for a given series of spectra, +the combination of rejection criteria, order, and iteration count +which works well on one spectrum will generally work well +on the other spectra. Note that the flatten operator attempts to +find a continuum level and normalize to that continuum, not to the +average value of the spectrum. +.PP +There are also the usual host of support operators - expansion, +overplotting, and so forth. There is also a pixel modifer mode +which connects two cursor positions. This forces a replot of the entire +spectrum after each pair of points has been entered. This should +probably be changed to inhibit auto-replot. +.PP +Some users have requested that all two cursor operators allow +an option to escape from the second setting in case the wrong +key was typed. I think this is a good idea, and might be implemented +using the "esc" key (although I could not seem to get this keystroke +through the GIO interface). +.PP +Another user request is the option to overplot many spectra with +autoscaling operational on the entire range. This is also a good +idea. Yet another improvement could be made by allowing the user +to specify the x and y range of the plot, rather than autoscaling. +.PP +There is one serious problem with respect to plotting spectra +corrected to a logarithmic wavelength scale. It would be nice to +plot these spectra using the logarithmic axis option, but this +option in GIO requires that at least one entire decade of x axis +be plotted. So for optical data, the x axis runs from 1000 Angstroms +to 10,000 Angstroms. Imagine a high dispersion plot having only 100 +Angstroms of coverage - the plot will look like a delta function! +The current version of SPLOT uses a linear axis but plots in +the log10 of wavelength. Not very good, is it. +.sp 1 +.SH +STANDARD +.PP +This task computes the sensitivity factor of the instrument +at each wavelength for which an a priori measured flux value is known +and within the wavelength range of the observations. +Sensitivity is defined as +[average counts/sec/Angstrom]/[average ergs/cm2/sec/Angstrom] +over the specified bandpass for which the star has been measured. +Both numerator and denominator refer to quantities above the +Earth's atmosphere and so the count rates must be corrected for +extinction. +The wavelengths of known measurements, the bandpasses, the +fluxes (in magnitudes), and the mean extinction table +are read from a calibration file whose name is specified +by the calib_file parameter (see LCALIB for a description of this +file). If a magnitude is exactly 0.0, it is assumed +that no magnitude is known for this star at the wavelength +having a 0.0 magnitude. This allows entries having incomplete +information. +.PP +As each observation is read, it is added into an accumulator for +its aperture. Or subtracted if it is a sky measurement. After +a pair of object and sky observations have been added, the +difference is corrected for extinction (as in BSWITCH), converted +to counts per second, and integrations performed over the bandpasses +for which flux measures are known. The bandpasses must be completely +contained within the spectrum - partial coverage of a bandpass +disqualifies it from consideration. The integrations are compared +with the known flux values and the ratio is written to a text +file (the "std" file) along with the wavelength of the measurement +and the total counts in the bandpass. The total counts value may +be used by SENSFUNC for weighting the measurements during averaging. +.PP +Many users are surprised by the order of the spectral names +printed out as STANDARD executes since the order is not necessarily +ascending through the spectrum list. This is because the name +printed is the name of the object spectrum most recently associated +with an object-sky pair. So if a sky pair is several spectra down the +list, an intervening object-sky pair taken through a different +instrument aperture may be processed in the meantime. +For example, say spectra 1-8 are taken so that object spectra +numbers 1 and 7 and sky spectra 3 and 5 are taken through aperture 0, +object spectra 4 and 6 and sky spectra 2 and 8 are taken through +aperture 1. [This is a very common pattern for IIDS/IRS users.] +Then spectrum 1 and 3 will pair up and be processed first (spectrum +name 1 will be printed). Then 4 and 2 (name 4 printed), then +7 and 5 (name 7 printed), and then 6 and 8 (name 6 printed). +So the order of names printed will be 1,4,7,6. Simple, isn't it? +.PP +If the input spectra are not taken in a beam-switched mode +then the parameter "beam_switch" should be set to no. +Then no sky subtraction will be attempted. +.PP +The user may enter sensitivity values directly into a file and use +it as the "std" file for a correction. +See the help file for STANDARD for a description of the entries in +the file, and see a typical file. +.PP +STANDARD offers a limited interactive mode. The first sky subtracted +spectrum is displayed and the bandpasses at which sensitivity +measurements are made will be shown as boxes. This provides a means +to see where the measurements are falling on the observational +data and to assess whether a bandpass may be including some +absorption edge which may be affecting the measurement. While it +is true that the wavelengths of the reference measurements should +fall in the same place, the effects of instrument resolution and +inaccuracies in the wavelength calibration may shift the positions +of the apparent bandpasses. The samples may then be biased. +.PP +The second purpose of the interactive mode is to allow the user +to artificially create new bandpasses on the fly. By placing the +cursor to bound a new wavelength region, STANDARD will interpolate +in the magnitude table of the reference star to estimate the magnitude +of the star at the bounded wavelength. The sensitivity will be calculated +at that wavelength just as if the bandpass had come from the calibration +file. This option should be exercised with care. Obviously, points +should not be generated between reference wavelengths falling on +strong absorption lines, or on a line either. This option is most useful +when at a high dispersion and few samples happen to fall in the +limited wavelength region. Sufficient space is allocated for 10 +artificial samples to be inserted. Once the artificial bandpasses +have been designated, they are applied to the entire sequence of +spectra for the current invocation of STANDARD. Once STANDARD +completes, the added bandpasses are forgotten. This prevents +an accidental usage of newly created bandpasses on stars of different +spectral types where a bandpass may fall in a region of continuum +for one star, but on an absorption line in another. +.sp 1 +.SH +SUBSETS +.PP +This is a simple task to subtract the second spectrum from the +first in a series of spectra. So if spectra 1-10 are input, +5 new spectra will be created from 1 minus 2, 3 minus 4, and so on. +This is a straight subtraction, pixel for pixel, with no +compensation for exposure time differences. +The header from the first spectrum of the pair is applied to the +output spectrum. +.sp 1 +.SH +The ONEDUTIL tasks +.PP +These utility tasks are logically separated from the ONEDSPEC +package. +.sp 1 +.SH +COEFS +.PP +This task reads the header parameters contained in comparison arc spectra +describing the wavelength solution generated by the mountain reduction +program and re-writes the solution parameters into a database +text file for use by DISPCOR. Otherwise those solutions would be +lost. COEFS assumes that the coefficients represent a Legendre +polynomial which is what the mountain reduction programs use. +.sp 1 +.SH +COMBINE +.PP +When an object has been observed over a wide range of wavelength +coverage by using more than one instrumental setup (such as +a blue and a red setting) or with different instruments (such +as IUE and the IRS), it is often desirable to combine the +spectra into a single spectrum. COMBINE will rebin a group of +spectra to new spectra having a single dispersion and average the +new spectra to create a single long spectrum. +If there are gaps in the composite spectrum, zeroes are used +as fillers. Ideally those pixels which have no known value +should be considered blank pixels. IRAF does not currently +support blank pixels, so zeroes are used for now. [One +might suggest using INDEF, but then all other routines will +have to check for this value.] +A side effect of choosing 0.0 is that during the averaging +of overlapping spectra, a true 0.0 will be ignored by COMBINE. +The basic rebinning algorithms used in DISPCOR are used in COMBINE +(and also REBIN). +.PP +The averaging can be weighted by exposure time, or by user assigned weights. +It would be better if each spectrum had an associated vector of +weights (one weight at each wavelength) so that the weighted averaging +could be done on a pixel basis. This is very expensive in terms +of both storage and file access overhead since each spectrum would +require twice the storage and number of files. +[Actually weights could be small 4 bit integers and take up very little space.] +.PP +A less ideal alternative would be to place a small number +(about 16) of weight parameters +in the header file which represent the approximate weights of that many +regions of the spectrum, and then one could interpolate in these parameters +for a weight appropriate to the pixel of interest. +.PP +A third solution (and even less ideal) +is to place a single parameter in the header which +represents an average weight of the entire spectrum. For the latter two cases, +the header weights could be derived from the average counts per +wavelength region - the region being the entire spectrum in the last case. +The weights must be entered into the header during the BSWITCH operation +since that is the last time that true counts are seen. [An implicit +assumption is that counts are proportional to photons. If data from +two different instruments are to be averaged, then the weights should be +expressed in photons because the ratio of counts to photons is highly +instrument dependent.] +.PP +COMBINE suffers from a partial pixel problem at the end points. +Interpolation at the ends can lead to an underestimate of the flux +in the last pixels because the final pixel is not filled. When averaging +in data from another spectrum or instrument, these pixels show up +as sharp drops in the spectrum. The problem appears due to the +rebinning algorithm and should be corrected someday (also in DISPCOR +and REBIN). +.sp 1 +.SH +LCALIB +.PP +This utility provides a means of checking the calibration files +containing the standard star fluxes and extinction table. +Any of the entries in the file may be listed out - the bandpasses, +extinction, standard star names, standard star fluxes in either +magnitudes, flambda, or fnu. For a description of the calibration +file format, see the help documentation for LCALIB. +.PP +The primary uses for LCALIB are to verify that new entries in +the tables are correct, to generate a list of standard star names +in a calibration file, and to produce a table of fluxes for a given standard +star. The table may then be used to generate a spectrum over a specified +wavelength region using SINTERP and overplotted with observational +data to check the accuracy of the reductions. +.sp 1 +.SH +MKSPEC +.PP +MKSPEC provides a way to generate a limited set of artificial +spectra. Noise generation is not available. The current options +are to generate a spectrum which is either a constant, a ramp, +or a black body. The spectrum may be two dimensional, but +all image lines will be the same. +.sp 1 +.SH +NAMES +.PP +This is the simplest task in the ONEDSPEC package. It +generates the image file names which are implied by a +root name and record string. The primary use for this +task is to generate a list of image names to be used +as input for some other program such as WFITS. +The output from NAMES can be redirected to file +and that file used with the "@file" notation for image +name input. An optional parameter allows an additional +string to be appended to the generated file name +to allow a subraster specification. +.sp 1 +.SH +REBIN +.PP +Spectra are rebinned to the wavelength parameters specified +by either matching to a reference spectrum or by user input. +The algorithms are those used by DISPCOR and the same options +for the interpolation method are available. REBIN is useful +when data are obtained with different instruments or setups +producing roughly comparable wavelength ranges and possibly +different dispersions, and the data are to be compared. +REBIN may also be used as a shift operator by specifying a +new starting wavelength. Or it may be used as a smoothing operator +by specifying a course dispersion. It may also be used +to convert between the two formats - linear in wavelength and +linear in the logarithm of wavelength. This latter option has +not been thoroughly exercised - proceed with caution. +.sp 1 +.SH +RIDSMTN +.PP +This task was stolen from the DATAIO package to make the following +modification: IIDS and IRS data are both written as 1024 pixel +spectra at the mountain. But the detectors do not produce a full +1024 pixels of acceptable data. In fact the IRS only has 936 pixels. +The data are written this way to conform to the IIDS ideal spectrum +which does have 1024 pixels, but the first few (about 6) are not usable. +To signal the good pixels, the IIDS/IRS header words NP1 and NP2 are +set to the beginning and ending good pixels. Actually NP1 points to +the first good pixel minus one. [Really actually NP1 and NP2 may be reversed, +but one is big and the other small so you can tell them apart.] +.PP +The version of RIDSMTN in ONEDUTIL keys off these parameters and writes +images containing only good pixels which means that the images will be +smaller than 1024 pixels. The user has the option of overriding the +header values with the task parameters "np1" and "np2". These may be +specified as 1 and 1024 to capture the entire set of pixels written to +tape or any other subset. Beware that np1 and np2 as task parameters +refer to the starting pixel and ending pixel respectively. None of this +nonsense about possible role reversals or "first good minus one" is +perpetuated. +.sp 1 +.SH +SINTERP +.PP +I think this is a handy little program. It provides a way to make +an IRAF spectral image from a table of values in a text file. +The table is interpolated out to any length and at any sampling +rate. A user can create a table of corrections to be applied to +a set of spectra, for example, use SINTERP to build a spectrum, +and run CALIBRATE to multiply a group of spectra by the correction. +.PP +The original raison d'etre for SINTERP was to create spectra of +standard stars from the listing of fluxes generated by LCALIB. +Using SPLOT the created spectrum can be overplotted with calibrated +observations to compare the true tabulated fluxes with the observed +fluxes. +.PP +SINTERP grew out of the task INTERP in the UTILITIES package +and works pretty much the same way. One major change is that +the table containing the x-y pairs is now stored in a dynamically +allocated array and can be as large as the user requests. The +default size is 1024 pairs, but the parameter tbl_size can +be set to a larger value. This then allows one to create a spectrum +from its tabulated values of wavelength and flux even if the +the table is several thousand elements long. +Note that the option to route the output from INTERP to +STDOUT has been retained if a new table is to be generated rather +than an IRAF image. +.PP +Another major change from INTERP is the use of the IRAF curve fitting +routines as an option. These were not originally available. +The choices now include linear or curvey interpolators, Legendre +or Chebyshev polynomial fits, and cubic or linear splines. +.sp 1 +.SH +WIDSTAPE +.PP +This task has vague origins in the DATAIO task WIDSOUT which writes +a tape having the format of the IDSOUT package which ran on the +CYBER (R.I.P.). For convenience to users this format has been +maintained for spectra having lengths up to 1024 pixels. +The version in DATAIO requires that the user enter all the header +parameters as task parameters. For several hundred spectra, this +approach is unwieldy. Because the ONEDSPEC package uses the header +parameters heavily, it is able to read them directly and write the +values to the tape file without user intervention. +.PP +The output tape (or diskfile) may be in either ASCII or EBCDIC format. +Spectra shorter than 1024 are zero filled. Each invocation of +the task write a new tape file followed by a tape mark (EOF). +.LP +.SH +3. Image Header Parameters +.PP +The ONEDSPEC package uses the extended image header to extract +information required to direct processing of spectra. If the +header information were to be ignored, the user would need to +enter observing parameters to the program at the risk of +typographical errors, and with the burden of supplying the +data. For more than a few spectra this is a tedious job, +and the image header information provides the means to eliminate +almost all the effort and streamline the processing. +.PP +However, this requires that the header information be present, +correct, and in a recognizable format. To meet the goal of +providing a functional package in May 1985, the first iteration +of the header format was to simply adopt the IIDS/IRS headers. +This allowed for processing of the data which would be first +used heavily on the system, but would need to be augmented at +a later date. The header elements may be present in any order, +but must be in a FITS-like format and have the following names +and formats for the value fields: +.sp 1 +.TS +l c l +l l l. +Parameter Value Type Definition + +HA SX Hour angle (+ for west, - for east) +RA SX Right Ascension +DEC SX Declination +UT SX Universal time +ST SX Sidereal time +AIRMASS R Observing airmass (effective) +W0 R Wavelength at center of pixel 1 +WPC R Pixel-to-pixel wavelength difference +NP1 I Index to first pixel containing good data (actually first-1) +NP2 I Index to last pixel containing good data (last really) +EXPOSURE I Exposure time in seconds (ITIME is an accepted alias) +BEAM-NUM I Instrument aperture used for this data (0-49) +SMODE I Number of apertures in instrument minus one (IIDS only) +OFLAG I Object or sky flag (0=sky, 1=object) +DF-FLAG I Dispersion fit made on this spectrum (I=nr coefs in fit) +SM-FLAG I Smoothing operation performed on this spectrum (I=box size) +QF-FLAG I Flat field fit performed on this spectrum (0=yes) +DC-FLAG I Spectrum has been dispersion corrected (0=linear, 1=logarithmic) +QD-FLAG I Spectrum has been flat fielded (0=yes) +EX-FLAG I Spectrum has been extinction corrected (0=yes) +BS-FLAG I Spectrum is derived from a beam-switch operation (0=yes) +CA-FLAG I Spectrum has been calibrated to a flux scale (0=yes) +CO-FLAG I Spectrum has been coincidence corrected (0=yes) +DF1 I If DF-FLAG is set, then coefficients DF1-DFn (n <= 25) exist +.TE +.PP +The values for the parameters follow the guidelines adopted for +FITS format tapes. All keywords occupy 8 columns and contain +trailing blanks. Column 9 is an "=" followed by a space. The value field +begins in column 11. Comments to the parameter may follow a "/" after +the value field. The value type code is as follows: +.RS +.IP SX +This is a sexigesimal string of the form '12:34:56 ' where the first +quote appears in column 11 and the last in column 30. +.IP R +This is a floating point ("real") value beginning in column 11 and +extending to column 30 with leading blanks. +.IP I +This is an integer value beginning in column 11 and extending to +column 30 with leading blanks. +.RE +.sp 1 +.PP +The parameters having FLAG designations all default to -1 to indicate +that an operation has not been performed. +The ONEDSPEC subroutines "load_ids_hdr" and "store_keywords" follow +these rules when reading and writing spectral header fields. +If not present in a header, load_ids_hdr will assume a value of zero +except that all flags are set to -1, and the object flag parameter +defaults to object. +.PP +When writing an image, only the above parameters are stored by store_keywords. +Other header information is lost. This needs to be improved. +.PP +Not all programs need all the header elements. The following table +indicates who needs what. Tasks not listed generally do not require +any header information. Header elements not listed are not used. +The task SLIST requires all the elements listed above. +The task WIDTAPE requires almost all (except NP1 and NP2). +The headings are abbreviated task names as follows: +.sp 1 +.nr PS 8 +.ps 8 +.TS +center; +l l | l l | l l. +ADD addsets COI coincor FIT flatfit +BSW bswitch COM combine REB rebin +CAL calibrate DIS dispcor SPL splot +COE coefs FDV flatdiv STA standard +.TE +.sp 1 +.TS +center, tab(/); +l | l | l | l | l | l | l | l | l | l | l | l | l. +Key/ADD/BSW/CAL/COE/COI/COM/DIS/FDV/FIT/REB/SPL/STA +_ +HA// X////////// X/ +RA// X////////// X/ +DEC// X////////// X/ +ST// X////////// X/ +UT// X////////// X/ +AIRMASS// X////////// X/ +W0// X/ X/// X//// X/ X/ X/ +WPC// X/ X/// X//// X/ X/ X/ +NP1/////////// X/// +NP2/////////// X/// +EXPOSURE/ X/ X/// X/ X///// X/// +BEAM-NUM// X/ X//// X/ X/ X// X/ X// +OFLAG// X////////// X/ +DF-FLAG//// X +DC-FLAG// X//// X//// X/ X/ X/ +QD-FLAG//////// X/ +EX-FLAG// X/ +BS-FLAG// X/ +CA-FLAG/ X// X//////// X/ +CO-FLAG///// X// +DFn//// X/ +.TE +.nr PS 11 +.ps 11 +.bp +.SH +Headers From Other Instruments +.PP +The header elements listed above are currently created only when reading +IIDS and IRS data from one of the specific readers: RIDSMTN and RIDSFILE. +The time-like parameters, (RA, DEC, UT, ST, HA), are created in a +compatible fashion by RCAMERA and RFITS (when the FITS tape is written +by the KPNO CCD systems). +.PP +For any other header information, the ONEDSPEC package is at a loss +unless the necessary information is edited into the headers with +an editing task such as HEDIT. This is not an acceptable long term +mode of operation, and the following suggestion is one approach to +the header problem. +.PP +A translation table can be created as a text file which outlines +the mapping of existing header elements to those required by the +ONEDSPEC package. A mapping line is needed for each parameter +and may take the form: +.sp 1 +.RS +.DC +1D_param default hdr_param key_start value_start type conversion +.DE +.RE +.sp 1 +where the elements of an entry have the following definitions: +.sp 1 +.TS +center, tab( ); +l lw(5i). +1D_param T{ +The name of the parameter expected by the ONEDSPEC package, +such as EXPOSURE, OFLAG, BEAM-NUM. +T} + +default T{ +A value to be used if no entry is found for this parameter or if +no mapping exists. +T} + +hdr_param T{ +The string actually present in the existing image header to be +associated with the ONEDSPEC parameter. +T} + +key_start T{ +The starting column number at which the string starts +in the header. +T} + +value_start T{ +The starting column number at which the string describing the +value of the parameter starts in the header. +T} + +type T{ +The format type of the parameter: integer, real, string, boolean, +sexigesimal. +T} + +conversion T{ +If the format type is string, a further conversion may +optionally be made to one of the formats listed under type. +The conversion may requires some expression evaluation. +T} +.TE +.sp 1 +.PP +Consider the example where the starting wavelength of a +spectrum is contained in a FITS-like comment and the object- +sky flag in a similar fashion: +.sp 1 +.DS + COMMENT = START-WAVE 4102.345 / Starting wavelength + COMMENT = OBJECT/SKY 'SKY '/ Object or Sky observation +.DE +.sp 1 +The translation file entries for this would be: +.sp 1 +.DS + W0 0.0 START-WAVE 12 24 R + OFLAG 0 OBJECT/SKY 12 25 S SKY=0;OBJECT=1 +.DE +.sp 1 +The first entry is fairly simple. The second requires an expression +evaluation and second conversion. +.PP +A translation file can be built for each instrument and its +special header format, and the file name can be associated with a +ONEDSPEC package parameter. The two subroutines in ONEDSPEC dealing +directly with the headers (load_ids_hdr and store_keywords) +can be modified or replaced to access this file and +translate the header elements. |