Initial commit

1 files changed, 576 insertions, 0 deletions

diff --git a/noao/imred/dtoi/doc/dtoi.ms b/noao/imred/dtoi/doc/dtoi.ms
new file mode 100644
index 00000000..4f999259
--- /dev/null
+++ b/noao/imred/dtoi/doc/dtoi.ms
@@ -0,0 +1,576 @@
+.RP
+.TL
+An Overview of the IRAF DTOI Package
+.AU
+Suzanne Hammond Jacoby
+.AI
+IRAF Group - Central Computer Services
+.K2 "" "" "*"
+February 1987
+.br
+Revised July 1988
+
+.AB
+This document describes the DTOI package, which contains tasks
+for determining and applying a density to intensity transformation to
+photographic data.  The transformation is determined from a set
+of calibration spots with known relative intensities.  A curve is
+interactively fit to the densities and intensities of the calibration
+spots.  The transformation is then applied and a new output image written.
+.AE
+
+.NH
+Introduction
+.PP
+The DTOI package contains tasks for computing and applying a density
+to intensity transformation to photographic data.  These tasks perform the
+standard steps in linearizing data: calculating HD data points from 
+calibration spots, fitting a curve to these points and applying the HD
+curve to the data.  It is also possible to combine related HD curves.
+Communication between the tasks is via text files which the user can 
+inspect or modify.  It is intended
+to be easy for users to introduce data from outside the DTOI package
+into the processing.
+.PP
+There are currently six tasks in the package.  They are:
+
+.ce
+The \fBDTOI\fR Package
+.TS
+center;
+n.
+spotlist \&- Calculate densities and weights of calibration spots.
+dematch \&- Match densities to log exposure values.
+hdfit \&- Fit characteristic curve to density, exposure data.
+hdtoi \&- Apply HD transformation to image data.
+hdshift \&- Align related characteristic curves.
+selftest \&- Test transformation algorithm.
+.TE
+.PP
+The DTOI package does not currently support the self calibration of images, 
+but the addition of this capability is planned.  This would involve
+determining the HD curve from the data itself, by assuming the point spread
+function scales linearly with intensity.
+.PP
+Upon entering the package, your calibration spots and images to be
+transformed should be on the disk in IRAF image format.
+
+.NH 
+Determining the HD Curve Data 
+.PP
+To determine the HD curve, you need two sets of data:  the
+measured photographic densities of a set of calibration spots and
+the log exposure values corresponding to these measurements.  The
+log exposure values must be known a priori.  Tasks \fIspotlist\fR and
+\fIdematch\fR are used to assemble these two data sets.
+.NH 2
+SPOTLIST
+.PP
+The first step is to calculate the density of
+the calibration spots, each of which is a separate IRAF image or image
+section.  The spot density is either the median of the spot pixels or
+the mean of the pixels when pixels more then a user specified number of
+standard deviations away from the mean have been rejected.  The numbers
+in the spot image must be scaled to density; parameter \fBspotlist.scale\fR
+is used such that density = input_value * scale.  Task \fIspotlist\fR also
+calculates the standard deviation of each spot and reports
+the number of good pixels, i.e., the number of pixels not rejected
+when determining the mean density.  
+The final product of this task is a record in the data base containing a
+density for each spot.  The scale factor used is also written to the data
+base; it will be read later in task \fIhdtoi\fR.
+.NH 2
+DEMATCH
+.PP
+Log exposure values must be matched to the measured density values.  These
+log exposure values must be known a priori and will be read from a file.
+Task \fIdematch\fR retrieves the proper exposure information by
+matching the wedge number, emulsion type and filter used.  Once a match
+has been made, the proper log exposure values are written to a record
+in the database.  
+.PP
+A database of log exposure values for the NOAO standard wedges is maintained 
+in a system file; the wedge/emulsion/filter combinations available are listed
+in last section of this document.  This file can be replaced with one specific 
+to any institution; the file name is supplied to task \fIdematch\fR as a 
+parameter.  In this way the wedge file can be personalized to any application 
+and not be lost when the system is updated.
+
+.NH
+Fitting the Curve
+.PP
+The HD curve, or characteristic curve, is a plot of density versus log 
+exposure.  This curve is determined from the data points generated by
+tasks \fIspotlist\fR and \fIdematch\fR.  The objective is to fit
+a curve to these points, such that Log exposure = F(Density).  The
+technique available in this package allows the independent variable of the
+fit to be a transformation of the density (log opacitance, for example).
+The log exposure and density values are
+read from the database.  If multiple entries for a particular record are 
+present in the database, the last one is used.
+.NH 2
+HDFIT
+.PP
+Task \fIhdfit\fR fits a characteristic curve to density and log exposure
+values in preparation for transforming an image from density to intensity.
+Five functional forms of the curve are available:
+.nf
+
+ 	  Power Series
+    	  Linear Spline
+    	  Cubic Spline
+    	  Legendre Polynomials
+    	  Chebyshev Polynomials
+ 	
+.fi
+.LP 
+It is possible to apply a transformation to the 
+independent variable (density above fog) prior to the fit.  The traditional 
+choice is to fit log exposure
+as a function of the log opacitance, rather than density directly.  This is
+sometimes referred to as the Baker, or Seidel, function.  Transforming
+the density has the effect of stretching the low density data points, which
+tend to be relatively oversampled. 
+In the DTOI package, four independendent variables are currently available:
+.nf
+
+ 	Density
+ 	Log Opacitance
+ 	K50 - (Kaiser* Transform with alpha = 0.50)
+ 	K75 - (Kaiser* Transform with alpha = 0.75)
+
+.fi
+.FS
+* Margoshes and Rasberry, Spectrochimica Acta, Vol 24B, p497, (1969)
+.FE
+Any combination of transformation type and fitting function can be used and
+changed interactively.  Two combinations of interest are discussed here.
+
+The default fit is a power series fit where the independent variable is 
+Log Opacitance.  That is:
+.LP
+.EQ
+
+	"Log Exposure = " sum from k=0 to {ncoeff - 1} {A sub k Y sup k}
+
+.EN
+.sp 1
+.EQ
+	"where Y = Log Opacitance = "Log sub 10 (10 sup Density - 1)
+.EN
+.LP
+A fit that is expected to best model a IIIA-J emulsion is a power series
+fit to a K75 transform of the density.  That is,
+.LP
+.EQ
+
+	"Log Exposure = "sum from k=0 to {ncoeff - 1} {A sub k Y sup k}
+
+.EN
+.sp 1
+.EQ
+"where Y = K75 transform = Density + 0.75 " Log sub 10 (1 - 10 sup -Density )
+.EN
+.LP
+Over the expected small dynamic range in variables of the fit, legendre
+and chebyshev functions offer no advantages over a simple power series
+functional form.  The cubic and linear spline fits may follow the data very
+closely, but with typically sparse data sets this is not desirable.  It
+is expected that power series fit will serve satisfactorily in all cases.
+
+.NH 3
+Interactive Curve Fitting
+.PP
+Task \fIhdfit\fR can be run interactively or not.  In interactive mode,
+points in the sample can be edited, added or deleted.  Weighting values
+can be changed as well as the fog value, the type of transformation
+and the fitting function chosen.  To obtain the best fit possible, interactive
+fitting is recommended.  A complete list of the available commands
+is printed here; this list is also available interactively with the 
+keystroke '\fL?\fR'.
+.TS
+center;
+c s s w(3.0i)
+c l s.
+
+	DTOI INTERACTIVE CURVE FITTING OPTIONS 
+
+\fL?\fR	Print options
+\fLa\fR	Add the point at the cursor position to the sample
+\fLc\fR	Print the coordinates and fit of point nearest the cursor
+\fLd\fR	Delete data point nearest the cursor
+\fLf\fR	Fit the data and redraw or overplot
+\fLg\fR	T{
+Redefine graph keys.  Any of the following data types may be along
+either axis:
+T}
+.T&
+l l l.
+ 	\fLx\fR  Independent variable	\fLy\fR  Dependent variable
+ 	\fLf\fR  Fitted value	\fLr\fR  Residual (y - f)
+ 	\fLd\fR  Ratio (y / f)	\fLn\fR  Nonlinear part of y
+ 	\fLu\fR  Density above fog
+
+Graph keys:	 	 
+.T&
+c l s.
+
+\fLh\fR	h = (x,y)  transformed density vs. log exposure
+\fLi\fR	i = (y,x)  log exposure vs. transformed density
+\fLj\fR	j = (x,r)  transformed density vs. residuals
+\fLk\fR	k = (x,d)  transformed density vs. the y(data)/y(fit) ratio
+\fLl\fR	l = (y,u)  log exposure vs. density above fog (HD Curve)
+
+\fLo\fR	Overplot the next graph
+\fLq\fR	T{
+Terminate the interactive curve fitting, updating the database file. 
+T}
+\fLr\fR	Redraw graph
+\fLu\fR	Undelete the deleted point nearest the cursor
+\fLw\fR	Set the graph window.  For help type 'w' followed by '?'.
+\fLx\fR	Change the x value of the point nearest the cursor
+\fLy\fR	Change the y value of the point nearest the cursor
+\fLz\fR	Change the weight of the point nearest the cursor
+
+.T&
+l s s w(3.0i).
+T{
+The parameters are listed or set with the following commands which may be
+abbreviated.  To list the value of a parameter type the command alone.
+T}
+
+.T&
+l l s.
+
+\fL:show \fR[\fIfile\fR]	Show the values of all the parameters
+\fL:vshow \fR[\fIfile\fR]	Show the values of all the parameters verbosely
+\fL:errors \fR[\fIfile\fR]	Print the errors of the fit (default STDOUT)
+\fL:reset \fR	T{
+Return to original conditions of x, y, wts and npts.
+T}
+\fL:ebars \fR[\fIerrors/weights\fR]	T{
+The size of marker type '[hv]ebars' can show either standard deviations or
+relative weights.
+T}
+\fL:function \fR[\fIvalue\fR]	T{
+Fitting function (power, chebyshev, legendre, spline3, or spline1)
+T}
+\fL:transform \fR[\fIvalue\fR]	Set the transform type (none, logo, k50, k75)
+\fL:fog \fR[\fIvalue\fR]	Change the fog level (or ":fog reset")
+\fL:order \fR[\fIvalue\fR]	Fitting function order
+\fL:quit \fR	Terminate HDFIT without updating database
+\fL:/mark \fRstring	T{
+Mark type (point, box, plus, cross, diamond, hline, vline, hebar, vebar, circle)
+T}
+
+.T&
+l s s.
+T{
+Additional commands are available for setting graph formats and manipulating
+the graphics.  Use the following commands for help.
+T}
+
+.T&
+l l s.
+\fL:/help\fR	Print help for graph formatting option
+\fL:.help\fR	Print cursor mode help
+
+.TE
+.PP
+The value of fog can be changed interactively if you have
+reason to override the value written in the database by \fIspotlist\fR.
+You can reset the fog to its original value with the command ":fog reset".
+A common problem with defining the HD curve is that some of 
+the calibration spot densities fall below fog.  This is caused by either
+the low signal to noise at low densities or by making a poor choice of 
+where to scan the fog level.  These points are rejected from the fit
+when a transformation of the density is being made, as the transform cannot
+be evaluated for negative density.  If the fog value or transformation 
+type is interactively changed so this problem no longer exists, 
+the spot densities are restored in the sample.
+
+The parameters of the final fit are written to a database which then
+contains the information
+necessary to reinitialize the curfit package for applying the transformation
+in \fIhdtoi\fR.  
+
+.NH
+Applying the Transform
+.PP
+.NH 2
+HDTOI
+.PP
+Once the HD curve has been defined, it is applied to a density image
+in task \fIhdtoi\fR.
+Here the transformation is applied, as described by the fit parameters
+stored in the database.  If more than one record of fit parameters is
+present, the last one is used.  This means task \fIhdfit\fR can be
+repeated until an acceptable solution is found; the last solution will
+be used by \fIhdtoi\fR.  On output, a new output image is written; the 
+input image is left intact.
+.PP
+The transformation is accomplished by using a look-up table.  All possible
+input values, from the minimum to maximum values found in the image, are
+converted to density using the scale value read from the database, and then 
+to intensity using the fit parameters determined by \fIhdfit\fR.  The input
+value is then the index into the intensity table: 
+intensity = look_up_table (input_value).
+.PP
+A scaling factor can be applied to the final intensities, as typically
+they will be < 1.0.  (The maximum log exposure in the NOAO wedge database
+is 0.0)  By default, a saturated density pixel will be assigned the "ceiling"
+intensity of 30000 and the other pixels are scaled accordingly.  
+The user is responsible for choosing a ceiling value
+that will avoid having significant digits truncated.
+The precision 
+of the transform is unaffected by scaling the
+final intensities, although caution must be used if the output image
+pixel type is an integer.  
+.PP
+The value of fog to be used is entered by the user, and can be either
+a number or a list of file names from which to calculate the fog value.
+The fog value is subtracted from the input image before the transformation
+takes place.
+Again, consider density values below fog.  Two choices are available for
+these densities: the calculated intensity can be equal to the constant 
+value 0.0 or equal -1.0 times the intensity determined for absolute (density).
+
+.NH
+Aligning Related HD curves
+.PP
+Calibration data sets from several plates can be combined once a shift
+particular to each set has been removed.  "Different spot exposures 
+define a series of HD curves which are parallel but mutually separated
+by arbitrary shifts in log exposure, produced by differing lamp intensities
+or exposure times.  Generally, Kodak spectroscopic plates can be
+averaged if [1] they come from the same emulsion batch and box, [2]
+they receive identical hypersensitizing, [3] they are exposed similarly and
+[4] they receive the same development." * 
+.FS
+* "Averaging Photographic Characteristic Curves", John Kormendy, from
+"ESO Workshop on Two Dimensional Photometry", Edited by P. Crane and
+K.Kjar, p 69, (1980), an ESO Publication.
+.FE
+.NH 2
+HDSHIFT
+.PP
+Procedure \fIhdshift\fR calculates and subtracts a zero point shift to 
+bring several related HD curves into alignment.  The individual shifts 
+are calculated by elimination of the first coefficient (Bevington, eqn 9-3):
+.EQ
+
+a0 = y bar - a sub 1 X bar - a sub 2 X bar sup 2 - ~ ...~ - a sub n X bar sup n
+
+.EN
+Here, the averages over y and X refer to individual calibration set averages; 
+the coefficients a1, ... an were previously calculated using data from all 
+calibration sets with task \fIhdfit\fR, and stored in the database.  The
+a0 term is calculated individually for each database; this term represents
+the zero point shift in log exposure and will be different for each database.
+
+On output, the log exposure values in each database have been 
+shifted to the zero point shift of the first database in the list.  The
+log exposure records are now aligned and it would be appropriate
+to run \fIhdfit\fR on the modified database list.
+.NH
+Testing the Transformation Algorithm
+.PP
+A test task is included to see if any numerical errors were introduced
+during the density to intensity transformation.  It also evaluates
+truncation errors produced when an output image with integer pixels,
+rather than reals, is written.
+.NH 2
+SELFTEST
+.PP
+An intensity vector is generated from a density vector in two different
+ways.  The first method uses the density vector and known coefficients
+to compute the intensity.  The second method uses the curfit package
+to generate a look up table of intensities as done in task \fIhdtoi\fR.  The
+residual of the two vectors is plotted; ideally the difference between
+the 'known' and 'calculated' intensity is zero.
+.PP
+Task \fIselftest\fR also plots intensity as a function of density for
+both integer and real output pixels.  The user should investigate the
+plot with the cursor zoom and expand capabilities to determine if
+truncation errors are significant.
+.NH
+The Wedgefile Database
+.PP
+Task \fIdematch\fR reads a database and retrieves log exposure information 
+for certain combinations of wedge number, photographic emulsion and filter.  
+Those combinations included in the NOAO database are listed in the next 
+section, although any calibration data can be included if the values are
+known.  To modify the database, it is recommended that
+you generate a new file rather than add records to the existing file.  This
+way, the modifications will not be lost when a new version of the IRAF
+system is released.  
+
+In the database, the information for each wedge makes up a separate record;
+each record starts with the word \fBbegin\fR.  Each record has a title field 
+and can have multiple emulsion/filter fields.  The number of log exposure
+values must be given, followed by the values written 8 per line.  The order 
+of the exposure data can be either monotonically increasing or decreasing.  
+Here is an example:
+.DS
+begin	115
+	title	MAYALL 4-M PF BEFORE 15APR74 (CHROME) [MP1-MP968]
+	IIIAJ/UG2	16
+		  0.000 -0.160 -0.419 -0.671 -0.872 -1.153 -1.471 -1.765
+		 -2.106 -2.342 -2.614 -2.876 -3.183 -3.555 -3.911 -4.058
+	IIAO/UG2	16
+		  0.000 -0.160 -0.418 -0.670 -0.871 -1.152 -1.468 -1.761
+		 -2.102 -2.338 -2.609 -2.870 -3.176 -3.547 -3.901 -4.047
+
+.DE
+.NH 2
+Contents of the NOAO Wedgefile
+.LP
+The following table lists the wedge/emulsion/filter combinations available in
+the NOAO wedgefile database.  
+.TS
+center;
+l l s s s
+l l l l l.
+
+\fBWedge  24 	CTIO SCHMIDT WESTON TUBE SENSITOMETER.            \fR
+ 	MONO/MONO					
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge  48 	PALOMAR 48-INCH SCHMIDT STEP WEDGE.               \fR
+ 	MONO/MONO			
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge  84 	OLD 84-INCH SPOT SENSITOMETER (1967)               \fR
+ 	MONO/MONO			
+
+.TE
+.TS
+l l s s s
+l l l l l.
+\fBWedge 101 	SPOT BOX 4, KEPT IN SCHOENING-S LAB.              \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4760	MONO/5200	MONO/5876        
+ 	MONO/6470			
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 115 	MAYALL 4-M PF BEFORE 15APR74 (CHROME) [MP1-MP968] \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4770	MONO/5200	MONO/5876        
+ 	MONO/6470			
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 117 	CTIO 4-METER P.F.                                 \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4770	MONO/5200	MONO/5876        
+ 	MONO/6470			
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 118 	CTIO 4-METER CASSEGRAIN                           \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4760	MONO/5200	MONO/5876        
+ 	MONO/6470	MONO/6900		
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 119	SPOT BOX 5, KEPT AT MAYALL 4-METER.               \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4760	MONO/5200	MONO/5876        
+ 	MONO/6470			
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 120 	SPOT BOX 6, KEPT AT 2.1-METER.                    \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4760	MONO/5200	MONO/5876        
+ 	MONO/6470			
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 121 	SPOT BOX 8, KEPT IN SCHOENING'S LAB.              \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4760	MONO/5200	MONO/5876        
+ 	MONO/6470        
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 122 	SPOT BOX 7, AVAILABLE AT KPNO NIGHT ASST'S OFFICE \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4760	MONO/5200	MONO/5876        
+ 	MONO/6470C			
+.TE
+.TS
+l l s s s
+l l l l l.
+\fBWedge 123 	MAYALL 4-M P.F. 15APR74 TO 21MAY74 [MP969-MP1051] \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4770	MONO/5200	MONO/5876        
+ 	MONO/6470        
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 129 	MAYALL 4-METER P.F. AFTER 21MAY74 [MP1052-->  ]   \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4760	MONO/5200	MONO/5876        
+ 	MONO/6470			
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 130 	MAYALL 4-METER CASS CAMERA.                       \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4760	MONO/5200	MONO/5876        
+ 	MONO/6470			
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 138 	TRAVELLING BOX AFTER 06JAN78.                    \fR
+ 	IIIAJ/UG2	IIAO/UG2	IIIAJ/*5113	IIAO/*5113       
+ 	IIAO/GG385	IIIAJ/CLEAR	IIIAJ/GG385	IIAD/GG495       
+ 	127/GG495	098/RG610	127/RG610	IVN/RG695       
+ 	MONO/4363	MONO/4770	MONO/5200	MONO/5876        
+ 	MONO/6470			
+
+.T&
+l l s s s
+l l l l l.
+\fBWedge 201 	TEN UCLA SPOTS (H. FORD, 10JAN78)                 \fR
+ 	MONO/MONO			
+.TE