aboutsummaryrefslogtreecommitdiff
path: root/sys/imio/doc
diff options
context:
space:
mode:
authorJoe Hunkeler <jhunkeler@gmail.com>2015-08-11 16:51:37 -0400
committerJoe Hunkeler <jhunkeler@gmail.com>2015-08-11 16:51:37 -0400
commit40e5a5811c6ffce9b0974e93cdd927cbcf60c157 (patch)
tree4464880c571602d54f6ae114729bf62a89518057 /sys/imio/doc
downloadiraf-osx-40e5a5811c6ffce9b0974e93cdd927cbcf60c157.tar.gz
Repatch (from linux) of OSX IRAF
Diffstat (limited to 'sys/imio/doc')
-rw-r--r--sys/imio/doc/IMH.hlp219
-rw-r--r--sys/imio/doc/Notes177
-rw-r--r--sys/imio/doc/bench.ms73
-rw-r--r--sys/imio/doc/imfort.doc72
-rw-r--r--sys/imio/doc/imio.2.ms331
-rw-r--r--sys/imio/doc/imio.doc232
-rw-r--r--sys/imio/doc/imio.hlp1185
-rw-r--r--sys/imio/doc/imio.ms295
8 files changed, 2584 insertions, 0 deletions
diff --git a/sys/imio/doc/IMH.hlp b/sys/imio/doc/IMH.hlp
new file mode 100644
index 00000000..0599843b
--- /dev/null
+++ b/sys/imio/doc/IMH.hlp
@@ -0,0 +1,219 @@
+.help imio Mar86 "Image I/O Modifications"
+.ce
+\fBImage I/O Modifications to Support Multiple Data Formats\fR
+.ce
+Doug Tody
+.ce
+March 16, 1986
+
+.nh
+Introduction
+
+ The primary purpose of this revision of IMIO was to add support for
+multiple disk data formats. This was done by defining a new interface
+called IMH, which hides the details of how images are stored on disk from
+the IMIO code. IMIO is concerned only with image i/o based on an internal
+image descriptor. IMIO calls IMH to perform all imagefile management
+operations, but accesses the pixel data directly using the FIO interface.
+The IMH interface initially supports only the old IRAF and SDAS image formats,
+but may be extended to support other formats in the future if necessary.
+The IMH interface should be reusable in the future when IMIO is layered
+upon DBIO.
+
+A secondary purpose of this revision was to make several minor enhancements
+to IMIO to better support groups of images. The image template syntax
+has been extended to provide a special selection syntax for specifying the
+subset of the images in a group or set of groups to be operated upon.
+Minor changes were made to the filenames of the files used to store IRAF
+format images to make identification of the header and pixel files easier.
+The image database interface (IDB) has been extended to permit the storage
+of one dimensional arrays in the image header. IMIO was modified to use the
+static file driver rather than the regular binary file driver to access
+pixel data.
+
+.nh
+Image Templates
+.nh 2
+Image Template Syntax
+
+ An image template is an expression used to specify the set or group of
+images to be operated upon. The current image template syntax is upwards
+compatible with the planned DBIO record select/project syntax. The full
+syntax is as follows.
+
+ images [,images ...]
+
+where \fIimages\fR is an expression built up from some combination of the
+following constructs:
+
+.nf
+ @listfile take strings from a listfile
+ pattern expand pattern against database
+ str // str concatenate strings
+ {select} select elements from a set
+ [section] append image section
+.fi
+
+
+These elements may appear in any order, although not all orderings make
+sense. The \fIpattern\fR string may contain any of the standard pattern
+matching characters, but the pattern matching meta-characters {} (ignore
+case) and [] (character class) must be escaped to avoid interpretation as
+the subset selection and image section operators. An \fIimages\fR string
+may contain at most one list construct, i.e., listfile reference, pattern,
+or selection set. The \fIselect\fR expression is limited to a range list
+at present. The range list syntax uses the : range syntax, i.e.,
+"from[:to[:by]]". Some examples are given below. None of templates shown
+in these examples need be quoted if entered in the CL command mode.
+
+
+.nf
+ pix one image
+ pix.0013 one image
+ @pics list of images
+ pix[*,-*] one image section
+ @pics[*,-*] list of image sections
+ pix.* all pix.whatever images in cwd
+ pix.*//.flat same, but append ".flat" to image name
+ pix.*//.flat[*,5] same, but append image section too
+ pix{1,4,9:21} pix.0001, pix.0004, and 9 through 21
+ pix{1,4,9:21}[1:10,5] same, but append section to each one
+.fi
+
+
+Note that \fIselect\fR expressions are expanded without checking to see if the
+named images actually exist. Image names are formed by concatenating the
+image number encoded as a four digit string, padding with zeroes at the left
+as in the examples.
+
+.nh 2
+Image Template Procedures
+
+ The image template package (IMT) is an existing package. The calling
+sequences are identical to those of the FNT (FIO filename template) package
+and have not been changed in this release of the package. Extensive changes
+have however been made internally, and the new package has been installed
+in IMIO. The old package has been removed from the XTOOLS library. To use
+the new version of the package, all one need do is relink.
+
+
+.ks
+.nf
+ list = imtopen (template)
+ nimages = imtlen (list)
+ imtrew (list)
+ nchars|EOF = imtgetim (list, fname, maxch)
+ imtclose (list)
+.fi
+.ke
+
+
+An image template is expanded into a list of image names or image sections
+with \fBimtopen\fR. The list is not globally sorted, however sublists
+generated by pattern matching are sorted before appending the sublist to
+the final list. The number of images or image sections in a list is given by
+\fBimtlen\fR. Images are read sequentially from the list with \fBimtgetim\fR,
+which returns EOF when the end of the list is reached. The list may be
+rewound with \fBimtrew\fR. An image template list should be closed with
+\fBimtclose\fR to return the buffers used to store the list and its
+descriptor.
+
+.nh
+The Image Header Access Interface (IMH)
+
+ The image header access interface (IMH) is a new interface in this release
+of IMIO. The purposes of the IMH interface are to hide knowledge of how images
+are stored on disk from the rest of the system, and to make it possible to
+support multiple image storage formats. IMIO is not aware that there are
+multiple image storage formats. When called to open an existing image,
+IMH determines the image format and calls the appropriate lower level access
+procedure to read the image header. A standard set of IMH callable interface
+procedures are required for each supported storage format.
+
+The IMH package is intended as an internal IMIO package and should not normally
+be called by packages other than IMIO.
+
+
+.ks
+.nf
+ im = imh_open (image, acmode, o_im) # open/create header
+ imh_opix (im, acmode) # open/create pixels
+ imh_update (im) # update header
+ imh_close (im) # close image
+
+ y/n = imh_access (image, type) # test if image exists
+ imh_delete (image) # delete an image
+ imh_rename (oldname, newname) # rename an image
+ imh_copy (oldname, newname) # copy an image
+.fi
+.ke
+
+
+The \fIimh_open\fR procedure will open an existing image, create a new
+image, or make a new copy of an existing image (preserving the header
+of the old image but not the pixels). A pointer to an IMIO binary image
+descriptor is returned; when opening an existing image, the primary
+function of \fIimh_open\fR is to map the disk image header, stored in
+whatever format, into the IMIO fixed format binary descriptor.
+
+The \fIimh_opix\fR procedure must be called after the header has been opened
+before any pixel i/o can be done to the image. In the case of a new image
+the size attributes of the new image are not fixed until \fIimh_opix\fR is
+called (giving the high level code time to set the size parameters in the
+image descriptor). It is not necessary to call \fIimh_opix\fR if only the
+header of an existing image is to be accessed.
+
+The \fIimh_access\fR procedure is provided to test if the named image
+exists (no test is made to determine if the image is also accessible).
+An integer code identifying the storage format of the image, e.g., old
+IRAF or SDAS, is returned in the \fItype\fR argument. Currently, the type
+of an image is indicated by the filename extension of the header file.
+
+The \fIimh_rename\fR and \fIimh_copy\fR procedures are in principle not
+required since the operations can be performed at a high level with the
+procedures already provided, but the IMH operators can carry out the
+operations more efficiently and without the possibility of information being
+lost, since they have knowledge of the physical storage format.
+
+.nh
+Physical Data Formats
+
+ The only two physical (disk) data formats currently supported are the
+old IRAF format and the STScI SDAS format. The IMH interface makes it
+relatively straightforward for other sites to interface their local format
+if necessary, or to support multiple versions of the same format.
+
+IMH will automatically read images stored in any of the supported formats.
+When making a NEW_COPY image, IMH will generate a new image in the same
+format as the existing input image. When making a NEW_IMAGE type image,
+the value of the environment variable \fBimtype\fR determines the type of
+image to be created. The recognized values of \fBimtype\fR are shown below.
+
+.ks
+.nf
+ not defined old iraf format
+ imtype = "oiraf" old iraf format
+ imtype = "sdas" SDAS/SOGS format
+.fi
+.ke
+
+The format in which an image is stored is indicated by the filename extension
+of the header file. The filename extension is not specified in image names
+or templates above the IMH interface, but is visible to the user in a
+directory listing.
+
+.ks
+.nf
+ .imh old iraf format header
+ .hhh SDAS/SOGS format header
+.fi
+.ke
+
+The current IMH interface is implemented in such a way that the semantics
+of the two image storage formats are essentially equivalent, i.e., applications
+programs should work consistently regardless of the storage format used.
+In order to achieve this, the SDAS group format is fully supported only
+when reading existing group format images. On output, images stored in
+SDAS format are stored in separate files (gcount=1). The IRAF image template
+is more flexible than the SDAS group format, simplifies programming, and
+provides much the same basic capability.
diff --git a/sys/imio/doc/Notes b/sys/imio/doc/Notes
new file mode 100644
index 00000000..b7f6675c
--- /dev/null
+++ b/sys/imio/doc/Notes
@@ -0,0 +1,177 @@
+IMIO modifications to support SDAS format imagefiles
+----------------------------------------------------------------
+
+1. EXISTING DATA FORMATS
+
+
+1.1 IRAF Data Format (pre-DBIO)
+
+Characteristics:
+
+ o Header: binary data structure + fits card image string buffer
+ o Pixels: binary pixhdr + pixels (can be block aligned)
+ o Many datatypes supported
+ o One image per imagefile
+ o Pixel file may be stored in different directory than header file
+ o Header file is protected from deletion
+
+Disadvantages:
+
+ o Can lose track of pixel file if header is deleted.
+ o Since each image is stored in a separate pair of files,
+ directories can be large.
+ o The storage format is machine dependent.
+
+Modifications in this release:
+
+ o Add .imh extension to image header file
+ o Add .pix extension to pixel file, make root name same as that
+ of the header file
+ o If IMDIR = "", put pixel file in same directory as the header.
+ This avoids use a pathname in the header, hence the images
+ are relocatable, but forces one to work on a scratch device.
+
+
+
+1.2 SDAS Data Format
+
+Characteristics:
+
+ o Images are physically stored in group format
+ o FITS group header + binary image headers
+ o Pixfile format: [pixels + group header] * ngroups
+ nothing is aligned on block boundaries
+ o Pixel and header files stored in same directory
+ o Header file extension .hhh, pixel file extension .hhd
+
+Disadvantages:
+
+ o Images cannot be added to a group; the number of images in a
+ group must be specified when the group is created.
+ o The individual images in a group cannot be deleted; only the
+ entire group can be deleted.
+ o The format of the image headers for the individual images in
+ a group is fixed at create time; new parameters cannot be
+ added to the individual image headers (new parameters can
+ however be added which apply to the group as a whole).
+ o The images in a group must all be of the same size and datatype
+ (this is probably not a serious disadvantage).
+ o The storage format is machine dependent.
+
+
+2. IMAGEFILE ACCESS
+
+ o Open/create image
+ o Make new_copy image
+ o Access pixel segment
+ o Close image
+ o Test if image exists (and determine type)
+ o Delete image
+ o Rename image
+
+
+2.1 Open Image
+
+ generate header filename
+ open/create header file
+ allocate image descriptor
+ if (existing image)
+ read image header into descriptor
+ else
+ initialize descriptor
+ return pointer to descriptor
+
+
+2.2 Access Pixel Segment
+
+ All image size parameters must be determined at pixfile creation
+ time.
+
+ if (new segment) {
+ fill in descriptor
+ if (new pixel file)
+ allocate pixel file
+ else
+ open pixel file r/w
+ update header
+ } else
+ open pixel file
+
+
+2.3 Close Image
+
+ if (header has been modified)
+ update header
+ close pixfile
+ close header
+
+
+
+3. SPECIFICATIONS
+
+3.1 Image Header Access
+
+ To minimize changes to existing code, the IMIO internal data structures
+will not be modified. The principal change to the structure of the existing
+interface is the replacement of the direct calls to the FIO open, close, read,
+write, etc. procedures called to access the image header in mass storage by
+calls to a new interface IH (Image Header access). The new interface will
+hide the disk image format from IMIO. Interface subroutines will be provided
+only for the IRAF and SDAS image formats, although in principle the interface
+will be extensible to other formats as well.
+
+Ideally the IH interface should be coded using only relatively low level
+VOS and kernel facilities (i.e., no high level FIO, no error handling) so that
+it may be used by the IMFORT interface and called from host Fortran programs,
+as well as by IMIO.
+
+
+ im = ihopen (image, group, acmode)
+ ihopix (im)
+ ihclose (im)
+
+
+IHOPEN returns the standard IMIO image descriptor, consisting of the internal
+IMIO fields, the binary image header structure IMHDR, and the "user area",
+a sequence of FITS card images stored in memory a string buffer, i.e.,
+each card image is represented as a stripped, newline delimited sequence of
+characters, with an EOS following the last card.
+
+The GROUP argument to IHOPEN permits access to the individual elements of
+a group format imagefile. Group format is supported for both imagefile
+formats, the principal difference being that the individual images are
+stored in separate files in the old IRAF format, and in a single pair of
+images in the SDAS format.
+
+The individual images in a group format imagefile appear as separate,
+independent images in IMIO. Several images in a group format imagefile may
+be simultaneously open (the files are physically opened only once).
+The group header parameters are duplicated for each image in the group.
+If the images are stored in the old IRAF format on disk the values of these
+parameters may vary from image to image, otherwise (SDAS format) they are
+the same for all members of the group. The number of images in a group is
+fixed at image creation time.
+
+
+3.2 Image Sections and Templates
+
+ The image section notation recognized by IMMAP may include a group
+specifier (set selection expression) as well as a section specifier.
+The full syntax is "image{group}[section]", e.g.,
+
+ pix{3}[*,5]
+
+where { is the set selection operator, and [ is the familiar array subscript
+or subsection operator.
+
+The image template notation has also been generalized to support group format
+image data. The general form of a template element is
+
+ image{groups}[section]
+
+where [section] applies to each image in the group. For example, the template
+
+ aa{4},bb{1,3:5},cc{12:15}[*,-*]
+
+expands as image 4 of group AA, images 1, 3, 4, and 5 of group BB, and images
+12 through 15 of group CC flipped in Y.
diff --git a/sys/imio/doc/bench.ms b/sys/imio/doc/bench.ms
new file mode 100644
index 00000000..05717208
--- /dev/null
+++ b/sys/imio/doc/bench.ms
@@ -0,0 +1,73 @@
+.OM
+.TO
+Steve Ridgway
+.FR
+Doug Tody
+.SU
+Performance of IRAF Image I/O
+.PP
+As Caty reported in her memo of 15 November, the timings of the \fIimarith\fR
+task were surprisingly poor, i.e., approximately 20 cpu seconds for the
+addition of two 200 column by 800 line short integer images, producing a
+short integer image as output (a "short" integer is 16 bits).
+A look at the code for \fIimarith\fR revealed
+that the internal computations were being done in double precision floating,
+regardless of the datatype of the images on disk.
+I was not aware of this and I appreciate having it brought to my attention.
+Fixing \fIimarith\fR took several hours and nearly cut the timings in half.
+.PP
+When I orginally implemented IMIO I planned to eventually make three major
+optimizations (as noted in the program plan and system interface reference
+manual):
+.RS
+.LP \(bu
+Optimize the special case of line by line i/o with no automatic type
+conversion, image sectioning, boundary extension, etc.
+.LP \(bu
+Provide direct access into the FIO file buffers when possible to eliminate
+the memory to memory copy to and from the IMIO and FIO buffers.
+.LP \(bu
+Implement a optimal static file driver for UNIX to eliminate the overhead
+of copying the data through the system buffer cache, and to permit
+overlapped i/o.
+.RE
+.LP
+I have gone ahead and implemented the first two optimizations; this took
+a day and the changes were entirely internal to the interface,
+requiring no changes to user code and no loss of machine independence.
+After these changes were made to IMIO I ran several benchmarks with the
+following results. All benchmarks were for images with 16 bit integer pixels.
+.TS
+center box tab(|);
+ci ci ci ci ci ci ci
+r n n n nb n n.
+operation|open/close|line ovhead|kernel op|total user time|%opt|systime
+-
+(c=a+b)[200,800]|.38|1.43|1.69|3.50|48%|3.82
+(c=a+b)[800,800]|.38|1.43|6.94|8.75|79%|12.16
+minmax[800,800]|.05|0.59|11.39|12.03|95%|2.66
+.TE
+.PP
+The columns in the table show the operation tested by the benchmark (two image
+additions, each involving three images, and a computation of the minimum and
+maximum of a single image), the overhead involved in opening and closing the
+images (same operation on a [1,1] image), the total overhead to process the
+image lines, the time consumed by the kernel operation, the total user time
+for the task, the degree of optimality (ratio of time spent in the kernel
+vector operation to the total time for the task),
+and the system (UNIX kernel) time required.
+.PP
+In short, the time required by the original benchmark has decreased from
+20 seconds to 3.5 seconds, disregarding the system time. In this worst
+case benchmark we still manage to come within 48% of the optimal time of
+1.69 seconds for a VAX 11/750.
+.PP
+The short integer vector addition kernel operator was hand optimized in
+assembler for these benchmarks to provide a true measure of the degree
+of optimality. The actual unoptimized UNIX vector addition operator is
+slightly slower.
+The last column, labelled "systime", shows the cpu time consumed
+by the UNIX kernel moving the pixels to and from disk; this is the time
+that will be eliminated by the static file driver optimization.
+Once the static file driver is optimized any further optimizations
+will be difficult.
diff --git a/sys/imio/doc/imfort.doc b/sys/imio/doc/imfort.doc
new file mode 100644
index 00000000..6eef0dc6
--- /dev/null
+++ b/sys/imio/doc/imfort.doc
@@ -0,0 +1,72 @@
+Jun 19: IRAF images may now be read and written from Fortran programs.
+ The interface is simple and efficient, but limited in capability.
+ If a more sophisticated interface is required one may call Fortran
+ subroutines from SPP main programs (templates are available for
+ accessing 1 and 2 dimensional images in this fashion), or program
+ directly in SPP.
+
+ 1. Documentation from the source file
+
+ IMFORT -- Fortran interface to IRAF images. This interface permits a
+ Fortran program to read or write an existing IRAF image. There is
+ currently no provision for creating new images or deleting old images
+ from Fortran. The interface routines are as follows:
+
+ im = imopen (image, mode, ndim, len_axes)
+ imclos (im)
+
+ imget[sr] (im, buf, x1, x2, linenum)
+ imput[sr] (im, buf, x1, x2, linenum)
+
+ where
+ input integer im, x1, x2, linenum
+ input character image, mode
+ output integer ndim, len_axes(7)
+ pixel buf(*)
+
+ imgets,imputs are for short integer (integer*2) pixels
+ imgetr,imputr are for real pixels
+
+ An image must be opened with IMOPEN before it can be accessed. Legal
+ access modes are 'r', 'w', and 'rw'. The number of dimensions and
+ the length of the axes are returned in ndim and len_axes; the latter
+ should be dimensioned for at least 7 dimensions. All coordinates are
+ 1-indexed. The variable "im" is an integer. The get and put routines
+ will perform datatype conversion if necessary. The imget and imput
+ routines will abort program execution if there is an error.
+
+
+ 2. Usage
+
+ Source files (minimal documentation in imfort.c header):
+
+ /iraf/sys/imio/mhdr.c.h
+ /iraf/sys/imio/imfort.c
+
+ Libraries:
+
+ /usr/lib/libiraf.a -liraf on f77 cmd line
+ /usr/lib/libvops.a -lvops on f77 cmd line
+
+ e.g.,
+ f77 myprog.f -liraf -lvops -o myprog
+
+ or if called in SPP
+
+ cl> xc myprog.x, lib=iraf
+
+
+ 3. Example
+
+ integer im
+ integer axlen(7), ndim
+ integer imopen
+ integer*2 pix(1024)
+
+ im = imopen ('/tmp2/iraf/images/m74', 'r', ndim, axlen)
+ write (*,*) ndim, axlen
+ call imgets (im, pix, 10,15, 5)
+ write (*,*) pix(1), pix(5)
+ stop
+ end
+
diff --git a/sys/imio/doc/imio.2.ms b/sys/imio/doc/imio.2.ms
new file mode 100644
index 00000000..0727179f
--- /dev/null
+++ b/sys/imio/doc/imio.2.ms
@@ -0,0 +1,331 @@
+.DA May 7, 1985
+.OM
+.TO
+distribution
+.FR
+Doug Tody
+.SU
+New release of image i/o, etc.
+.PP
+The new release of IMIO has been installed in the system for a week now and
+appears to be bug free. This memo summarizes the changes/additions in this
+new version of the interface, and introduces the new "image database" tools
+\fIhedit\fR and \fIhselect\fR as well.
+.NH
+Summary of Changes in the Current Release
+.PP
+The following changes or additions have been made to the IMIO interface and
+the \fIimages\fR package.
+.RS
+.IP \(bu
+IMIO now has the ability to perform (optionally) automatic boundary extension
+to satisfy out of bounds pixel references.
+.IP \(bu
+A preliminary database interface has been added to IMIO.
+.IP \(bu
+Image headers are now variable length and use only the amount of disk space
+required to store the header (excluding byte packing).
+.IP \(bu
+Two new database utility tasks \fIhedit\fR and \fIhselect\fR have been
+added to the \fIimages\fR package. Both use the new library subroutine
+\fIevexpr\fR, now installed in the FMTIO package.
+.IP \(bu
+A new task \fIimshift\fR has been added to the \fIimages\fR package to
+perform shifts of two dimensional images using full two dimensional
+interpolation. The related tasks \fIgeomap\fR and \fIgeotran\fR are
+currently being worked on. Some filtering and convolution tasks should
+also be available soon. All of these tasks use the new boundary extension
+feature of IMIO.
+.RE
+.PP
+The new release of IMIO is upward compatible with previous versions and should
+require no changes to or even recompilation of existing code. The basic image
+header structure is unchanged hence existing images and image headers are still
+accessible. Copying of old images still on disk with \fIimcopy\fR may however
+be desirable to reduce disk consumption (the old headers were wasteful of
+storage).
+.PP
+This release of IMIO introduces some database tools and concepts which
+should help us understand the role the DBIO interface and DBMS package will
+play in image processing applications in the near future. The current database
+interface has serious functional limitations and is inefficient for operations
+which operate upon entire databases (e.g., the \fIselect\fR operation),
+but does provide a basic and much needed image database capability.
+.NH
+Planned Future Developments
+.PP
+This new release of IMIO is expected to remain unchanged until DBIO is
+completed, at which time a new version of the interface will be released.
+This next release is expected to be upward compatible with the current
+interface except in cases where the applications task has detailed knowledge
+of the current image header structure. Applications which directly access
+the "user area" of the current header, or which use certain header fields
+such as IM_HISTORY, will have to be modified as these data structures will
+change in the next release.
+.PP
+Applications which use only \fIimmap\fR, \fIimunmap\fR, IM_PIXTYPE,
+IM_NDIM, IM_LEN, and the basic i/o procedures should not have to be changed.
+The new interface will provide different facilities to do the same things
+but we can probably emulate the old interface to allow plenty of time to
+convert the old code. Of course, the new interface will provide new facilities
+which we did not formerly have and which we will want to use, and therefore
+we will eventually have to modify all existing image tasks.
+.PP
+Perhaps more seriously, we are not going to be able to maintain the ability
+to read the existing binary image files when the DBIO version of IMIO is
+released. At that time, all disk resident images will have to be processed
+to FITS format and thence into the new DBIO image format. We will keep the
+old binary for the FITS writer task around for an indefinite period after
+the changeover to make this possible.
+
+.NH
+Modifications to the Current Interface
+.NH 2
+Boundary extension
+.PP
+Automatic boundary extension is useful in applications such as filtering via
+convolution, since the convolution kernel will extend beyond the boundary of
+the image when near the boundary, and in applications which operate upon
+subrasters, for the same reason. When reading from an image with boundary
+extension in effect, IMIO will generate artificial values for the out of
+bounds pixels using one of the techniques listed below. When writing to an
+image with boundary extension in effect, the out of bounds pixels are
+discarded.
+.PP
+By default, an out of bounds pixel reference will result in an error message
+from IMIO. Consider an image line of length 5 pixels. The statement
+.DS
+\fL
+ buf = imgs1r (im, -1, 7)
+\fR
+.DE
+references out of bounds by 2 pixels on either end of the image line,
+referencing a total of 5+2+2=9 pixels. If boundary extension is enabled
+and the get section completes successfully then \fIMemr[buf]\fR will reference
+the pixel at X = -1, and \fIMemr[buf+2]\fR will reference the first inbounds
+pixel.
+.PP
+When an image is first opened zero pixels of boundary extension are
+selected, and any out of bounds references will result in an error.
+To enable boundary extension \fIimseti\fR must be called on the open
+image to specify the number of pixels of boundary extension permitted
+before an out of bounds error occurs.
+.DS
+\fL
+ include <imset.h>
+ call imseti (im, IM_NBNDRYPIX, 2)
+.DE
+\fR
+.LP
+If boundary extension is enabled the type of boundary extension desired
+should also be set. The possibilities are defined in \fI<imset.h>\fR and
+are summarized below.
+.DS
+\fL
+ BT_CONSTANT return constant if out of bounds
+ BT_NEAREST return nearest boundary pixel
+ BT_REFLECT reflect back into image
+ BT_WRAP wrap around to other side
+ BT_PROJECT project about boundary
+.ce
+\fR
+\fBTypes of Boundary Extension\fR
+.DE
+.LP
+The type of boundary extension is set with the imset parameter IM_TYBNDRY.
+If the BT_CONSTANT option is selected the constant value should be set with
+an \fIimseti\fR or \fIimsetr\fR call to set the parameter IM_BNDRYPIXVAL.
+Boundary extension works for images of any dimension up to 7 (the current
+IMIO limit). A single IM_NBNDRYPIX value is used for all dimensions.
+This value is used only for bounds checking, hence the value should be set
+to the maximum out of bounds reference expected for any dimension.
+Larger values do not "cost more" than small ones. An actual out of bounds
+reference is however more expensive than an inbounds reference.
+.NH 2
+Image Database Interface
+.PP
+The image database interface is the IMIO interface to the database
+containing the image headers. In this implementation the image header is
+a variable length binary structure. The first, fixed format, part of the
+image header contains the standard fields in binary and is fixed in size.
+This is followed by the so called "user area", a string buffer containing
+a sequence of variable length, newline delimited FITS format keyword=value
+header cards. When an image is opened a large user area is allocated to permit
+the addition of new parameters without filling up the buffer. When the
+header is subsequently updated on disk only as much disk space is used as
+is needed to store the actual header.
+.PP
+The new header format is upwards compatible with the old image header format,
+hence old images and programs do not have to be modified to use the latest
+release of IMIO. In the future image headers will be maintained under DBIO,
+but the routines in the image header database interface described in this
+section are not exected to change.
+The actual disk format of images will of course change when we switch
+over to the DBIO headers. While the physical storage format of header will
+change completely under DBIO, the logical schema will change very little,
+i.e., our mental picture of an image header will be much as it is now.
+The main difference will be the consolidation of many images into a few files,
+and real support in the image header for bad pixels, history, and coordinate
+transformations. In addition a number of restrictions on the "user fields"
+will be lifted, the remaining distinctions between the standard and user
+fields will disappear, and database operations will be much more efficient
+than they are now.
+.NH 3
+Library Procedures
+.PP
+The IMIO library procedures comprising the current image database interface
+are summarized in the table below.
+.DS
+\fL
+ value = imget[bcsilrd_] (im, field)
+ imgstr (im, field, outstr, maxch)
+ imput[bcsilrd_] (im, field, value)
+ impstr (im, field, value)
+ imadd[bcsilrd_] (im, field, def_value)
+ imastr (im, field, def_value)
+ imaddf (im, field, datatype)
+ imdelf (im, field)
+ y/n = imaccf (im, field)
+
+ list = imofnl[su] (im, template)
+ nchars/EOF = imgnfn (list, fieldname, maxch)
+ imcfnl (list)
+
+where
+ pointer im, list
+ char[] field, outstr, datatype, template, fieldname
+\fR
+.ce
+\fBImage Database Interface Procedures\fR
+.DE
+.PP
+New parameters will typically be added to the image header with either
+one of the typed \fIimadd\fR procedures or with the lower level \fIimaddf\fR
+procedure.
+The former procedures permit the parameter to be created and the value
+initialized all in one call, while the latter only creates the parameter.
+In addition, the typed \fIimadd\fR procedures may be used to update the values
+of existing parameters, i.e., it is not considered an error if the parameter
+already exists. The principal limitation of the typed procedures is that
+they may only be used to add or set parameters of a standard datatype.
+The \fIimaddf\fR procedure will permit creation of parameters with more
+descriptive datatypes (abstract datatypes or domains) when the interface is
+recut upon DBIO. There is no support in the current interface for domains.
+.PP
+The value of any parameter may be fetched with one of the \fIimget\fR functions.
+\fIBe careful not to confuse \fBimgets\fI with \fBimgstr\fI
+(or \fBimputs\fI with \fBimpstr\fI) when
+fetching or storing the string value of a field\fR. Full automatic type
+conversion is provided. Any field may be read or written as a string,
+and the usual type conversions are permitted for the numeric datatypes.
+.PP
+The \fIimaccf\fR function may be used (like the FIO \fIaccess\fR procedure)
+to determine whether a field exists. Fields are deleted with \fIimdelf\fR;
+it is an error to attempt to delete a nonexistent field.
+.PP
+The field name list procedures \fIimofnl[su]\fR, \fIimgnfn\fR,
+and \fIimcfnl\fR procedures are similar to the familiar file template
+facilities, except that the @file notation is not supported. The template
+is expanded upon an image header rather than a directory. Unsorted lists
+are the most useful for image header fields. If sorting is enabled each
+comma delimited pattern in the template is sorted separately, rather than
+globally sorting the entire template after expansion. Minimum match is
+permitted when expanding the template, another difference from file
+templates. Only actual, full length field names are placed in the output
+list.
+.NH 3
+Standard Fields
+.PP
+The database interface may be used to access any field of the image header,
+including the following standard fields. Note that the nomenclature has
+been changed slightly to make it more consistent with FITS. Additional
+standard fields will be defined in the future. These names and their
+usage may change in the next release of IMIO.
+.DS
+\fI
+ keyword type description
+\fL
+ i_ctime l time of image creation
+ i_history s history string buffer
+ i_limtime l time when limits (minmax) were last updated
+ i_maxpixval r maximum pixel value
+ i_minpixval r minimum pixel value
+ i_mtime l time of last modify
+ i_naxis i number of axes (dimensionality)
+ i_naxis[1-7] l length of an axis ("i_naxis1", etc.)
+ i_pixfile s pixel storage file
+ i_pixtype i pixel datatype (SPP integer code)
+ i_title s title string
+\fR
+.ce
+\fBStandard Header Fields\fR
+.DE
+.PP
+The names of the standard fields share an "i_" prefix to reduce the possibility
+of collisions with user field names, to identify the standard fields in
+sorted listings, to allow use of pattern matching to discriminate between the
+standard fields and user fields, and so on. For the convenience of the user,
+the "i_" prefix may be omitted provided the resultant name does not match the
+name of a user parameter. It is however recommended that the full name be
+used in all applications software.
+.NH 3
+Restrictions
+.PP
+The use of FITS format as the internal format for storing fields in this
+version of the interface places restrictions on the size of field names and
+of the string value of string valued parameters. Field names are currently
+limited to eight characters or less and case is ignored (since FITS requires
+upper case). The eight character limit does not apply to the standard fields.
+String values are limited to at most 68 characters. If put string is passed
+a longer string it will be silently truncated. Trailing whitespace and
+newlines are chopped when a string value is read.
+
+.NH
+Database Utility Tasks
+.PP
+Two image database utility tasks have been implemented, \fIhedit\fR and
+\fIhselect\fR. \fIHedit\fR is the so called header editor, used to modify,
+add, or delete selected fields of selected images. The \fIhselect\fR task
+is used to select images that satisfy a selection criteria given as a boolean
+expression, printing a subset of the fields of these images on the standard
+output in list form. Manual pages are attached.
+.PP
+Both of these tasks gain most of their power from use of the \fIevexpr\fR
+utility procedure, now available in FMTIO. The \fIevexpr\fR procedure takes
+as input an algebraic expression (character string), parses and evaluates
+the expression, and returns as output the value of the expression.
+.DS
+\fL
+ include <evexpr.h>
+ pointer evexpr()
+
+ o = evexpr (expr, getop, ufcn)
+
+where
+ o Is a pointer to an operand structure
+ expr Is a character string
+ getop Is either NULL or the \fIlocpr\fL address
+ of a user supplied procedure called during
+ expression evaluation to get the value of
+ an external operand.
+ ufcn Is either NULL or the \fIlocpr\fL address
+ of a user supplied procedure called during
+ expression evaluation to satisfy a call to
+ an external function.
+\fR
+.DE
+The operand structure is defined in \fB<evexpr.h>\fR. The best documentation
+currently available for the operators and functions provided by \fIevexpr\fR
+will be found in the manual page(s) for \fIhedit\fR. Additional documentation
+will be found with the sources. The expression evaluation procedure is
+probably the single largest procedure in the system (in terms of kilobytes
+added to an executable) and should not be used unless it is needed, but it can
+greatly increase the power of a task in the right application.
+.CT
+IRAF
+Larry Goad
+George Jacoby
+Richard Wolff
+Steve Ridgway (fyi)
+Jeanette Barnes (fyi)
+Ed Anderson (fyi)
diff --git a/sys/imio/doc/imio.doc b/sys/imio/doc/imio.doc
new file mode 100644
index 00000000..daa72a52
--- /dev/null
+++ b/sys/imio/doc/imio.doc
@@ -0,0 +1,232 @@
+ IRAF IMIO OVERVIEW
+ 7 May 1986
+
+
+
+1. DATA MANAGEMENT ROUTINES
+
+include <imhdr.h>
+
+ im = immap (image, mode, oim)
+ imunmap (im)
+
+ imdelete (image)
+ imrename (oldname, newname)
+
+where
+ struct imhdr *im, *oim; image header/descriptor
+ char image[]; image name or image section
+ int mode; ro, wo, rw, new_image, new_copy
+
+important header parameters:
+
+ im->im_naxis number of axes
+ im->im_axlen[i] axis lengths
+ im->im_pixtype pixel datatype
+ im->im_datamin min pixel value
+ im->im_datamax max pixel value
+ im->im_title title string
+
+
+Existing images are normally opened either read only or read write.
+New images are opened either new_image or new_copy. In the latter case,
+the third argument is a pointer to the image descriptor of an existing
+image, with the new image inheriting the non-data attributes of the
+existing image header. This latter feature is important for data
+independence.
+
+The IMIO interface supports images of up to naxis=7. In a sense, all images
+are multidimensional, with the higher, unused axis lengths set to 1.
+An N dimensional image may therefore be accessed by a program coded to
+operate upon an M dimensional image.
+
+The image section facility greatly inceases the flexibility of the IMIO
+interface. Image sections are specified as part of the image name input
+to IMOPEN, and are not visible to the applications program, which sees
+a somewhat smaller image, or an image of lesser dimensionality. Some examples
+are shown below.
+
+
+ section refers to
+
+ pix[] whole image
+ pix[i,j] the pixel value (scalar) at [i,j]
+ pix[*,*] whole image, two dimensions
+ pix[*,-*] flip y-axis
+ pix[*,*,b] band B of three dimensional image
+ pix[*,*:s] subsample in y by S
+ pix[*,l] line L of image
+ pix[c,*] column C of image
+ pix[i1:i2,j1:j2] subraster of image
+ pix[i1:i2:sx,j1:j2:sy] subraster with subsampling
+
+
+Sections are implemented by defining a linear transformation upon the
+pixel coordinates input when image i/o takes place. All image data
+transfers can be represented as subrasters defined by corner points
+pointed to by the vectors VS and VE, each of length NAXIS. If an image
+section is specified, the IMIO i/o routines merely transform these
+vectors into PVS and PVE, the physical coordinates of the referenced
+subraster, before doing any i/o.
+
+
+2. IMIO OPTIONS
+
+ IMIO options may be set and queried with the IMSET and IMSTAT procedures,
+shown below.
+
+ imseti (im, option, int_value)
+ imsetr (im, option, real_value)
+
+ int = imstati (im, option)
+ real = imstatr (im, option)
+
+The options currently supported are shown below (from <imset.h>).
+
+
+# IMSET.H -- Definitions for IMIO user settable options
+
+define IM_ADVICE 1 # RANDOM or SEQUENTIAL
+define IM_NBUFS 2 # number of input buffers
+define IM_COMPRESS 3 # align lines on device blocks?
+define IM_NBNDRYPIX 4 # width of boundary region
+define IM_TYBNDRY 5 # type of boundary extension
+define IM_FLAGBADPIX 6 # set bad pix to INDEF
+define IM_PIXFD 7 # pixfile fd (special devices)
+define IM_WHEADER 8 # update image header at unmap time
+define IM_BNDRYPIXVAL 9 # for option IM_CONSTANT
+
+
+# Types of Boundary Extension
+
+define BT_CONSTANT 1 # return constant if out of bounds
+define BT_NEAREST 2 # return nearest boundary pixel
+define BT_REFLECT 3 # reflect back into image
+define BT_WRAP 4 # wrap around to other side
+define BT_PROJECT 5 # project about boundary
+
+
+The most useful options are the multiple input buffers and boundary extension,
+used to implement filtering operators.
+
+
+3. IMIO I/O ROUTINES
+
+ There are two basic approaches used in image interfaces: images may be
+mapped into virtual memory, or accessed via conventional file i/o. IMIO
+provides both but emphasizes the latter, since it is more portable, more
+efficient for sequential image operations, and because it provides data
+independence. All buffering is handled internally by the interface to
+simplify the interface (externally), and to provide the control necessary
+for sophisticated features and optimizations.
+
+IMIO currently provides three classes of i/o routines: [1] get/put line
+random, [2] get/put line sequential, and [3] get/put subraster random.
+
+
+3.1 Get/Put Line Random (for images of known dimension)
+
+ ptr = im[gp]l[123][usilrdx] (im, line [, band [, ...]])
+e.g.,
+ (short *) = imgl1s (im) # get line from 1d short image
+ (short *) = imgl2s (im, lineno) # get line from 2d short image
+ (real *) = imgl2r (im, lineno) # get line from 2d real image
+ (short *) = impl2s (im, lineno) # put line to 2d short image
+ (real *) = imgl3r (im, line, band) # get from 3d image
+
+
+3.2 Get/Put Line Sequential (for images of any dimension)
+
+ int = im[gp]nl[usilrdx] (im, ptr, v)
+e.g.,
+ (EOF?) = imgnlr (im, ptr, v) # get next line, real
+ (EOF?) = impnls (im, ptr, v) # put next line, short
+
+Here, STAT is either EOF or not EOF, with EOF being returned when the last
+line of the image has been read. The output argument PTR is set to point
+to the buffer containing the input pixels, or the buffer into which the
+output pixels are to be written. The vector V, of length IM_MAXDIM, points
+to the next line of the image to be read. It is set initially to [1,1,1,...]
+by the user (assuming the entire image is to be accessed), and is automatically
+updated by IMIO in each call.
+
+
+3.3 Get/Put Subraster Random
+
+ ptr = im[gp]s[123][usilrdx]
+e.g.,
+ (short *) = imgs2s (im, x1,x2, y1,y2) # get 2d subraster, short
+ (real *) = imps1r (im, x1,x2) # put 1d subraster, real
+
+These routines (and indeed all the i/o routines) can be used for either
+sequential or random accesses. The subraster routines must be used to
+reference outside the boundary in X.
+
+
+3.4 Other I/O Routines
+
+ Other, lower level routines are provided for unusual applications for which
+the above routines are not suited.
+
+ ptr = im[pg]gs[usilrdx] (im, vs, ve, ndim)
+
+The above puts/gets a general section of any dimension. The vectors VS and VE
+define the starting and ending corners of the subraster to be accessed.
+An IMFLUSH routine is provided for flushing the output buffer (remember the
+delayed write).
+
+
+In all cases, no buffers are allocated until i/o takes place, allowing IMSET
+calls to set options after the image has been opened. In the case of a new
+image (or new copy image), the pixel file is not allocated until i/o takes
+place, giving the user time to set the number of axes, size of each axis,
+pixel type, etc. after the image has been opened.
+
+
+4. STORAGE FORMATS
+
+ IMIO stores images on disk in line storage mode, like a multidimensional
+Fortran array. Image lines are normally padded out to an integral number
+of disk blocks to increase i/o efficiency. We store the header information
+separately from the pixels, since the header is variable length. The pixel
+storage file is preallocated and fixed in size. We call this a "static file".
+A special FIO driver is provided for static files to provide optimal i/o.
+Since the file is not dynamically extended at run time and the physical
+blocks allocated for the file do not move about, it is possible to bypass
+the host files system to directly access the data with a low level interface.
+
+
+5. EXAMPLE (SPP)
+
+
+# IMCOPY -- Copy an image. The header information is preserved. The output
+# image has the same size, dimensionality, and pixel type as the input image.
+# An image section may however be used to copy a subsection of the input image.
+
+procedure imcopy (in_image, out_image)
+
+char in_image[ARB]
+char out_image[ARB]
+
+int npix
+long v1[IM_MAXDIM], v2[IM_MAXDIM]
+pointer in, out, l1, l2
+pointer immap(), imgnlr(), impnlr()
+
+begin
+ # Open/create the images.
+ in = immap (in_image, READ_ONLY, 0)
+ out = immap (out_image, NEW_COPY, im_a)
+
+ # Initialize position vectors to line 1, column 1, band 1, ...
+ call amovkl (long(1), v1, IM_MAXDIM)
+ call amovkl (long(1), v2, IM_MAXDIM)
+ npix = IM_LEN(in,1)
+
+ # Copy the image.
+ while (imgnlr (in, l1, v1) != EOF && impnlr (out, l2, v2) != EOF)
+ call amovr (Memr[l1], Memr[l2], npix)
+
+ call imunmap (in)
+ call imunmap (out)
+end
diff --git a/sys/imio/doc/imio.hlp b/sys/imio/doc/imio.hlp
new file mode 100644
index 00000000..8eb96159
--- /dev/null
+++ b/sys/imio/doc/imio.hlp
@@ -0,0 +1,1185 @@
+.help imio May83 "Image I/O Routines"
+.sh
+The Image Header
+
+ The major difference between the prototype IMIO interface, and the final
+interface, concerns the way in which the image header is implemented and
+accessed. In the prototype version, we will simply read the entire header
+into core and access it as an ordinary (dynamically allocated) structure.
+
+
+.nf
+ ptr = immap (fname, mode, hdrsize/hdrptr)
+ imunmap (hdrptr)
+.fi
+
+
+The final resolution of how image headers are implemented depends on how
+we decide to implement virtual structures in the spp language. The immap
+calls, and the techniques used to access the fields of the image header,
+can be expected to change.
+
+.sh
+Pixel I/O
+
+ The calling sequences for the i/o routines, described below, hopefully will
+not have to be changed. We will eventually add GETPIX and PUTPIX statements
+to the subset preprocessor language, to automatically generate the appropriate
+low level calls.
+
+A generic, polymorphic GETPIX or PUTPIX statement is translated into a
+reference to a low level Fortran function. The transformation is governed
+by the following subprogram name generating function:
+
+
+.rj (108 total)
+GETPIX, PUTPIX --> im[gp][pls][123][silrdx]
+
+
+.ks
+.nf
+For example (get, type real):
+
+ ptr = imgp1r (hdrptr, x, npix) # get pixels
+ ptr = imgp2r (hdrptr, x, y, npix)
+ ptr = imgp3r (hdrptr, x, y, z, npix)
+
+ ptr = imgl1r (hdrptr) # get line
+ ptr = imgl2r (hdrptr, y)
+ ptr = imgl3r (hdrptr, y, z)
+
+ ptr = imgs1r (hdrptr, x1, x2) # get section
+ ptr = imgs2r (hdrptr, x1, x2, y1, y2)
+ ptr = imgs3r (hdrptr, x1, x2, y1, y2, z1, z2)
+.fi
+.ke
+
+
+The IM?P?? procedures access a list of pixels, the coordinates of which
+are given by the X, Y, Z, etc. arrays given as arguments. Note that random
+access of individual pixels is provided as a special case (npix=1).
+
+The IM?L?? routines access the lines of an image, and the IM?S?? routines
+operate on general, but connected, subsections of images.
+
+
+.sh
+Restrictions Imposed by the Initial Prototype:
+
+ IMMAP, IMMAPNC, IMUNMAP will be implemented for image headers that are
+simple binary structures (not self describing), subject to the restriction
+that a file may contain only a single header. An arbitrary selection of user
+defined fields will follow the standard header. The entire header will
+be read into core and accessed as a simple incore structure.
+
+The pixels, and other variable size image substructures, will be stored
+in separate files, as in the general plan. All of the standard data types
+will be implemented in the disk space. The initial implementation will
+support only type REAL pixels in program space.
+
+The following i/o routines will be implemented in the first release of
+the prototype:
+
+.rj (12 total)
+ im[gp][sc][123][r]
+
+In words, we will be able to read and write lines and sections, with the
+applications program manipulating type REAL pixels internally. The full
+range of data types will be permitted in the image file as stored on disk.
+Up to three dimensional images are permitted.
+
+.sh
+IMSET Options
+
+ The prototype need not provide multiple buffering and boundary extension
+initially.
+
+.sh
+Implementation
+
+ Little effort should be made to make the prototype optimal. All
+buffering will be locally allocated, and data will be copied to and from
+the FIO buffers (the FIO buffers will not be directly accessed). Special
+cases will not be optimized. The most general entry points are IMGGSR
+and IMPGSR (get/put general section). Initially, all of the other entry
+points can be defined in terms of these.
+
+
+.ks
+.nf
+Structure of the input procedures (type REAL):
+
+ imgl1r
+ imgl2r
+ imgl3r
+ imgs1r
+ imgs2r
+ imgs3r
+ imggsr
+ imggsc
+ imgibf
+ imopsf
+ calloc
+ imcssz
+ realloc
+ malloc
+ mfree
+ imsslv
+ imrdpx
+ imsoob
+ imnote
+ seek
+ read
+ imflip
+ imupkr
+
+ (datatype dependent) | (datatype independent)
+.fi
+.ke
+
+
+
+The output procedures are structured somewhat differently, because the
+transfer of a section occurs sometime after a "put section" returns,
+rather than immediately as in the input procedures. Since the output
+is buffered for a delayed write, we must have an IMFLUSH entry point, and
+IMUNMAP must flush the output buffer before unmapping an image.
+
+
+.ks
+.nf
+Structure of the output procedures (type REAL):
+
+ impl1r
+ impl2r
+ impl3r
+ imps1r imunmap
+ imps2r |
+ imps3r |
+ impgsr |
+ imflush
+ imflsr
+ imflsh
+ imflip
+ imwrpx
+ imsoob
+ imnote
+ imwrite
+ fstatus
+ seek
+ write
+ impakr
+
+ imgobf
+ imopsf
+ calloc
+ imcssz
+ realloc
+ malloc
+ mfree
+
+ (datatype dependent) | (datatype independent)
+.fi
+.ke
+
+
+.sh
+Semicode for the Basic I/O Routines
+
+ The IMGGS? and IMPGS? procedures get and put general N-dimensional
+image sections of a specific datatype. There is no intrinsic limit on
+the maximum number of dimensions, and the full range (8) of disk datatypes
+are easily supported. The subscript for a particular dimension may run
+either forward or backward. The semicode is written generically, allowing
+code to be machine generated for all program space datatypes (6).
+
+We do not address the problems of boundary extension and multiple buffering
+here, but these features can be easily added in the future. This version
+of IMIO assumes that pixels are stored on disk in line storage mode, with
+no interlacing of bands.
+
+
+IMGGS? gets a general section, and converts it to the datatype indicated
+by the type suffix.
+
+
+.ks
+.nf
+pointer procedure imggs$t (imdes, vs, ve)
+
+imdes pointer to image descriptor structure
+vs,ve coordinates of starting and ending points
+
+begin
+ bp = imggsc (imdes, vs, ve, TY_PIXEL, totpix)
+ if (imdes.pixtype != TY_PIXEL)
+ call imupk$t (*bp, *bp, totpix, imdes.pixtype)
+ return (coerce (bp, TY_CHAR, TY_PIXEL))
+end
+.fi
+.ke
+
+
+
+IMGGSC gets a general section from an imagefile into a buffer. Upon
+exit, the buffer contains the requested section, with the pixels still
+in the same datatype they were in the imagefile. The buffer is made
+large enough to accommodate the pixels in either datatype.
+
+
+.ks
+.nf
+pointer procedure imggsc (imdes, vs, ve, dtype, totpix)
+
+imdes pointer to image descriptor structure
+vs,ve coordinates of starting and ending points
+dtype datatype of pixels required by calling program
+bp pointer to CHAR buffer to hold pixels
+
+begin
+ # Get an (input) buffer to put the pixels into. Prepare the
+ # section descriptor vectors V, VINC.
+
+ bp = imgibf (imdes, vs, ve, dtype)
+ call imsslv (imdes, vs, ve, v, vinc, npix)
+
+ # Extract the pixels. IMRPIX reads a contiguous array of
+ # pixels into the buffer at the specified offset, incrementing
+ # the offset when done. The pixels are type converted if
+ # necessary.
+
+ offset = 0
+
+ repeat {
+ call imrdpx (imdes, *(bp+offset), v, npix)
+ if (vinc[1] < 0)
+ call imflip (*(bp+offset), npix, sizeof(imdes.pixtype))
+ offset = offset + npix
+
+ for (d=2; d <= imdes.ndim; d=d+1) {
+ v[d] += vinc[d]
+ if ((v[d] - ve[d] == vinc[d]) && d < imdes.ndim)
+ v[d] = vs[d]
+ else {
+ d = 0
+ break
+ }
+ }
+ } until (d >= imdes.ndim)
+
+ totpix = offset
+ return (bp)
+end
+.fi
+.ke
+
+
+
+
+
+Prepare the section descriptor vectors V and VINC. V is a vector specifying
+the coordinates at which the next i/o transfer will take place. VINC is
+a vector specifying the loop step size.
+
+
+.ks
+.nf
+procedure imsslv (imdes, vs, ve, v, vinc, npix)
+
+begin
+ # Determine the direction in which each dimension is to be
+ # traversed.
+
+ do i = 1, imdes.ndim
+ if (vs[i] <= ve[i])
+ vinc[i] = 1
+ else
+ vinc[i] = -1
+
+ # Initialize the extraction vector (passed to IMRDS? to read a
+ # contiguous array of pixels). Compute length of a line.
+
+ do i = 1, imdes.ndim
+ v[i] = vs[i]
+
+ if (vs[1] > ve[1]) {
+ v[1] = ve[1]
+ npix = vs[1] - ve[1] + 1
+ } else
+ npix = ve[1] - vs[1] + 1
+end
+.fi
+.ke
+
+
+
+
+
+The put-section procedure must write the contents of the output buffer
+to the image, using the section parameters saved during the previous call.
+The new section parameters are then saved, and the buffer pointer is
+returned to the calling program. The calling program subsequently fills
+the buffer, and the sequence repeats.
+
+
+.ks
+.nf
+pointer procedure impgs$t (imdes, vs, ve)
+
+imdes pointer to image descriptor structure
+vs,ve coordinates of starting and ending points
+
+begin
+ # Flush the output buffer, if appropriate. IMFLUSH calls
+ # one of the IMFLS? routines, which write out the section.
+
+ call imflush (imhdr)
+
+ # Get an (output) buffer to put the pixels into. Save the
+ # section parameters in the image descriptor. Save the epa
+ # of the typed flush procedure in the image descriptor.
+
+ bp = imgobf (imdes, vs, ve, TY_PIXEL)
+ imdes.flush_epa = loc (imfls$t)
+
+ return (bp)
+end
+.fi
+.ke
+
+
+
+Flush the output buffer, if a put procedure has been called, and the
+buffer has not yet been flushed. The output buffer is flushed automatically
+whenever a put procedure is called, when an image is unmapped, or when
+the applications program calls IMFLUSH.
+
+
+.ks
+.nf
+procedure imfls$t (imdes)
+
+begin
+ # Ignore the flush request if the output buffer has already been
+ # flushed.
+
+ if (imdes.flush == YES) {
+ bdes = imdes.obdes
+ bp = bdes.bufptr
+
+ # Convert datatype of pixels, if necessary, and flush buffer.
+ if (imdes.pixtype != TY_PIXEL)
+ call impak$t (*bp, *bp, bdes.npix, imdes.pixtype)
+ call imflsh (imdes)
+
+ imdes.flush = NO
+ }
+end
+.fi
+.ke
+
+
+.ks
+.nf
+procedure imflsh (imdes)
+
+begin
+ # Determine the direction in which each dimension is to be
+ # traversed.
+
+ bdes = imdes.obdes
+ call imsslv (imdes, bdes.vs, bdes.ve, v, vinc, npix)
+
+ # Write out the pixels. IMWRPX writes a contiguous array of
+ # pixels at the specified offset.
+
+ offset = 0
+
+ repeat {
+ if (vinc[1] < 0)
+ call imflip (*(bp+offset), npix, sizeof(imdes.pixtype))
+ call imwrpx (imdes, *(bp+offset), v, npix)
+ offset = offset + npix
+
+ for (d=2; d <= imdes.ndim; d=d+1) {
+ v[d] += vinc[d]
+ if ((v[d] - ve[d] == vinc[d]) && d < imdes.ndim)
+ v[d] = vs[d]
+ else {
+ d = 0
+ break
+ }
+ }
+ } until (d >= imdes.ndim)
+end
+.fi
+.ke
+
+
+
+
+Read a contiguous array of NPIX pixels, starting at the point defined by
+the vector V, into the callers buffer.
+
+
+.ks
+.nf
+procedure imrdpx (imdes, buf, v, npix)
+
+begin
+ # Check that the access does not reference out of bounds.
+
+ if (imsoob (imdes, v, npix))
+ call imerr (imname, subscript_out_of_range)
+
+ # Seek to the point V in the pixel storage file. Compute size
+ # of transfer.
+
+ call seek (imdes.pfd, imnote (imdes, v))
+ nchars = npix * sizeof (imdes.pixtype)
+
+ # Read in the data.
+ if (read (imdes.pfd, buf, nchars, junk) != nchars)
+ call imerr (imname, missing_pixels)
+end
+.fi
+.ke
+
+
+
+Write a contiguous array of NPIX pixels, starting at the point defined by
+the vector V, into the pixel storage file.
+
+
+.ks
+.nf
+procedure imwrpx (imdes, buf, v, npix)
+
+begin
+ # Check that the access does not reference out of bounds.
+
+ if (imsoob (imdes, v, npix))
+ call imerr (imname, subscript_out_of_range)
+
+ # Seek to the point V in the pixel storage file. Note that
+ # when writing to a new image, the next transfer may occur
+ # at a point beyond the current end of file. If so, write
+ # out zeros until the desired offset (which is in bounds)
+ # is reached.
+
+ file_offset = imnote (imdes, v)
+ if (file_offset > imdes.file_size)
+ [write zeros until the desired offset is reached]
+ else
+ call seek (imdes.pfd, file_offset)
+
+ # Compute size of transfer. If transferring an entire line,
+ # increase size of transfer to the physical line length,
+ # to avoid having to enblock the data. NOTE: buffer must
+ # be large enough to guarantee no memory violation.
+
+ if (v[1] == 1 && npix == imdes.len[1])
+ nchars = imdes.physlen[1] * sizeof (imdes.pixtype)
+ else
+ nchars = npix * sizeof (imdes.pixtype)
+
+ call write (imdes.pfd, buf, nchars)
+ imdes.file_size = max (imdes.file_size, file_offset+nchars)
+end
+.fi
+.ke
+
+
+
+
+IMNOTE computes the physical offset of a particular pixel in the
+pixel storage file. If the disk datatype is UBYTE, this is the offset
+of the char containing the subscripted byte.
+
+
+.ks
+.nf
+long procedure imnote (imdes, v)
+
+begin
+ pixel_offset = v[1]
+ for (i=2; i <= imdes.ndim; i=i+1)
+ pixel_offset += (v[i]-1) * imdes.physlen[i]
+
+ char_offset0 = (pixel_offset-1) * sizeof (imdes.pixtype)
+ return (imdes.pixoff + char_offset0)
+end
+.fi
+.ke
+
+
+
+Convert a vector of any datatype to type PIXEL ($t). The input and
+output vectors may be the same, without loss of data. The input and
+output datatypes may be the same, in which case no conversion is
+performed.
+
+
+.ks
+.nf
+procedure imupk$t (a, b, npix, dtype)
+
+begin
+ switch (dtype) {
+ case TY_USHORT:
+ call achtu$t (a, b, npix)
+ case TY_SHORT:
+ call achts$t (a, b, npix)
+ case TY_INT:
+ call achti$t (a, b, npix)
+ case TY_LONG:
+ call achtl$t (a, b, npix)
+ case TY_REAL:
+ call achtr$t (a, b, npix)
+ case TY_DOUBLE:
+ call achtd$t (a, b, npix)
+ case TY_COMPLEX:
+ call achtx$t (a, b, npix)
+ default:
+ call syserr (unknown_datatype_in_imagefile)
+ }
+end
+.fi
+.ke
+
+
+
+Convert a vector of type PIXEL ($t) to any datatype. The input and
+output vectors may be the same, without loss of data. The input and
+output datatypes may be the same, in which case no conversion is
+performed.
+
+
+.ks
+.nf
+procedure impak$t (a, b, npix, dtype)
+
+begin
+ switch (dtype) {
+ case TY_USHORT:
+ call acht$tu (a, b, npix)
+ case TY_SHORT:
+ call acht$ts (a, b, npix)
+ case TY_INT:
+ call acht$ti (a, b, npix)
+ case TY_LONG:
+ call acht$tl (a, b, npix)
+ case TY_REAL:
+ call acht$tr (a, b, npix)
+ case TY_DOUBLE:
+ call acht$td (a, b, npix)
+ case TY_COMPLEX:
+ call acht$tx (a, b, npix)
+ default:
+ call syserr (unknown_datatype_in_imagefile)
+ }
+end
+.fi
+.ke
+
+
+.sh
+Data Structure Management
+
+ When an image is mapped, buffer space is allocated for a copy of
+the image header, and for the image descriptor (used by IMIO while an
+image is mapped). When the first i/o transfer is done on an image,
+either an input or an output data buffer will be created. The size of
+this buffer is governed by the size of the transfer, and by the datatypes
+of the pixels on disk and in program space.
+
+If a new image is being written, the pixel storage file is created at
+the time of the first PUTPIX operation. The physical characteristics
+of the new image, defined by the image header of the new image, are
+unalterable once the first i/o operation has occurred. Accordingly,
+the number of dimensions, length of the dimensions, datatype of the
+pixels on disk, and so on must be set (in the image header structure)
+before writing to the new image.
+
+The only exception to this rule may be the addition of new lines to a
+two dimensional image stored in line storage mode, or the addition of
+new bands to a multiband image stored in band sequential (noninterlaced)
+mode. It is not always possible to modify the dimensions or size of
+an existing image.
+
+It is possible to preallocate space for an image (using FALOC). This
+may result in a more nearly contiguous file, and may make writing a
+new image slightly more efficient, since it will not be necessary
+to write blocks of zeros in IMPGS?. Preallocation will occur
+automatically in systems where it is desirable.
+
+.sh
+Pixel Buffer Management
+
+ There may be any number of input buffers per image, but only a single
+output buffer. By default there is only a single input buffer. The input
+and output buffers are distinct: the same buffer is never used for both
+input and output (unlike FIO).
+
+The size of a buffer may range from one pixel, to the entire image (or
+larger if boundary extension is in use). If multiple buffers are in use,
+all buffers do not have to be the same size. The size of a buffer may
+vary from one GETPIX or PUTPIX call to the next.
+
+If multiple input buffers are in use, buffers are allocated in a strictly
+round robin fashion, one per GETPIX call. Several buffers may contain
+data from the same part of the image. Once the desired number of buffers
+have been filled, a buffer "goes away" with each subsequent GETPIX call.
+
+
+IMGIBF gets an input buffer. When called to get a line or section,
+the vectors VS and VE specify the subsection to be extracted.
+This information is saved in the buffer descriptor, along with the
+datatype of the pixels and the dimension of the section.
+
+When IMGIBF is called to get a list of pixels, VS and VE would have to be
+replaced by a set of NPIX such vectors, to fully specify the section.
+It is impractical to save this much information in the buffer descriptor,
+so when creating a buffer to contain a list of pixels, VS and VE are faked
+to indicate a one dimensional section of the appropriate size.
+
+
+
+
+.ks
+.nf
+pointer procedure imgibf (imdes, vs, ve, dtype)
+
+imdes image descriptor
+vs,ve define the number of pixels to be buffered
+dtype the datatype of the pixels in the program
+
+begin
+ # If first input transfer, allocate and initialize array of
+ # input buffer descriptors.
+
+ if (imdes.ibdes == NULL) {
+ call imopsf (imdes)
+ call calloc (imdes.ibdes, LEN_BDES * imdes.nbufs, TY_STRUCT)
+ }
+
+ # Compute pointer to the next input buffer descriptor.
+ # Increment NGET, the count of the number of GETPIX calls.
+
+ bdes = &imdes.ibdes [mod (imdes.nget, imdes.nbuf) + 1]
+ imdes.nget += 1
+
+ # Compute the size of the buffer needed. Check buffer
+ # descriptor to see if the old buffer is the right size.
+ # If so, use it, otherwise make a new one.
+
+ nchars = imcssz (imdes, vs, ve, dtype)
+
+ if (nchars < bdes.bufsize)
+ call realloc (bdes.bufptr, nchars, TY_CHAR)
+ else if (nchars > bdes.bufsize) {
+ call mfree (bdes.bufptr, TY_CHAR)
+ call malloc (bdes.bufptr, nchars, TY_CHAR)
+ }
+
+ # Save section coordinates, datatype in buffer descriptor, and
+ # return buffer pointer to calling program.
+
+ bdes.bufsize = nchars
+ bdes.dtype = dtype
+ bdes.npix = totpix
+
+ do i = 1, imdes.ndim {
+ bdes.vs[i] = vs[i]
+ bdes.ve[i] = ve[i]
+ }
+
+ return (coerce (bdes.bufptr, TY_CHAR, dtype)
+end
+.fi
+.ke
+
+
+
+
+.ks
+.nf
+pointer procedure imgobf (imdes, vs, ve, dtype)
+
+imdes image descriptor
+vs,ve define the number of pixels to be buffered
+dtype the datatype of the pixels in the program
+
+begin
+ # If first write, and if new image, create pixel storage file,
+ # otherwise open pixel storage file. Allocate and initialize
+ # output buffer descriptor.
+
+ if (imdes.obdes == NULL) {
+ call imopsf (imdes)
+ call calloc (imdes.obdes, LEN_BDES, TY_STRUCT)
+ }
+
+ bdes = imdes.obdes
+
+ # Compute the size of buffer needed. Add a few extra chars
+ # to guarantee that there won't be a memory violation when
+ # writing a full physical length line.
+
+ nchars = imcssz (imdes, vs, ve, dtype) +
+ (imdes.physlen[1] - imdes.len[1]) * sizeof (imdes.pixtype)
+
+ if (nchars < bdes.bufsize)
+ call realloc (bdes.bufptr, nchars, TY_CHAR)
+ else if (nchars > bdes.bufsize) {
+ call mfree (bdes.bufptr, TY_CHAR)
+ call malloc (bdes.bufptr, nchars, TY_CHAR)
+ }
+
+ # Save section coordinates, datatype of pixels in buffer
+ # descriptor, and return buffer pointer to calling program.
+
+ bdes.bufsize = nchars
+ bdes.dtype = dtype
+ bdes.npix = totpix
+
+ do i = 1, imdes.ndim {
+ bdes.vs[i] = vs[i]
+ bdes.ve[i] = ve[i]
+ }
+
+ return (coerce (bdes.bufptr, TY_CHAR, dtype)
+end
+.fi
+.ke
+
+
+
+Given two vectors describing the starting and ending coordinates
+of an image section, compute and return the amount of storage needed
+to contain the section. Sufficient storage must be allocated to
+hold the largest datatype pixels which will occupy the buffer.
+
+
+
+.ks
+.nf
+long procedure imcssz (imdes, vs, ve, dtype)
+
+begin
+ pix_size = max (sizeof(imdes.pixtype), sizeof(dtype))
+ npix = 0
+
+ do i = 1, imdes.ndim
+ if (vs[i] <= ve[i])
+ npix *= ve[i] - vs[i] + 1
+ else
+ npix *= vs[i] - ve[i] + 1
+
+ return (npix * pix_size)
+end
+.fi
+.ke
+
+
+.sh
+Mapping and unmapping Image Structures
+
+ An imagefile must be "mapped" to an image structure before the
+structure can be accessed. The map operation associates a file with
+a defined structure.
+
+The IMMAP procedure must allocate a buffer for the image header
+structure, and for the image descriptor structure. If an existing
+imagefile is being mapped, the header is copied into memory from
+the imagefile. If a new image is being mapped, the header structure
+is allocated and initialized.
+
+If an image is being mapped as a "new copy", a new header
+structure is created which is a copy of the header of an image which
+has already been mapped. The entire image header, including any
+application specific fields, is copied.
+
+After copying an image header for a NEW_COPY image, the header field
+containing the name of the pixel storage file is cleared. A "new copy"
+image structure does not inherit any pixels. Any similar substructures
+which describe the attributes of the pixels (i.e., the blank pixel
+list, the histogram) must also be initialized.
+
+Note that the "image descriptor" buffer allocated below actually
+contains the image descriptor, followed by the standard image header
+(at offset IMHDR_OFF), followed by any user fields. If an existing
+image structure is being mapped, the caller supplies the length of
+the user area of the header as the third argument to IMMAP.
+
+IMMAP returns a pointer to the first field of the standard header
+as the function value. The image descriptor is invisible to the
+calling program.
+
+
+
+.ks
+.nf
+pointer procedure immap (fname, mode, hdr_arg)
+
+begin
+ # Add code here to handle section suffixes in imagefile
+ # name strings (e.g. "image[*,5]").
+
+ # Open image header file.
+ hfd = open (fname, mmap[mode], BINARY_FILE)
+
+ # Allocate buffer for image descriptor/image header. Note
+ # the dual use of the HDR_ARG argument.
+
+ if (mode == NEW_COPY)
+ sz_imhdr = hdr_arg.sz_imhdr
+ else
+ sz_imhdr = (LEN_IMHDR + int(hdr_arg)) * SZ_STRUCT
+
+ call calloc (imdes, SZ_IMDES + sz_imhdr, TY_STRUCT)
+ imhdr = imdes + IMHDR_OFF
+
+ [initialize the image descriptor, including the default
+ image section (optionally set by user with suffix above).]
+
+ # Initialize the mode dependent fields of the image header.
+ switch (mode) {
+ case NEW_COPY:
+ call im_init_newcopy (imdes, hdr_arg)
+ case NEW_IMAGE:
+ [initialize the image header]
+ default:
+ call seek (hfd, BOFL)
+ n = read (hfd, Memi[imhdr], sz_imhdr)
+ if (n < SZ_IMHDR || strne (IM_MAGIC(imhdr), "imhdr")) {
+ call mfree (imdes)
+ call imerr (fname, file_not_an_imagefile)
+ } else if (mode == READ_ONLY)
+ call close (hfd)
+ }
+
+ [initialize those fields of the image header which are not
+ dependent on the mode of access.]
+
+ return (imhdr_pointer)
+end
+.fi
+.ke
+
+
+
+
+.ks
+.nf
+procedure imunmap (imhdr)
+
+begin
+ imdes = imhdr - IMHDR_OFF
+
+ # Flush the output buffer, if necessary.
+ call imflush (imhdr)
+
+ # Append the bad pixel list.
+ if (the bad pixel list has been modified) {
+ if (file_size < blist_offset)
+ [write out zeros until the offset of the bad pixel
+ list is reached]
+ [append the bad pixel list]
+ [free buffer space used by bad pixel list]
+ }
+
+ call close (imdes.pfd)
+
+ # Update the image header, if necessary (count of bad pixels,
+ # minimum and maximum pixel values, etc.).
+
+ if (imdes.update == YES) {
+ if (no write permission on image)
+ call imerr (imname, cannot_update_imhdr)
+ call imuphdr (imdes)
+ call close (imdes.hfd)
+ }
+
+ # Free buffer space.
+ for (i=1; i <= imdes.nbufs; i=i+1)
+ call mfree (imdes.ibdes[i].bufptr, TY_CHAR)
+ call mfree (imdes.obdes.bufptr, TY_CHAR)
+ call mfree (imdes, TY_STRUCT)
+end
+.fi
+.ke
+
+
+IMFLUSH indirectly references a typed flush procedure, the entry point
+address of which was saved in the image descriptor at the time of the
+last IMPGS? call. The problem here is that IMFLUSH must work properly
+regardless of the data type of the pixels in the output buffer. To
+ensure this, and to avoid having to link in the full matrix of 48 type
+conversion routines, we call LOC in the put-section procedure to reference
+the appropriate typed flush routine.
+
+
+
+.ks
+.nf
+procedure imflush (imhdr)
+
+begin
+ if (imdes.flush == YES)
+ call zcall1 (imdes.flush_epa, imdes)
+end
+.fi
+.ke
+
+
+
+The following procedure is called by the IMGOBF and IMGIBF routines
+to open the pixel storage file, during the first PUTPIX operation on
+a file.
+
+
+.ks
+.nf
+procedure imopsf (imdes)
+
+begin
+ switch (imdes.mode) {
+ case READ_ONLY, READ_WRITE, WRITE_ONLY, APPEND:
+ imdes.pfd = open (imdes.pixfile, imdes.mode, BINARY_FILE)
+ if (read (imdes.pfd, pix_hdr, SZ_PIXHDR) < SZ_IMMAGIC)
+ call imerr (imname, cannot_read_pixel_storage_file)
+ else if (strne (pix_hdr.im_magic, "impix"))
+ call imerr (imname, not_a_pixel_storage_file)
+
+ case NEW_COPY, NEW_FILE, TEMP_FILE:
+ # Get the block size for device "imdir$", and initialize
+ # the physical dimensions of the image, and the absolute
+ # file offsets of the major components of the pixel storage
+ # file.
+
+ dev_block_size = fdevblk ("imdir$")
+ [initialize im_physlen, im_pixels, im_hgmoff fields
+ in image header structure]
+
+ # Open the new pixel storage file (preallocate space if
+ # enabled on local system). Call FADDLN to tell FIO that
+ # the pixfile is subordinate to the header file (for delete,
+ # copy, etc.). Save the physical pathname of the pixfile
+ # in the image header, in case "imdir$" changes.
+
+ call mktemp ("imdir$im", temp, SZ_FNAME)
+ call fpathname (temp, imhdr.pixfile, SZ_PATHNAME)
+
+ if (preallocation of imagefiles is enabled)
+ call falloc (imhdr.pixfile, sz_pixfile)
+ imdes.pfd = open (imdes.pixfile, NEW_FILE, BINARY_FILE)
+ call faddln (imdes.imname, imdes.pixfile)
+
+ # Write small header into pixel storage file. Allows
+ # detection of headerless pixfiles, and reconstruction
+ # of header if it gets lost.
+
+ [write pix_hdr header structure to pixel storage file]
+
+ default:
+ call imerr (imname, illegal_access_mode)
+ }
+end
+.fi
+.ke
+
+
+.sh
+Data Structures
+
+ An imagefile consists of two separate files. The first file contains
+the image header. In the prototype, there may be only a single header per
+header file. The header consists of the standard image header, followed
+by an arbitrary number of user defined fields.
+
+The standard part of the image header has a fixed structure. All the variable
+size components of an image are stored in the pixel storage file. The
+name of the pixel storage file, and the offsets to the various components
+of the image, are stored in the image header. The name of the image header
+file is in turn stored in the header area of the pixel storage file,
+making it possible to detect headerless images.
+
+The pixel storage file contains a small header, followed by the pixels
+(aligned on a block boundary), optionally followed by a fixed size
+histogram, and a variable size bad pixel list.
+
+
+.ks
+.nf
+ Structure of an Imagefile
+
+ --------- ---------
+ | <---- |
+ standard ----> header
+ image |
+ header PIXELS
+ | |
+ user histogram (optional)
+ fields |
+ | bad
+ \|/ pixel (optional)
+ ---------- list
+ |
+ \|/
+ ---------
+.fi
+.ke
+
+
+The image header file, which is small, will reside in the users own
+directory. The pixel storage file is generated and manipulated
+transparently to the applications program and the user, and resides
+in the temporary files system, in the logical directory "imdir$".
+
+Storing the parts of an image in two separate files does cause problems.
+The standard file operators, like DELETE, COPY, RENAME, and so on,
+either cannot be used to manipulate imagefiles, or must know about
+imagefiles.
+
+To solve this problem, without requiring FIO to know anything about IMIO
+or VSIO data structures, two operators will be added to FIO. The first
+will tell FIO that file 'A' has a subordinate file 'B' associated with
+it. Any number of subordinate files may be associated with a file.
+The information will be maintained as a list of file names in an invisible
+text file in the same directory as file 'A'.
+
+The second operator will delete the link to a subordinate file. The FIO
+procedures DELETE and RENAME will check for subordinate files, as will CL
+utilities like COPY.
+
+.sh
+The Standard Image Header
+
+ The standard fields of an image header describe the physical
+characteristics of the image (required to access the pixels), plus
+a few derived or historic attributes, which are commonly associated
+with images as used in scientific applications.
+
+
+.ks
+.nf
+struct imhdr {
+ char im_magic[5] # contains the string "imhdr"
+ long im_hdrlen # length of image header
+ int im_pixtype # datatype of the pixels
+ int im_ndim # number of dimensions
+ long im_len[MAXDIM] # length of the dimensions
+ long im_physlen[MAXDIM] # physical length (as stored)
+ long im_pixels # offset of the pixels
+ long im_hgmoff # offset of hgm pixels
+ long im_blist # offset of bad pixel list
+ long im_szblist # size of bad pixel list
+ long im_nbpix # number of bad pixels
+ long im_cdate # date of image creation
+ long im_mdate # date of last modify
+ real im_max # max pixel value
+ real im_min # min pixel value
+ struct histogram im_hgm # histogram descriptor
+ struct coord_tran im_coord # coordinate transformations
+ char im_pixfile[SZ_PATHNAME] # name of pixel storage file
+ char im_name[SZ_IMNAME] # image name string
+ char im_history[SZ_IMHIST] # history comment string
+}
+.fi
+.ke
+
+
+
+The histogram structure, if valid, tells where in the pixel storage file
+the histogram is stored, and in addition summarizes the principal
+attributes of the histogram. All of these quantities are directly
+calculable, except for the last three. The modal value is determined
+by centering on the (major) peak of the histogram. LCUT and HCUT define
+an area, centered on the modal value, which contains a certain fraction
+of the total integral.
+
+
+.ks
+.nf
+struct histogram {
+ int hgm_valid # YES if histogram is valid
+ int hgm_len # number of bins in hgm
+ long hgm_npix # npix used to compute hgm
+ real hgm_min # min hgm value
+ real hgm_max # max hgm value
+ real hgm_integral # integral of hgm
+ real hgm_mean # mean value
+ real hgm_variance # variance about mean
+ real hgm_skewness # skewness of hgm
+ real hgm_mode # modal value of hgm
+ real hgm_lcut # low cutoff value
+ real hgm_hcut # high cutoff value
+}
+.fi
+.ke
+
+
+
+The coordinate transformation descriptor is used to map pixel coordinates
+to some user defined virtual coordinate system, (useful when displaying the
+contents of an image). For lack of a significantly better scheme, we have
+simply adopted the descriptor defined by the FITS standard.
+
+
+.ks
+.nf
+struct coord_tran {
+ real im_bscale # pixval scale factor
+ real im_bzero # pixval offset
+ real im_crval[MAXDIM] # value at pixel
+ real im_crpix[MAXDIM] # index of pixel
+ real im_cdelt[MAXDIM] # increment along axis
+ real im_crota[MAXDIM] # rotation angle
+ char im_bunit[SZ_BUNIT] # pixval ("brightness") units
+ char im_ctype[SZ_IMCTYPE,MAXDIM] # coord units
+}
+.fi
+.ke
+
+
+
+The image and buffer descriptors are used internally by IMIO while
+doing i/o on a mapped image. The image descriptor structure is
+allocated immediately before the image header, is transparent to the
+applications program, and is used to maintain runtime data, which
+does not belong in the image header.
+
+
+.ks
+.nf
+struct image_descriptor {
+ long file_size # size of pixfile
+ long nget # number getpix calls
+ int nbufs # number of in buffers
+ int flush # flush outbuf?
+ int update # update header?
+ int pfd # pixfile fd
+ int hfd # header file fd
+ int flush_epa # epa of imfls? routine
+ struct buffer_descriptor *ibdes # input bufdes
+ struct buffer_descriptor *obdes # output bufdes
+ char imname[SZ_FNAME] # imagefile name
+}
+.fi
+.ke
+
+
+
+.ks
+.nf
+struct buffer_descriptor {
+ char *bufptr # buffer pointer
+ int dtype # datatype of pixels
+ long npix # number of pixels in buf
+ long bufsize # buffer size, chars
+ long vs[MAXDIM] # section start vector
+ long ve[MAXDIM] # section end vector
+}
+.fi
+.ke
diff --git a/sys/imio/doc/imio.ms b/sys/imio/doc/imio.ms
new file mode 100644
index 00000000..2302a8ff
--- /dev/null
+++ b/sys/imio/doc/imio.ms
@@ -0,0 +1,295 @@
+.ce
+\fBThe IRAF Image I/O Interface\fR
+.ce
+\fIDesign Strategies\fR
+.ce
+\fIStatus and Plans\fR
+.sp
+.ce
+Doug Tody
+.ce
+November 1983
+.sp 3
+.NH
+Introduction
+.PP
+Bulk data arrays are accessed in IRAF SPP programs via the Image I/O
+(IMIO) interface. IMIO is used to create, read, and write IRAF
+\fBimagefiles\fR. The term \fBimage\fR refers to data arrays of one, two,
+or more dimensions. Each "imagefile" actually consists of two files:
+the \fBheader file\fR and the \fBpixel storage file\fR. Seven disk datatypes
+are currently supported.
+.PP
+The IMIO calling sequences are summarized in the \fIProgrammer's Crib
+Sheet\fR. There is as yet no Reference Manual or User's Guide for the package.
+Our intention in this document is merely to introduce IMIO, to summarize its
+capabilities, and note what is planned for the future.
+.NH
+Structure
+.PP
+The basic structure of an applications program which uses IMIO is shown
+below. In the current implementation of IMIO the image header is a simple
+binary structure, but this will change when DBIO (the database interface)
+is implemented. The pixel storage file is accessed via FIO (the IRAF File
+I/O interface) which permits arbitrarily large buffers and double or multiple
+buffered i/o. All buffers are dynamically allocated and deallocated using
+the facilities provided by the MEMIO interface.
+
+.DS
+.cs 1 22
+Command Language
+ (applications program)
+ IMIO
+ DBIO
+ FIO
+ OS |
+ MEMIO | (operating system)
+ OS |
+
+
+ (system independent) | (system dependent)
+.DE
+.cs 1
+
+.NH
+Summary of What is Provided by the Current Interface
+.PP
+The IMIO interface code is mostly concerned with pixel buffer allocation and
+manipulation, and with mapping requests to read and write image sections
+into file i/o calls. FIO handles all low level i/o. The efficiency of FIO
+for sequential image access stems from the fact that the FIO buffers may
+be made as large as desired transparently to the outside world (i.e., IMIO),
+the number of FIO buffers is variable, and full read-ahead and write-behind
+are implemented (provided the OS provides asynchronous i/o facilities).
+.PP
+IMIO currently provides the following functions/features:
+
+.RS
+.IP (1)
+7 disk datatypes (ushort, silrdx).
+.IP (2)
+6 in-core datatypes (the standard silrdx).
+.IP (3)
+Images of up to 7 dimensions are currently supported. The maximum
+dimensionality is a sysgen parameter.
+.IP (4)
+Fully automatic multidimensional buffer allocation, resizing,
+and deallocation. There is no fixed limit on the size of a buffer (a subraster
+may actually exceed the size of the image if boundary extension is employed).
+The size of an image is limited only by the resources of the machine.
+.IP (5)
+An arbitrary number of input buffers (default 1) may be used to access an
+image. Buffers are allocated in a round robin fashion, and need not be the
+same size, dimension, or datatype. This feature is especially useful for
+convolutions, block averaging, and similar operators.
+.IP (6)
+Fully automatic type conversion on both input and output. Conversion occurs
+only when data is accessed, so one need not type convert the entire image
+to access a subraster.
+.IP (7)
+IMIO implements general image sections (described below), coordinate flip,
+and subsampling.
+.IP (8)
+The dimensionality of the image expected by the applications code and the
+actual dimension of an image need not agree. If an operator expects a one
+dimensional image, for example, it may be used to operate on any line, column,
+or pillar of a three dimensional image, on both input and output (see
+discussion on image sections below).
+.IP (9)
+Both "compressed" and "block aligned" storage modes are supported, with IMIO
+automatically selecting the optimal choice during image creation (if the
+packing efficiency is not above a certain threshold then image lines are
+not block aligned). The device blocksize is determined at runtime and
+devices with different blocksizes may coexist.
+.IP (10)
+IMIO may be advised if i/o is to be either highly sequential or highly
+random; the buffering strategy will be modified to increase i/o efficiency.
+.IP (11)
+Pixel storage files may reside on special devices if desired. For example,
+the current \fBdisplay\fR routine accesses the image display device as a random
+access imagefile via the standard IMIO interface. This was easy to do
+because FIO is device independent and allows new devices to be interfaced
+dynamically at run time (other examples of special "devices" are the CL,
+magtapes, and strings).
+.IP (12)
+The image header file, which is small, is normally placed in the user's
+own directory system. The pixel storage file, on the other hand, is often
+very large and is normally placed in a different filesystem. This is
+transparent to the user, and has the advantage that bulk data does not
+have to be backed up on tape when the user disk is backed up, and throughput
+is often higher because the pixel filesystem can be optimized for large
+transfers and more nearly contiguous files.
+.IP (13)
+An image opened with the mode "new_copy" inherits the full image header
+of an existing image, including all user defined fields, minus the pixels
+and minus all fields which depend on the actual values of the pixels.
+.RE
+
+.PP
+The basic i/o facilities are described in the crib sheet. In short, we
+have procedures to get or put pixels, lines, or sections. The put calls
+are identical to the get calls and all buffer allocation and manipulation
+is performed by IMIO. The pixel access routines access a list of pixels
+(described by one, two, or more integer arrays giving the coordinates of
+the pixels, which are fetched in storage order to minimize seeks).
+An additional set of calls are available for accessing all of the lines
+in an image sequentially in storage order, regardless of the dimensionality
+of the image (as in the FITS reader).
+.NH
+Planned Enhancements to IMIO
+.PP
+The following enhancements are currently planned for IMIO; they are
+arranged more or less with the highest priority items first. The DBIO
+header, boundary extension facilities, and bad pixel list features are
+of the highest priority and will be implemented within the next few months.
+
+.RS
+.IP (1)
+Replacement of the current rather rigid binary header by the highly
+extensible yet efficient DBIO header.
+.IP (2)
+Automatic boundary extension by any of the following techniques:
+nearest neighbor, reflection, projection, wrap around, indefinite,
+constant, apodize. Useful for convolutions and subraster extraction
+near the boundary of an image.
+.IP (3)
+Bad pixel list manipulation. A list of bad pixels will optionally be
+associated with each image. The actual value of each "bad" pixel in the
+image will be a reasonable, artificially generated value. Programs which
+do not need to know about bad pixels, such as simple pointwise image
+operators, will see only reasonable values. IMIO will provide routines to
+merge (etc.) bad pixel lists in simple pointwise image operations.
+Operators which need to be able to deal with bad pixels, such as surface
+fitting routines, will advise IMIO to replace the bad pixels with the
+value INDEF upon input.
+.IP (4)
+Implement the pixel access routines (\fBimgp__\fR and \fBimpp__\fR).
+Currently only the line and section routines are implemented. The section
+routines may be used to access individual pixels, but this involves quite
+a bit of overhead and disk seeks are not optimized.
+.IP (5)
+Optimization to the get/put line procedures to work directly
+out of the FIO buffers when possible for increased efficiency.
+.IP (6)
+IMIO (and FIO) dynamically allocate all buffers. Eventually we will add
+an "advice" option permitting buffers to be allocated in a region
+of memory which is \fIshared\fR with a bit-mapped array processor.
+The VOPS primitives, already used extensively for vector operations,
+will be interfaced to the AP and applications sofware will then make use
+of the AP without modification and without introducing any device
+dependence. Note that CSPI is currently marketing a 7 Mflop bit-mapped
+AP for the VAX, and Masscomp provides a similar device for their 680000 based
+supermicro.
+.IP (6)
+Support for the unsigned byte disk datatype.
+.DE
+
+.PP
+Long range improvements include language support for image sections in
+the successor to the SPP (subset) language compiler, and extensions for
+block storage mode of images on disk. Currently all images are stored on
+disk in line storage mode (i.e., like a Fortran array).
+.NH
+Image Sections
+.PP
+Image sections are used to specify the region of an image to be operated
+upon. The essential idea is that when the user passes the name of an
+image to a task, a special notation is employed which specifies the section
+of the image to be operated upon. The image section is decoded by IMIO
+at "immap" time and is completely transparent to the applications code
+(when a section is used, the image appears smaller to the applications
+program). If no section is specified then the entire image is accessed.
+.PP
+For example, suppose we want to display the image "pix" in frame 1 of the
+image display, using all the default parameters:
+
+.nf
+ cl> display pix, 1
+.fi
+
+This works fine as long as "pix" is a one or two dimensional image. If it
+is a three dimensional image, we will see only the first band. To display
+some other band, we must specify a two-dimensional \fIsection\fR of the
+three dimensional image:
+
+.nf
+ cl> display pix[*,*,5], 1
+ cl> display pix[5], 1
+.fi
+
+Either command above would display band 5 of the three dimensional image
+(higher dimensional images are analogous). To display a dimensional image
+with the columns flipped:
+
+.nf
+ cl> display pix[*,\(mi*], 1
+.fi
+
+This command flips the y-axis. To display a subraster:
+
+.nf
+ cl> display pix[30:40,310:300], 1
+.fi
+
+would display the indicated eleven pixel square subraster. To display a
+2048 square image on a 512 square display by means of subsampling:
+
+.nf
+ cl> display pix[*:4,*:4], 1
+.fi
+.NH
+Use of Virtual Memory
+.PP
+The current implementation of IMIO does not make use of any virtual memory
+facilities. We have had little incentive to do so because 4.1BSD Berkeley
+UNIX does not have a very good implementation of virtual memory (few systems
+do, it seems - DG/AOS, which is what CTIO runs, does not have a
+good implemenation either). Various strategies can, however, be employed
+to take advantage of virtual memory on a machine which provides good
+virtual memory facilities.
+.PP
+One technique is to use IMIO to "extract a subraster" which is in fact
+the entire image. The current implementation of IMIO would copy rather
+than map the image, but \fIif\fR no type conversion were required,
+if no section was specified, if the image was not block-aligned,
+and if referencing out of bounds was not required,
+IMIO could instead map the image directly into virtual memory.
+This would be an easy enhancement to make to IMIO because all data is
+accessed with pointers. The code fragment in the following example
+demonstrates how this is done in the current version of IMIO.
+
+.DS
+.cs 1 22
+int ncols, nlines
+pointer header, raster
+pointer immap(), imgs2r()
+
+begin
+ # Open or "map" the image. "Imagefile" is a file name
+ # or a file name with section subscript appended.
+
+ header = immap (imagefile, READ_ONLY, 0)
+
+ ncols = IM_LEN (header, 1)
+ nlines = IM_LEN (header, 2)
+
+ # Read or map entire image into memory. Pixels are
+ # converted to type real if necessary.
+
+ raster = imgs2r (header, 1, ncols, 1, nlines)
+
+ # Call SPP or Fortran subroutine to process type real
+ # image. Note how the pointer "raster" is dereferenced.
+
+ call subroutine (Memr[raster], ncols, nlines)
+ ...
+.DE
+.cs 1
+
+.PP
+Another, slightly different approach would be to allocate a single FIO
+buffer and map it onto the entire file. This would require no modifications
+to IMIO, rather one would modify the "file fault" code in FIO.
+This scheme would more efficiently support random access (to image lines or
+subrasters) on a virtual machine without introducing a real dependence
+on virtual memory.