diff options
Diffstat (limited to 'pkg/dataio/doc/import.hlp')
| -rw-r--r-- | pkg/dataio/doc/import.hlp | 631 |
1 files changed, 631 insertions, 0 deletions
diff --git a/pkg/dataio/doc/import.hlp b/pkg/dataio/doc/import.hlp new file mode 100644 index 00000000..da8047b7 --- /dev/null +++ b/pkg/dataio/doc/import.hlp @@ -0,0 +1,631 @@ +.help import Oct94 dataio +.ih +NAME +import -- create an IRAF image from an arbitrary binary file +.ih +USAGE +import binfiles images +.ih +PARAMETERS +.ls binfiles +The list of input binary files to be read. +.le +.ls images +The list of output IRAF images to be written. This parameter only needs to +be specified when generating an output image (see the \fIoutput\fR parameter +description). +.le +.ls format = "sense" +The type of format to be processed. In default mode, i.e. \fIsense\fR, +the format database is searched for a format identifier that evaluates +truly for the current binary file, the input file parameters are then +derived from the database entry. A specific format name in the database may +alternatively be given in which case the input params are derived from that +entry in the database. If \fIformat\fR=\fInone\fR the task parameters +are used to describe the input file. +.le + +.ce +INPUT PARAMETERS +.ls dims = "" +The input file dimension string. This is a space or comma delimited string +containing the length of the file in each dimension, e.g. "512,512,3". +.le +.ls pixtype = "" +Input pixel type. This is a comma delimited string giving the type and size +of each pixel, and an optional tag name to be used in the \fIoutbands\fR +expressions. The syntax for the pixtype entry is +.ls <type><nbytes>[:tag],<type><nbytes>[:tag],[....] + +where +.nf + type = b # byte (no conversions) + u # unsigned integer + i # signed integer + r # ieee floating point + n # native floating point + x # ignore (skip) + + nbytes = 1, 2, 4, or 8 + + tag is something like 'r','g','b' (color triplets), 'r', + 'i' (complex data), etc. If no tags are given one will + automatically be assigned of the form 'b1', 'b2', etc. + +.fi +.le +.le +.ls interleave = 0 +Pixel interleave type. If the \fIpixtype\fR parameter is a composite then +the input pixel are pixel-interleaved (i.e. each pixel in a band is stored +together, as with RGB triplets) and this parameter is ignored. If +the \fIpixtype\fR is an atomic value and \fIinterleave\fR is a positive +number the image is line interleaved (e.g. a line of 'R', followed by a +line of 'G', and so on). If the \fIpixtype\fR is atomic and \fIinterleave\fR +is zero, the no data interleaving is assumed and each band in the file +is stored sequentially. +.le +.ls bswap = "no" +Type of byte-swapping to perform. By default no byte swapping is done, +if \fIbswap\fR is "yes" then all input values are byte swapped, if \fIbswap\fR +is "i2" then only short integers are byte swapped, if \fIbswap\fR is "i4" then +only long integers are swapped. A combination of "i2,i4" can be used to +swap only integer values, floating point numbers will not be swapped. +.le +.ls hskip = 0 +Number of bytes preceding pixel data to skip. +.le +.ls tskip = 0 +Number of bytes to skip at end of file. +.le +.ls bskip = 0 +Number of bytes between image bands to skip. +.le +.ls lskip = 0 +Number of bytes to skip at font of each line. +.le +.ls lpad = 0 +Number of bytes to skip at end of each line. +.le + +.ce +OUTPUT PARAMETERS +.ls output = "image" +Type of output to generate. Possible values include "none" process the files +but not generate an output image (e.g. to check the parameter values for +correctness), "image" to generate an output image, "list" to generate a +pixel listing of the file as would be produced by the \fILISTPIX\fR task +on the image if were converted (no image is created with this option), +or "info" to print information about the file. The \fIimages\fR parameter +is only used for \fIoutput\fR=image. +.le +.ls outtype = "" +The data type of the output image. May be one of 's' for a short image, 'i' +for an integer image, 'l' for a long image, 'r' for a real image, and 'd' +for a double precision image. If no \fIouttype\fR is specified then the +datatype of the \fIoutbands\fR expression is used. This parameter is only +used when \fIoutput\fR is set to "image". +.le +.ls outbands = "" +Output image band expressions. If no expressions are given then all of the +input pixels will be converted. The number of output bands may be more or +less than the number of input bands. See the \fIOUTBANDS EXPRESSIONS\fR +section for a more complete description of this parameter. +.le +.ls imheader = "" +Image or header keyword data file. If an image is given then the image header +is copied. If a file is given then the FITS format cards are copied. +This only applies to new images. The data file consists of lines +in FITS format with leading whitespace ignored. A FITS card must begin +with an uppercase/numeric keyword. Lines not beginning with a FITS +keyword such as comments or lower case are ignored. The user keyword +output of \fBimheader\fR is an acceptable data file. See \fBmkheader\fR +for further information. +.le + +.ls database = "imcnv$lib/images.dat" +The format database. This may also be a list of files to be searched (e.g. +so that user-defined databases may be included), which will be treated as +a single database. +.le +.ls verbose = yes +Print verbose output during the conversion? +.le +.ls buffer_size = 64 +Number of image lines \fIper band\fR to buffer in memory before writing to +disk. Image buffering can increase task performance by as much as a factor +of 30 for some formats but requires more memory. +.le + +.ih +DESCRIPTION + + The \fIimport\fR task is used to convert arbitrary raster binary +files to IRAF format images. The input format may be specified either +through the task parameters (\fIformat\fR set to 'none'), or as an entry +in a database of known formats (\fIformat\fR set to the name of the entry). +If the format of the image is not known a priori, the database can be +searched and each record will be evaluated for an expression which +identifies the format (\fIformat\fR set to "sense"). The task will +output either an IRAF image, a list of pixel values +in a manner similar to the \fILISTPIX\fR task, or information about the +file format if it is supported in the database. + +.ih +Input File Specification + The input raster is assumed to be at most three dimensional, with +pixels of various sizes that can be interleaved in a variety of ways. +No compression schemes are yet supported, except in the case of builtin +formats where special code has been written to handle to format. +Byte-swapping and floating point conversion of pixels (from IEEE to +native) is also supported. + + The \fIpixtype\fR and \fIinterleave\fR parameters define the pixel +storage in the binary file. \fIPixtype\fR is a comma delimited string, +the elements of which define the type and size of each pixel. An optional +'tag' name may be given to each pixel for use in the \fIoutbands\fR +expressions. If no tag is given one will automatically be assigned. +For composite pixtypes (i.e. when more than one element is listed), the +data are assumed to be pixel interleaved (e.g. stored as { {RGB}, {RGB} ...} +triplets). For atomic (i.e. single) pixtypes, a positive value of +\fIinterleave\fR indicates that the data are stored in a line-interleaved +manner (e.g. a line of R, a line of G, ...). If \fIinterleave\fR is +zero and \fIpixtype\fR is atomic, then no interleaving is done and the +image bands are thought to be stored sequentially. Minimal error +checking is done to make sure the +combination of these parameters is correct. + + The file may contain arbitrary padding around the pixels as +defined by the \fItskip\fR, \fIbskip\fR, \fIlskip\fR, and \fIlpad\fR +parameters, header information may be skipped by setting the \fIhskip\fR +parameter. Additionally, pixels may be ignored on input while still +specifying the full format. +.ih +Output Parameters + Once a format has been found, the task may output an IRAF image +by setting \fIoutput\fR to "image", a list of the pixels in the file +can be written to STDOUT by setting \fIoutput\fR to "list", or information +about the input file can be printed by setting \fIoutput\fR to "info". +If \fIoutput\fR is set to "none" then no output will be generated, this +can be used to check for read errors on the input file to verify task +parameters. The datatype of the output image can be set by specifying +the \fIouttype\fR parameter. + + The \fIoutbands\fR parameter is a list of expressions which are +evaluated to compute the pixels in each band of the output image. Operands +in these expressions consist of numeric constants and the pixtype tags +(either user-supplied tags or the automatic tags), general arithmetic +expressions are supported, which can include any of the special functions +listed below. The simplest expression is the name of a tag itself. +Regardless of the storage of pixels in the input file, each image band is +separated on output unless an expression is given which combines them. +See below for more details on \fIoutbands\fR. + + Header information may be added to an output image by naming +either a keyword file or an existing image header listing in the +\fIimheader\fR parameter. A header keyword data file consists of lines +of FITS format cards. Leading whitespace is ignored. Lines not recognized +as FITS cards are ignored. A valid FITS card is defined as beginning with +a keyword of up to 8 uppercase, digit, hyphen, or underscore characters. If +less than 8 characters the remaining characters are blanks. The +ninth character may be an equal sign but must be immediately followed +by a blank. Such value cards should be in FITS format though no +attempt is made to enforce this. Any other ninth character is also +acceptable and the line will be treated as a comment. Note that this +way of recognizing FITS parameters excludes the case of comments +in which the first 8 characters are blank. The reason for allowing +leading whitespace and eliminating the blank keyword case is so that +the long output of \fBimheader\fR may be used directly as input. + +.ih +OUTBANDS EXPRESSIONS + + The outbands parameter is a comma delimited list of expressions, the +simplest of which is the name of a tag itself (or the default names of the +tags if none are provided in the \fIpixtype\fR param). +The input pixels, regardless of how they are stored in the binary file, +are always stored as separate bands in the output IRAF image. +The outbands expressions will be evaluated to compute the pixels in each +band of the output image. This means that e.g. RGB triplets in an input +file will be separated into different bands in the output image, unless a +single expression is given that combines them. The components named +in \fIpixtype\fR may be eliminated or re-ordered in \fIoutbands\fR to +exclude certain input bands, or to change the channel order. For example +the commands: + +.nf +cl> import file img pixtype="u1:a,u1:r,u1:g,u1:b" outbands="g,r,a" +cl> import file img pixtype="u1,u1,u1,u1" outbands="b3,b2,b1" +.fi + +both convert an input 32-bit image with ARGB components. In the first case +the output image is an IRAF image where the B component has been eliminated +and the channel order reversed. The second case is the same as the first but +uses the automatic tag names. A combination of user-supplied tags and +defaults could also be used. + + General interpreted arithmetic expressions are supported and can +contain any of the standard expression evaluator functions (see +the \fIimexpr\fR help page for more details). Special functions in +expressions also include: +.nf + + flipx (arg) - flip image in X + flipy (arg) - flip image in Y + gr[ea]y (r,g,b) - RGB to grayscale using the NTSC Y formula + red (arg) - get the red component of a colormap image + green (arg) - get the green component of a colormap image + blue (arg) - get the blue component of a colormap image + gamma (arg, gamma) - apply a gamma correction to the image + +.fi +The two flip functions can change the image orientation by reversing the order +of pixels within a line (a flipx() call), or it can flip an image from top- +to-bottom (a flipy() call). The flipping will apply to all bands of the out- +put image even if it was only used in one expression. To reverse the channel +order simply change the order of the tags in the outbands parameter. RGB +images may be converted to a single grayscale image using the NTSC formula: +.nf + + gray = (0.289 * r) + (0.587 * G) + (0.114 * B) + +.fi +Note that a similar grayscale conversion can be done by explicitly defining +a similar equation in \fIoutbands\fR and supplying different coefficients. + + The \fIred()\fR, \fIgreen()\fR, or \fIblue()\fR functions can be used +to get a single color component from a colormap image rather than the +grayscale equivalent of the colormap. For example, to separate an 8-bit +GIF color image into it's RGB components one could specify an outbands +parameter such as +.nf + +cl> import foo.gif bar format=gif outbands="red(b1),green(b1),blue(b1)" + +.fi + + Functions may also be nested in complex expressions such as: + +.nf + flipy (gray(r,g,b)) - convert to grayscale, flip in Y + flipx (flipy (gray (r,g,b))) - convert to grayscale, flip in X & Y + gray (r,g,255) - use constant 255 as the B band + gray (r,g+100,-b) - add constant to G, negate B +.fi + +.ih +FORMAT DATABASE + + The format database is a text file named as a task parameter. +Each record of a database entry is of the form: + +.nf + <format_name>: + <alias>: + keyword = <expr> + keyword = <expr> + ...and so on +.fi + +A database record begins with the format name at the beginning of a line. +Whitespace at the beginning of a line is considered the continuation of a +previous line. Comments may be inserted in the database using the normal '#' +character, the remainder of the line is considered a comment. Blank lines +and comments are ignored, a record ends at the next line with a format name +at the beginning of the line. The task \fIdatabase\fR parameter +defines the text files to be +scanned as the database. If the parameter is a list of files then each file +in the list will be concatenated to a single database file used by the task. + + The format_name field is a string identifying each entry in the +database, any number of aliases may also be given to identify the same +format possibly known by another name. Supported keywords include: + +.nf + image_id - A boolean expression identifying the image type + id_string - Verbose name of file format + bswap - is file byte-swapped? (See Below) + dims - a whitespace/comma delimited string of dimensions + pixtype - pixel type, size [and tag], may be a composite + interleave - describes how pixels are stored + hskip - # of bytes of header info to skip + tskip - # of bytes of trailing info to skip at end of file + bskip - # of bytes of info to skip between image bands + lskip - # of bytes of info to skip at front of each line + lpad - # of bytes of info to skip at end of each line + error - A condition that would cause a file read error, + returns a string with the error message, otherwise + returns the string "okay" +.fi + +The 'image_id' string is an expression to be evaluated which, if true, +uniquely identifies the file format (such as a comparison to a "magic number"). +The 'id_string' is a verbose name of the format. +The 'error' keywords use the "? :" conditional syntax to +define a boolean expression which, when true, returns an error message and is +used to indicate a condition in a format which isn't supported. The remaining +keywords have the same meaning as the task parameters. Keywords not present +in the database record will take the default parameter value. + + Expressions consist of any valid string that may be evaluated with the +standard system expression evaluator evvexpr(). (See the documentation for this +procedure or the \fIIMEXPR\fR task help page for details of builtin functions +and operators.) Operators within expressions may be boolean, arithmetic, +or the string operators '?=' (substring equality) and '//' (concatenation). +Operands may be the special functions named below, previously defined +keywords, constants (numeric or strings), and the special operands + +.ls $FSIZE +The size of the binary file in bytes. In expressions this operand has an +integer datatype. For formats with variable header sizes this can be used +to determine the size of the header, since the size of the data can be +derived from the image dimensions and subtracted from the total size of the +file. +.le +.ls $FNAME +The name of the binary file. In expressions this operand has a character +datatype. As a last resort for images without any identifying features the +file name may possibly be used to determine the format from a file name +extension. +.le + + +.ih +Special Functions: + + In addition to the intrinsic functions already provided there are a +number of input and utility functions for the database. These are: +.nf + + \fIINPUT FUNCTIONS\fR + + ctocc ([offset]) - convert byte to printable char constant + ctod ([offset]) - convert string to double precision real + ctoi ([offset]) - convert string to integer + ctol ([offset]) - convert string to long + ctor ([offset]) - convert string to single precision real + ctowrd ([offset]) - get 1st white-space delimited word from str + + getstr ([offset,] len) - get a string at offset + getb ([offset]) - get a byte at offset + getu ([offset]) - get an unsigned short int at offset +geti[24] ([offset]) - get a signed int at offset +getr[48] ([offset]) - get an IEEE fp number at offset +getn[48] ([offset]) - get a native fp number at offset + + locate ([offset,] pat) - find an offset to a pattern + line (n) - offset of line N + + \fIUTILITY FUNCTIONS\fR + + skip (nbytes) - move offset by N-bytes + bswap (arg) - byte swap the argument + substr (str, c1, c2) - extract a substring from argument + stridx (test, str) - get 1st occurrence of 'test' w/in 'str' + +parameter (param) - return the current task parameter + default (param) - return the default task parameter + lsb_host () - returns true if host is little-endian + msb_host () - returns true if host is big-endian +.fi + +.ls ctocc ([offset]) [string] +Convert byte at the given offset to printable char constant. +If no offset argument is given the current offset is used. +.le +.ls ctod ([offset]) [double] +Convert string to double precision real. +The function reads a string from +the file and converts it up to the first unrecognized character. +If no offset argument is given the current offset is used. +.le +.ls ctoi ([offset]) [int] +Convert string to integer. +The function reads a string from +the file and converts it up to the first unrecognized character. +If no offset argument is given the current offset is used. +.le +.ls ctol ([offset]) [long] +Convert string to long. +The function reads a string from +the file and converts it up to the first unrecognized character. +If no offset argument is given the current offset is used. +.le +.ls ctor ([offset]) [real] +Convert string to single precision real. +The function reads a string from +the file and converts it up to the first unrecognized character. +If no offset argument is given the current offset is used. +.le +.ls ctowrd ([offset]) [string] +Get 1st white-space delimited word from str, leading whitespace is skipped. +If no offset argument is given the current offset is used. +.le +.ls getstr ([offset,] len) [string] +Get a string at offset. +If no offset argument is given the current offset is used, the length of +the string must be specified. +.le +.ls getb ([offset]) [int] +Get a byte at offset. +If no offset argument is given the current offset is used. +.le +.ls getu ([offset]) [int] +Get an unsigned short integer at offset. +If no offset argument is given the current offset is used. +.le +.ls geti[24] ([offset]) [int] +Get a signed int at offset. +If no offset argument is given the current offset is used. +Long integers values can be read by specifying the function as geti4(), +the names geti() and geti2() return short integers. +.le +.ls getr[48] ([offset]) [real/double] +Get an IEEE floating point number at an optional offset. +If no offset argument is given the current offset is used. +Double precision values can be read by specifying the function as getr8(), +the names getr() and getr4() return single precision real. +.le +.ls getn[48] ([offset]) [real/double] +Get a native floating point number at an optional offset. +If no offset argument is given the current offset is used. +Double precision values can be read by specifying the function as getn8(), +the names getn() and getn4() return single precision real. +.le +.ls locate ([offset,] pat) [int] +Compute an offset. +If no offset argument is given the current offset is used. +.le +.ls line (N) [int] +Offset of line N in bytes. The database is rewound and the offset of the +requested line number is returned, line are delimited by the '\n' character. +.le +.ls skip (nbytes) [int] +Move current offset by N-bytes. The number of bytes skipped is returned as +the function value. +.le +.ls bswap (arg) [type of arg] +Byte swap the argument. +.le +.ls substr (str, first, last) [string] +Extracts a substring from string \fIstr\fR. The first character in +the string is at index 1. +.le +.ls stridx (test, str) [int] +Finds the position of the first occurrence of any character found +in \fItest\fR in the string \fIstr\fR, returning 0 if the match fails. +.le +.ls parameter (param) [param type] +Return the current task parameter. The parameter is specified as a string +containing the name of a task parameter, the type of the returned value is +the parameter type +.le +.ls default (param) [param type] +Return the default task parameter. The parameter is specified as a string +containing the name of a task parameter, the type of the returned value is +the parameter type +.le +.ls lsb_host () [bool] +Returns true if host is little-endian. +This function can be used as the \fIbswap\fR keyword expression for formats +with a specified byte order. +.le +.ls msb_host () [bool] +Returns true if host is big-endian. +This function can be used as the \fIbswap\fR keyword expression for formats +with a specified byte order. +.le + +.ih +BYTE SWAPPING + + The 'bswap' database entry is similar to the task parameter, it may +be used to set byte swapping for the whole file, or for only certain data +types. The value is a string parameter that may be "yes" to byteswap the +whole file, "no" to not swap anything, or a comma delimited string of types +described below to enable swapping for only those values. +.nf + + bswap = { no | yes | i2 i4 } + + no # no swapping (default) + yes # byte swap whole file + i2 # byte swap short ints only + i4 # byte swap long ints only +.fi + + The \fIbswap\fR task parameter applies only to the pixel data, +but the bswap keyword in a database record sets byte-swapping +for the header information: arguments to the input and conversion functions +will be byteswapped prior to being evaluated by the function. The bswap() +special function can be used to negate byteswapping for a particular +argument if it is or is not set by the keyword (the default is no byte +swapping). + +.ih +EXAMPLES +.nf + +Get a list of known input formats: + + cl> import "" "" output=info + +Get a list of known input formats, including those defined by the user: + + cl> import "" "" output=info database="dev$images.dat,mydb.dat" + +Get a list of the file formats of each image in the directory: + + cl> import file* "" format="sense" output=info verbose- + file1.ras Sun rasterfile + file1.eps unknown format + file1.pgm 8-bit PGM file + : : + +Get a list of the file formats of each image in the directory and +print out some information about each file: + + cl> import file* "" format="sense" output=info verbose+ + file1.ras: Sun Rasterfile + Resolution: 320 x 200 + Pixel type: 8-bit unsigned integer + Pixel storage: non-interleaved + Header length: 137 bytes + Byte swapped: no + ... : + +Read a raw 8-bit file of pixels into an unsigned short IRAF image: + + cl> import file img format="none" dims="512,512" pixtype="b1" \ + >>> outtype="u" outbands="b1" + +Read a JPL VICAR image or 8-bit Sun rasterfile: + + cl> import file img format="vicar" + cl> import file img format="sunras" + +Concatenate three separate red, blue, and green images and convert + to a single grayscale image: + + cl> concat pic.[rgb] > rgb + cl> import rgb img format=none dims="640,480,3" \ + >>> pixtype="u1" interleave=0 outbands="gray(b1,b2,b3)" + +Read an 8-bit colormap GIF image and separate the RGB colors into + separate bands in the output image: + + cl> import file.gif img outbands="red(b1),green(b1),blue(b1)" + +Read three 8-bit rasterfiles with 200 byte-headers as if they were + a single image, and combine the images to a single output band: + + cl> concat pix.* > rfiles + cl> import rfiles img dims="512,512,3" pixtype="b1" \ + >>> hskip=200 bskip=200 interleave=0 outbands="gray(b1,b2,b3)" + +Read a FITS image with one header record in which the data bytes + are incorrectly swapped, but the header info is in the right order: + + cl> rfits nite1.fits "" nite1 + File: nite1 1866-A Size = 640x480 + cl> imheader nite1 l+ > imheader.dat # Save the header info + cl> imdel nite1.imh + cl> import nite1.fits nite1 format="none" dims="640,480" \ + >>> bswap+ hskip=2880 pixtype="i2" outtype="s" imheader="imheader.dat" + +.fi + +.ih +BUGS +Bitmap images are not yet supported. Their most logical use would be as +pixel masks but there hasn't been much call for these formats so they may +be implemented at a later time. +.ih +REVISIONS +.ls IMPORT V2.11 +This is a new task in this version. +.le +.ih +SEE ALSO +export. imexpr, hedit, default image database imcnv$lib/images.dat +.endhelp |