.help import Oct94 dataio .ih NAME import -- create an IRAF image from an arbitrary binary file .ih USAGE import binfiles images .ih PARAMETERS .ls binfiles The list of input binary files to be read. .le .ls images The list of output IRAF images to be written. This parameter only needs to be specified when generating an output image (see the \fIoutput\fR parameter description). .le .ls format = "sense" The type of format to be processed. In default mode, i.e. \fIsense\fR, the format database is searched for a format identifier that evaluates truly for the current binary file, the input file parameters are then derived from the database entry. A specific format name in the database may alternatively be given in which case the input params are derived from that entry in the database. If \fIformat\fR=\fInone\fR the task parameters are used to describe the input file. .le .ce INPUT PARAMETERS .ls dims = "" The input file dimension string. This is a space or comma delimited string containing the length of the file in each dimension, e.g. "512,512,3". .le .ls pixtype = "" Input pixel type. This is a comma delimited string giving the type and size of each pixel, and an optional tag name to be used in the \fIoutbands\fR expressions. The syntax for the pixtype entry is .ls [:tag],[:tag],[....] where .nf type = b # byte (no conversions) u # unsigned integer i # signed integer r # ieee floating point n # native floating point x # ignore (skip) nbytes = 1, 2, 4, or 8 tag is something like 'r','g','b' (color triplets), 'r', 'i' (complex data), etc. If no tags are given one will automatically be assigned of the form 'b1', 'b2', etc. .fi .le .le .ls interleave = 0 Pixel interleave type. If the \fIpixtype\fR parameter is a composite then the input pixel are pixel-interleaved (i.e. each pixel in a band is stored together, as with RGB triplets) and this parameter is ignored. If the \fIpixtype\fR is an atomic value and \fIinterleave\fR is a positive number the image is line interleaved (e.g. a line of 'R', followed by a line of 'G', and so on). If the \fIpixtype\fR is atomic and \fIinterleave\fR is zero, the no data interleaving is assumed and each band in the file is stored sequentially. .le .ls bswap = "no" Type of byte-swapping to perform. By default no byte swapping is done, if \fIbswap\fR is "yes" then all input values are byte swapped, if \fIbswap\fR is "i2" then only short integers are byte swapped, if \fIbswap\fR is "i4" then only long integers are swapped. A combination of "i2,i4" can be used to swap only integer values, floating point numbers will not be swapped. .le .ls hskip = 0 Number of bytes preceding pixel data to skip. .le .ls tskip = 0 Number of bytes to skip at end of file. .le .ls bskip = 0 Number of bytes between image bands to skip. .le .ls lskip = 0 Number of bytes to skip at font of each line. .le .ls lpad = 0 Number of bytes to skip at end of each line. .le .ce OUTPUT PARAMETERS .ls output = "image" Type of output to generate. Possible values include "none" process the files but not generate an output image (e.g. to check the parameter values for correctness), "image" to generate an output image, "list" to generate a pixel listing of the file as would be produced by the \fILISTPIX\fR task on the image if were converted (no image is created with this option), or "info" to print information about the file. The \fIimages\fR parameter is only used for \fIoutput\fR=image. .le .ls outtype = "" The data type of the output image. May be one of 's' for a short image, 'i' for an integer image, 'l' for a long image, 'r' for a real image, and 'd' for a double precision image. If no \fIouttype\fR is specified then the datatype of the \fIoutbands\fR expression is used. This parameter is only used when \fIoutput\fR is set to "image". .le .ls outbands = "" Output image band expressions. If no expressions are given then all of the input pixels will be converted. The number of output bands may be more or less than the number of input bands. See the \fIOUTBANDS EXPRESSIONS\fR section for a more complete description of this parameter. .le .ls imheader = "" Image or header keyword data file. If an image is given then the image header is copied. If a file is given then the FITS format cards are copied. This only applies to new images. The data file consists of lines in FITS format with leading whitespace ignored. A FITS card must begin with an uppercase/numeric keyword. Lines not beginning with a FITS keyword such as comments or lower case are ignored. The user keyword output of \fBimheader\fR is an acceptable data file. See \fBmkheader\fR for further information. .le .ls database = "imcnv$lib/images.dat" The format database. This may also be a list of files to be searched (e.g. so that user-defined databases may be included), which will be treated as a single database. .le .ls verbose = yes Print verbose output during the conversion? .le .ls buffer_size = 64 Number of image lines \fIper band\fR to buffer in memory before writing to disk. Image buffering can increase task performance by as much as a factor of 30 for some formats but requires more memory. .le .ih DESCRIPTION The \fIimport\fR task is used to convert arbitrary raster binary files to IRAF format images. The input format may be specified either through the task parameters (\fIformat\fR set to 'none'), or as an entry in a database of known formats (\fIformat\fR set to the name of the entry). If the format of the image is not known a priori, the database can be searched and each record will be evaluated for an expression which identifies the format (\fIformat\fR set to "sense"). The task will output either an IRAF image, a list of pixel values in a manner similar to the \fILISTPIX\fR task, or information about the file format if it is supported in the database. .ih Input File Specification The input raster is assumed to be at most three dimensional, with pixels of various sizes that can be interleaved in a variety of ways. No compression schemes are yet supported, except in the case of builtin formats where special code has been written to handle to format. Byte-swapping and floating point conversion of pixels (from IEEE to native) is also supported. The \fIpixtype\fR and \fIinterleave\fR parameters define the pixel storage in the binary file. \fIPixtype\fR is a comma delimited string, the elements of which define the type and size of each pixel. An optional 'tag' name may be given to each pixel for use in the \fIoutbands\fR expressions. If no tag is given one will automatically be assigned. For composite pixtypes (i.e. when more than one element is listed), the data are assumed to be pixel interleaved (e.g. stored as { {RGB}, {RGB} ...} triplets). For atomic (i.e. single) pixtypes, a positive value of \fIinterleave\fR indicates that the data are stored in a line-interleaved manner (e.g. a line of R, a line of G, ...). If \fIinterleave\fR is zero and \fIpixtype\fR is atomic, then no interleaving is done and the image bands are thought to be stored sequentially. Minimal error checking is done to make sure the combination of these parameters is correct. The file may contain arbitrary padding around the pixels as defined by the \fItskip\fR, \fIbskip\fR, \fIlskip\fR, and \fIlpad\fR parameters, header information may be skipped by setting the \fIhskip\fR parameter. Additionally, pixels may be ignored on input while still specifying the full format. .ih Output Parameters Once a format has been found, the task may output an IRAF image by setting \fIoutput\fR to "image", a list of the pixels in the file can be written to STDOUT by setting \fIoutput\fR to "list", or information about the input file can be printed by setting \fIoutput\fR to "info". If \fIoutput\fR is set to "none" then no output will be generated, this can be used to check for read errors on the input file to verify task parameters. The datatype of the output image can be set by specifying the \fIouttype\fR parameter. The \fIoutbands\fR parameter is a list of expressions which are evaluated to compute the pixels in each band of the output image. Operands in these expressions consist of numeric constants and the pixtype tags (either user-supplied tags or the automatic tags), general arithmetic expressions are supported, which can include any of the special functions listed below. The simplest expression is the name of a tag itself. Regardless of the storage of pixels in the input file, each image band is separated on output unless an expression is given which combines them. See below for more details on \fIoutbands\fR. Header information may be added to an output image by naming either a keyword file or an existing image header listing in the \fIimheader\fR parameter. A header keyword data file consists of lines of FITS format cards. Leading whitespace is ignored. Lines not recognized as FITS cards are ignored. A valid FITS card is defined as beginning with a keyword of up to 8 uppercase, digit, hyphen, or underscore characters. If less than 8 characters the remaining characters are blanks. The ninth character may be an equal sign but must be immediately followed by a blank. Such value cards should be in FITS format though no attempt is made to enforce this. Any other ninth character is also acceptable and the line will be treated as a comment. Note that this way of recognizing FITS parameters excludes the case of comments in which the first 8 characters are blank. The reason for allowing leading whitespace and eliminating the blank keyword case is so that the long output of \fBimheader\fR may be used directly as input. .ih OUTBANDS EXPRESSIONS The outbands parameter is a comma delimited list of expressions, the simplest of which is the name of a tag itself (or the default names of the tags if none are provided in the \fIpixtype\fR param). The input pixels, regardless of how they are stored in the binary file, are always stored as separate bands in the output IRAF image. The outbands expressions will be evaluated to compute the pixels in each band of the output image. This means that e.g. RGB triplets in an input file will be separated into different bands in the output image, unless a single expression is given that combines them. The components named in \fIpixtype\fR may be eliminated or re-ordered in \fIoutbands\fR to exclude certain input bands, or to change the channel order. For example the commands: .nf cl> import file img pixtype="u1:a,u1:r,u1:g,u1:b" outbands="g,r,a" cl> import file img pixtype="u1,u1,u1,u1" outbands="b3,b2,b1" .fi both convert an input 32-bit image with ARGB components. In the first case the output image is an IRAF image where the B component has been eliminated and the channel order reversed. The second case is the same as the first but uses the automatic tag names. A combination of user-supplied tags and defaults could also be used. General interpreted arithmetic expressions are supported and can contain any of the standard expression evaluator functions (see the \fIimexpr\fR help page for more details). Special functions in expressions also include: .nf flipx (arg) - flip image in X flipy (arg) - flip image in Y gr[ea]y (r,g,b) - RGB to grayscale using the NTSC Y formula red (arg) - get the red component of a colormap image green (arg) - get the green component of a colormap image blue (arg) - get the blue component of a colormap image gamma (arg, gamma) - apply a gamma correction to the image .fi The two flip functions can change the image orientation by reversing the order of pixels within a line (a flipx() call), or it can flip an image from top- to-bottom (a flipy() call). The flipping will apply to all bands of the out- put image even if it was only used in one expression. To reverse the channel order simply change the order of the tags in the outbands parameter. RGB images may be converted to a single grayscale image using the NTSC formula: .nf gray = (0.289 * r) + (0.587 * G) + (0.114 * B) .fi Note that a similar grayscale conversion can be done by explicitly defining a similar equation in \fIoutbands\fR and supplying different coefficients. The \fIred()\fR, \fIgreen()\fR, or \fIblue()\fR functions can be used to get a single color component from a colormap image rather than the grayscale equivalent of the colormap. For example, to separate an 8-bit GIF color image into it's RGB components one could specify an outbands parameter such as .nf cl> import foo.gif bar format=gif outbands="red(b1),green(b1),blue(b1)" .fi Functions may also be nested in complex expressions such as: .nf flipy (gray(r,g,b)) - convert to grayscale, flip in Y flipx (flipy (gray (r,g,b))) - convert to grayscale, flip in X & Y gray (r,g,255) - use constant 255 as the B band gray (r,g+100,-b) - add constant to G, negate B .fi .ih FORMAT DATABASE The format database is a text file named as a task parameter. Each record of a database entry is of the form: .nf : : keyword = keyword = ...and so on .fi A database record begins with the format name at the beginning of a line. Whitespace at the beginning of a line is considered the continuation of a previous line. Comments may be inserted in the database using the normal '#' character, the remainder of the line is considered a comment. Blank lines and comments are ignored, a record ends at the next line with a format name at the beginning of the line. The task \fIdatabase\fR parameter defines the text files to be scanned as the database. If the parameter is a list of files then each file in the list will be concatenated to a single database file used by the task. The format_name field is a string identifying each entry in the database, any number of aliases may also be given to identify the same format possibly known by another name. Supported keywords include: .nf image_id - A boolean expression identifying the image type id_string - Verbose name of file format bswap - is file byte-swapped? (See Below) dims - a whitespace/comma delimited string of dimensions pixtype - pixel type, size [and tag], may be a composite interleave - describes how pixels are stored hskip - # of bytes of header info to skip tskip - # of bytes of trailing info to skip at end of file bskip - # of bytes of info to skip between image bands lskip - # of bytes of info to skip at front of each line lpad - # of bytes of info to skip at end of each line error - A condition that would cause a file read error, returns a string with the error message, otherwise returns the string "okay" .fi The 'image_id' string is an expression to be evaluated which, if true, uniquely identifies the file format (such as a comparison to a "magic number"). The 'id_string' is a verbose name of the format. The 'error' keywords use the "? :" conditional syntax to define a boolean expression which, when true, returns an error message and is used to indicate a condition in a format which isn't supported. The remaining keywords have the same meaning as the task parameters. Keywords not present in the database record will take the default parameter value. Expressions consist of any valid string that may be evaluated with the standard system expression evaluator evvexpr(). (See the documentation for this procedure or the \fIIMEXPR\fR task help page for details of builtin functions and operators.) Operators within expressions may be boolean, arithmetic, or the string operators '?=' (substring equality) and '//' (concatenation). Operands may be the special functions named below, previously defined keywords, constants (numeric or strings), and the special operands .ls $FSIZE The size of the binary file in bytes. In expressions this operand has an integer datatype. For formats with variable header sizes this can be used to determine the size of the header, since the size of the data can be derived from the image dimensions and subtracted from the total size of the file. .le .ls $FNAME The name of the binary file. In expressions this operand has a character datatype. As a last resort for images without any identifying features the file name may possibly be used to determine the format from a file name extension. .le .ih Special Functions: In addition to the intrinsic functions already provided there are a number of input and utility functions for the database. These are: .nf \fIINPUT FUNCTIONS\fR ctocc ([offset]) - convert byte to printable char constant ctod ([offset]) - convert string to double precision real ctoi ([offset]) - convert string to integer ctol ([offset]) - convert string to long ctor ([offset]) - convert string to single precision real ctowrd ([offset]) - get 1st white-space delimited word from str getstr ([offset,] len) - get a string at offset getb ([offset]) - get a byte at offset getu ([offset]) - get an unsigned short int at offset geti[24] ([offset]) - get a signed int at offset getr[48] ([offset]) - get an IEEE fp number at offset getn[48] ([offset]) - get a native fp number at offset locate ([offset,] pat) - find an offset to a pattern line (n) - offset of line N \fIUTILITY FUNCTIONS\fR skip (nbytes) - move offset by N-bytes bswap (arg) - byte swap the argument substr (str, c1, c2) - extract a substring from argument stridx (test, str) - get 1st occurrence of 'test' w/in 'str' parameter (param) - return the current task parameter default (param) - return the default task parameter lsb_host () - returns true if host is little-endian msb_host () - returns true if host is big-endian .fi .ls ctocc ([offset]) [string] Convert byte at the given offset to printable char constant. If no offset argument is given the current offset is used. .le .ls ctod ([offset]) [double] Convert string to double precision real. The function reads a string from the file and converts it up to the first unrecognized character. If no offset argument is given the current offset is used. .le .ls ctoi ([offset]) [int] Convert string to integer. The function reads a string from the file and converts it up to the first unrecognized character. If no offset argument is given the current offset is used. .le .ls ctol ([offset]) [long] Convert string to long. The function reads a string from the file and converts it up to the first unrecognized character. If no offset argument is given the current offset is used. .le .ls ctor ([offset]) [real] Convert string to single precision real. The function reads a string from the file and converts it up to the first unrecognized character. If no offset argument is given the current offset is used. .le .ls ctowrd ([offset]) [string] Get 1st white-space delimited word from str, leading whitespace is skipped. If no offset argument is given the current offset is used. .le .ls getstr ([offset,] len) [string] Get a string at offset. If no offset argument is given the current offset is used, the length of the string must be specified. .le .ls getb ([offset]) [int] Get a byte at offset. If no offset argument is given the current offset is used. .le .ls getu ([offset]) [int] Get an unsigned short integer at offset. If no offset argument is given the current offset is used. .le .ls geti[24] ([offset]) [int] Get a signed int at offset. If no offset argument is given the current offset is used. Long integers values can be read by specifying the function as geti4(), the names geti() and geti2() return short integers. .le .ls getr[48] ([offset]) [real/double] Get an IEEE floating point number at an optional offset. If no offset argument is given the current offset is used. Double precision values can be read by specifying the function as getr8(), the names getr() and getr4() return single precision real. .le .ls getn[48] ([offset]) [real/double] Get a native floating point number at an optional offset. If no offset argument is given the current offset is used. Double precision values can be read by specifying the function as getn8(), the names getn() and getn4() return single precision real. .le .ls locate ([offset,] pat) [int] Compute an offset. If no offset argument is given the current offset is used. .le .ls line (N) [int] Offset of line N in bytes. The database is rewound and the offset of the requested line number is returned, line are delimited by the '\n' character. .le .ls skip (nbytes) [int] Move current offset by N-bytes. The number of bytes skipped is returned as the function value. .le .ls bswap (arg) [type of arg] Byte swap the argument. .le .ls substr (str, first, last) [string] Extracts a substring from string \fIstr\fR. The first character in the string is at index 1. .le .ls stridx (test, str) [int] Finds the position of the first occurrence of any character found in \fItest\fR in the string \fIstr\fR, returning 0 if the match fails. .le .ls parameter (param) [param type] Return the current task parameter. The parameter is specified as a string containing the name of a task parameter, the type of the returned value is the parameter type .le .ls default (param) [param type] Return the default task parameter. The parameter is specified as a string containing the name of a task parameter, the type of the returned value is the parameter type .le .ls lsb_host () [bool] Returns true if host is little-endian. This function can be used as the \fIbswap\fR keyword expression for formats with a specified byte order. .le .ls msb_host () [bool] Returns true if host is big-endian. This function can be used as the \fIbswap\fR keyword expression for formats with a specified byte order. .le .ih BYTE SWAPPING The 'bswap' database entry is similar to the task parameter, it may be used to set byte swapping for the whole file, or for only certain data types. The value is a string parameter that may be "yes" to byteswap the whole file, "no" to not swap anything, or a comma delimited string of types described below to enable swapping for only those values. .nf bswap = { no | yes | i2 i4 } no # no swapping (default) yes # byte swap whole file i2 # byte swap short ints only i4 # byte swap long ints only .fi The \fIbswap\fR task parameter applies only to the pixel data, but the bswap keyword in a database record sets byte-swapping for the header information: arguments to the input and conversion functions will be byteswapped prior to being evaluated by the function. The bswap() special function can be used to negate byteswapping for a particular argument if it is or is not set by the keyword (the default is no byte swapping). .ih EXAMPLES .nf Get a list of known input formats: cl> import "" "" output=info Get a list of known input formats, including those defined by the user: cl> import "" "" output=info database="dev$images.dat,mydb.dat" Get a list of the file formats of each image in the directory: cl> import file* "" format="sense" output=info verbose- file1.ras Sun rasterfile file1.eps unknown format file1.pgm 8-bit PGM file : : Get a list of the file formats of each image in the directory and print out some information about each file: cl> import file* "" format="sense" output=info verbose+ file1.ras: Sun Rasterfile Resolution: 320 x 200 Pixel type: 8-bit unsigned integer Pixel storage: non-interleaved Header length: 137 bytes Byte swapped: no ... : Read a raw 8-bit file of pixels into an unsigned short IRAF image: cl> import file img format="none" dims="512,512" pixtype="b1" \ >>> outtype="u" outbands="b1" Read a JPL VICAR image or 8-bit Sun rasterfile: cl> import file img format="vicar" cl> import file img format="sunras" Concatenate three separate red, blue, and green images and convert to a single grayscale image: cl> concat pic.[rgb] > rgb cl> import rgb img format=none dims="640,480,3" \ >>> pixtype="u1" interleave=0 outbands="gray(b1,b2,b3)" Read an 8-bit colormap GIF image and separate the RGB colors into separate bands in the output image: cl> import file.gif img outbands="red(b1),green(b1),blue(b1)" Read three 8-bit rasterfiles with 200 byte-headers as if they were a single image, and combine the images to a single output band: cl> concat pix.* > rfiles cl> import rfiles img dims="512,512,3" pixtype="b1" \ >>> hskip=200 bskip=200 interleave=0 outbands="gray(b1,b2,b3)" Read a FITS image with one header record in which the data bytes are incorrectly swapped, but the header info is in the right order: cl> rfits nite1.fits "" nite1 File: nite1 1866-A Size = 640x480 cl> imheader nite1 l+ > imheader.dat # Save the header info cl> imdel nite1.imh cl> import nite1.fits nite1 format="none" dims="640,480" \ >>> bswap+ hskip=2880 pixtype="i2" outtype="s" imheader="imheader.dat" .fi .ih BUGS Bitmap images are not yet supported. Their most logical use would be as pixel masks but there hasn't been much call for these formats so they may be implemented at a later time. .ih REVISIONS .ls IMPORT V2.11 This is a new task in this version. .le .ih SEE ALSO export. imexpr, hedit, default image database imcnv$lib/images.dat .endhelp