diff options
author | Joseph Hunkeler <jhunkeler@gmail.com> | 2015-07-08 20:46:52 -0400 |
---|---|---|
committer | Joseph Hunkeler <jhunkeler@gmail.com> | 2015-07-08 20:46:52 -0400 |
commit | fa080de7afc95aa1c19a6e6fc0e0708ced2eadc4 (patch) | |
tree | bdda434976bc09c864f2e4fa6f16ba1952b1e555 /pkg/tbtables/doc | |
download | iraf-linux-fa080de7afc95aa1c19a6e6fc0e0708ced2eadc4.tar.gz |
Initial commit
Diffstat (limited to 'pkg/tbtables/doc')
-rw-r--r-- | pkg/tbtables/doc/Notes | 46 | ||||
-rw-r--r-- | pkg/tbtables/doc/README | 10 | ||||
-rw-r--r-- | pkg/tbtables/doc/calls.doc | 190 | ||||
-rw-r--r-- | pkg/tbtables/doc/cfitsio.install | 57 | ||||
-rw-r--r-- | pkg/tbtables/doc/descrip.doc | 62 | ||||
-rw-r--r-- | pkg/tbtables/doc/ex.x | 109 | ||||
-rw-r--r-- | pkg/tbtables/doc/example.doc | 122 | ||||
-rw-r--r-- | pkg/tbtables/doc/fileformat.doc | 91 | ||||
-rw-r--r-- | pkg/tbtables/doc/tbtcpy.lis | 30 | ||||
-rw-r--r-- | pkg/tbtables/doc/text_tables.doc | 234 | ||||
-rw-r--r-- | pkg/tbtables/doc/versions.doc | 29 |
11 files changed, 980 insertions, 0 deletions
diff --git a/pkg/tbtables/doc/Notes b/pkg/tbtables/doc/Notes new file mode 100644 index 00000000..fef2c22a --- /dev/null +++ b/pkg/tbtables/doc/Notes @@ -0,0 +1,46 @@ +18 Sep 1987 get/put element routines + + The user-callable get/put element routines are tbegt[tbird] and +tbept[tbird]. In outline, each of these routines calls tbeoff to get +the offset to the element (i.e. row & column) to be gotten or put, +then calls a get/put primitive routine (tbegp[], tbepp[]) of the data +type of the column, and finally converts the data type if that is +necessary. If the data type of the column is the same as that of the +I/O buffer then the value is read directly into or from that buffer. +No separate null flag is returned by tbegt[]; if the table value is +undefined the output value will be INDEF. + + A subroutine call could have been saved by writing tbeoff into +each of the get/put primitives. That would have been very reasonable, +but there were a couple of reasons for not doing it that way. I would +like to replace the seek & read or seek & write in get/put row (tbrgt[], +tbrpt[]) by calls to these get/put primitives, and in that case the +offset does not need to be recomputed from scratch for each column. +If we ever have 3-D tables (i.e. entries which are arrays) then these +primitives could still be used for getting/putting single elements in +the arrays. + + I should also write get/put array primitives for get/put column +or for arrays in 3-D tables. Eventually it would be nice to include +conversion between machine-dependent and machine-independent formats +in these primitives. + + The use of TB_CURROW and TB_OFFSET in tbrgt[] and tbrpt[] seems +rather clumsy. I think it would be better to just call tbxoff or tbeoff. + +1 Oct 1987 tbpset & tbpsta for FIO buffer size + + I have not handled this very well. It is supposed to be not possible +to set the FIO buffer size after the first I/O operation to a file. When +an existing table is opened by tbtopn, the size-information record is read. +For a new table tbtopn does not actually call open (but perhaps it should!), +so the buffer size can't be set before calling tbtcre, which does call open +but which also writes the size-info record to the table. So in neither case +is it possible to change the FIO buffer size. My solution, in the routine +tbtfst, was to close the table file (using close, not tbtclo), reopen it, +and then call fset before any I/O operation is done. This precludes calling +tbpset to set the buffer size (e.g. advice=RANDOM) for a new table between +calls to tbtopn and tbtcre. This is just where you would expect to be able +to set it, however, so one of these days I will have to make that possible. +This could be done by calling open in tbtopn even for a new table but not +writing to the table until the call to tbtcre. diff --git a/pkg/tbtables/doc/README b/pkg/tbtables/doc/README new file mode 100644 index 00000000..4271bf9e --- /dev/null +++ b/pkg/tbtables/doc/README @@ -0,0 +1,10 @@ + 2328 Sep 11 14:53 cfitsio.install info about installing CFITSIO + 5988 Apr 20 1999 calls.doc user-callable table I/O routines + 3328 Jul 30 1997 ex.x a complete example + 3579 Jul 30 1997 example.doc example + 4463 Aug 17 1999 fileformat.doc description of stsdas table file format + 582 Jul 30 1997 tbtcpy.lis info about tbtcpy +11898 Aug 17 1999 text_tables.doc info about text tables + 1605 Aug 17 1999 versions.doc changes to stsdas table format + 2654 Jul 30 1997 Notes not up to date + 2858 Jul 30 1997 descrip.doc not up to date diff --git a/pkg/tbtables/doc/calls.doc b/pkg/tbtables/doc/calls.doc new file mode 100644 index 00000000..5ebd45dc --- /dev/null +++ b/pkg/tbtables/doc/calls.doc @@ -0,0 +1,190 @@ + 1999 April 20 +opening and closing tables: + + tbtopn - initialize (and open the table if not NEW_FILE or NEW_COPY) + tbtcre - create a new table (after initializing with tbtopn) + tbtclo - close a table + +table parameters: + + tbpset - set a table parameter + tbpsta - get the value of a table parameter (e.g. number of rows) + +row and column selectors: + + tbcdes - get the pointer to a column selector descriptor + tbsirow - get the actual row number from the selected row number + +columns: + + tbcdef - define columns + tbcdef1 - define one column + tbcfnd - find columns from their names + tbcfnd1 - find a column from its name + tbcinf - get information about a column + tbcig[ti] - get specific info about a column (e.g. name or data type) + tbcnel - get number of elements in column + tbcga[rd] - get all elements in column + +get and put: + + tbegt[tbirds] - get a value from the table + tbept[tbirds] - put a value into the table + tbagt[tbirds] - get an array from a row and column location + tbapt[tbirds] - put an array at a row and column location + tbrgt[tbirds] - get values from a row + tbrpt[tbirds] - put values into a row + tbcgt[tbirds] - get values from a column + tbcpt[tbirds] - put values into a column + tbrudf - set values in a row to undefined + +header parameters: + + tbhgt[tbird] - get a header parameter + tbhad[tbird] - add a new header parameter or replace existing one + tbhpt[tbird] - replace an existing header parameter + tbhcal - copy all header parameters + tbhgnp - get Nth header parameter as a string + tbhgcm - get a comment associated with a header parameter + tbhpcm - add or replace a comment for a header parameter + +table files: + + tbtcpy - copy a table + tbtdel - delete a table + tbtren - rename a table + tbtacc - test for the existence of a table + tbtext - append default extension (if it's not already there) + tbtnam - get the name (including extension) of the table + tbtflu - flush FIO buffer for table + tbfpri - copy primary header of FITS table, if appropriate + tbparse - get file name from table name + +miscellaneous: + + tbtchs - change allocated space of any/all portions of a table + tbrcpy - copy an entire row (only for tables with identical columns) + tbrcsc - copy a row, but copy only selected columns + tbrswp - swap two rows + tbtsrt - sort an index for the table rows + tbrdel - delete a range of rows + tbrnll - set all columns in a range of rows to INDEF + tbcnam - change the name of a column + tbcfmt - change the format for printing a column + tbcnit - change the units for a column + tbcnum - get the column pointer from the column number + +Calling sequences: + + nret = tbagt[] (tp, cp, rownum, buffer, first, nelem) + nret = tbagtt (tp, cp, rownum, buffer, lenstr, first, nelem) + tbapt[] (tp, cp, rownum, buffer, first, nelem) + tbaptt (tp, cp, rownum, buffer, lenstr, first, nelem) + + tbcdef (tp, cptr, + colnames, colunits, colfmt, datatype, nelem, numcols) + tbcdef1 (tp, cp, + colname, colunits, colfmt, datatype, nelem) + + descrip = tbcdes (tp, cp) + + tbcfmt (tp, cp, colfmt) + + tbcfnd (tp, colnames, cptr, numcols) + tbcfnd1 (tp, colname, cp) + + int = tbcga[rd] (tp, cp, buffer, nelem) + + tbcgt[] (tp, cp, buffer, nullflag, firstrow, lastrow) + tbcgtt (tp, cp, buffer, nullflag, lenstr, firstrow, lastrow) + + tbcinf (cp, + colnum, colname, colunits, colfmt, datatype, lendata, lenfmt) + + int = tbcigi (cp, param) + tbcigt (cp, param, outstr, maxch) + + tbcnam (tp, cp, colname) + + int = tbcnel (tp, cp) + + tbcnit (tp, cp, colunits) + + cp = tbcnum (tp, colnum) + + tbcpt[] (tp, cp, buffer, firstrow, lastrow) + tbcptt (tp, cp, buffer, lenstr, firstrow, lastrow) + + tbegt[] (tp, cp, rownum, buffer) + tbegtt (tp, cp, rownum, buffer, maxch) + + tbept[] (tp, cp, rownum, buffer) + + tbfpri (inname, outname, copied) + + tbhad[] (tp, keyword, value) + + tbhcal (itp, otp) + + tbhgcm (tp, keyword, comment, maxch) + tbhpcm (tp, keyword, comment) + + tbhgnp (tp, parnum, keyword, datatype, str) + + bool = tbhgtb (tp, keyword) + double = tbhgtd (tp, keyword) + int = tbhgti (tp, keyword) + real = tbhgtr (tp, keyword) + tbhgtt (tp, keyword, text, maxch) + + tbhpt[] (tp, keyword, value) + + int = tbparse (tablename, filename, extname, maxch, hdu) + + tbpset (tp, param, value) + + int = tbpsta (tp, param) + + tbrcpy (itp, otp, irownum, orownum) + + tbrcsc (itp, otp, icptr, ocptr, irownum, orownum, numcols) + + tbrdel (tp, firstrow, lastrow) + + tbrgt[] (tp, cptr, buffer, nullflag, numcols, rownum) + tbrgtt (tp, cptr, buffer, nullflag, lenstr, numcols, rownum) + + tbrnll (tp, firstrow, lastrow) + + tbrpt[] (tp, cptr, buffer, numcols, rownum) + tbrptt (tp, cptr, buffer, lenstr, numcols, rownum) + + tbrswp (tp, row1, row2) + + tbrudf (tp, cptr, numcols, rownum) + + tbsirow (tp, selrow, rownum) + + int = tbtacc (tablename) + + tbtchs (tp, maxpar, maxcols, rowlen, allrows) + + tbtclo (tp) + + tbtcpy (inname, outname) + + tbtcre (tp) + + tbtdel (tablename) + + tbtext (inname, outname, maxch) + + tbtflu (tp) + + tbtnam (tp, tblname, maxch) + + tp = tbtopn (tablename, iomode, template) + + tbtren (oldname, newname) + + tbtsrt (tp, numcols, cp, fold, nindex, index) diff --git a/pkg/tbtables/doc/cfitsio.install b/pkg/tbtables/doc/cfitsio.install new file mode 100644 index 00000000..3fb2afb5 --- /dev/null +++ b/pkg/tbtables/doc/cfitsio.install @@ -0,0 +1,57 @@ +This file describes how to install a new version of CFITSIO in TABLES. + +CFITSIO can be obtained from the High Energy Astrophysics Archive Research +Center, HEASARC: + + http://heasarc.gsfc.nasa.gov/fitsio + +The directory containing CFITSIO in the TABLES package is: + + tables$lib/tbtables/cfitsio/ + +Before installing a new version, save the mkpkg file from the above +directory, since a new distribution may include a file with the same +name but with very different contents. + +After saving mkpkg, the files in the distribution can then be installed +in tables$lib/tbtables/cfitsio/. Copy the saved version of mkpkg into +this directory, clobbering the version from the distribution. This saved +version of mkpkg may need to be modified to include new files, as described +below. + +These are the files that may need to be modified (unless these changes +have been incorporated into future versions of CFITSIO): + + eval_l.c + fitsio2.h + +Changes to eval_l.c and fitsio2.h: + +On a VMS machine, if the linker gives an error about strcasecmp and +strncasecmp being redefined, remove vms and __vms from this section +(this is near the end of both files): + +#if defined(vms) || defined(__vms) || defined(WIN32) || defined(__WIN32__) || defined(macintosh) + +/* ================================================================== */ +/* A hack for nonunix machines, which lack strcasecmp and strncasecmp */ +/* ================================================================== */ + +changing it to this: + +#if defined(WIN32) || defined(__WIN32__) || defined(macintosh) + +/* ================================================================== */ +/* A hack for nonunix machines, which lack strcasecmp and strncasecmp */ +/* ================================================================== */ + +Changes to mkpkg: + +There will be files in the CFITSIO distribution that are not needed by +the TABLES library, such as test programs; these should not be included +in mkpkg. However, there may be new source files that contain functions +that are called by functions that are called by the table I/O routines. +If the link fails for the ttools package, find the source files containing +the missing modules, and add these files to the list of dependents in mkpkg +for libtbtables.a. It may require more than one iteration to find all the +required source files. diff --git a/pkg/tbtables/doc/descrip.doc b/pkg/tbtables/doc/descrip.doc new file mode 100644 index 00000000..3905bdba --- /dev/null +++ b/pkg/tbtables/doc/descrip.doc @@ -0,0 +1,62 @@ + This note describes some specific characteristics and limitations +of STSDAS tables. + +1. Six data types are supported: + + single-precision real + double-precision real + integer (same number of bytes as real) + short integer (int*2) + boolean (Fortran logical; same size as real) + text (character strings) in multiples of four characters up to + 160 characters + +Type conversion is done implicitly if the data type of the column differs +from that of the buffer. Boolean true and false are converted to YES and NO +respectively if they are read as integer values. + +2. Header parameters: + + The same data types are supported for header (user) parameters as for +table data, except for short integers, and text strings are limited to 70 +characters. Type conversion is done for numeric and boolean parameters, +but text parameters may only be gotten as text. The I/O routines for header +parameters get and put single parameters, not arrays. A user program may +construct array elements by appending numbers to a root portion of a keyword, +and the array elements may be gotten or put one at a time. + + The table must exist in order to get or put header parameters. + +3. Undefined elements are flagged by using special values. There is at +present no undefined value for a boolean element; the default value is false +(or no). A null character string is regarded as undefined. Trailing blanks +are truncated by the Fortran-callable routines (ut...) but not by the +SPP-callable (tb...) routines. + +4. The format for printing table values supports the more common Fortran +formats with a few differences and some additional options: + + h: H:M:S.d + m: H:M.d (or M:S.d) + i-: left justified (but SPP uses d instead of i) + i0: fill the field with zeros on the left; Fortran also allows filling + a portion of the field (e.g. I4.2) + o: octal + x: hexadecimal + b: "yes" or "no" instead of "T" or "F" + +5. The header parameters and column descriptors are stored in the table +together with the data, so if more parameters are written or more columns +defined than there is space in the table, the table will be rewritten to +allocate more room. If the table is large, this can be a problem either +due to the time involved or because of disk space limitations. When a new +table is created it is possible to specify the amount of space to be +allocated for header parameters and for column descriptors. + + A table may be row-ordered (which is the default) or column-ordered. +If the table is row-ordered then rows may be written at the end of the table +without ever specifying the maximum number of rows. Adding new columns after +creating the table, however, requires rewriting the table unless space was +allocated for new columns. + + There is no intrinsic limit on the number of columns or rows. diff --git a/pkg/tbtables/doc/ex.x b/pkg/tbtables/doc/ex.x new file mode 100644 index 00000000..1e9438de --- /dev/null +++ b/pkg/tbtables/doc/ex.x @@ -0,0 +1,109 @@ +task ttt + +# Example task for creating a table from a text file. The text file +# is assumed to contain five columns per row, containing an integer +# catalog number, a text string, right ascension, declination, and +# magnitude. An example of three lines from the text file could be: +# +# 0172 abcd 0:01:17.865 -89:43:17.62 14.7 +# 0213 "a b" 0:02:29.775 -84:43:17.64 12.8 +# 0490 "" 2:19:21.000 -84:46:22.98 11.5 + +define NCOLS 5 # number of columns to create +# column numbers: +define ID 1 # catalog ID +define NAME 2 # name +define RA 3 # right ascension +define DEC 4 # declination +define MAG 5 # magnitude + +procedure ttt() + +char input[SZ_FNAME] # name of input file +char outtable[SZ_FNAME] # name of output table +#-- +pointer tp # pointer to table struct +pointer cp[NCOLS] # pointers to column info +pointer tbtopn() + +char name[SZ_FNAME] # star name +double ra, dec # coordinates +double mag # magnitude +int cat_id # catalog ID +int row # row number + +char lbuf[SZ_LINE] # buffer for reading from input +int ip, ctoi(), ctod(), ctowrd() +int fd # fd for input file +int open(), getline() + +errchk tbtopn, open + +begin + call clgstr ("input", input, SZ_FNAME) + call clgstr ("outtable", outtable, SZ_FNAME) + + # Open output table (file not created yet, though). + tp = tbtopn (outtable, NEW_FILE, NULL) # NULL --> no template + + # Define columns. The "Name" column is a string up to 20 char long. + call tbcdef (tp, cp[ID], "catalog_ID", "", "%8d", TY_INT, 1, 1) + call tbcdef (tp, cp[NAME], "Name", "", "", -20, 1, 1) + call tbcdef (tp, cp[RA], "RA", "hours", "%12.2h", TY_DOUBLE, 1, 1) + call tbcdef (tp, cp[DEC], "DEC", "degrees", "%12.1h", TY_DOUBLE, 1, 1) + call tbcdef (tp, cp[MAG], "V", "Vmag", "%6.2f", TY_REAL, 1, 1) + # ^ ^ ^ ^ ^ + # output col name units format datatype + + # Create the output table file. + call tbtcre (tp) + + # Add a history record. + call tbhadt (tp, "history", "created as an example") + + # Open the input file. + fd = open (input, READ_ONLY, TEXT_FILE) + + # Read each line from the input file. + row = 0 # initialize + while (getline (fd, lbuf) != EOF) { + + if (lbuf[1] == '#' || lbuf[1] == '\n' || lbuf[1] == EOS) + next # ignore comment or blank lines + + ip = 1 # beginning of line + + # Read the catalog ID. + if (ctoi (lbuf, ip, cat_id) < 1) + next # ignore comment or bad line + + # Read the star name. + if (ctowrd (lbuf, ip, name, SZ_FNAME) < 1) + name[1] = EOS + + # Read the right ascension and declination. + if (ctod (lbuf, ip, ra) < 1) + call error (1, "can't read right ascension") + if (ctod (lbuf, ip, dec) < 1) + call error (1, "can't read declination") + + # Read the magnitude (may be missing). + if (ctod (lbuf, ip, mag) < 1) + mag = INDEFD + + row = row + 1 # increment row number + + # Write the information to the table. The "Vmag" column is + # real in the table, but we're passing it a double, so we use + # tbeptd instead of tbeptr. + call tbepti (tp, cp[ID], row, cat_id) + call tbeptt (tp, cp[NAME], row, name) + call tbeptd (tp, cp[RA], row, ra) + call tbeptd (tp, cp[DEC], row, dec) + call tbeptd (tp, cp[MAG], row, mag) + } + + # Close input file and output table. + call close (fd) + call tbtclo (tp) +end diff --git a/pkg/tbtables/doc/example.doc b/pkg/tbtables/doc/example.doc new file mode 100644 index 00000000..b5f81c78 --- /dev/null +++ b/pkg/tbtables/doc/example.doc @@ -0,0 +1,122 @@ + Here is a sample program that demonstrates creating a new table, +opening an existing table, and creating a new table based on a template. +The example can be compiled and linked using: + +xc -p tables test.x -ltbtables + +You can use xc regardless of whether you are logged into the cl or not; +it doesn't matter. + + +include <tbset.h> + +task ttt + +define MAXROWS 10 # just for local buffer size + +procedure ttt() + +pointer tp, template # pointers to table descriptors +int nrows +int k +pointer outdec # column pointer for output column "Dec" +pointer inra, outra # column pointers for input & output "RA" +bool nullflag[MAXROWS] +double ra, dec, racol[MAXROWS] +pointer tbtopn() +int tbpsta() + +begin +# +# This section creates a new table (without using a template): +# + tp = tbtopn ("ex1", NEW_FILE, 0) # initialize for a new table + + call tbcdef (tp, outra, # define column with name "RA" + "RA", "degrees", "%12.1h", TY_DOUBLE, 1, 1) + + call tbtcre (tp) # open the table + + do k = 1, MAXROWS { # put RA in each row + ra = k/60. + call tbrptd (tp, outra, ra, 1, k) + } + call tbtclo (tp) # close the table +# +# Example for an existing table. +# + tp = tbtopn ("ex1", READ_ONLY, 0) # initialize and open the table + + call tbcfnd (tp, "RA", inra, 1) # find column "RA" + if (inra ==NULL) + call eprintf ("column RA not found\n") + + nrows = tbpsta (tp, TBL_NROWS) # how many rows? + + do k = 1, nrows { + call tbrgtd (tp, inra, ra, nullflag, 1, k) # read RA from each row + if (nullflag[1]) { + call eprintf ("column is null for row %d\n") + call pargi (k) + } else { + call printf ("ra = %12.1h\n") + call pargd (ra) + } + } + call tbtclo (tp) +# +# Example for creating a new table using a template to define column(s). +# + # Initialize and open the template table. + template = tbtopn ("ex1", READ_ONLY, 0) + + # Initialize using template. + tp = tbtopn ("ex2", NEW_COPY, template) + + call tbcdef (tp, outdec, # define new column "Dec" + "Dec", "degrees", "%12.0h", TY_DOUBLE, 1, 1) + + call tbtcre (tp) # open the output table + + call tbcfnd (template, "RA", inra, 1) # find col "RA" in input table + if (inra == NULL) + call eprintf ("column RA not found in input table\n") + + call tbcfnd (tp, "RA", outra, 1) # find col "RA" in output table + if (inra == NULL) + call eprintf ("column RA not found in output table\n") + + nrows = tbpsta (template, TBL_NROWS) # how many rows in template? + if (nrows > MAXROWS) + call error (1, + "input table has too many rows for size of input buffer") + + # Copy column "RA" from template to output. + call tbcgtd (template, inra, racol, nullflag, 1, nrows) + call tbcptd (tp, outra, racol, 1, nrows) + + call tbtclo (template) # we're done with the template + + do k = 1, nrows { # put Dec in each row + dec = -k/60.D0 + call tbrptd (tp, outdec, dec, 1, k) + } + + call tbtclo (tp) # close the output table +end + + The tprint task gave the following output for table ex2.tab: + +# ex2 has 10 rows and 2 columns + +(row) RA Dec + 1 0:01:00.0 -0:01:00 + 2 0:02:00.0 -0:02:00 + 3 0:03:00.0 -0:03:00 + 4 0:04:00.0 -0:04:00 + 5 0:05:00.0 -0:05:00 + 6 0:06:00.0 -0:06:00 + 7 0:07:00.0 -0:07:00 + 8 0:08:00.0 -0:08:00 + 9 0:09:00.0 -0:09:00 + 10 0:10:00.0 -0:10:00 diff --git a/pkg/tbtables/doc/fileformat.doc b/pkg/tbtables/doc/fileformat.doc new file mode 100644 index 00000000..3c9d3ef6 --- /dev/null +++ b/pkg/tbtables/doc/fileformat.doc @@ -0,0 +1,91 @@ + 1999 August 17 + + This note describes the file format for STSDAS tables. See +text_tables.doc for information on support for text tables in the +TABLES package. + +There are four sections to an STSDAS table: + +(1) a size-information record which gives the number of rows, etc +(2) optional header-parameter records +(3) a record for each column that describes the column +(4) the table data + + +1. The size-information record is 12 integers in length. Only the first +ten are used at the moment, and these have the following meanings: + + 1 The number of header parameters that have been written to the table. + 2 The maximum number of header parameters for which space has been allocated. + 3 The number of rows that have been written to the table. + 4 The allocated number of rows (relevant only for a column-ordered table). + 5 The number of columns that have been defined. + 6 The maximum number of column descriptors for which space has been + allocated. + 7 The length of the portion of the row that is used by columns that have + been defined; unit = SZ_CHAR (= two bytes). + 8 The allocated row length; unit = SZ_CHAR. This is relevant only for + a row-ordered table. + 9 Table type: 11 implies row-ordered; 12 implies column-ordered. + 10 Table software version number. Originally this was zero, and the + current value is three. The differences between versions are + described in the file "versions.doc". + + +2. The header parameters are FITS-like records for storing information such +as comments or numerical values. They are not used by the table routines +at all. There need not be any header parameters, and there need not be any +space allocated for them; that is, words one and two of the size-information +record may be zero. Each header-parameter record is 80 bytes in length and +contains the following: + +bytes description +----- ----------- + 1-8 Keyword; from one to eight characters padded on the right with blanks. + All letters will be in upper case. + 9 Data type; a single lower-case letter: t, b, i, r, d indicating a + type of text, boolean, integer, real, or double-precision respectively. + The data type has little real meaning since the value is ASCII. +10-80 The value, a left-justified ASCII string terminated by an ASCII null + and followed by garbage. Boolean true and false are represented by + one and zero respectively. + + +3. There is a column descriptor for each column that has been defined. Each +column-descriptor record has a length of 16 integer words and contains the +following: + + word description + ---- ----------- + 1 The column number. + 2 The offset from the start of the row; unit = SZ_CHAR (= two bytes). + This is the sum of the widths of all previous columns, or zero for + the first column. + 3 The amount of space (unit = SZ_CHAR) required to store one element. + 4 The data type: + 6 = single-precision real + 7 = double-precision real + 4 = integer + 3 = short integer + 1 = boolean + -n = character string containing up to n characters + 5-9 The column name, up to 20 characters, left justified. If the name + is shorter than 20 characters, it will be terminated by an ASCII + NULL; otherwise, the NULL will be omitted. Upper and lower case + are allowed, but they are not distinguished. + 10-14 The units, up to 20 characters, left justified. If the units string + is shorter than 20 characters, it will be terminated by an ASCII + NULL; otherwise, the NULL will be omitted. Upper and lower case + are allowed. + 15-16 The format for printing the column, up to eight characters. If the + string is shorter than eight characters, it will be terminated by + an ASCII NULL; otherwise, the NULL will be omitted. This format + string is an SPP-style format without the leading %. + + +4. The table data may be either row-ordered or column-ordered. Integer, +single- and double-precision reals, and boolean (logical) are stored in the +Fortran binary format of the host machine. For character data type, each +element will be terminated with an ASCII null if there is room for the null; +if an element fills the entire column width (a multiple of two bytes) then +the null will not be present. diff --git a/pkg/tbtables/doc/tbtcpy.lis b/pkg/tbtables/doc/tbtcpy.lis new file mode 100644 index 00000000..a371a618 --- /dev/null +++ b/pkg/tbtables/doc/tbtcpy.lis @@ -0,0 +1,30 @@ +Here is the algorithm for determining the type of output table that +will be created when calling tbtcpy: + + if output == STDOUT then + + type = text + + else if extension of output file is .fits (or .fit or .??f) then + + type = fits + + else if extension of output file is .qp + + type = row # but in a qpoe file + + else if input is text file or STDIN then + + # text --> text, except for extensions .tab, .fits, .qp + + if extension of output file is .tab + type = row + else # .fits & .qp taken care of above + type = text + end if + + else + + type = row + + end if diff --git a/pkg/tbtables/doc/text_tables.doc b/pkg/tbtables/doc/text_tables.doc new file mode 100644 index 00000000..a20a93c3 --- /dev/null +++ b/pkg/tbtables/doc/text_tables.doc @@ -0,0 +1,234 @@ + Text Tables 1999 August 17 + +The TABLES package I/O routines support text tables (ascii files in row +and column format) as well as FITS binary tables and STSDAS format binary +tables. There are limitations on size because the entire file is read +into memory when a text table is opened. Text tables are not as flexible +and certainly not as fast as binary tables, but for small files the ability +to use the table tools and other tasks can be very handy. + +Text tables can be plain ascii files with default column names (c1, c2, c3, +etc.) and no header keywords. However, the text table I/O routines now also +support explicit column definitions and/or header keywords. + +Header keywords have the following syntax: + +#k keyword = value comment + +The "#k " must be the first three characters of the line, and the space +following "k" is required. The "k" is not case sensitive. Header keywords +can be added to any text table, and they can appear anywhere in the file. +For a text string keyword, quotes around the value are needed if there is +a comment, in order to distinguish value from comment. Everything following +the value is considered to be the comment. + +Column definitions have the following syntax: + +#c column_name data_type print_format units + +The "#c " must be the first three characters of the line, and the space +following "c" is required. The "c" is not case sensitive. Aside from the +"#c ", the syntax is the same as the output from tlcol or the input cdfile +for tcreate. Only the column name is required, although in most cases you +will also need to give the data type (the default is d, double precision). + +Adding column definitions to a text table makes it a different "subtype" +(tinfo now prints this). If any column is defined this way, all columns +in the file must be defined, and all column definitions must precede the +table data. + +The print format is used for displaying the table or writing it back out +if the table was modified. The file is still read in free format, with +whitespace (blank or tab) separated columns. This means that text string +columns must be enclosed in quotes if they contain embedded blanks. + +A task that opens a simple text table read-write may change the table to one +with explicit column definitions. This will happen if the task changes a +column name to something other than "c" followed by an integer, or sets the +units to a non-null value, or if it creates a new column with non-default +name or units. In this case, column definitions will be written for all +columns, but the names for columns that weren't modified will still be c1, +c2, c3, etc. Tasks such as tchcol, tcalc and tedit can do this, for example. +Therefore, an easy way to add this information to a simple text table is to +run tchcol and change a column name, say from "c1" to "x". You can then edit +those "#c " lines to set the column names, print format and units. You can +change the data type, too, though it must be consistent with the data in the +file; for example, you could change i to d (integer to double), or ch*3 to +ch*8. + +Here are a couple of examples. + +#This is a simple text table (no column definitions), but it does have +#keywords. Some of the keywords have comments; anything following the +#value is a comment. +#k pi 3.14 +#k keywords "rootname opt_elem cenwave" these are the keywords we need +#k rootname = "o47s01k7m" rootname of the observation set +#k cenwave = 1307 Angstroms +#k opt_elem "E140H" grating name +1 2 3 +4 5 6 + +# This example has explicit column definitions as well as a header keyword. +#c rootname ch*9 +#c description ch*15 "" notes +#c cenwave i i4 angstrom +#c texpstrt d f20.8 "Modified Julian Date" +#k opt_elem = E140H +o47s01k9m "lost data" 1234 5.067942601191E+04 +o47s01kbm "" 1416 5.067945625487E+04 +o47s01kdm OK 1598 5.067949325747E+04 + +For a text table that does not contain explicit column definitions (referred +to as a simple text table), the column names are c1, c2, c3, etc., the data +types and print format are inferred from the data, and there are no units. +Columns should be separated by blanks or tabs. The supported data types are +double precision, integer and character string. Use a ":" to separate parts +of a sexagesimal value, e.g. 3:18:26.2. Except as described above, the "#" +sign is the comment character. Each line of the file is treated as a separate +table row (unless the newline is escaped with a backslash), and the total row +length may be as long as 4096 characters. + +The table routines determine the data type of each column in a simple text +table by examining the values in the column. If the value is numerical but +doesn't contain a decimal point, colon, or exponent, the column is taken to +be integer. You can use INDEF for undefined elements in numerical columns +and "" (or quotes enclosing blanks) for undefined string elements. For an +integer column, however, use INDEFI to indicate the data type. All columns +must be defined in the first line; that is, no other line may have more +columns than the first line has. To a certain extent, this serves as a check +to distinguish ordinary text files from text tables. + +For a simple text table, the print format for each column is determined from +the values in that column. (This is a good reason for using explicit column +definitions.) The precision is set by counting digits in each value, including +trailing zeroes. The field width of a column may be increased by inserting +spaces in front of a value in any row, and the precision may be increased by +appending zeroes to any value in the column. An output table or one opened +read-write is written out using this format, and the intention is that the +result should closely resemble the input table, rather than being reformatted +with a lot of extra space and more digits than are useful. G format is used +for floating point data, except that h and m formats (for HH:MM:SS.d and +HH:MM.d respectively) are also supported. This usually works well for tables +containing only numerical data or when the string columns follow the numerical +columns. Problems determining the field width typically arise when a floating +point column follows a string column, and the strings vary in length. In this +case, each time you open the table read-write the width of the floating point +column expands because of the extra space after the shortest string in the +previous string column. A hard upper limit to the width of about 25 stops +the expansion eventually. + +A character string in an input text table must be quoted if the string +contains whitespace, so that the table I/O routines will be able to tell +that the whole phrase is one table element. This is the case regardless +of whether the table contains explicit column definitions or not. Strings +in an output (or read-write) text table will be enclosed in quotes if they +contain whitespace, when the table is written back to disk. Strings in text +tables may not contain embedded quotes. The upper limit for the length of +a string is 1023 characters (SZ_LINE). + +Blank lines and lines beginning with # are comments (except for the #c and +#k cases described above) and will be ignored on input. For files opened +read-write or new-copy, the comments will be saved and written out at the +beginning of the file. In-line comments are not saved; they will be lost +if a table is opened read-write. + +While the name of a binary table must include an extension, with ".tab" as +the default, the name of a text table need not include an extension. For +this reason it is necessary to specify the extension explicitly for a text +table, even if it is ".tab". STDIN and STDOUT are acceptable names for input +and output text tables, but not for tables opened read-write. Thus you +cannot use STDIN or STDOUT for tcalc because it opens the table read-write. +Other table tools such as tquery, tselect, and tproject can read from STDIN +and write to STDOUT, so you can pipe text through these tasks. + +When running tcalc on a text table, it is generally advisable to create a new +column because the table is modified in-place, and it is possible to clobber +values when changing an existing column. For example, suppose a floating +point column contains three-digit values, and you add 1000000 to that column +using tcalc. The print format could be G6.3, which would be OK for the +original values, but you would need seven digits of precision for the modified +values. The result would be displayed as "1.00E6". Putting the output in a +new column, however, gives you full control over the print format. The +default print format (tcalc.colfmt = "") displays full precision. + +To prevent accidental deletion of text files, tdelete will not delete +text tables unless verify=yes. Tcopy will copy text tables, but it makes +more sense to use copy. + + +Notes about the system subroutines: + +While a text table is being read into memory (by tbzopn), tbcadd is called +to "create" columns, which means that column descriptors are allocated and +filled in, and memory is allocated for the column data. This may be done +even if the table is opened read-only, but we can't call tbcdef for a +read-only table. + +The upper limit on the line length for an input text table is set to 4096 +in tbltext.h. The macro SZ_TEXTBUF is SZ_LINE longer than 4096 because of +the way getlline works. + +BUGS: + +Get text, put text for a non-text input column but text output column does not +work very well. The value is sometimes lost off the end of the string. + +Summary of the text table routines: + +tbzgt.x get element; called by tbegt, tbzcg. +tbzpt.x put element; called by tbept, tbzcp. + +tbzopn.x read an existing text table into memory; + called by tbuopn; calls tbzsub, tbzrds, tbzrdx. +tbzsub.x determines table subtype (explicit or simple); + called by tbzopn; calls tbzlin, tbzkey, tbbcmt. +tbzrds.x read a simple text table into memory; + called by tbzopn; calls tbzlin, tbbcmt, tbzkey, tbzcol, tbzmem. +tbzrdx.x read a text table with explicit column definitions into memory; + called by tbzopn; calls tbzlin, tbbcmt, tbzkey, + tbbecd, tbcadd, tbzmex. +tbzlin.x read (getlline) a line of text, check if comment; + called by tbzsub, tbzrds, tbzrdx. +tbzcol.x define columns (except for print format) based on + values in first row; called by tbzrds; calls tbbwrd, tbcadd. +tbzmem.x read values from line and copy to memory; update info + for print format; called by tbzrds; calls tbbwrd, + tbzt2t, tbzd2t, tbzi2t, tbzi2d, tbzpbt. +tbzmex (in tbzmem.x) reads values from one line, for a table with explicit + column definitions; called by tbzrdx; calls tbzpbt. + +tbbwrd.x read one "word" from input line; interpret as to data type, + field width and precision. +tbzd2t.x change data type of a column from double to text, used + when actual data type was not clear from first row; + called by tbzmem. +tbzi2d.x change data type of a column from integer to double; + called by tbzmem. +tbzi2t.x change data type of a column from integer to character; + called by tbzmem. +tbzt2t.x increase allocated width of a character column; + called by tbzmem. + +tbznew.x open a new text file and call tbzadd to allocate memory + for each column for which we have a descriptor; + called by tbtcre; calls tbzadd. + +tbzadd.x check (& correct) data type; allocate memory for column + values and assign INDEF to each element; + called by tbcadd and tbznew. + +tbzsiz.x reallocate buffers for column values to change the + allocated size (number of rows) of a text table; + called by tbtchs. + +tbzsft.x shift a set of rows either up or down; + called by tbrsft; calls tbznll. + +tbznll.x set all columns in a range of rows to INDEF; called by tbzsft +tbzudf.x set specified columns to INDEF in one row; called by tbrudf. + +tbzclo.x call tbzwrt and deallocate memory; + called by tbtclo; calls tbzwrt. +tbzwrt.x write column values back to text file, and close the file; + called by tbzclo. diff --git a/pkg/tbtables/doc/versions.doc b/pkg/tbtables/doc/versions.doc new file mode 100644 index 00000000..256d8886 --- /dev/null +++ b/pkg/tbtables/doc/versions.doc @@ -0,0 +1,29 @@ + 1999 August 17 + +This file describes changes in the STSDAS table format for the various +software versions, from 0 to 3. For an STSDAS table, the software version +number can be found in the size-information record; see fileformat.doc. + +version = 0 (before stsdas and tables package version 1.3): +This is the original version. Data types supported were single and double +precision real, integer, logical, and character strings. Each table element +was a multiple of four bytes in length (i.e. the size of single precision +real, integer, or logical). The row length was also a multiple of four +bytes. Data type short (integer*2) was not supported. + +version = 1 (1993 May 5, tables package version 1.3): +Data type short is supported, and the length of a character string is rounded +up to a multiple of two instead of four. The row length (used or allocated) +can be an odd multiple of SZ_CHAR (i.e. a multiple of two bytes instead of +four bytes). + +version = 2 (1995 Oct 11, tables package version 1.3.4): +Header parameters can have associated comments. The value of a header +parameter that is of type text may optionally be enclosed in single quotes; +this is to allow a comment following the value. Header parameters with +keyword HISTORY, COMMENT or blank are not in quotes and may not have comments. + +version = 3 (1998 Apr 14, tables package version 2.0.2): +The character strings in column definitions can have one more character +than before. This was done by not including the end-of-string character if +the string fills the available space. |