diff options
Diffstat (limited to 'unix/os/doc/os.ms')
| -rw-r--r-- | unix/os/doc/os.ms | 4249 |
1 files changed, 4249 insertions, 0 deletions
diff --git a/unix/os/doc/os.ms b/unix/os/doc/os.ms new file mode 100644 index 00000000..61aa14fe --- /dev/null +++ b/unix/os/doc/os.ms @@ -0,0 +1,4249 @@ +.RP +.ND +.TL +A Reference Manual +.br +for the +.br +IRAF System Interface +.AU +Doug Tody +.AI +.K2 "" "" "*" +May 1984 +.AB +The IRAF system interface is the interface between the transportable IRAF +system and the host operating system. An overview of the software design +of the IRAF system is presented, naming the major interfaces and +discussing their relationships. The system interface is shown to consist +of a language interface, the subset preprocessor (SPP), and a procedural +interface, the IRAF kernel. The reasoning which led to the choice of +a Fortran preprocessor for the language interface is reviewed. A reference +manual for the IRAF kernel is presented, followed by the detailed technical +specifications for the kernel procedures. +.AE + +.NH +Introduction +.PP +The IRAF system libraries currently total 42 thousand lines of code. +The applications software adds another 75 thousand lines +of code, about half of which was imported (i.e., the graphics utilities and +math library routines). The UNIX implementation of the machine dependent +portion of the IRAF system consists of some 3300 lines of code, or about +3 percent of the current system. The remainder of the system, i.e., +approximately 97 percent of the current system, is machine and device +independent. It is this 3 percent of the IRAF system which is machine +dependent, known as the \fBsystem interface\fR, which is the focus of this +document. +.PP +The importance of maximizing the transportability of a large software system +cannot be overemphasized. The IRAF system is required to run on +a variety of different computers and operating systems from the time of +its first release to the end of its useful life. The computer the IRAF +system is being developed on is already old technology. Two years from +now when IRAF is a mature system, it will almost certainly contain 20 to +30 manyears of software. With the increasing dependence on computers for +scientific data analysis, and the demand for increasingly powerful software, +it is no longer possible to keep up with demand if we have to throw out +our systems every 5 or 10 years and start over. +.PP +Commercially developed operating systems/programming environments such as +UNIX and ADA offer some hope for the future. At present, however, ADA is +not widely available and there are at least half a dozen versions of UNIX +in use, with no clear cut standard yet to emerge. The different versions +of UNIX resemble each other, but there are many differences. UNIX has been +steadily evolving for ten years and there is no reason to expect that the +process will stop now. +.PP +Many manufacturers offer machine dependent extensions to get around the +inefficiencies of basic UNIX, and it is desirable to be able to take advantage +of these in a production system. Even in a hardcore UNIX system like 4.2BSD, +significant efficiency gains are possible by taking advantage of the +peculiarities of the 4.2BSD kernel, e.g., by always doing file transfers +in units of the machine page size, into buffers aligned on page boundaries. +In a large system it is difficult to take advantage of such machine +peculiarities unless the system has a well defined and well isolated interface +to the host system. There is no denying the fact that despite the many +attractive aspects of UNIX, it is nice to have the option of switching to +a more efficient operating system if IRAF is to be used in a production +environment. +.PP +Perhaps the most powerful argument is that despite the increasingly widespread +use of UNIX, many of the sites for which IRAF is targeted do not or can not +run UNIX on their current computers. The increasing availability of +transportable operating systems will make transporting IRAF easier, but is no +substitute for a well defined and well isolated system interface. +.PP +The \fBsystem interface\fR is the interface between the machine independent +IRAF software and the host operating system. +The system interface consists of a procedural interface, the IRAF +\fBkernel\fR, and a language interface, the IRAF Subset Preprocessor +language (SPP). Both types of interface are required to isolate the +IRAF software from the host system. +.PP +We first present an overview of the structure of the IRAF system, naming the +major interfaces and explaining their functions. The system interface is +introduced and the role it plays in the full system is described. +The choice of an implementation language for IRAF is next discussed, +concentrating on the role the language interface plays in addressing the +transportability problem. Next we define the kernel and discuss its attributes. +The conceptual model of the host operating system, or \fBvirtual machine +model\fR assumed by IRAF is presented. The design and functioning of the +kernel is discussed in detail, followed by the detailed specifications +(\fBmanual pages\fR) for the subroutines constituting the actual interface. +.PP +It is not necessary to carefully read the first part of this document +to implement the IRAF system interface for a new host operating system. +The first part of this document (the \fBreference manual\fR) concentrates on +the role the system interface plays in IRAF, the principles underlying +the design of the interface, and on how and why the interface came to be +what it is. The second part of the document (the \fBmanual pages\fR) +presents the detailed specifications for the individual routines comprising +the system interface, and is intended to be self contained. The reference +manual tells what modules the interface consists of, how they work together, +and why they are designed the way they are. The manual pages tell +precisely \fIwhat\fR each routine does, with no attempt to explain why, +or how the routine fits into the system. +.PP +Interfacing to new \fBgraphics devices\fR is likely to be one of the most +difficult problems to be solved in porting the IRAF system. Nonetheless +the graphics device interface is not (or should not be) part of the system +interface, and is not discussed here. Ideally, a graphics device will be +interfaced to IRAF file i/o as a binary file. The device should +have a data driven interface, rather than a control interface, i.e., +all device control should be effected by inserting control instructions in +the data stream transmitted to the device, rather than by calling system +dependent subroutines. The software required to drive the device should +be device independent, table driven, and fully portable. Virtually all +graphics devices either fit this model, or can be made to fit this model +by writing system dependent device drivers to interpret metacode generated +by the high level, machine and device independent software. + +.NH +Structure of the IRAF System Software +.PP +The major components of the IRAF system are +the high level applications and systems \fBprograms\fR and \fBpackages\fR, +the Command Language (CL), also known as the \fBuser interface\fR, +the \fBprogram interface\fR (library procedures called by programs), +and the \fBsystem interface\fR. The system interface is further subdivided +into the \fBkernel\fR and the \fBlanguage interface\fR. +Other system modules not relevant to our discussion are the math library +and the graphics device interfaces. +.PP +The relationship and relative importance of these modules is strongly +dependent upon one's point of view; all points of view are equally valid. +From the point of view of the (highly idealized) user, looking at the +system from the top level, the user is at the center of the system and +is in command. The CL appears to be the central piece of software, and +applications packages are mere extensions of the CL. To the extent that the +user is aware of the host system, the CL appears to be the interface +between the user and the host system. The user expects the system to behave +predictably and reliably, and be responsive to their commands. +To a first approximation, the user does not care what language programs +are written in, or how they are interfaced to the host system (real users, +of course, are often programmers too, and do care). +.PP +From the point of view of the applications programmer, the program they +are writing is at the center of the system and is in command (and indeed +the program does control the system when it is run). The programmer sees +only the abstract problem to be solved and the environment in which the +solution must be constructed, defined by the program interface and the SPP +language. The CL is an abstract data structure, accessed via the CLIO +library in the program interface. The fact that there is a host system +underlying the program interface is irrelevant, and is of no concern to the +applications programmer. As far as the applications programmer is concerned, +the CL could be CL1, CL2, a file, the host OS command interpreter, Forth, +Lisp, or anything else. Ideally, the applications programmer does not know +that the target language of the SPP is Fortran. +.PP +From the point of view of the systems programmer, the kernel is the center +of the system, with the host operating system below and the program interface +above. The system software is subservient to the program which calls it, +and does exactly what it is told to do and nothing more. The CL and the SPP +compiler are \fIapplications programs\fR which use only the facilities +of the program and system interfaces. +.PP +The structural design of the IRAF software is outlined in the figure below. +In general, control and parameters flow downward and data flows both both +downward and upward (mostly upward). The procedures at one level do +not call procedures at a higher level, e.g., kernel routines are +not permitted to call procedures in the program interface libraries. +A procedure may call other procedures at the same level or in the next +lower level, but calls to routines more than one level lower are avoided, +i.e., a program should not bypass the program interface and talk directly +to the kernel. IRAF applications programs \fInever\fR bypass the system +interface to talk directly to the host system. + +.KS +.ce +\fBStructure of the Major IRAF Interfaces\fR +.sp +.nf + \fIuser interface\fR + high level program + \fIprogram interface\fR + \fIsystem interface\fR + host operating system +.fi +.KE + +.PP +This structure chart illustrates only the control hierarchy of the major +interfaces. Additional structure is imposed on the actual system. +Thus, the CL uses only a small portion of the facilities provided by the +program interface, calling external programs to perform most functions. +Few system or applications programs, on the other hand, use the process +control facilities, the part of the program interface most heavily used +by the CL (apart from file i/o). The CL and external applications +programs are minimally coupled, using only human readable text files for +interprocess communication (interprocess communication is implemented as +a special file in IRAF). +.PP +These restrictions tend to result in more functional programs which can be +combined in many ways at the CL level, and which can be used quite +productively without the CL. In particular, any IRAF program can easily +be debugged without the CL, using the host system debugger, and any IRAF +program can be used productively on a system which cannot support the CL. +The system perceived by the user at the CL level can easily be extended +or modified by the user or by a programmer. +.PP +The \fBprocess structure\fR of the IRAF system, with the CL serving +up and networking all processes, while fundamental to the design of +the IRAF system, is not relevant to a discussion of the system interface +because it is all handled above the system interface, i.e., in the machine +independent code. Many quite different process structures are possible +using the one IRAF system interface. Concentration of most or all of the +complex logic required to implement process control, the CL process cache, +the CL/program interface, pseudofiles, multiprocessing, i/o redirection, +exception handling and error recovery, efficient file access, etc., +into the machine \fIindependent\fR code was a major goal of the IRAF +system design. + +.NH +The IRAF System Interface +.PP +As has already been noted, the IRAF system interface consists of both a +procedural interface or kernel (library of Fortran callable subroutines), +and a language interface (preprocessor for Fortran). All communication +with the host system is routed through the kernel, minimizing the machine +dependence of the system and providing scope for machine dependent +optimizations. Similarly, all code other than existing, imported numerical +Fortran procedures is processed through the language interface, further +minimizing the machine dependence of the system. The language interface, +or subset preprocessor (SPP), isolates IRAF programs from the peculiarities +of the host Fortran compiler, provides scope for optimization by making it +possible to take advantage of the nonstandard features of the compiler +without compromising transportability, and does much to correct for the +defects of Fortran as a programming language. +.NH 2 +The Language Interface +.PP +The kernel alone is not sufficient to solve all the problems of +transporting a large software system. There remain many problems associated +with the programming language itself. The many problems of transporting +even purely numerical software are well known and we will not attempt to +discuss them in detail here. The reasoning which led to the decision to +implement the IRAF system in a Fortran preprocessor language (SPP) is less +obvious, and is probably worth recounting. In the process of retracing +the logic which led to the decision to develop SPP, we will come to understand +the purpose of the language interface. +.PP +For some reason programming languages are one of the most controversial +topics in all of programming. No language is perfect, and it is important +to try to objectively gauge the advantages and disadvantages of a language +for a particular application. It is difficult to make such comparisons +without the injection of opinion, and for that we apologize. In the final +analysis the choice of a language is probably not all that important, +provided the cost of the project is minimized and the resultant code is +reliable, portable, readable, and efficient. Only the Fortran and C languages +are discussed; these were the only candidates seriously considered in 1982, +when the decision to implement the IRAF system in a Fortran preprocessor +language was made. +.NH 3 +Fortran +.PP +Consider the typical scientific applications program. Such a program may +need to talk to the CL, access files, access images, access databases, +dynamically allocate memory, generate graphics, perform vector operations, +and so on. These are all functions provided by the program interface. +In addition, in a scientific program there will often be some central +transformation or numerical computation which will almost certainly be +performed by a Fortran subprogram. There is an enormous amount of high +quality Fortran numerical and graphics software available, both commercially +and in the public domain, which IRAF programs must have access to. A third +of the current IRAF system consists of highly portable, numerical (no i/o) +Fortran code, all of it in the public domain. +.PP +To be useful, these general purpose, numerical Fortran subroutines must +be integrated into an IRAF program to perform some specific function. +Here we run into the first serious problem: while Fortran is great for +numerical procedures, it has many defects as a general purpose programming +language. When it comes to complex systems software, Fortran is vastly +inferior to a modern programming language such as C or Pascal. It is +very difficult to implement complex nonnumerical applications in +standard Fortran; the temptation to use a manufacturer's nonstandard +language extensions is often too difficult to resist, and a nonportable +program results. Clearly, Fortran is not the language of choice for the +IRAF system software, in particular the program interface. +.PP +The next question is what to do about the high level part of the scientific +applications program, the part which talks to the program interface, +performs applications specific functions, and eventually calls the numerical +Fortran procedures. In a large scientific package Fortran subprograms +will almost certainly be used somewhere in the package, sometimes quite +heavily, but the bulk of the software is often very similar to systems +software, concerned with allocating resources, managing data structures, +doing i/o of various kinds, and directing the flow of control. +.PP +This is all nonnumerical programming, for which Fortran is poorly suited. +Many Fortran compilers provide nonstandard language extensions which make +it easier to code nonnumerical applications in Fortran. Most applications +programmers and scientists are not intimately familiar with the Fortran +standard, and are more interested in getting a program working than in making +it portable, and nonportable code will result. In any case, programmers +and scientists should not have to struggle to code an application in +a language which was designed for another purpose. We conclude that Fortran +is not the language of choice for general scientific programming within a +complex system. +.PP +A more serious problem with using Fortran for general scientific programming, +however, is that the high level portion of an IRAF scientific program depends +heavily on the capabilities of the program interface. To produce sophisticated +scientific programs with minimum effort we must have a large and powerful +program interface, providing the capabilities most often needed by scientific +programs. We also need a large and powerful program interface for +\fIsystem\fR programs, and in fact the capabilities required by scientific +programs are not all that different than those of system programs, so +clearly it is desirable if the same program interface can support both +kinds of programs. +.PP +The next step is to explore the implications of the heavy dependence of +a systems or scientific program on the program interface. The program +interface is a large interface, consisting of a dozen subsystems containing +several hundred procedures. To have a sophisticated, high level, efficient +interface, it is necessary to use "include" files to parameterize argument +lists, machine parameters, and data structures, we must use dynamic memory +management (pointers) for buffer allocation, and something has to be done +about error handling and recovery. In short, there is a lot of communication +between high level programs and the program interface. This level of +communication is only feasible if the program interface and the high level +program are written in the \fIsame language\fR. Standard Fortran provides +\fIalmost no language support\fR for these facilities. +.sp +.KS +.LP +To summarize our reasoning to this point: +.RS +.IP \(bu +Fortran must be used extensively in the scientific applications. +.IP \(bu +Fortran is not the language of choice for IRAF systems software, +in particular the program interface. +.IP \(bu +Fortran is not the language of choice for general scientific programming, +because most scientific programming is nonnumerical in nature, i.e., +much like systems programming. +.IP \(bu +A single program interface should support both systems programs and +scientific programs. +.IP \(bu +The level of communication required between a high level program and the +program interface requires that both be written in the same language. +.IP \(bu +Standard Fortran provides almost no language support for include files, +globally defined parameters, dynamic memory management and pointers, +data structures, or error recovery. These facilities are required by +both systems and applications software. +.RE +.KE +.NH 3 +Mixing C and Fortran in the same System +.PP +All of our logic to this point forces us to conclude that standard Fortran +just is not suitable for the bulk of the IRAF software. The complexity of +large applications packages, not to mention that of the system software, +would be unmanageable in straight Fortran. Nonetheless we must +still face the requirement that a third or so of the system be existing, +imported numerical Fortran procedures. The obvious question to be answered +is, if Fortran is such a poor choice for the main programming language +of IRAF, can we program in an appropriate modern structured language like C, +calling the numerical Fortran functions and subroutines from within C programs? +.PP +The answer is sure we can, if our target system supports both a Fortran +compiler and a C compiler, but there are serious portability implications. +Furthermore, when examined closely C turns out to be not that great a +language for general scientific programming either, and the defects of C +cannot easily be fixed, whereas those of Fortran can. +.PP +Mixing two different languages in the same program is straightforward on +many operating systems and possible with difficulty on others. No language +standard could attempt to specify how its subprograms would by called by a +completely different, unspecified language, so the one thing we can be +sure of is that the method used will be system dependent. Even on systems +where mixing subprograms from different languages is straightforward, +there are many potential problems. +.PP +Argument lists present many problems. +A potentially deadly problem is the fact that C is a recursive +language while (standard) Fortran is not. C expects the arguments to a +procedure to be passed on the stack, while Fortran was designed with static +storage of argument lists in mind; static storage is somewhat more efficient. +Arguments pushed on a stack are usually pushed in the reverse of the order +used for static allocation. C is call by value; Fortran is call by +reference. Returning a function value is trivial on many systems, +but can be a problem. The Fortran standard requires that a function be +called as a function (in an expression) and not as a subroutine, while +C permits either type of call (all C procedures are functions). +.PP +The method used to implement Fortran character strings is machine dependent; +some machines may pass two items in the argument list to represent a +character string, while others pass only one. On some machines a Fortran +character string is implemented with a count byte, on others an end of string +marker is used. The C standard requires that a character string be delimited +by a trailing zero byte. Thus, on some systems C and Fortran character strings +will be equivalent; programs written on such a system are not portable +to to systems where strings are implemented differently in the two languages. +.PP +Global variables are likely to be a problem, because Fortran common blocks +and C external variables are quite different types of data structures. +Though it would be poor programming practice to use global variables +or common blocks to pass data between C and Fortran procedures, it is +sometimes justified for control parameters, in particular in connection +with error handling. A C include file cannot be accessed from a Fortran +program. +.PP +Finally, external identifiers have a set of problems all of their own. +On some systems, e.g. VMS and AOS, C and Fortran external identifiers are +equivalent, and a procedure is referred to by the same name in both +languages. This makes it easy to mix calls in the two languages, +but can lead to serious name conflicts in libraries. On other systems, +e.g. UNIX, the external identifiers generated for the two languages are +\fInot\fR equivalent, and a C procedure cannot be called from Fortran +unless special provisions are taken (the UNIX Fortran compiler adds an +underscore to all Fortran external identifiers). +.PP +The problem is not that it is hard to mix the two languages, but that +every one of the points mentioned above is a potential machine dependency +which is not covered by any standard. We conclude that mixing C and +Fortran in the same program is inevitably going to be machine dependent. +The problem is controllable only if the number of procedures common to +the two languages is small, or if some automated technique can be developed +for interfacing the two languages. The former solution is straightforward, +the second requires use of some form of \fIpreprocessor\fR to isolate the +machine dependencies. +.PP +C and Fortran should not be mixed in applications software because the +number of Fortran procedures involved is potentially very large. +Since the number of procedures is large, the effort required to port the +applications is also potentially large, and it is the applications which +change most between system releases. If the applications total, say, +200 thousand lines of code, a new release of the system occurs every 3 months, +and it takes two weeks to make all the changes in the high level software +necessary to port it, then we have a bad situation. Porting the system +should ideally be a simple matter of reading a tape, possibly modifying a +config file or two, and running diagnostics. +.PP +C and Fortran should not be mixed in the program interface (in the system +libraries) because it introduces machine dependency into the program +interface where formerly there was none. Less obviously, the problem +discussed in \(sc3.1.2 of communication between modules written in different +languages is very serious. The modules of the program interface use +global \fBinclude\fR files to communicate with one another, and to +parameterized the characteristics of the host system. To ensure a reliable +and modifiable system, there must be only one copy of these include files +in the system. Furthermore, the IRAF error handling scheme, employed +in all code above the kernel, is based on the use of a Fortran common +and to access this from C code would be awkward and would introduce additional +machine dependence. +.PP +The only remaining alternative is to write applications programs entirely in +Fortran and the system software in C, using a small program interface between +the two parts of the system. The problems with this approach were noted in +the last section. The small program interface will inevitably prove too +restrictive, and more and more scientific programs will be written as +"system" programs. These programs will inevitably want to use the numerical +Fortran libraries, and the problem of a large interface between two languages +resurfaces at a different level in the system. +.NH 3 +Critique of C as a Scientific Language +.PP +The major strengths of C as a programming language are that it lends itself +well to structured, self documenting programming, has great expressive power, +strong compile time type checking, tends to result in highly transportable +code (when not mixed with other languages), and is efficient for a large +range of applications. C has been widely used in computer science research +for a decade, and many high quality systems applications are available in +the public domain. +.PP +As a scientific programming language, however, C has serious shortcomings; +C was not designed to be a scientific programming language. The major problem +with C as a scientific programming language is that the scientific community +already has such a large investment in Fortran, and it is difficult to mix +the two languages, as we have already discussed. Upon close analysis we find +that there are additional problems, and these deserve mention. +.PP +C does not support multidimensional arrays. C provides something similar +using pointers, but the feature is really just a side effect of the +generality of pointers, and is not fully supported by the language. +A multidimensional array cannot be passed to a subprogram along with its +dimensions as it can in Fortran. Few C compilers optimize loops, i.e., +evaluate common subexpressions (such as array subscripts) only once, +or remove constant expressions from inner loops. These problems can +be overcome by sophisticated use of pointers, but such hand optimization +of code is extra work and increases the complexity of the code. +On a machine such as the Cray, which has a Fortran compiler that +recognizes vector operations, the problem would be even more severe. +.PP +Char and short integer (8 and 16 bit) expressions are evaluated using +integer instructions (32 bits on a VAX), and single precision floating +expressions are evaluated using double precision instructions. +In a tight loop, this will require the addition of a type conversion +instruction to promote a variable to the higher precision datatype, +and another to convert back to the original datatype after the evaluation. +This alone can double the size and execution +time of a tight loop. In addition, on many machines double precision +floating is not well supported by the hardware and is several times more +expensive than single precision floating. C does not support the +\fBcomplex\fR datatype, important in some scientific applications. +.PP +The C language does not include support for \fBintrinsic\fR and \fBgeneric\fR +functions. These are used heavily in scientific applications. Typed function +calls are required to access the scientific functions, and to perform +exponentiation. I am not aware of any C standard for the scientific functions, +though most systems appear to have adopted the Fortran standard. +The scientific functions, e.g., the trigonometric functions, are evaluated +in double precision. This could lead to a serious degradation +of performance in a large class of applications. +.PP +Despite these quite serious shortcomings, faced with the task of coding a +large and complex application I would prefer C to Fortran, if I had only +the two languages to choose from. Fortunately there is a third alternative, +the use of a Fortran preprocessor. +This approach preserves the best features of Fortran while providing many +of the nice features of a modern language such as C, and in addition allows +us to provide language level support for the IRAF i/o facilities. +The preprocessor approach provides a means of isolating both systems and +applications code from the underlying host compiler, making portability +a realistic goal. +.NH 3 +The IRAF Subset Preprocessor Language +.PP +The Subset Preprocessor language (SPP) is a precursor to a full language +scheduled for development in 1986. The subset language is a fully defined, +self contained language, suitable both for general programming and for +numerical scientific programming. The basic language is modeled after +both C and Ratfor but is a distinct language and should not be confused +with either. SPP is fully integrated into the IRAF system, i.e., +SPP provides substantial language support for the program interface, +the IRAF system itself is written in SPP, and the SPP compiler is an IRAF +applications program written in SPP and using the facilities provided by +the program interface (this is not true of the original preprocessor but +that is not relevant to the design). The syntax of the SPP language is +nearly identical to that of the IRAF command language. +.PP +The SPP language is defined in the document \fIA Reference Manual for the +IRAF Subset Preprocessor Language\fR. The language provides modern +control flow constructs, a wide range of datatypes, support for both system +and user \fBinclude\fR files, a macro definition facility, free format +input, long (readable) identifiers, C-like character constants and strings, +Fortran-like arrays, access to the standard Fortran intrinsic and generic +functions, powerful error handling facilities, and limited but adequate +support for pointers, automatic storage allocation, and data structuring. +Since the target language is Fortran, there is no problem calling Fortran +subprograms from SPP programs or vice versa. We do require, however, +that the Fortran subprograms be purely numerical in nature, i.e., +no Fortran i/o is permitted. +.PP +The function of the preprocessor is to translate an SPP source file +into a highly portable subset of ANSI-66 Fortran. The transformation is +governed by a set of machine dependent tables describing the characteristics +of the host computer and of the target Fortran compiler. These tables +must be edited to port the preprocessor to a new machine; the tables are +ideally the only part of the preprocessor which is machine dependent. +.PP +Even if it should turn out that all of the necessary machine dependence +has not been concentrated into the tables, however, it will often be possible +to port the hundreds of thousands of lines of code in the system by +modifying or adding a few lines of code to the preprocessor, +\fIbecause we have placed an interface between the language of the IRAF system +and that of the host computer\fR. The language interface provides both +a solution to the problem of transporting software between different +contemporary machines, and protection from future changes in the Fortran +language. +.PP +The principal motivation of the preprocessor approach taken in IRAF is +that it provides a real solution to the transportability problem, i.e., +one which does not depend upon perfect programmers. An additional incentive +is that by defining our own language we can provide excellent support the +IRAF i/o facilities, i.e., they can be more than just subroutines and +functions. +.PP +If one has the freedom of being able to modify the programming language +used by applications programs, one can do things that are impractical to +do any other way. In other words, it becomes feasible to solve problems +that were formerly too difficult to address. An example is the use of +lookup tables to implement blocked storage of images, an alternative to +line storage mode which is superior in a large class of image processing +applications. To do this well requires language support, and it is unlikely +that such support will be found in any standard, general purpose programming +language. The SPP is intended partially to provide a path for the future +development of the IRAF system, in the hope that the system will be able +to evolve and be competitive with new systems in coming years. +.NH 3 +Limitations of the Subset Preprocessor +.PP +No compiled language, SPP included, can guarantee transportability. +SPP programs developed on one machine will normally have to tested on +one or more other machines before they can be declared portable. +SPP suffers from many of the same portability problems as standard +Fortran, the main difference being that the output Fortran is very +simple, and nonstandard language extensions are strictly controlled. +Further discussion of the portability aspects of SPP programs is given +in the document \fIIRAF Standards and Conventions\fR. +.PP +We hope to eventually have IRAF running on several types of machines +at the development centers. New releases will be brought up and tested +on several different machines before distribution. +No large software package developed on a single system is portable; +if it is tested on even two different systems, that is much better. +.PP +The SPP language also depends on certain side effects which are not +specified in the Fortran standard, but which many other systems also +depend upon and which are commonly permitted by Fortran compilers. +These include: +.RS +.IP [1] +It must be possible to reference beyond the bounds of an array. +.IP [2] +It must be possible to reference a subarray in a call to a subprocedure, +i.e., "call copyarray (a[i], b, npix)". +.IP [3] +The compiler should permit a procedure to be called with an actual +argument of type different than that of the corresponding dummy argument +(except type \fBcharacter\fR: SPP does not use this type anywhere). +.IP [4] +It must be possible to store a machine address or the entry point +address of an external procedure in an integer variable. +.RE +.LP +Common language extensions used by the preprocessor in output code include +the nonstandard datatypes such as INTEGER*2, any associated type coercion +functions (INT2), and the boolean intrinsic functions if provided by the +target compiler. +.PP +The output of the current preprocessor is ANSI-66 Fortran only to a first +approximation. The following features of Fortran 77 are also used: +.DS +general array subscripts +zero-trip do loop checking +\fBsave\fR statement +\fBentry\fR statement +generic intrinsic functions (\fBmax\fR rather than \fBamax0\fR, etc.) +.DE +All of these extensions are correctable in the preprocessor itself +except the use of the \fBentry\fR statement, if it should ever prove +necessary. + +.NH 2 +Bootstrapping the System +.PP +Since the SPP is fully integrated into the system -- the program interface +is written in SPP and SPP uses the program interface, one may have been +wondering how to go about getting it all set up in the first place. +The basic procedure for porting IRAF to a new system is as follows. +The actual procedure has not yet been fully defined and will be tailored +to the individual target systems, i.e., there will be a separate +installation guide and distribution package for UNIX, VMS, DG/AOS, +and any other systems supported by the IRAF development team. +.RS +.IP [1] +Implement and test the kernel routines. +.IP [2] +Compile the bootstrap SPP, which has already been preprocessed for the +target machine (the datatype used to implement \fBchar\fR must match that +expected by the kernel procedures). +.IP [3] +Edit the system dependent files \fBiraf.h\fR, \fBconfig.h\fR, and the +preprocessor tables to define the characteristics of the host machine. +.IP [4] +Preprocess and compile the production SPP. +.IP [5] +Do a full \fBsysgen\fR of the system libraries (the program interface etc.). +This takes a couple hours on the VAX/UNIX 11/750 development system; get +the config tables right the first time. +.IP [6] +Run diagnostics on the system library procedures. +.IP [7] +\fBMake\fR the applications packages. +.RE +.PP +Once the system has been ported, installing a new release requires only +steps 5 through 7 (sometimes step 4 may also be required), after reading +the distribution tape. + +.NH 2 +The IRAF Kernel +.PP +The IRAF kernel is a set of Fortran callable subroutines. +Every effort has been made to make these primitives as simple as possible; +the kernel primitives provide raw functionality with minimum overhead. +Ideally the kernel primitives should map directly to the kernel of the +host operating system. The implementation of the kernel primitives +should emphasize simplicity and efficiency; these primitives embody the +machine dependence of IRAF and there is little reason to try to make them +machine independent. Critical routines should be coded in assembler +if a substantial gain in efficiency will result. In IRAF, \fIall\fR +communication with the host system is routed through the kernel, +so kernel efficiency is paramount. +.PP +With very few exceptions, applications programs and high level systems +modules do not talk directly to the kernel. The kernel primitives are +called only by the routines comprising the core of the IRAF \fBprogram +interface\fR, the lowest level of the machine independent part of the +IRAF system. The core of the program interface, i.e, file i/o, process +control, exception handling, memory management, etc., combined with the +kernel, constitute a \fBvirtual operating system\fR, the heart of IRAF. +The virtual operating system approach is the key to maximizing +transportability without sacrificing either functionality or efficiency. +.PP +Ideally all of the machine dependence of the IRAF system is concentrated +into the kernel, which should be as small and efficient as possible while +offering sufficient raw functionality to support a large and sophisticated +system. In practice it is possible to come quite close to this ideal, +although the range of host systems on which the kernel can be implemented is +finite, being inversely proportional to the richness of function +provided by the kernel. We did not consider it acceptable to provide +transportability at the expense of a restrictive and limited program +interface, so IRAF has a large and quite sophisticated program interface +which depends upon a sizable kernel. The IRAF kernel embodies much of +the functionality provided by the typical minicomputer operating system, +and should be implementable on a wide range of such systems. +.NH 2 +The Virtual Machine Model +.PP +The virtual machine model is the conceptual model of the host machine assumed +by the kernel. The difficulty of implementing the kernel on a given host +depends on how closely the model matches the host operating system. In general, +the older large batch oriented machines do not match the model well, +and it will be difficult to port the full IRAF system to such a machine. +At the other end of the scale are the small 16 bit minicomputers; these +machines do not have a sufficiently large memory addressing range to run IRAF. +In the middle are the multiprocessing, multiuser, terminal oriented supermicro +and minicomputers with large address spaces and large physical memories: +these are the machines for which the IRAF system was primarily designed. +.PP +Our intent in this section is to summarize the most important features of +the virtual machine model. Much of the material given +here will be presented again in more detail in later sections. The design +of the IRAF system is such that there are actually two distinct virtual +machine models, the full model and a subset model. The subset model assumes +little more than disk file i/o, and will be presented first. +.NH 3 +The Minimal Host Machine +.PP +Even though it may be difficult to run the \fIfull\fR IRAF system on a large +batch oriented (timesharing) machine, it should still be possible to run most +of the science software on such a system. The IRAF system was designed such +that the science software is placed in separate processes which can be run +independently of the CL and of each other. +These processes actually use only a portion of +the full system interface; most of the system and device dependence is in +the CL and the graphics control processes, i.e., in the user interface, +which is what one has to give up on a batch machine. +.PP +On the typical batch oriented timesharing system, the IRAF applications +programs would be run under control of the host job control language, +reading commands and parameters from an input file, and spooling all +textual output into an output file. The IRAF command language would not +be used at all; this mode of operation is built into the present system. +Little more than file i/o is required to run a program on such a system, +though dynamic memory allocation is highly desirable. Exception handling +and error recovery can be done without if necessary; process control is not +used by applications processes. The i/o subsystems required by an +applications program are CL i/o, image i/o, and database i/o, each of +which is built upon file i/o. +.PP +Applications programs that produce graphics write device independent +metacode instructions, rather than talking directly to a graphics device, +so a graphics device interface is not required to run an applications +program. Graphics output can be discarded, or the output file can be +postprocessed to generate graphics hardcopy. +Graphics input (cursor readback) is parameterized, so any program that +normally reads a cursor can just as easily read coordinates directly from +the input file. +.PP +All of the software in the science modules is Fortran or Fortran based +(C is used only in the CL and in some kernels), so only a Fortran compiler +is required. A C compiler is currently required to compile the command +language, though it is not necessary to have such a compiler on every +physical machine. We can compile the CL on one machine and distribute +the object code or executables to other machines of the same type; this +will be done, for example, for VAX/VMS. +.PP +If it is necessary to port IRAF to a machine which +does not have a C compiler, it is feasible to code a basic, stripped down +command language in SPP in a few weeks. A command language capable of +executing external processes, parsing argument lists, managing parameter +files, and redirecting i/o is sufficient to run most of the IRAF software, +the main loss being CL scripts. Virtually all of the system and science +software is external to the CL and would be unaffected by substitution of +a new CL. +.NH 3 +The Ideal Host Machine +.PP +The minimal IRAF target machine therefore need offer little more than +file i/o and a Fortran compiler. One would have to do without a nice user +interface, but it should still be possible to do science in the fashion it +has traditionally been done on such machines. +Fortunately, however, minicomputers of modern design are widely available today +and will be increasingly available in the future. We expect that most IRAF +implementations will be ports of the full system onto a modern supermicro or +minicomputer. Such a host system should provide the following general classes +of facilities: + +.DS +.RS +.IP \(bu +file i/o, file management +.IP \(bu +process control +.IP \(bu +exception handling +.IP \(bu +memory management +.IP \(bu +date and time +.IP \(bu +bit and byte primitives +.RE +.DE + +Most of the complexity of the kernel is in file i/o, process control, +and exception handling. File management (file deletion, renaming, etc.) +is straightforward on virtually any system. Memory management can be +some work to implement, but many systems provide dynamic allocation +primitives in which case the interface is trivial. The date and time +facilities assume that the host provides some sort of facilities for +reading the clock time (ideally as a high precision integer) and the cpu +time consumed by a process. The bit and byte primitives do not actually +do any i/o, and are included in the interface primarily because Fortran +is difficult to use for such applications. +.PP +The IRAF \fBfile i/o\fR system deals with two types of files. +\fBText files\fR contain only character data, are read and written in +units of records (lines of text), and are maintained in a such a form +that they can be edited with a host editor. +Writing may occur only at the end of file. +Reading is normally sequential, but seeking to the beginning of a line +prior to a read is permitted. The user terminal is interfaced +as a text file, opened by the IRAF main at process startup. Terminal i/o +is generally line oriented, but character at a time input is used by some +programs, and the system expects to be able to send control codes to the +terminal. Text files are accessed synchronously. Character data is always +ASCII within IRAF, with the kernel routines mapping to and from the host +character set (e.g. EBCDIC). +.PP +The second file type is the \fBbinary file\fR. A binary file is an +extendible array of machine bytes. There are two types of binary files, +\fBblocked\fR (random access) binary files and \fBstreaming\fR (sequential) +binary files. Transfers to and from blocked binary files are +always aligned on device block boundaries and are asynchronous. +The size of a transfer may be any integral multiple of the device block +size, up to a device dependent maximum transfer size. +The IRAF file i/o system (FIO) assumes that a file can be extended by +overwriting the end of file, and that a partial record can be written at +the end of file without the host filling the record to the size of a +device block. The device block size is assumed to be device dependent. +Devices with different block sizes may coexist on the same system. +.PP +Streaming binary files are for devices like magtapes and the interprocess +communication (IPC) facilities. Seeks are not permitted on streaming files, +and there are no restrictions on block size and alignment of transfers, +other than an upper limit on the transfer size. The ideal host system will +initiate an asynchronous transfer from either type of binary file directly +from the mass storage device into the buffer pointed to by the kernel +(or vice versa). +.PP +The model does not assume that the host system provides device independent +file i/o. A different set of kernel routines are provided for each device +interfaced to FIO. On a system which does provide device independent i/o +the kernel routines may be coded as calls (or multiple entry points) to a +single set of interface subroutines. Standard file devices include disk +resident text and binary files, the IPC facilities, magtapes, line printers, +and (currently) the image display devices and the batch plotters. +The special devices are normally interfaced to FIO as binary files. +The IPC files are streaming binary files. +.PP +Although it is highly desirable that the host provide a hierarchical files +system, it is not required. IRAF tends to generate and use lots of small files. +FIO maps virtual filenames into machine dependent filenames, and will pass +only filenames acceptable to the host system to the kernel routines. +It should be possible to read filenames from a host directory, +determine if a file exists and is accessible, +delete a file, and rename a file. The model assumes that there are two types +of filenames: files within the current directory (directory name omitted), +and files within a particular directory, and that both types of files can be +simultaneously accessed. The directory name is assumed to be a string which +can be prepended to the filename to produce a pathname. Multiple versions +of a file are neither assumed nor supported. +.PP +\fBMagnetic tape\fR devices are interfaced as streaming binary files. +A magnetic tape is either empty or consists of one or more files, +each delimited by an end of file (EOF) mark, with an end of tape (EOT) +mark following the last file on the tape. +The kernel routine opens the device positioned to +the first record of a specific file or EOT. Tape density may be manually +set on the device, or may be set at allocate time or at open time (the IRAF +software will work regardless of which method is used). A separate file +open is required to access each file on the tape, i.e., FIO will not try +to read or write beyond a tape mark. +.PP +Record and file skipping primitives are desirable for tape positioning in +the kernel open procedure, but are not assumed by the model. Tape records +may be variable in length. No attempt will be made to position the tape +beyond EOT. FIO knows nothing about labeled tapes or multivolume tapes; +if it is necessary to deal with such tapes, the details should be handled +either by the host system or by the kernel routines. All IRAF programs +which read and write magtape files can also be used to read and write disk +files. +.PP +The virtual machine model assumes that a parent process can spawn one or +more child \fBsubprocesses\fR, to execute concurrently with the parent, +with bidirectional streaming binary communications channels connected to +the parent. The IPC facilities are used only to communicate with child +processes; the model does not assume that any one process can talk to any +other process. The model assumes that an IPC channel can only be opened +when a child process is spawned; the two functions are bound into the +same kernel primitive. A child process is assumed to inherit the same +current working directory as the parent. The child is not assumed to +inherit environment or logical name tables, open files, or the parent's +address space. +.PP +\fBException handling\fR is very machine dependent and is difficult to model. +Fortunately the IRAF system will probably still be usable even if the host does +not entirely fit the model, since exceptions are not the norm. +A parent process is assumed to be able to interrupt a child process. For error +recovery and process shutdown to occur properly control should transfer +to an interrupt handler in the child process when the interrupt signal is sent. +.PP +All exceptions occurring during execution of a process should be caught by +the kernel and mapped into the exception classes assumed by the kernel. +The model assumes that all exceptions can be caught, that control can be +transferred to an exception handler, and that execution can resume following +processing of the exception. The host and the kernel should let the high +level software process all exceptions and handle error recovery. +.PP +In summary, IRAF can be used to do science on a limited, batch oriented host +machine, at the expense of a limited user interface and considerable hacking +of the released system. The ideal host system will provide a hierarchical +files system, large asynchronous file transfers directly into process memory, +multiprocessing, efficient binary interprocess communication facilities, +dynamic memory management facilities, and high level exception handling. + +.NH +A Reference Manual for the IRAF Kernel +.PP +The kernel is a set of SPP or Fortran callable subroutines. The syntax +and semantics of these routines, i.e., the external specifications of the +interface, are the same for all machines. The code beneath the interface +will in general be quite different for different operating systems. +Any language may be used to implement the kernel routines, provided the +routines are Fortran callable. Typed functions are avoided in the kernel +where possible to avoid the problems of passing a function value between +routines written in different languages. +.PP +The method chosen to implement a kernel routine for a given host should be +dictated by the characteristics of the host and by the external specifications +of the routine, not by the method chosen to implement the same kernel +routine for some other host. Nonetheless it is often possible to reuse +code from an existing interface when coding an interface for a new host. +This occurs most often in the numerical procedures, e.g., the bit and byte +primitives. Furthermore, some routines are placed in the kernel only +because they are potentially machine dependent; these routines need only be +examined to see if they need to be modified for the new host. +.PP +The kernel routines are found in the \fBOS package\fR, the interface to the +host Operating System (OS). The OS package is maintained in the logical +directory \fBsys$os\fR, and the kernel routines are archived in the +system library \fBlib$libos.a\fR. + +.NH 2 +Conventions +.PP +At the kernel level, data is accessed in units of \fBmachine bytes\fR. +The size of a machine byte in bits and the number of bytes per SPP data +type are both parameterized and are assumed to be machine dependent. +It is fundamentally assumed throughout IRAF that an integral number of bytes +will fit in each of the language datatypes. Conversion between byte units +and SPP units is handled by the high level code; the kernel routines deal +with data in byte units. +.PP +All offsets in IRAF are \fBone-indexed\fR, including in the kernel routines. +Thus, the first byte in a file is at offset 1, and if the device block size +is 512 bytes, the second device block is at offset 513. The first bit in a +word is bit number 1. Many operating systems employ zero-indexing instead +of one-indexing, and the implementor must be especially careful to avoid off +by one errors on such systems. +.PP +All \fBcharacter strings\fR are packed in the high level code before +transmission to the kernel, and strings returned by the kernel are unpacked +by the high level code into SPP strings. The packed string is passed in an +array of SPP type \fBchar\fR. +The format of a packed string is a sequence of zero or more characters +packed one character per byte and delimited by end-of-string (EOS). +SPP strings are ASCII +while packed strings use the host character set. The EOS delimiter occupies +one character unit of storage but is not counted in the length of the string. +Thus if a kernel routine returns a packed string of at most \fImaxch\fR +characters, up to \fImaxch\fR characters are returned followed by an EOS. +.PP +Kernel procedures should call only other kernel procedures or the host +system. Calls to program interface routines are forbidden to avoid problems +with \fBreentrancy\fR and backwards library references (and because code which +references upwards is usually poorly structured). A program crash should +never occur as a result of a call to a kernel procedure. Illegal operations +should result in return of an error status to the routine which called the +kernel procedure. The SPP error handling facilities must not be used in +kernel procedures, even if a kernel procedure can be coded in SPP. +This is because the high level code does not error check kernel procedures, +and because printing an error message involves calls to FIO and possible +reentrancy. +.PP +Any kernel procedure which can fail to perform its function returns an +integer status code as its final argument. Other integer codes are used +to parameterize input arguments, e.g., the file access modes. Codes which +are used only in a particular procedure are documented in the specifications +for that procedure. The codes used extensively in the kernel are shown in +the table below. The magic integer values given in the table must agree +with those in the SPP global include file \fBiraf.h\fR. +.sp 0.08i +.TS +center box; +cb s s +ci | ci | ci +l | c | l. +Kernel Constants +_ +name value usage += +ERR \(mi1 function was unsuccessful +EOS '\\\\0' end of string delimiter +OK 0 function successfully completed +NO 0 no (false) +YES 1 yes (true) +.TE +.sp 0.08i +.PP +The names of kernel routines never exceed six characters, and only alphanumeric +letters are used. The Fortran implicit datatyping convention (I through N for +integer identifiers) is not used in IRAF. The IRAF naming convention is +package prefix plus function plus optional type suffix letter. This convention +is little used in the kernel because there are few functions, but it does occur +in a few places, e.g. the bitwise boolean functions \fBand\fR and \fBor\fR. +The procedure naming convention and other IRAF conventions are further +discussed in the document \fIIRAF Standards and Conventions\fR. + +.NH 2 +Avoiding Library Conflicts +.PP +Only documented OS interface routines should be callable from the high level +SPP and Fortran code. If at all possible, all non-interface kernel +subprocedures and host system calls referenced in the kernel should be +named such that they are not Fortran callable. All external identifiers +used in IRAF code adhere to the Fortran standard, i.e., at most six +alphanumeric characters, the first character being a letter. Non-interface +kernel procedures are guaranteed not to cause library conflicts with the +high level software provided they are not legal Fortran identifiers. +.PP +For example, on UNIX systems, the Fortran compiler appends a hidden underscore +character to all Fortran external identifiers. No standard C library or system +procedures have names ending in an underscore, so library conflicts do not +arise. On a VMS system, the VMS system service procedures all have external +names beginning with the package prefix "sys$", hence the system service +procedures are guaranteed not to cause library conflicts with standard Fortran +identifiers. +.PP +If there is no way to avoid library conflicts by using a naming convention +at the kernel level, it is possible to modify the SPP to map identifiers in +a way which avoids the conflicts (e.g., by use of \fBdefine\fR statements +in the global include file \fBiraf.h\fR). This approach is less desirable +because it involves modification of high level code, and because it does +nothing to avoid library conflicts with Fortran subprograms which are not +preprocessed. Furthermore, it is important that the mapping of SPP +identifiers to plain vanilla Fortran identifiers be simple and predictable, +to ease interpretation of host Fortran compiler error messages, and to make +the host system debugger easy to use with SPP programs. + +.NH 2 +File I/O +.PP +The file i/o subsystem is the most critical i/o subsystem in IRAF. +No process can run without file i/o, and the high level system code and +applications programs are all built upon file i/o. Many programs are i/o +intensive, and an efficient file i/o system is vital to the functioning +of such programs. The high level of functionality and device independence +provided by the IRAF file i/o subsystem is critical to minimizing the +complexity and maximizing the flexibility of all code which uses file i/o. +In particular, the database and image i/o subsystems are heavily dependent +upon file i/o; the IRAF file i/o system was designed expressly to provide +the kinds of facilities required by these and similar applications. +.PP +Most of the complexity of the file i/o system is in the machine independent +FIO interface. FIO handles all buffer allocation and management including +management of buffer caches, record blocking and deblocking, and read ahead +and write behind. FIO makes all files and file storage devices look the same, +and allows new devices to be interfaced dynamically at run time without +modifying the system. The database facilities rely on FIO for efficient +random file access, which requires use of a buffer cache to to minimize file +faults. Image i/o relies on FIO for efficient sequential file access, +which requires asynchronous i/o and large buffers. +.PP +Kernel support for file i/o consists of a few simple file management +primitives, e.g. for file deletion and renaming, plus the "device drivers" for +the standard devices. There are two basic types of files, \fBtext files\fR +and \fBbinary files\fR. The device driver for a text device consists of +8 subroutines; the driver for a binary device consists of 6 subroutines. +A different set of driver subroutines are required for each device interfaced +to FIO. All system and device dependence is hidden within and beneath these +subroutines. Most devices are interfaced to FIO as binary files. Kernel +device drivers are closely matched in capabilities to the actual device +drivers found on many systems. +.PP +The file access modes, device parameters, and other codes used to communicate +with the kernel file i/o primitives are summarized in the table below. + +.TS +center box; +cb s s +ci | ci | ci +l | c | l. +FIO Kernel Constants +_ +name value usage += +BOF \(mi3 beginning of file +EOF \(mi2 end of file +_ +READ_ONLY 1 file access modes +READ_WRITE 2 +WRITE_ONLY 3 +APPEND 4 write at EOF +NEW_FILE 5 create a new file +_ +TEXT_FILE 11 file types +BINARY_FILE 12 +_ +FSTT_BLKSIZE 1 device block size +FSTT_FILSIZE 2 file size, bytes +FSTT_OPTBUFSIZE 3 optimum transfer size +FSTT_MAXBUFSIZE 4 maximum transfer size +.TE +.sp 0.08i + +.NH 3 +Text Files +.PP +A \fBtext file\fR is a sequence of lines of text, i.e., of characters. +Examples of text files are parameter files, list files, program source files, +and \fBterminals\fR. Although it is not strictly required, it is desirable +that text files be maintained in such a form that they can be accessed by the +host system file editor and other host utilities. The principal function +of the text file primitives is to convert text data from the host format +to the IRAF internal format, and vice versa. +.PP +The physical representation of a text file is hidden beneath the kernel +interface and is not known to an IRAF program. +The logical (IRAF) and physical (host system) representations of a text +file will in general be quite different. On some systems +it may be possible to represent a single logical text file in any of several +different physical representations, and the kernel primitives will have to +be able to recognize and deal with all such representations. On other systems +there may be only a single physical format for text files, or there may be no +distinction between text files and binary files. +.PP +The \fBlogical representation\fR of a text file is a sequence of lines of text. +Each line of text consists of zero or more ASCII characters terminated by +the \fBnewline\fR character. The newline character defaults to ASCII LF +(linefeed), but some other character may be substituted if desired. +IRAF assumes that any ASCII character can be stored in a text file; +in particular, case is significant, and control characters may be embedded +in the text. There is no fixed limit on the number of characters per line. +It may not be possible to edit a file containing arbitrarily long lines +or embedded control characters with some host system editors, but such files +are rare. +.PP +Character data is represented within IRAF with the SPP datatype \fBchar\fR. +On many systems, char is implemented as the (nonstandard) Fortran datatype +INTEGER*2. The read and write primitives +for a text file must convert SPP arrays of ASCII char to and from the +internal host representation. This conversion usually involves a packing +or unpacking operation, and may also involve conversion between ASCII and +some other character set, e.g., EBCDIC. Regardless of the precision of the +datatype used to implement char on a given host system, characters are +limited to ASCII values, i.e., 0 to 127 decimal (negative valued characters +are permitted only in SPP variables and arrays). +.PP +The kernel primitives used to access ordinary disk resident text files, +i.e., the "device driver" primitives for an ordinary text file, are shown below. +The calling sequences for other text file devices are identical if the +two character device code for the new device is substituted for the "tx" +suffix shown. A device driver is installed in FIO by passing the entry +points of the subroutines to FIO with \fBfopntx\fR or \fBfdevtx\fR; +the entry point addresses of the 8 subroutines are saved in the FIO device +table. +.PP +The \fBzopntx\fR primitive opens a text file or creates a new one, +returning the channel number (an integer magic number) or ERR as +its status value. +All subsequent references to the file are by this channel number. +The file access modes are listed in the table in \(sc4.3. +Output text is assumed to be buffered; \fBzflstx\fR is called by FIO to flush +any buffered output to the device when the file is closed, or when file output +is flushed by the applications program. +.sp 0.08i +.TS +center; +cb s +n l. +Text File Primitives +.sp +zopntx \&(osfn, mode, chan) open or create a textfile +zclstx \&(chan, status) close a textfile +zgettx \&(chan, text, maxch, status) get next record +zputtx \&(chan, text, nchars, status) put next record +zflstx \&(chan, status) flush output +znottx \&(chan, loffset) note file position +zsektx \&(chan, loffset, status) seek to a line +zstttx \&(chan, param, lvalue) get file status +.TE +.sp 0.08i +.PP +If the physical file is record oriented, there will normally be one newline +delimited line of text per record. A sequence of characters output with +\fBzputtx\fR is however not necessarily terminated with a newline. +The \fBzputtx\fR primitive is called to write the FIO output buffer when +(1) newline is seen, (2) the buffer fills, (3) the output is flushed, +or (4) the file is closed. Thus if a very long line is written, several +calls to \fBzputtx\fR may be required to output the full line. Conversely, +if the input record contains more than \fImaxch\fR characters, \fBzgettx\fR +should return the remainder of the record in the next call. In no case should +more than maxch characters be returned, as the output buffer would be overrun. +.PP +A \fBzgettx\fR call with \fBmaxch=1\fR has a special meaning when the input +device is a terminal. This call will switch the terminal from line mode to +\fBcharacter mode\fR, causing \fBzgettx\fR to return immediately each time a key +is typed on the terminal. This highly interactive mode is useful for programs +like screen editors, and is discussed further in \(sc4.3.4.1. Character mode +applies only to terminal input; if individual characters are to be output, +the output must be flushed after each character is written. +.PP +Text files are virtually always accessed sequentially. Writing is permitted +only at end of file (EOF). A file opened for reading is initially positioned +to the beginning of file (BOF). Seeking to the beginning of any line in the +file is permitted prior to a read. The seek offset must be BOF, EOF, or an +offset returned by a prior call to the \fBznottx\fR primitive, which returns +the file offset of the last text line read or of the next text line to be +written. Seeks on text files are restricted to lines; seeks to individual +characters are not permitted. The long integer file offset returned by +\fBznottx\fR is a magic number, i.e., the value is assumed to be machine +dependent. +.NH 3 +Binary Files +.PP +A \fBbinary file\fR is an extendible array of bytes accessed in segments the +size of a device block. The principal difference between a text file and +a binary file is that character data is converted in some machine dependent +fashion when a text file is accessed, whereas data is copied to and from +a binary file without change. A second difference is that only binary files +are randomly accessible for both reading and writing at any offset. +Binary files are used to implement interprocess communication (IPC) files, +database files (datafiles), picture storage files (imagefiles), and so on. +Special devices such as magtape, the line printer, image display devices, +and process memory are interfaced to FIO as binary files. +.PP +The fundamental unit of storage in a binary file is the \fBmachine byte\fR. +The number of bits per byte is presumed to be machine dependent, although +IRAF has thus far been used only on machines with 8 bit bytes. IRAF assumes +only that there are an integral number of bytes in each SPP or Fortran datatype. +The SPP datatype \fBchar\fR should not be confused with the machine byte. +The char is the fundamental unit of storage in SPP programs; the number +of machine bytes per SPP char is greater than or equal to one and is given +by the machine dependent constant SZB_CHAR, defined in \fBiraf.h\fR. +Though the distinction between chars and machine bytes is important in the +high level system code, it is of no concern in the kernel since the kernel +routines access binary files only in units of machine bytes. +.PP +Binary files are further differentiated into \fBblocked\fR (random access) +binary files and \fBstreaming\fR (sequential) binary files. The most +common blocked binary file is the binary random access disk file. +IPC files and magtape files are typical streaming binary files. +The \fBdevice block size\fR is used to differentiate between blocked and +streaming binary files. A device with a block size greater than or equal +to one byte is understood to be blocked, whereas a device with a block size +of zero is understood to be a streaming file. Transfers to and from blocked +devices are always aligned on device block boundaries. There are no alignment +restrictions for streaming files. +.sp 0.08i +.TS +center; +cb s +n l. +Binary File Primitives +.sp +zopnbf \&(osfn, mode, chan) open or create a binary file +zclsbf \&(chan, status) close a binary file +zardbf \&(chan, buf, maxbytes, loffset) initiate a read at loffset +zawrbf \&(chan, buf, nbytes, loffset) initiate a write at loffset +zawtbf \&(chan, status) wait for transfer to complete +zsttbf \&(chan, param, lvalue) get file status +.TE +.sp 0.08i +.PP +The kernel primitives used to access ordinary disk resident binary files, +i.e., the "device driver" primitives for a binary file, are shown above. +The calling sequences for other binary file devices are identical if the +two character device code for the new device is substituted for the "bf" +suffix shown. The device driver for a binary file is particularly simple +since all buffering is performed at a high level. A binary file is opened +or created with \fBzopnbf\fR, which returns the channel number or ERR as +its final argument. All subsequent references to the file are by channel +number. +.PP +The kernel primitives for a binary file closely approximate the functionality +of the typical host system device driver. +Ideally an asynchronous kernel read or write +to a binary file will translate into a DMA transfer directly from the data +buffer to the device, or vice versa. FIO guarantees that only a single transfer +will be pending on a channel at a time, i.e., that a new i/o request will not be +issued until any previous transfer is completed. There is no \fBseek\fR +primitive for binary files since the absolute file offset is specified in +a read or write request (the file offset argument should be ignored for a +streaming file). There is no \fBnote\fR primitive since there is no concept +of the current file position for a binary file at the kernel level. +.PP +The \fBzardbf\fR and \fBzawrbf\fR (asynchronous read and write) primitives +should initiate a transfer and return immediately. No status value is +returned by these primitives: rather, the number of bytes read or written +or ERR is returned in the next call to \fBzawtbf\fR (asynchronous wait). +A byte count of zero on a read indicates end of file. It is not an error +if fewer than \fBmaxbytes\fR bytes can be read; \fBzardbf\fR should return +immediately with whatever data it was able to read, rather than try to +read exactly maxbytes bytes. In no case should more than maxbytes bytes +be returned, as this would overflow the caller's buffer. If the size of the +input block or record is greater than maxbytes bytes when reading from a +streaming file, data is assumed to be lost. An attempt to read or write +before BOF or after EOF is illegal and will be caught by FIO. +.PP +The status value ERR should be returned for all illegal requests or +i/o errors. FIO will always call \fBzawtbf\fR after a transfer for +synchronization and to check the status value. Repeated calls to \fBzawtbf\fR +after a single i/o transfer should continue to return the same status. +Errors should not "stick", i.e., an error status should be cleared when +the next transfer is initiated. +.PP +It is fundamentally assumed that it is possible to extend a file by +overwriting EOF in a call to \fBzawrbf\fR. It is further assumed that the +last block in a file need not be full. For example, suppose the device +block size is 512 bytes, the FIO buffer size is 512 bytes, and we are +writing to a file 1024 bytes long. We write 18 bytes at file offset 1025, +the fourth block in the file, and then close the file. +When the file is subsequently reopened, FIO will call \fBzsttbf\fR to get +the file size, which should be 1042 bytes. If the program calling FIO +then attempts to write 50 bytes at EOF, FIO will call \fBzardbf\fR to initiate a +read of 512 bytes at file offset 1025, and a subsequent call to \fBzawtbf\fR +will return a byte count of 18. FIO will copy the 50 bytes into the buffer +starting at byte offset 19 and eventually write the buffer to the file +at file offset 1025, overwriting EOF and extending the file. +.PP +.NH 3 +Specifying Device Parameters +.PP +Each device interfaced to FIO has a unique status primitive callable while +the file is open to obtain values for the device parameters. +These parameters reflect the characteristics of the \fIdevice\fR +or \fIfilesystem\fR on which the file is stored. +The status primitives for disk resident text and binary +files are \fBzstttx\fR and \fBzsttbf\fR. These primitives are the only file +status primitives available for a file while it is open; \fBzfinfo\fR is not +called to get information on open files. +.PP +The device block size and file size parameters are currently not used for +text files, although they are read when the file is opened. +The file size parameter is not needed for text files because \fBzsektx\fR +is used to seek to EOF on a text file. All four parameters are required for +binary files. +.sp +.in 1.0i +.ti -0.5i +FSTT_BLKSIZE +.sp 0.04i +The device block size in bytes (binary files only). A block size of zero +indicates a streaming device; no alignment checking will be performed, +and only sequential i/o will be permitted. +If the block size is greater than or equal to one, the device is understood +to be a random access binary file with the indicated device block size. +File reads and writes will be aligned on device block boundaries, +although if the block size is given as 1 byte (e.g., if process memory is +accessed as a file) there is effectively no restriction on block alignment. +.sp +.ti -0.5i +FSTT_FILSIZE +.sp 0.04i +The file size in bytes; zero should be returned for a new file. +Not used for streaming files. FIO will ask for this once, when the file +is opened, if the file is a regular disk resident binary file. +Thereafter FIO keeps track of the file size itself. If necessary the +kernel can get the file size from the host system before opening the file. +.sp +.ti -0.5i +FSTT_OPTBUFSIZE +.sp 0.04i +The optimum transfer or buffer size for "normal" file access. This parameter +defines the default FIO buffer size for both read and write access. +The optimum transfer size typically depends on the characteristics of +the i/o system of the host, and may also depend on the characteristics +of the device or of the file system on which the file is found. +For example, the optimum transfer size of a file system configured for +the storage of images (large files) may well be larger than that for a +file system configured for general use (predominantly small files). +.sp +.ti -0.5i +FSTT_MAXBUFSIZE +.sp 0.04i +The maximum permissible transfer size in a single read or write request. +This parameter determines the maximum FIO buffer size, and hence the +maximum size of a FIO read or write request. +.in -1.0i +.sp +.PP +The optimum buffer size for magtape devices is usually different (larger) +for reading than for writing; the default magtape buffer sizes are set by +system tuning parameters in \fBconfig.h\fR (\(sc6). FIO automatically +adjusts the internal FIO buffer size for a file to be an integral multiple +of the device block size. The FIO buffer size may be further controlled +at a high level by advising FIO that i/o to a file is to be highly random +or highly sequential. With all this going on in the high level code, +it is inadvisable to try to tune the system by adjusting the device +parameters. The file status parameters should reflect the physical +characteristics of the device or files system on which the file is resident. +.PP +For example, consider a random access binary disk file on a UNIX system. +The device block size will typically be 512 bytes and is generally wired +into the status primitive as a constant. The file size may be obtained +at any time from the inode for the file. The optimum buffer size is +512 bytes on V7 UNIX, 1024 bytes on 4.1BSD, and dependent on how a filesystem +is configured on 4.2BSD. There is no maximum transfer size for a disk +file, so the maximum integer value is returned. If the file happens to +be a pipe, the block size would be given as 0, the file size is ignored, +the optimum transfer size is arbitrary, e.g. 2048 bytes, and the maximum +transfer size is typically 4096 bytes. +.NH 3 +Standard File Devices +.PP +The kernel routines for the ordinary text and binary disk files have +already been presented. An arbitrary number of other devices may be +simultaneously interfaced to FIO. The standard devices are disk, memory, +terminal, line printer, IPC, magtape, and the pseudofiles (STDIN, STDOUT, +etc.). The memory and pseudofile interfaces are machine independent and +will not be discussed here. The IRAF development system currently also +supports file interfaces for image display devices and plotters, but the +graphics device interfaces are being redesigned to use the ISO standard +Graphical Kernel System (GKS) graphics device interface, so we will not +discuss those devices here. +.PP +Each device is interfaced with a distinct set of kernel interface routines +to give the implementor maximum scope for tailoring the interface to a +device. If the host system provides device independent file i/o at a low level, +it may be possible to use the same kernel routines for more than one device. +For example, the text file driver might be used for both disk resident text +files and terminals, and the IPC, magtape, and line printer devices might +resolve into calls to the kernel routines for a disk resident binary file. +This approach offers maximum flexibility for minimum effort and should be +followed if the host system permits. On the other extreme, a host might +not have something fundamental like IPC channels, and it might be +necessary to build the driver from the ground up using non-file resources +such as shared memory. +.NH 4 +The User Terminal +.PP +Terminal devices are interfaced to FIO as text files. The device code +is "ty". The driver subroutines are shown below. The legal access +modes for a terminal are READ_ONLY, READ_WRITE, WRITE_ONLY, and APPEND. +Seeking to offsets other than BOF or EOF is illegal; seeks to BOF and +EOF should be ignored. +.sp 0.08i +.TS +center; +cb s +n l. +Terminal Driver +.sp +zopnty \&(osfn, mode, chan) open a terminal file +zclsty \&(chan, status) close a terminal +zgetty \&(chan, text, maxch, status) get next record +zputty \&(chan, text, nchars, status) put next record +zflsty \&(chan, status) flush output +znotty \&(chan, loffset) not used +zsekty \&(chan, loffset, status) not used +zsttty \&(chan, param, lvalue) get file status +.TE +.sp 0.08i +.PP +When an IRAF process is run from the CL it communicates with the CL via +IPC files; when not run from the CL, an IRAF process assumes it is talking +to a terminal. The terminal driver is therefore linked into every IRAF main. +The main assumes that the terminal is already open when an IRAF process +starts up; the \fBzopnty\fR and \fBzclsty\fR routines are used only when +a terminal is directly accessed by a program. +.PP +If possible the terminal driver should be set up so that input can come +from either a terminal or an ordinary text file, allowing IRAF processes +to be run in batch mode taking input from a file. On a batch oriented +system the "terminal" driver would be the same as the text file driver, +and input would always come from a file. +.PP +Terminal input is normally line oriented. The host terminal driver +accumulates each input line, handling character, word, and line deletions and +other editing functions, echoing all normal characters, checking for control +characters (e.g. interrupt), and returning a line of text to \fBzgetty\fR +when carriage return is hit. The line returned by \fBzgetty\fR to the +calling program should always be terminated by a \fBnewline\fR. +.PP +If \fBzgetty\fR is called with \fBmaxch=1\fR the terminal is put into raw +character mode. In this mode \fBzgetty\fR returns each character as it +is typed, control characters have no special significance (as far as possible), +and characters are not automatically echoed to the terminal. A subsequent +call with maxch greater than one causes a mode switch back to line input +mode, followed by accumulation of the next input line. +.PP +The IRAF system includes device independent software for terminal control +and vector graphics, and expects to be able to send device dependent control +sequences to the terminal. Any program which does anything out of the ordinary +with a terminal, e.g., clearing the screen or underlining characters, uses +the TTY interface to generate the device dependent control sequences necessary +to control the terminal. Ordinary output to the terminal, however, is not +processed with the TTY interface. +.PP +The control characters commonly present in ordinary text are \fBnewline\fR, +\fBcarriage return\fR, and \fBtab\fR. The \fBnewline\fR character delimits +lines of text and should result in a carriage return followed by a line feed. +\fBZputtx\fR may be called repeatedly to build up a line of text; the output +line should not be broken until newline is sent. The carriage return character +should cause a carriage return without a line feed. If the host system terminal +driver can conditionally expand tabs, tab characters present in the text +should be passed on to the host driver. The terminal should be allowed +to expand tabs if possible as it is much faster, especially when working +from a modem. On many systems it will be necessary to map newlines upon +output, but all other control characters should be left alone. +.NH 4 +The Line Printer Device +.PP +Line printers are interfaced at the kernel level as binary files. +At the FIO level a line printer may be opened as either a text file or +a binary file. If opened as a text file at the FIO level, textual +output is processed under control of the device independent TTY interface, +generating the control sequences necessary to control the device, +then the output is packed and written to the device as a binary byte +stream. If the printer is opened as a binary file at the high level, +binary data is passed from the applications program through FIO and on +to the kernel without modification. +.sp 0.08i +.TS +center; +cb s +n l. +Line Printer Driver +.sp +zopnlp \&(osfn, mode, chan) open printer or spoolfile +zclslp \&(chan, status) close printer +zardlp \&(chan, buf, maxbytes, notused) initiate a read +zawrlp \&(chan, buf, nbytes, notused) initiate a write +zawtlp \&(chan, status) wait for transfer to complete +zsttlp \&(chan, param, lvalue) get file status +.TE +.sp 0.08i +.PP +The line printer is a streaming device. Currently only APPEND mode +is used, although there is nothing in the FIO interface to prevent +reading from a line printer device. The filename argument is the +\fBlogical name\fR of the printer device, as defined by the CL environment +variable \fBprinter\fR and as found in the \fBdev$termcap\fR file. +These logical device names are quite system dependent, and in general +it will be necessary to add new device entries to the termcap file, +and change the name of the default printer device by modifying the +\fBset printer\fR declaration in \fBlib$clpackage.cl\fR. +.PP +On some systems or for some devices it may be desirable to spool printer +output in an ordinary binary file opened by \fBzopnlp\fR, disposing of +the file to the host system when \fBzclslp\fR is called, or at some +later time. This is desirable when the line printer device is so slow +that asynchronous printing is desired, or when the printer device is +located on some other machine and the spoolfile must be pushed through +a network before it can be output to the device. +.PP +In general it should not be necessary to modify printer data upon output. +The high level code processes form feeds, expands tabs, generates control +sequences to underline characters, breaks long lines, maps newline into +the device end of line sequence, pads with nulls (or any other character) +to generate delays, and so on, as directed by the \fBdev$termcap\fR file. +If the host system driver insists on processing printer output itself, +it may be necessary to modify the termcap entry for the printer to generate +whatever control sequences the host system requires (the newline sequence +is likely to be a problem). The termcap entry for a printer is potentially +machine dependent since raw output to a line printer may not be feasible on +some systems, and it may be easier to edit the termcap file than to filter +the output stream in the driver. +.PP +Part of the reason for implementing the printer interface as a binary file +was to provide a convenient and efficient means of passing bitmaps to printer +devices. If a bitmap is to be written to a printer device, ideally the +device will be data driven and it will be possible to pass data directly +to the device without translation. If this is not the case, the driver +must make the device look like it is data driven by scanning the data +stream for a control sequence indicating a change to bitmap mode, +then poking the host driver to change to bitmap mode. Since the device +is data driven at the kernel level it will still be possible to spool +the output in a file and process it on a remote network node. +.NH 4 +Interprocess Communication +.PP +Interprocess communication (IPC) channels are necessitated by the multiprocess +nature of IRAF. When a subprocess is spawned by the CL (or by any IRAF +process) it is connected to its parent by two IPC channels, one for reading +and one for writing. An IPC channel is a record oriented streaming binary +file. +.sp 0.08i +.TS +center; +cb s +n l. +IPC Driver +.sp +zopnpr \&(osfn, mode, chan) not used +zclspr \&(chan, status) not used +zardpr \&(chan, buf, maxbytes, notused) initiate a read +zawrpr \&(chan, buf, nbytes, notused) initiate a write +zawtpr \&(chan, status) wait for transfer to complete +zsttpr \&(chan, param, lvalue) get file status +.TE +.sp 0.08i +.PP +The IPC channels are set up when a subprocess is spawned, and a process may +use IPC facilities only to talk to its parent and its children (the process +structure is a tree, not a graph). Since the opening of an IPC channel is +bound to the spawning of a subprocess, the open and close primitives are +not used for IPC files. The \fBprconnect\fR procedure (not a kernel primitive), +called by the parent to spawn a subprocess, sets up the IPC channels and +installs the IPC driver in FIO, +returning two binary file descriptors to the calling program. +The \fBprconnect\fR procedure is very much like a file \fBopen\fR, except that +the "file" it opens is active. The IRAF main in the child process senses +that it has been spawned by an IRAF process, and installs the same IPC driver +in its FIO, connecting the IPC channels to the streams CLIN and CLOUT. +.PP +Since the IPC channels are read and written by concurrent processes, +some care is necessary to ensure synchronization and to avoid deadlocks. +Fortunately most of the necessary logic is built into the high level protocols +of the CL interface, and an understanding of these protocols is not necessary +to implement the IPC driver. It is however essential that the low level +protocols of an IPC channel be implemented properly or deadlock may occur. +.PP +An IPC channel is \fBrecord oriented\fR. This means that if \fBzawrpr\fR +is called by process A to write N bytes, and \fBzardpr\fR is called by +process B to read from the same channel, N bytes will be read by process B. +If the IPC facilities provided by the host are sufficiently sophisticated, +records may be \fBqueued\fR in an IPC channel. The writing process should +block when the IPC channel fills and no more records can be queued. The +reading process should block when it attempts to read from an empty channel. +.PP +For example, suppose process A writes an N byte record and then an M byte +record. If \fBzardpr\fR is called by process B to read from the channel, +it should return to its caller the first record of length N bytes. +A second call will be required to read the next record of length M bytes. +On some systems, e.g. UNIX, the IPC facilities are not record oriented and the +first read might return either N bytes or N+M bytes, depending on unpredictable +system timing details. Hence the IPC driver for a UNIX system must impose +a record structure upon the UNIX "pipe" used as the IPC channel. +.PP +On other systems the IPC facilities may be limited to the transfer of single +records, i.e., process B will have to read a record +before process A can transmit the next record. This is the lowest common +denominator, and hence the protocol chosen for the IPC driver. Despite the +use of the lowest common denominator for the low level protocol, a high +bandwidth can be achieved for IPC channels if the maximum transfer size +is large and records are queued. The high level protocol ensures that only +one process will be writing or reading at a time, thus preventing deadlock. +The high level protocol also uses special data records as semaphores to achieve +synchronization and permit record queuing. +.PP +Most modern operating systems provide some sort of interprocess communications +facilities which the IPC driver can use. UNIX calls it a pipe or a socket, +VAX/VMS calls it a mailbox, and DG/AOS calls it an IPC port. If the host +system has no such facility, or if the facility provided by the host is +inefficient, an IPC driver can often be built using shared memory (use a +circular buffer to implement the queue). As a last resort, a real driver +can be coded and installed in the host system. On \fBmulti-processor\fR +systems the IPC facilities should allow the parent and child processes to +reside on different processors. +.NH 4 +Imagefile Access +.PP +Imagefiles, i.e., bulk data files, are a special kind of binary file. +The ordinary disk binary file driver may be used to access imagefiles, +but imagefiles have certain properties which can be exploited on some +systems for increased i/o efficiency. The image i/o software (IMIO) therefore +uses a special kernel driver to access imagefiles. Since this driver is +a special case of the ordinary binary file driver, a transportable version +of the driver which simply calls the ordinary binary file driver is included +in the standard distribution. The transportable driver may easily be replaced +by a machine dependent version to optimize image i/o for the host system. +.PP +Imagefiles differ from ordinary binary files in that the size of the image +is known when the \fBpixel storage file\fR is created. Furthermore, images +do not dynamically change in size at run time. On many systems it is possible +to preallocate the pixel storage file before writing any data into it, +rather than creating the file by writing at EOF. +.PP +Preallocation of a file makes it feasible for the host system to allocate +\fBcontiguous storage\fR for the file. Use of a preallocated, fixed size +file also makes it possible on some systems to map the file into +\fBvirtual memory\fR. If image access is expected to be sequential, or if +the host system does not support virtual memory, it is often possible to +\fBdirectly access\fR the file via the host system device driver, bypassing +the host files system software and significantly reducing the overhead of +file access (e.g., eliminating any intermediate buffering by the host system). +.sp 0.08i +.TS +center; +cb s +n l. +Static File Driver +.sp +zopnsf \&(osfn, mode, chan) open static file +zclssf \&(chan, status) close static file +zardsf \&(chan, buf, maxbytes, loffset) initiate a read +zawrsf \&(chan, buf, nbytes, loffset) initiate a write +zawtsf \&(chan, status) wait for transfer to complete +zsttsf \&(chan, param, lvalue) get file status +.sp +zfaloc \&(osfn, nbytes, status) preallocate a binary file +.TE +.sp 0.08i +.PP +The use of a file i/o interface to implement virtual memory access to files +is desirable to minimize the machine dependence of applications which use +virtual memory. The functional behavior of the static file driver is the +same whether it maps file segments into virtual memory or copies file +segments into physical memory. If IRAF is to be used on a system which does +not provide virtual memory facilities, the image processing software +will work without modification, provided the physical memory requirements +of the software are reasonable. +.PP +FIO divides a file up into segments of equal size, where the size of a segment +is equivalent to the size of a file buffer and is an integral multiple of the +virtual memory page size. IMIO ensures that the pixel data in the pixel +storage file begins on a block boundary, and is an integral number of pages +in length. Furthermore, when FIO allocates a file buffer, it ensures that +the buffer is aligned on a virtual memory page boundary. The virtual memory +page size is parameterized in \fBconfig.h\fR, and is set to 1 on a nonvirtual +machine. +.PP +Provided that the buffer and the file data are both properly aligned, +\fBzardsf\fR may be used to map a file segment into memory. The file buffer +pages are first deleted and then remapped onto the new file segment. +If the buffer is written into, \fBzawrsf\fR will eventually be called to +update the segment, i.e., flush modified pages to the image file (the pages +should not be unmapped). If desired a single FIO buffer may be allocated +the size of the entire image and all reads and writes will reference this +single buffer with minimum overhead. Alternatively the image may be mapped +in segments; reusing the same buffers avoids flushing the system page cache +when sequentially accessing a large image. +.PP +When an image section is read or written by IMIO, the interface returns +a pointer to a buffer containing the pixels. If all of the necessary +conditions are met (e.g., no subsampling, no datatype conversion, etc.), +IMIO will return a pointer directly into the file buffer, otherwise IMIO +extracts the pixels from the file buffer into a separate buffer. If the file +buffer is mapped onto the imagefile, IMIO thus returns a pointer directly into +the imagefile without performing any i/o (until the data is referenced). +Thus it is possible to exploit virtual memory for image access without +restricting the flexibility of programs which operate upon images; general image +sections may be referenced, datatypes need not agree, etc., yet i/o will still +be optimal for simple operations. +.PP +For example, suppose an entire large image is to be mapped into virtual +memory. FIO does not allocate any buffers until the first i/o on a file +occurs. IMIO will be called by the applications program to read a "subraster" +the size of the entire image. If the image can be directly accessed, +IMIO will set the FIO buffer size to the size of the image, then issue a +\fBseek\fR and a \fBread\fR to read the pixels. +FIO will allocate the buffer, aligned on a page boundary, then call +\fBzardsf\fR which maps the buffer onto the pixel storage file. +.PP +If all the necessary conditions are met, IMIO will return a pointer into the +FIO buffer and hence to the segment of memory mapped onto the pixel storage +file. If this is not possible, IMIO will allocate a new buffer of its own +and perform some transformation upon the pixels in the FIO buffer, writing +the transformed pixels into the IMIO buffer. The IMIO pointer will be +dereferenced in an subprogram argument list in the applications program, +and the SPP or Fortran subprogram will see what appears to be a static array. +.PP +Most virtual memory implementations are designed more for random access than +for \fBsequential access\fR. +Some systems, e.g. VAX/VMS, allow a variable number of pages (the page fault +cluster) to be read or written when a page fault occurs. +Other systems, e.g., DG/AOS, read or write a single page for each fault. +Even when the page fault cluster can be made large to minimize faulting +when sequentially accessing a large image, i/o is not optimal because paging +is not asynchronous, and because the heavy faulting tends to flush the process +and system page caches. Thus for sequential image operations conventional +double buffering with large buffers and large DMA transfers direct from +disk to memory is preferable. This level of i/o is available via QIO calls +on VAX/VMS and is feasible via \fIphysio\fR calls on UNIX, if images are +static and contiguous or nearly contiguous. +.PP +If the host system provides both virtual memory facilities and low level +asynchronous i/o, the static file driver should ideally be capable of +performing i/o by either technique. The choice of a technique may be based +upon the alignment criteria and upon the size of the transfer. +If the alignment criteria are not met or if the size of the transfer is +below a threshold, conventional i/o should be used. If the size of the +transfer is large, e.g., some sizable fraction of the working set size, +virtual i/o should be used. Virtual memory should only be used for images +which are to be accessed randomly, so the page fault cluster should be small. +.NH 4 +Magtape Devices +.PP +The magnetic tape device interface is the most complex file device interface +in the IRAF system. Operating systems vary greatly in the type of i/o +facilities provided for magtape access, making it difficult to design a +machine independent interface. Some systems provide primitive access to +the drive, permitting file and record skipping, writing of tape marks, +and so on, while others permit only sequential access in the forward direction. +Magtape access is further complicated by the storage of multiple files on +a tape, by variable size records, the occasional need to swap bytes, +and the need to specify the density. Error recovery is particularly difficult +for magtape devices because it is possible to lose track of the position +of the tape: whereas most binary devices are accessed by absolute offset, +magtapes are accessed relative to the current position. +.PP +To avoid having to deal with this level of complexity in the kernel, +the magtape device driver has been subdivided into a machine independent +part and a unique magtape device interface. The standard streaming +binary file driver subroutines are coded in SPP and are portable. An inner +set of six "zz" routines are defined especially for accessing magtape +devices. The portable driver routines constitute the MTIO interface and +are not part of the kernel. MTIO opens and initializes multiple magtape +devices, keeps track of the file position, and handles error recovery. +The kernel routines are responsible for physically positioning the tape +and for reading and writing records. +.sp 0.08i +.TS +center; +cb s +n l. +Magtape Driver +.sp +zopnmt \&(osfn, mode, chan) open a magtape file +zclsmt \&(chan, status) close magtape device +zardmt \&(chan, buf, maxbytes, notused) initiate a read +zawrmt \&(chan, buf, nbytes, notused) initiate a write +zawtmt \&(chan, status) wait for transfer to complete +zsttmt \&(chan, param, lvalue) get file status +.TE +.sp 0.08i +.PP +A magtape device must be \fBallocated\fR at the CL level before the device +can be accessed. A file on a magtape device is opened by calling the program +interface procedure \fBmtopen\fR in an applications program. +Once opened, a magtape file is accessed via the FIO interface and +hence benefits from the buffer management facilities provided by FIO. +Use of the FIO interface also provides device independence, allowing programs +which access magtape to be used to (sequentially) access any other binary +file. In particular, any IRAF program which commonly accesses magtape may +also be used to access a disk file. This permits use of the FITS reader and +writer, for example, for image transmission between stranger machines in a local +area network. +.PP +When a magtape device is allocated a device \fBlock file\fR is written into +the IRAF public directory \fBdev$\fR by the high level code. In addition to +telling other IRAF processes that the device has been allocated, the lock file +is used to keep track of the tape position while the device is closed. +When a device is closed, either normally or during error recovery, a new lock +file is written recording the current position of the drive as well as various +statistics (owner, time of last access, number of records read or written, +etc.). When a device is opened the lock file is read to determine the +current position. The \fBsystem.devstatus\fR program prints the contents +of the device lock file; if there is no lock file, the IRAF system assumes +that the device has not been allocated. +.PP +Each file on a reel must be opened individually, just as each file on a disk +must be opened individually. A pair of calls to \fBmtopen\fR and \fBclose\fR +(FIO) are required to access each file. +Drives are referred to by the logical names "mta", "mtb", and so on. +The assignment of logical drives to physical devices is system dependent. +The logical drive number, density, absolute file number on the tape, +absolute record number within the file, access mode, and FIO buffer size +(most of which can be defaulted) are specified in the file "name" argument +when the file is opened. For example, the filespec "mtb1600[3,10]" refers +to record 10 of file 3 of logical drive "mtb" at 1600 bpi. The minimum +filespec consists of just the logical drive name; everything else is optional. +.PP +The \fBmtopen\fR procedure parses the magtape filespec to determine whether +a magtape device or a disk binary file is being referenced. If a disk file +is named, \fBmtopen\fR reduces into a conventional call to \fBopen\fR. +If a magtape file is named, the filespec is parsed to determine the logical +drive, density, and the file and record numbers to which the tape is to +be opened. If this information is legal, if the drive is allocated, +and if the drive is not already open, \fBmtopen\fR reads the lock file to +determine the current position. FIO is then called to open the device. +A global common is used to pass device dependent information from \fBmtopen\fR +to \fBzopnmt\fR, since device dependent information cannot be passed through +FIO. +.PP +The magtape kernel primitives are shown below. Our intent here is only to +introduce the routines and discuss the role they fulfill in the MTIO interface. +Detailed specifications for the routines are given in the manual pages. +.sp 0.08i +.TS +center; +cb s +n l. +Magtape Kernel Primitives +.sp +zzopmt \&(drive, density, mode, oldrec, oldfile, newfile, chan) open +zzclmt \&(chan, mode, nrecords, nfiles, status) close +zzrdmt \&(chan, buf, maxbytes) aread +zzwrmt \&(chan, buf, nbytes) awrite +zzwtmt \&(chan, nrecords, nfiles, status) await +zzrwmt \&(chan, status) arewind +.TE +.sp 0.08i +.PP +The \fBzopnmt\fR procedure is called by FIO to open a magtape device. +The only legal access modes for magtape files are READ_ONLY and WRITE_ONLY. +The device parameters are retrieved from the global common prepared by +\fBmtopen\fR and passed on to the kernel primitive \fBzzopmt\fR to physically +open the drive. The kernel open primitive sets the density of the drive and +opens the drive with the desired access mode, leaving the tape positioned to +record 1 of the desired file. +.PP +The exact position of the tape at open time, i.e., both file and record +numbers (one-indexed), is passed to \fBzzopmt\fR to facilitate positioning +the tape. Many systems can skip records and tapemarks in either the forward +or reverse direction, and it is easy to position the tape on such systems +given the current position and the new position. The current record number +is needed to tell \fBzzopmt\fR when the current file is already rewound. +On some systems it is not possible to backspace to the last tapemark, +and the only way to rewind the current file or position to a previous file +is to rewind the tape and read forward. +.PP +Upon input to \fBzzopmt\fR, the \fInewfile\fR argument specifies the number +of the file to which the tape is to be positioned, or the magic number EOT. +Upon output, \fInewfile\fR contains the number of the file to which the +tape was actually positioned. The high level code assumes that the +tape does not move when the device is closed and subsequently reopened; +if this is not the case, \fBzzopmt\fR should ignore the old position arguments. +.PP +The \fBzzrdmt\fR and \fBzzwrmt\fR primitives initiate an asynchronous +read or write of a record at the current tape position. The number of bytes +read or written and the number of records and or files skipped in the operation +are returned in the next call to \fBzzstmt\fR. If a tape mark is seen when +attempting to read the next record, \fBzzwtmt\fR should return a byte count +of zero to signal EOF. It does not matter whether the tape is left positioned +before or after the tape mark, provided the record and file counts are +accurate. +.PP +A file containing zero records marks logical EOT. If physical EOT is seen +either the host system or the kernel should signal the operator to mount +the next reel, returning only after a record has been read or written on +the next reel. The high level code does not try to reread or rewrite +records. Error retry should be handled either by the kernel routines or +preferably by the host driver. The kernel should return ERR only if it +cannot read or write a record. It is not an error if less data is read +than was requested, or if more data was available in the record than was +read, resulting in loss of data. Loss of data is unlikely because FIO +generally allocates a large (16K or 32K) input buffer when reading magtapes. +.PP +If an error occurs when accessing the magtape device, e.g. a keyboard +interrupt, the high level code will mark the position of the tape as +undefined and call \fBzzclmt\fR to close the device. When the device is +subsequently reopened, \fBmtopen\fR will see that the position of the tape +is undefined and will rewind the tape before calling \fBzzopmt\fR to open +and position the drive. Since this is handled by MTIO, the kernel routines +need not be concerned with error recovery except possibly to abort a tape +motion if an interrupt occurs, to prevent runaway (hopefully this will not +be necessary on most systems). +.PP +The \fBzzclmt\fR primitive will be called to close the device upon normal +termination or during error recovery. MTIO assumes that \fBzzclmt\fR will +write an EOT mark (two tapemarks) \fIat the current tape position\fR when +a tape opened with write permission is closed. This is the only way in which +MTIO can write EOF and EOT marks on the tape. To avoid clobbering tapes, +\fBzzopmt\fR may need to open the drive read-only while positioning the tape. +Since an interrupt may occur while the tape is being positioned, \fBzzopmt\fR +should return the OS channel argument immediately after the channel has been +opened, before positioning the tape. +.PP +In summary, the magtape kernel primitives are highly machine independent +because they permit access to only a single file at a time, reading or +writing sequentially in the forward direction. No primitive tape positioning +commands are required except \fBzzrwmt\fR, and that can be implemented +with a call to \fBzzopmt\fR if necessary (the difference is that \fBzzrwmt\fR +may be asynchronous). No assumptions are made about where the tape is left +positioned if an error occurs or if a tapemark is read or written. +All writing of tapemarks is left to the kernel or to the host system. + +.NH 2 +Filename Mapping +.PP +The syntax of a filename is highly system dependent. This poses a major +obstacle to transporting a large system such as IRAF, since the standard +distribution consists of several thousand files in several dozen directories. +Many of those files are referred to by name in the high level program sources, +and use of host system dependent filenames in such a context would make the +IRAF system very difficult to transport. +.PP +To avoid this problem only \fBvirtual filenames\fR (VFNs) are used in IRAF +source files. FIO converts a VFN into a host system dependent filename (OSFN) +whenever a filename is passed to a kernel routine. Conversely, when FIO reads +a directory it converts the list of OS filenames in the host directory into +a list of virtual filenames. The kernel routines see only machine dependent +filenames packed as Fortran character constants. +.PP +While it is not necessary to study filename mapping to implement the kernel, +filename mapping is a vital part of the system interface and an understanding +of the mapping algorithm employed is necessary to adjust the machine dependent +parameters controlling the mapping. The filename mapping parameters are given +in the system configuration file \fBlib$config.h\fR. +.NH 3 +Virtual Filenames +.PP +A VFN consists of three fields, the \fBdirectory\fR, the \fBroot\fR, +and the \fBextension\fR. Directories are specified by a \fBlogical +directory\fR name followed by a \fBpathname\fR to a subdirectory. +Either the logical directory name or the pathname may be omitted. +If the logical directory field is omitted the current directory is assumed. +The extension field is optional and is used to specify the file type, +e.g., CL source, SPP source, object module, and so on. Either the logical +directory delimiter character \fB$\fR or the subdirectory delimiter +character \fB/\fR may delimit the directory field, and a period delimits +the root and extension fields. +.DS + \fIdir\fR \fB$\fR \fIpath\fR \fB/\fR \fIroot\fR \fB.\fR \fIextn\fR +.DE +.PP +The combined length of the root and extension fields is limited to 32 +characters. The legal character set is \fBA-Za-z0-9_.\fR, i.e., +the upper and lower case alphanumerics (case is significant), underscore, +and period. Other characters may be permitted on some systems, but if +present the filename is machine dependent. The first character of a filename +is not special, i.e., the first character may be a number, underscore, +or any other filename character. Purely numeric filenames are permitted. +The VFN syntax does not support VAX/VMS-like version numbers. A file naming +syntax would not be sufficient to emulate versions; extensive FIO support +would also be required on systems other than VMS. +.PP +The following are all legal virtual filenames (avoiding directory specifications +for the moment). +.DS + 20 + 02Jan83 + Makefile + 10.20.11 + M92_data.Mar84 + extract_spectrum.x + _allocate +.DE +.NH 4 +Logical Directories and Pathnames +.PP +The use of logical directories and pathnames is perhaps best explained by +an example. Consider the VFN \fBplot$graph.x\fR, specifying the file +\fBgraph.x\fR in the logical directory \fBplot\fR. +The logical directory \fBplot\fR is +defined in the CL environment (file \fBlib$clpackage.cl\fR) as follows. +.DS + \fBset plot = "pkg$plot/"\fR + \fBset pkg = "iraf$pkg/"\fR +.DE +These definitions state that \fBplot\fR is a subdirectory of \fBpkg\fR, +and that \fBpkg\fR is a subdirectory of \fBiraf\fR, the root directory of +the IRAF system. The definition for the root directory is necessarily +both machine and configuration dependent. On a VAX/VMS system \fBiraf\fR +might be defined as follows: +.DS + \fBset iraf = "dra0:[iraf]"\fR +.DE +Recursively expanding the original VFN produces the following partially +machine dependent filename: +.DS + \fBdra0:[iraf]pkg/plot/graph.x\fR +.DE +The final, fully translated, machine dependent filename is produced by +folding the subdirectory names into the VMS directory prefix, mapping +the remaining filename (which does not change in this case), and concatenating: +.DS + \fBdra0:[iraf.pkg.plot]graph.x\fR +.DE +.PP +The important thing here is that while there may be many directories in the +system, \fIonly the definition of the IRAF root directory is machine +dependent\fR. Filenames in package script tasks, in the \fBhelp\fR database, +in makefiles, and so on inevitably include references to subdirectories, +hence the VFN syntax must recognize and map subdirectory references to fully +address the problems of machine independence. +.PP +Even when porting the system to another host running the same operating +system the root directory may (and usually will) change. +Since all logical directories and filenames are defined in +terms of the root directory, since the root is defined at runtime, and since +filenames are mapped at runtime, the system may be ported to another machine +running the same operating system by editing only one file, \fIwithout having +to recompile the system\fR. The importance of not having to recompile the +system becomes clear when the local hardware configuration changes or when +installing periodic updates at a site with multiple host computers all running +the same operating system. +.sp 0.08i +.TS +center; +cb s +n l. +Filename Mapping Primitives +.sp +zfsubd \&(osdir, subdir, new_osdir, maxch, nchars) fold subdir into osdir +zfxdir \&(osfn, osdir, maxch, nchars) get directory prefix +zfpath \&(osfn, pathname, maxch, nchars) get absolute pathname +.TE +.sp 0.08i +.PP +The primitives used to map filenames are shown above. The \fBzfsubd\fR +primitive folds a subdirectory name into a machine dependent directory +name (OSDIR), producing the OSDIR name of the subdirectory as output. +The subdirectory name ".." refers to the next higher directory in the +hierarchy, allowing upwards directory references. The form of a host +directory name is undefined, hence the primitive \fBzfxdir\fR is required +to extract a machine dependent directory prefix from a host filename (OSFN). +Directory expansion does not guarantee that the OSFN produced is independent +of the current working directory, hence the \fBzfpath\fR primitive is provided +to convert OSFNs into absolute pathnames. +.NH 4 +Filename Extensions +.PP +Filename \fBextensions\fR, like directories, pose a problem because different +operating systems use different extensions for the same logical file types. +Thus a Fortran source file might have the extensions ".f", ".f77", and +".for" on various systems. IRAF defines a standard set of file extensions +to be used in virtual filenames. Filename extensions are mapped by string +substitution when a file is referenced; unrecognized extensions are left alone. +The standard extensions used in IRAF virtual filenames are essentially those +used by UNIX, plus extensions for the special IRAF file types (e.g., +CL script files and parameter files). +.PP +To illustrate the mapping of filename extensions, consider the IRAF system +library \fBlib$libos.a\fR, which contains the kernel routines in object form. +On a UNIX system this might be expanded as "\fB/usr/iraf/lib/libos.a\fR", +whereas on a VMS system it might be converted to \fBdra0:[iraf.lib]libos.olb\fR. +.PP +The standard IRAF filename extensions are listed in the table below. +Those which are system dependent and are normally mapped are marked +at the right. +.sp +.TS +center box; +cb s s +c | c c +c | l c. +Standard Filename Extensions += +Extension Usage Mapped +_ +\\.a Library file (archive) ** +\\.c C language source ** +\\.cl Command Language script file +\\.com Global common declaration +\\.db database file +\\.e executable image ** +\\.f Fortran 77 source file ** +\\.h SPP header file +\\.hlp \fILroff\fR format help text +\\.ms \fITroff\fR format text +\\.o object module ** +\\.par CL parameter file +\\.pix pixel storage file +\\.x SPP language source +.TE +.sp +.PP +For the convenience of the user working interactively within the IRAF +environment, FIO permits virtual and host system dependent filenames +to be used interchangeably. Any arbitrary file or directory in the host +system may be referenced as an argument to an IRAF program, even if the +host directory has not been assigned a logical name. The filename mapping +scheme thus provides indirect support for pathnames, by permitting the +use of OS dependent pathnames to specify files when working interactively. +If a machine dependent filename is given, mapping of the root and extension +fields is disabled. +.NH 3 +Filename Mapping Algorithm +.PP +The primary requirement for filename mapping is that the process be +reversible, i.e., it must be possible to map a VFN to an OSFN and later +recover the original VFN by applying the reverse mapping. The following +additional requirements led to the selection of the algorithm described +in this section. +.sp 0.08i +.RS +.IP \(bu +There should be no efficiency penalty for simple filenames. +.IP \(bu +The algorithm must permit multiple processes to access the same directory +without contention. +.IP \(bu +The mapping must be transparent, i.e., reversible by inspection of the +host system directory, making it easy to work with directories and files +at the host system level. +.IP \(bu +The reverse mapping (OSFN to VFN) should be efficient, i.e., there must +not be a serious degradation of performance for template expansion and +directory listings. +.IP \(bu +The mapping should permit use of IRAF, with some loss of efficiency, +on a computer with a flat directory system. +.RE +.sp 0.08i +.PP +The algorithm selected consists of two phases. If the maximum information +content of a host system filename is sufficiently large, the first phase will +succeed in generating a unique mapping with no significant overhead. +If the first phase fails, the second phase guarantees a unique mapping on +any system with minimal overhead. The first phase maps the VFN into the +OSFN character set using \fBescape sequences\fR to map non-OSFN characters, +preserving the information content of a filename by increasing its length. +If the length of the OSFN thus generated exceeds the maximum filename length +permitted by the host system, the second phase accesses an \fBauxiliary +hidden file\fR to recover the excess information. +.PP +The operation of the mapping algorithm differs slightly depending on whether +an existing file is to be accessed or a new file is to be created. +The procedure followed to generate a unique OSFN when opening an existing +file is outlined below. The complications caused by multiple logical +directories, filename extensions, and transparency to machine dependent +filenames are not relevant to the algorithm and are omitted. +.sp +.DS +.cs 1 18 +\fBalgorithm\fR vfn_to_osfn +.sp .05 +\fBbegin\fR + # Phase one: encode VFN using only legal OSFN characters. +.sp 0.05i + map vfn to osfn using escape sequence encoding + \fBif\fR (length of osfn is within host limit) + \fBreturn\fR (osfn) +.sp 0.05i + # Phase two. Access or read auxiliary file to get OSFN. +.sp 0.05i + squeeze osfn to legal host filename length + \fBif\fR (squeezed osfn is degenerate) { + extract unique_osfn for named vfn from mapping file + \fBreturn\fR (unique_osfn) + } \fBelse\fR + \fBreturn\fR (osfn) +\fBend\fR +.DE +.cs 1 +.sp +.PP +\fBEscape sequence encoding\fR is a technique for mapping illegal characters +into sequences of legal characters. A single illegal character is mapped into +two legal characters. Strings of illegal characters, e.g., a sequence of +characters of the wrong case, are prefixed by a font change sequence. +For example, suppose the host system permits only upper case alphanumeric +characters in filenames. Lower case is the dominant case in VFNs, so case +will be inverted in the mapping and upper case in a VFN must be escaped. +If we pick the letter Y as our escape character, the following mappings might +be established (these are the defaults for VMS and AOS): +.sp 0.08i +.DS +.TS +ci ci ci +c c l. +vfn osfn usage +.sp +y Y0 the escape character itself +tolower Y1 switch to primary case +toupper Y2 switch to secondary case +\\_ Y3 underscore +\\. Y4 period +A-Z YA-YZ upper case letters +.TE +.DE +.sp 0.08i +.PP +The use of escape sequences can result in confusing mappings, +but if the escape character is chosen carefully such cases will be rare. +Most filenames are quite ordinary and will map with at most a case conversion. +Some examples of filenames which do not map trivially are given below. +The maximum length of a filename extension on the host system is assumed +to be 3 characters in these examples. +Any host limit on the maximum number of characters in the root is ignored. +For the purposes of illustration, we assume that the first character of the +OS filename cannot be a number, necessitating use of a no-op sequence. +.sp 0.08i +.in 0.8i +.TS +l l. +20 Y120 +02Jan83 Y102YJAN83 +Makefile YMAKEFILE +10.20.11 Y110Y420.11 +M92_data.Mar84 YM92Y3DATAY4YMAR84 +extract_spectrum.x EXTRACTY3SPECTRUM.X +_allocate Y3ALLOCATE +yy.tab.c Y0Y0Y4TAB.C +README Y2README +.TE +.in -0.8i +.sp 0.08i +.PP +Escape sequence encoding will probably suffice for most filename mapping, +particularly if the host system permits long filenames (e.g., AOS/VS currently +permits filenames of up to 32 characters). If the encoded filename is +too long for the host system, auxiliary files must be used to store the +excess information. A single \fBmapping file\fR is used for the entire +directory to permit efficient inverse mapping when expanding filename templates +and listing directories, and to avoid wasting disk space by generating many +small files. +.PP +If escape sequence encoding produces an OSFN longer than the maximum OS +filename length N, then characters must be discarded to produce an N character +filename. This is done by squeezing the long OSFN, preserving the first few +and final characters of the root, and the first character of the extension +(if any). For example, if the OSFN is CONCATENATE.PAR and N is 9, +the squeezed OSFN will be CONCATEEP.PAR (the mapping is the same as that +employed for long identifiers in the SPP, except that the first character +of the extension is appended). Once this is done, of course, the mapping +is no longer guaranteed to be unique. +.PP +More often than not a squeezed OSFN will be unique within the context +of a single directory. If this is the case it is not necessary to read the +mapping file to convert a VFN to an OSFN, although it is always necessary to +read the mapping file to carry out the inverse transformation. If the mapping +is unique within a directory, a null file with the same root name as the +primary file but with the (default) extension \fB.zmu\fR is created to +indicate that the mapping is unique (e.g., CONCATEEP.ZMU and CONCATEEX.ZMU). +If a second file is created with the same root OSFN and the mapping is no +longer unique, the \fB.zmu\fR directory entry is simply deleted, and the +mapping file will have to be read whenever either file is accessed. +.PP +The utility of the \fB.zmu\fR file is based on the assumption that the +determining the existence of a file is a much less expensive operation on +most systems than opening, reading, and closing the mapping file. +Furthermore, the \fB.zmu\fR file is a null length file, i.e., just an +entry in a directory, so no disk space is wasted. +Use of the advisory \fB.zmu\fR file does however involve an assumption that the +host system permits filename extensions. If this is not the case, +set the maximum length of a filename extension to zero in \fBconfig.h\fR, +and FIO will not generate the files. +.PP +The mapping file is effectively an extension of the directory and hence +will lead to contention problems when \fBconcurrent processes\fR try to +access the same directory. +A process must not be allowed to read the mapping file +while another process is modifying it, and under no circumstances may two +processes write to the mapping file at the same time. This requires that +a process which wishes to modify the mapping file place a lock on the +file before accessing it, and that a process be capable of waiting if the +mapping file is locked. The mapping data must not be buffered, i.e., +the file should be reread every time a (degenerate) file is accessed. +Fortunately contention should be rare since most file accesses to not +require use of the mapping file. +.PP +In summary, the overhead of the filename mapping algorithm should be +insignificant when (1) accessing files with simple names, +and (2) accessing files with long names for which the mapping is unique. +A small but fixed overhead is incurred when a file with a long name is +created or deleted, when a directory is read, and when a file is accessed +for which the mapping is degenerate. If the host computer has a decent +files system the algorithm will incur negligible overhead for all operations. + +.NH 2 +Directory Access +.PP +The capability to read filenames from a host directory is required for +the expansion of filename templates and for the directory listing program. +At the program interface level a directory appears to be a simple text file, +i.e., an unordered list of virtual filenames. A directory file is opened +at the applications level with \fBdiropen\fR and successive VFNs are read +with \fBgetline\fR. The driver procedures used to interface a directory to +FIO as a text file are machine independent. The kernel primitives called +to read OS filenames from a directory are machine dependent and are summarized +below. +.sp 0.08i +.TS +center; +cb s +n l. +Directory Access Primitives +.sp +zopdir \&(osfn, chan) open a directory +zcldir \&(chan, status) close directory +zgfdir \&(chan, osfn, maxch, status) get next OSFN +.TE +.sp 0.08i +.PP +Directory files are read-only and are accessed sequentially. +A single filename is returned in each call to \fBzgfdir\fR as a +packed string, returning as status the length of the string or EOF. +Filenames may be returned in any order; all filenames in the directory +should be returned (there are no "hidden" files at this level). +Raw OS dependent filenames should be returned. The inverse mapping +from OSFN to VFN is carried out in the machine independent code. +.PP +If the host system does not permit direct access to a directory file, +or does not provide a primitive which returns successive filenames, +it may be necessary to read the entire contents of the directory +into a buffer at \fBzopdir\fR time, returning successive filenames +from the internal buffer in \fBzgfdir\fR calls, and deleting the buffer +at \fBzcldir\fR time. + +.NH 2 +File Management Primitives +.PP +The kernel provides a number of general file management primitives for +miscellaneous operations upon files and directories. These are summarized +in the table below. These primitives are all alike in that they operate +upon files by name rather than by channel number. +The file management primitives +read, write, create, and delete the \fIdirectory entries\fR for files; +none access the actual file data. No primitive which operates upon a file +by name will be called while the file is open for i/o. +.sp 0.08i +.TS +center; +cb s +n l. +File Management Primitives +.sp +zfacss \&(osfn, mode, type, status) access file +zfchdr \&(new_directory, status) change directory +zfdele \&(osfn, status) delete a file +zfinfo \&(osfn, out_struct, status) get info on a file +zfmkcp \&(old_osfn, new_osfn, status) make null copy of a file +zfpath \&(osfn, pathname, maxch, status) osfn to pathname +zfprot \&(osfn, prot_flag, status) file protection +zfrnam \&(old_osfn, new_osfn, status) rename a file +.TE +.sp 0.08i +.PP +The \fBzfacss\fR primitive is used to determine whether or not a file +exists, is accessible with the given permissions, or is a text or binary +file. The \fBzfinfo\fR primitive returns a data structure defining the +file type, access modes, owner, size, creation date, time of last modify, +and so on. Information not returned includes whether the file is a text +or binary file, and whether or not the file is protected from deletion, +because this information is expensive to determine on some systems. +The \fBzfprot\fR primitive is called to place or remove delete protection +on a file, and to test whether or not a file is protected. +.PP +Primitives for accessing directories are limited to \fBzfpath\fR, which +returns the pathname of a file or of the current working directory, +and \fBzfchdr\fR, which changes the current directory. There are no +primitives for creating or deleting new subdirectories: thus far no program +has needed such a primitive. Directory creation and manipulation is +probably best left to the host system. +.PP +The assumption that there are only two basic file types, text and binary, +is overly simplistic when it comes to copying an arbitrary file. If an +executable file is copied as a binary file, for example, the copy will +not be executable. The problem is that a conventional binary file copy +operation copies only the file data: the directory entry is not copied, +and information is lost. The \fBzfmkcp\fR primitive makes a zero length +file which inherits all the system dependent attributes of another file, +excluding the filename, length, and owner. The new file is subsequently +opened for appending as either a text or binary file, and the file data +is copied in the conventional manner. Directory files cannot be copied. + +.NH 2 +Process Control +.PP +Process control, interprocess communication, exception handling, and error +recovery are probably the most complex and subtle services provided by the +IRAF virtual operating system, and the most likely to be machine dependent. +Despite the effort to isolate the machine dependence into the kernel and to +make the kernel primitives as simple and self contained as possible, a high +level understanding of the subtleties of process control may be necessary +to debug system problems. An introduction to process control in IRAF is +therefore presented to supplement the specifications for the kernel primitives. +It should be possible for a systems programmer implementing the kernel to +skim or skip most of this section, referring to it only to resolve ambiguities +in the kernel specifications. +.PP +The CL is currently the only process in the IRAF system which spawns other +processes. In this section we present an overview of process control in the +CL, defining important terms, discussing the conceptual model of a subprocess +as a command file, and describing the architecture of the process control +subsystem. The synchronous protocol for communicating with subprocesses +is discussed, as is the asynchronous file oriented protocol for communicating +with background jobs. The function of the IRAF main is described, including +error recovery and implementation strategies for interfacing asynchronous +processes to a host system window manager or job status terminal. The kernel +primitives for process control are presented at the end of the section. +.NH 3 +Overview and Terminology +.PP +From the point of view of the CL, an executable program is a \fBcommand file\fR +containing a sequence of commands to be parsed and executed. Once opened or +\fBconnected\fR, a compiled program is equivalent to a script task or the +user terminal; the CL does not know or care where the commands are coming from. +Any CL command that can be executed from the user terminal can also be executed +by a compiled program or a script task. Calls to external programs, script +tasks, the terminal, or any other \fBlogical task\fR may be nested until the +CL runs out of file descriptors or stack space. +.PP +A \fBprogram\fR is a compiled logical task. An arbitrary number of programs +may be linked together to form a single physical \fBprocess\fR, i.e., +executable file. When an IRAF process is executed the \fBIRAF Main\fR +(main routine or driver procedure) in the subprocess initializes the process +data structures and then enters an interpreter loop awaiting a command from +the input file, which may be an IPC file (the CL), a terminal, or a text file. +Hence, not only does a subprocess look like a file to the CL, the CL looks +like a file to a subprocess. +.PP +Task termination occurs when the CL reads either end of file (EOF) or the +command \fBbye\fR. When a script task terminates the script file is closed; +when a program terminates the associated process may be \fBdisconnected\fR. +A subprocess is disconnected by sending the command \fBbye\fR to the +IRAF Main in the subprocess. The IRAF main cleans up the files system and +dynamic memory, calls any procedures posted by the user program with +\fBonexit\fR, and then returns to its caller (the \fBprocess main\fR), +causing process exit. +.PP +When a process spawns a subprocess, the original process is called +the \fBparent\fR process, and the subprocess is called the \fBchild\fR process. +A parent process may have several child processes, but a child process may +have only a single parent, hence the process structure of IRAF is a rooted +tree. The root process is the interactive CL; the user terminal (or a window +manager process) is the parent of the CL. Since the CL is the only process +which spawns other processes, an IRAF process tree has only two levels. +.PP +Processes communicate with each other only via \fBIPC channels\fR or +ordinary disk files. Since the only way to open an IPC channel is to connect +a subprocess, a process may communicate only with its parent or with one of its +children. Separate channels are used for reading and writing, rather than +a single read-write channel, to conform to the logical model of a subprocess +as a text file and to facilitate record queueing in the write channel. +Since only IPC channels and ordinary files are used for interprocess +communication and synchronization, the parent and child processes may reside +on separate processors in a \fBmultiple processor\fR system configuration. +.PP +The IPC channels are used to pass commands, data, and control parameters. +While a program is executing it sends commands to the CL, +most commonly to get or put the values of \fBCL parameters\fR. +In a get parameter operation the CL responds by printing the value of the +parameter on its output, just as it does when the +same command is typed in interactively. The output of the CL is connected +to the input IPC channel of the child process, which reads and decodes the +value of the parameter, returning the binary value to the calling program. +.PP +The standard input, standard output, standard error output, standard graphics +output, etc. of the child process are known as \fBpseudofiles\fR because the +actual files are opened and controlled entirely by the CL. A read or write +to a pseudofile in the child process is converted into a command to read or +write a binary block of data and sent over the IPC channel along with the +binary data block. Hence, even though most traffic on the IPC channels is +ASCII text, the channels are implemented as binary files. A large transfer +block size is used for all pseudofiles except STDERR to maximize throughput. +All parameter i/o and pseudofile i/o is multiplexed into a single command stream +and transmitted over the two IPC channels. +.PP +To minimize process connects, the CL maintains a \fBprocess cache\fR of +connected but generally idle subprocesses. A process is connected and placed +in the cache when a program in that process is run. A process will remain +in the cache, i.e., remain connected to the CL process, until either a new +process connect forces the process out of the cache or until the cache +is flushed. Since programs execute serially, at most one cached process will be +active at a time. Since several subprocesses may simultaneously be connected +and each subprocess may contain an arbitrary number of programs, the cache +can greatly reduce the average overhead required to run an external program, +while permitting dynamic linking of programs at run time. +.PP +The CL executes programs serially much as a program executes subroutines +serially. The protocol used to communicate with a connected subprocess is +synchronous. To execute a program or general command block as a +\fBbackground job\fR, i.e., asynchronously, the CL spawns a copy of itself +which executes independently of the parent CL. The child CL inherits the +full interior state of the parent CL, including the metacode for the command +to be executed, all loaded packages and parameter files, the environment list, +and the dictionary and stacks. Open files are not inherited, the child CL +is not connected to the parent CL, and the CL subprocess is not placed in the +process cache of the parent. The child CL manages its own process cache +and executes external programs using the synchronous protocol. A child CL +may spawn a child CL of its own. +.NH 3 +Synchronous Subprocesses +.PP +The sequence of actions required to synchronously execute an external compiled +program are summarized below. Only those actions required to execute a +program are shown; the process cache is a local optimization hidden within the +CL which is not relevant to a discussion of the synchronous subprogram protocol. +Everything shown is machine independent except the process main and the +IPC driver. Only a single process may be in control at any one time. +.PP +A process may be spawned by another IRAF process or by the host command +interpreter or JCL. When a process is spawned the host system transfers +control to a standard place known as the \fBprocess main\fR. The process +main must determine what type of parent it has and open the type of input +and output channels required by the parent. If the parent is another process +IPC channels are opened. The process main then calls the IRAF Main which +initializes IRAF i/o and enters the Main interpreter loop to read commands +from the parent. +.sp 0.05i +.DS +.ce +\fBProcess Startup\fR +.sp +\fIParent process:\fR + spawn the subprocess + open IPC channels between parent and child + send commands to initialize environment list in child +.sp +\fIProcess Main in child:\fR + determine whether input device is a terminal, text file, or process + open input and output channels (e.g. the IPC channels) + call the IRAF Main +.sp +\fIIRAF Main in child:\fR + save process status for error restart + initialize file i/o, dynamic memory, and error handling + post default exception handlers + enter interpreter loop, reading commands from parent +.DE +.sp +.PP +An IRAF process will interpret and execute successive commands in its input +file until it encounters either EOF or the command \fBbye\fR. The first +block of commands read by the Main will normally be a sequence of \fBset\fR +statements initializing the process environment (defining logical directories +and devices, etc.). A number of calls to the programs resident in the process +will normally follow. When a program runs it assumes control and begins +issuing commands to the parent to read and write parameters and pseudofiles. +The IRAF i/o system is reset to its default initial state each time a program +terminates (this can be overridden by a program if desired). +.sp +.DS +.ce +\fBProcess Execution\fR +.sp +\fIParent process:\fR + send name of program to be run to the IRAF Main in the child process + redirect command input to child, i.e., transfer control to the + program in the child process +.sp +\fIProgram in child process:\fR + (we were called by the interpreter in the IRAF Main) + execute, sending commands to parent to read and write parameters + and pseudofiles + return to caller (the IRAF Main) when done +.sp +\fIIRAF Main:\fR + flush STDOUT, close any open files, etc. + send the command \fBbye\fR to parent to signal that program has + completed and to return control to the parent + enter interpreter loop, reading commands from parent +.DE +.sp +.PP +A process shuts down or exits only when commanded to do so by the parent, +i.e., when an input file read returns EOF or the command \fBbye\fR is executed. +Any user defined procedures posted with \fBonexit\fR calls during process +execution will be executed during process shutdown. Control eventually +returns to the process main which takes whatever system dependent actions +are required to terminate the process. +.sp +.DS +.ce +\fBProcess Shutdown\fR +.sp +\fIParent process:\fR + if no further programs are to be run, send the command \fBbye\fR + to the child to initiate process shutdown +.sp +\fIIRAF Main:\fR + disable IPC output (parent is no longer reading) + call any user procedures posted with \fBonexit\fR, i.e., flagged + to be executed upon process shutdown + return to caller (the process main) when done +.sp +\fIProcess Main:\fR + terminate subprocess +.sp +\fIParent process:\fR + disconnect child process +.DE +.sp 0.05i +.PP +If a process issues a read request on an IPC channel +and there is no input in the channel, the reading process will block, hence +reading from an empty IPC channel causes process synchronization. Successive +writes are queued until the channel is full, hence writing is generally +asynchronous and some degree of overlapped execution is possible. +Traffic on the IPC channels is restricted to the small set of commands +described in the next section. +.NH 3 +Standard IPC Commands +.PP +Although in principle a subprocess may send any legal CL command to the CL +process, in practice only a small subset of commands are permitted in order to +minimize the size of the interface. A larger interface would mean more +dependence upon the characteristics of a particular CL, making it more +difficult to modify the CL and to support several different versions of the CL. +.PP +The IPC interface commands described in this section are a high level protocol +implemented entirely above the kernel routines to support execution of external +programs by the CL. If the parent process were not the CL and if a new IRAF +Main were implemented (the IRAF Main is an ordinary SPP procedure), then a +quite different protocol could be devised. +.PP +The \fBIRAF Main requests\fR, i.e., the IPC commands sent to the IRAF Main by +the parent process, are shown below in the order in which they are normally +sent to the child process. Italicized text denotes dummy parameters to be +replaced by the name or value of the actual parameter when the command is +issued. Keywords are shown in boldface. Optional characters or arguments are +delimited by square brackets. All commands are ASCII lines of text terminated +by the \fBnewline\fR character. +.sp +.in 1.0i +.KS +.ti -0.5i +\fBset\fR \fIvariable\fR = \fIstring\fR +.ti -0.5i +\fBset\fR @\fIfname\fR +.sp 0.04i +Set the value of an environment variable or set the environment from a file. +If the variable does not exist it is created; if it does exist the new value +silently replaces the old value. The \fBset\fR statement is used to pass +the environment list of the parent to the child process when the subprocess +is connected. The second form of the \fBset\fR statement reads a list of +\fBset\fR declarations from a text file, and is especially useful in debug +mode. +.KE +.sp +.KS +.ti -0.5i +\fB?\fR +.sp 0.04i +Print the names of all user programs linked into the process in tabular +form (i.e. print a menu) on the standard output. This command is not +currently used by the CL; it is most useful when debugging a process run +directly from the host command interpreter. +.KE +.sp +.ti -0.5i +[\fB$\fR] \fIprogram\fR [<[\fIfname\fR]], [[\fIstream\fR[(T|B)]]>[\fIfname\fR]], [[\fIstream\fR]>>[\fIfname\fR]] +.sp 0.04i +Execute the named program. The environment should have been initialized by +the time a program is run. If a dollar sign is prefixed to the command +name, the cpu and clock time consumed by the process are printed on the +standard error output when the task terminates. +If a pseudofile stream has been redirected by the parent or is to be +redirected by the child, this should be indicated on the command line. +Thus the IRAF Main command +.DS +count < +.DE +would run the program \fIcount\fR, informing the IRAF Main that the standard +input has already been redirected by the parent (some programs need to know). +If redirection to or from a named file is indicated, the IRAF Main will open +the file and redirect the indicated stream before running the program. +Pseudofiles streams are denoted by the numerals 1 through 6, corresponding +to STDIN, STDOUT, STDERR, STDGRAPH, STDIMAGE, and STDPLOT. If output is +being redirected into a new file and \fBT\fR or \fBB\fR appears in the +argument (e.g., "4B>file"), a text or binary file will be created as specified. +If the file type suffix is omitted and the output stream is STDOUT or STDERR +a text file will be created, otherwise a binary file is created. +For example, the command +.DS +count <, > file +.DE +directs the Main to flag the standard input as redirected, open the new text +file "file" as the standard output of the the program \fIcount\fR, and then +run the program. When the program terminates the Main will automatically +close the output file. +.sp +.KS +.ti -0.5i +\fBbye\fR +.sp 0.04i +Commands the subprocess to shutdown and exit. The subprocess must not read +or write the IPC channels once this command has been received; the CL +disconnects the subprocess immediately after sending \fBbye\fR. User +procedures posted with \fBonexit\fR are called during process shutdown. +If an irrecoverable error occurs during normal process shutdown it will cause +an immediate \fBpanic shutdown\fR of the process. The kernel writes an +error message to the process standard error channel (e.g. the user terminal) +when a panic shutdown occurs. +.KE +.sp +.in -1.0i +.PP +Although it might appear that initialization of the process environment +list via a sequence of \fBset\fR commands is inefficient, +the \fBset\fR commands are buffered and transmitted to the child process +in large binary IPC blocks to minimize the overhead. +The amount of data transmitted is not significantly +different than it would be if the environment list were transmitted as a +binary array, and matching of the internal environment list data structures in +the two processes is not required. Furthermore, the \fBset\fR command is +ideal when debugging a process or when running a process in batch mode +with a previously prepared command input file. +.PP +The IRAF Main commands are used both by the CL to run an external compiled +program and by the programmer when debugging a process at the host system +level. The IRAF Main knows whether it is being used interactively or not, +and modifies the interface protocol slightly when used interactively to +provide a better user interface. For example, the Main issues a command +prompt only when being used interactively. The form of a parameter request +and of a pseudofile read or write request is also slightly different in the +two cases. +.PP +The \fBCL requests\fR, i.e., the commands sent by a running program to the CL +(or any other parent process) are shown below. These are the only commands +which the child can legally send to the parent, and hence the only commands the +interpreter in the parent need recognize. The noninteractive syntax is shown. +If the parent process is not the CL a completely different protocol can be +used. When a subprocess is run interactively the \fBxmit\fR and \fBxfer\fR +requests are omitted (only the data is sent) and the newline is omitted after a +parameter read request. +.sp +.in 1.0i +.KS +.ti -0.5i +\fIparam\fR = +.sp 0.04i +The parent process is directed to print the single-line value of the +named parameter in ASCII on the child's input IPC channel. The child +decodes the response line and returns a binary value to the program. +.KE +.sp +.KS +.ti -0.5i +\fIparam\fR = \fIvalue\fR +.sp 0.04i +The parent process is directed to set the value of the named parameter to +the indicated ASCII value. The child does not expect a response, and parameter +write requests may be queued in the output IPC channel. +.KE +.sp +.KS +.ti -0.5i +\fBxmit\fR (\fIpseudofile\fR, \fInchars\fR) +.sp 0.04i +The parent process is directed to read exactly \fInchars\fR chars of binary +data from the IPC channel and transmit it without interpretation to the +indicated pseudofile. The child does not expect a response, and pseudofile +write requests may be queued in the output IPC channel. Pseudofiles are +denoted by the numerals 1 through 6, corresponding to STDIN, STDOUT, STDERR, +STDGRAPH, STDIMAGE, and STDPLOT. +.KE +.sp +.KS +.ti -0.5i +\fBxfer\fR (\fIpseudofile\fR, \fImaxchars\fR) +.sp 0.04i +The parent process is directed to read up to \fImaxchars\fR chars of binary +data from the indicated pseudofile and transmit it without interpretation to the +input IPC channel of the child. The binary data block should be preceded +by an ASCII integer count of the actual number of chars in the data block. +.KE +.sp +.KS +.ti -0.5i +\fBbye\fR +.sp 0.04i +Normal program termination. Control is transferred from the +child to the parent. The child returns to the interpreter loop in the +IRAF Main, awaiting the next command from the parent. +.KE +.sp +.KS +.ti -0.5i +\fBerror\fR (\fIerrnum\fR, "\fIerrmsg\fR") +.sp 0.04i +Abnormal program termination. +An irrecoverable error has occurred during the execution of the program +(or of the IRAF Main), and the CL is directed to take an error action for +error number \fIerrnum\fR. The child returns to the interpreter loop in the +IRAF Main, awaiting the next command from the parent. If the error is not +caught and handled by an error handler in a CL script, the error message +\fIerrmsg\fR is printed on the standard error output of the CL and the +child process is commanded to shutdown. +.KE +.sp +.in -1.0i +.NH 3 +Example +.PP +By this point process control probably sounds much more complicated than it +actually is. A brief example should illustrate the simplicity +of the CL/IPC interface. Consider the CL command +.DS +cl> \fBset | match tty\fR +.DE +which prints the values of all environment entries containing the substring +\fBtty\fR. The CL task \fBset\fR is a builtin function of the CL and +hence does not use the IPC interface. We assume that the process +\fBsystem$x_system.e\fR, which contains the program \fImatch\fR, +has already been connected so that it is not necessary to pass the +environment to the child. The traffic over the IPC channels is shown below. +If a running IRAF system is available the process side of this example can +be duplicated by typing \fBecho=yes\fR followed by the command shown above. +.sp 0.08i +.TS +center; +ci ci +l l. +CL Process +.sp +\fBmatch <\fR + \fBpattern=\fR +tty + \fBmetacharacters=\fR +yes + \fBstop=\fR +no + \fBxfer(1,1024)\fR +1024 +\fI(1024 chars of data sent to child)\fR + \fBxfer(1,1024)\fR +368 +\fI(368 chars of data sent to child)\fR + \fBxmit(2,63)\fR + \fI(63 chars of data sent to CL)\fR + \fBbye\fR +.TE +.sp 0.08i +.PP +Each line of text shown in the example is transmitted through the appropriate +IPC channel as a single distinct record. Commands are shown in boldface. +The italicized records represent raw data blocks. +The process \fBsystem$x_system.e\fR contains the fifty or so executable programs +in the \fBsystem\fR package and hence is a good example of the use of +multitasking and the process cache to minimize process connects (as well as +disk space for executable images). +.NH 3 +Background Jobs +.PP +IRAF process control does not support fully asynchronous subprocess execution +for the following reasons: +.sp +.RS +.IP \(bu +The parent and child processes are tightly bound, i.e., while an external +program is executing the CL process is subservient to the applications program. +The fact that the CL is a separate process is an irrelevant detail to +the applications program. From the point of view of the applications program +the CL is a database interface called by CLIO. Applications programs are not +fully functional unless connected to a CL at run time, and a synchronous, +interactive interface between the two processes is assumed. +.IP \(bu +From the point of view of the user or of a CL script, external programs are +subroutines. Subroutines execute serially in the context of the calling +program. In this case the context is defined by the state of the data +structures of the CL, i.e., the dictionary, environment list, loaded packages +and tasks, parameters, and so on. +.IP \(bu +The user does not care whether a task is a subprocess or a CL script, +and tends to think in terms of \fBcommand blocks\fR rather than individual +commands. A command block is a user specified sequence of commands to +be compiled and executed as a single unit. Asynchronous subprocesses +are not interesting; what we want is an asynchronous command block. +.IP \(bu +It is much more difficult to define a machine independent process control +interface for asynchronous subprocesses than for synchronous subprocesses. +The problem is similar to that of designing +a multiprocessing operating system, with the CL acting as the operating system +kernel and the user as the cpu. Asynchronous subprocess execution is +inconsistent with the conceptual model of subprocesses and users as command +files, and is extremely difficult to implement in a portable system in any case. +.RE +.sp +.PP +For these and other reasons, background job execution is implemented in +IRAF by spawning a copy of the foreground CL which executes as a +\fBdetached process\fR, rather than as a connected subprocess. +The child CL manages its own process cache independently of the parent. +All connected subprocesses execute synchronously, i.e., only one process +in a tree of connected processes may be active at a time. +Since the child is never connected to the parent, the background CL may +execute at any time (e.g. in a batch queue), and may continue to execute +after the parent process has terminated (if the host system permits). +.PP +The child CL inherits the data structures of the parent as they existed +immediately after translating the command block into metacode and just prior +to execution. The parent's data structures are propagated to the child by +writing them into a binary file which is subsequently opened and read by the +child. Open files are not inherited. The command block executes in the +child in exactly the same context as it would have had if executed in the +parent, with exactly the same results. +.PP +On many systems background job execution will be predominantly noninteractive, +particularly if background jobs are placed into a batch queue. +Even if a background job is run noninteractively, however, there is no +guarantee that the job will not require interaction during execution, +for example if the user forgot to set the value of a parameter when the +job was submitted. Rather than aborting a background job which needs to +query for a parameter, the CL provides a limited but portable method for +servicing queries from background jobs. Extensive interaction with background +jobs is beyond the capabilities of the portable IRAF system but is not ruled +out; interfacing to \fBwindow management\fR facilities is straightforward +if the host system provides such facilities, and is described in the next +section. +.PP +The CL has a builtin capability for generating and servicing queries from +noninteractive background jobs. Such queries might be normal and expected, +e.g. if a background job executes concurrently with the interactive CL and +limited interaction is desired, or might be a failsafe, e.g. if a background +job has consumed several hours of cpu time and the job would have to be +resubmitted if it were to abort because it could not satisfy a parameter +request. +.PP +The CL will automatically initiate a query request sequence whenever it is +executing in the background and an attempt to service a query by reading +from the process standard input returns EOF. To initiate a query request +the CL writes the query prompt into a \fBservice request file\fR, +writes a status message to the standard error output of the CL process +noting that the job is stopped waiting for parameter input, +and enters a loop waiting for the \fBquery response file\fR to be created. +When the query response file becomes accessible the CL opens it, +reads the contents to satisfy the original read request, +deletes the file, and continues normal execution. +.PP +If there is no response the background CL will eventually timeout, writing +a fatal error message to the standard error output of the process. +Queries from background jobs are normally satisfied from an interactive CL +using the builtin task \fBservice\fR, which types the service request file +on the terminal, deletes the file, reads the user's response from the terminal, +and writes the response into the query response file. +If the query response is unacceptable another query will be generated and +the interchange is repeated. The use of simple text files for interprocess +communication makes the technique very general, and in principle there is +nothing to prevent the technique from being used to service requests from +jobs run either as detached subprocesses or in batch queues. +.NH 3 +The Process and IRAF Mains +.PP +The roles of the Process and IRAF Mains should already be clear from the +previous sections. The process main is machine dependent and is called by +the host operating system when a process is executed. The IRAF Main is +a portable SPP procedure which is called by the process main +during process startup, which acts as a simple command interpreter during +process execution, and which returns control to the process main during +process shutdown. +.NH 4 +The Process Main +.PP +The \fBprocess main\fR is a part of the kernel, but unlike any other kernel +procedure it is not Fortran callable (in fact it is not necessarily a procedure +at all). The process main further differs from any other kernel procedure +in that it calls a high level procedure, the IRAF Main. Since the process +main is not Fortran callable, however, there is no possibility of recursion. +.PP +The primary functions of the process main are to open and initialize the +process i/o channels and to call the IRAF Main. +The process i/o channels are the standard input, standard output, and standard +error output of the process. The process or device to which the channels +are connected is both system dependent and dependent on how the process was +spawned. +.PP +The process main can be coded in assembler if necessary on almost any system. +Typically the host operating system will upon process entry transfer control +to a predefined address or external identifier, and this entry point should be +the process main. On many modern systems it will be possible to code the +main in a high level language; this is desirable provided the high level +language does not have a main of its own which necessitates loading a lot +of extraneous code which will never be used (since IRAF does all i/o via +the IRAF kernel). On a UNIX system, for example, the process main is +implemented as the C procedure "main" with no overhead, so there is nothing +to be gained by coding the process main in assembler. +.sp +.cs 1 18 +.nf +\fBprocedure\fR process_main + +input_chan: process standard input channel +output_chan: process standard output channel +errout_chan: process standard error output channel + +\fBbegin\fR + # Determine type of output device and connect channels. + + \fBif\fR (we are a connected subprocess) { + connect input_chan and output_chan to IPC channels + connect errout_chan to the user terminal (i.e, to the + standard error output channel of the parent process) + + } \fBelse if\fR (we are a detached process) { + \fBif\fR (window management facilities are available) + connect all channels to window manager + \fBelse\fR { + connect input_chan such that a read will return EOF + \fBif\fR (we are executing in a batch queue) + connect output channels to files + \fBelse\fR { + connect both output_chan and errout_chan to the user + terminal (i.e, to the standard error output channel + of the parent process, if the terminal can be + written to by multiple processes) + } + } + + } \fBelse if\fR (we were run from the host command interpreter) { + \fBif\fR (we were called interactively) + connect channels to user terminal + \fBelse\fR { + connect input_chan and output_chan to job input + and output files, and errout_chan to operator + console or system dayfile. + } + } + + # Call the IRAF Main, the command interpreter or driver of an + # IRAF process. + + call iraf_main, passing the channel numbers and identifying + the driver and protocol to be used by the Main + + # We get here only after the parent has commanded the IRAF + # Main to shutdown, and after shutdown has successfully + # completed. Fatal termination occurs elsewhere. + + close channels if necessary + normal exit, i.e., terminate process +\fBend\fR +.cs 1 +.fi +.sp +.PP +An IRAF process may easily be used in a completely batch mode by connecting +the process channels to text files. On an interactive system the channels +of a detached process may be connected directly to the user terminal, +but it can be annoying for the user if background processes are intermittently +writing to the terminal while the user is trying to do something else +(e.g. trying to edit a file using a screen editor). Having multiple processes +trying to simultaneously read from a terminal is disastrous. +.PP +The best solution to the problem of multiple processes trying to read or +write from the user terminal is some sort of \fBwindow manager\fR. +A simple window manager which can handle output from multiple simultaneous +IRAF processes but which will only allow a single process to read is not +difficult to code on many systems, provided one admits that the capability +is machine dependent. A full up window manager such as is provided on many +modern microcomputers is a much more difficult problem and should not be +attempted as an add-on, as it really needs to be integrated into the +operating system to work well. A better approach is to buy a microcomputer +which comes with a bit-mapped terminal and a fully integrated window manager, +or to buy a smart terminal which has window management capabilities. +.PP +If a window manager is to be provided as an add-on to a system which does +not already have one, it should be implemented as a single process handling all +i/o to the terminal. The CL will be run from the window manager process +and detached processes will talk directly to the window manager process +using multiplexed IPC channels. Such an add-on window manager is unlikely +to be fully usable for non-IRAF processes (e.g. the host system screen editor) +unless the host operating system is modified in fundamental ways. If the +window manager has to be built from scratch consider coding it as an IRAF +process with an extended system interface, so that it will be at least +partially portable. +.NH 4 +The IRAF Main +.PP +The IRAF Main is the "main program" of an IRAF process. The primary function +of the Main is to interpret and execute commands from the standard input +of the CL process (the stream CLIN) until either EOF is seen or the command +\fBbye\fR is received by the Main (as opposed to a program called by the Main). +The Main is mechanically generated by the SPP compiler when the \fBtask\fR +statement is encountered in an SPP program; the source is in the file +\fBmain.x\fR in the logical directory \fBsys$system\fR. +.PP +The secondary functions of the Main are to initialize the IRAF i/o system +and participate in error recovery. The first time the Main is called it +initializes the i/o system and posts a default set of \fBexception handlers\fR. +The Main can only be called again (without recursion) during an \fBerror +restart\fR. If an irrecoverable error occurs during error restart, +a panic exit occurs, i.e., the process dies. +.PP +Error restart takes place when a uncaught hardware or software exception +occurs, or when an error action is taken by a program and no user +\fBerror handler\fR is posted. All exceptions and errors may ideally be +caught and processed by a user exception handler or error handler, without error +restart occurring. When error restart occurs the hardware stack is +reset and control transfers to the marked position within the process main. +The process main calls the IRAF Main, which knows that it has been called +during error restart. +.PP +When the IRAF Main is called during error restart the first thing it does +is call any user procedures posted with \fBonerror\fR. If an irrecoverable +error occurs during execution of an \fBonerror\fR error recovery procedure, +\fBerror recursion\fR occurs and a panic exit results. +When the \fBonerror\fR procedures have successfully executed the Main sends +the \fBerror\fR statement to the CL (i.e., to the stream CLOUT) and reenters +its interpreter loop, awaiting the next command from the CL. +If no user error handler is posted at the CL level (error handling was not +implemented at the CL level at the time when this was written), then the +CL will direct the child process to shutdown to ensure that dynamic memory +space is reclaimed, and to ensure that a user program is not left in a bad +state by the error. +.NH 3 +Process Control Primitives +.PP +We are now in a position to define and understand the kernel primitives +necessary to implement process control. There are 9 such primitives, +excluding the process main and the exception handling primitives. +The mnemonic "pid" refers to the \fBprocess id\fR, a unique magic integer +assigned by the host operating system at process creation time. +.sp 0.08i +.TS +center; +cb s +n l. +Process Control Primitives +.sp +zopcpr \&(process, inchan, outchan, pid) open connected subprocess +zclcpr \&(pid, exit_status) close connected subprocess +zintpr \&(pid, exception, status) interrupt connected subprocess +.sp +zopdpr \&(process, bkgfile, jobnum) open or queue detached process +zcldpr \&(jobnum, killflag, exit_status) close or dequeue detached process +.sp +zgtpid \&(pid) get process id of current process +zpanic \&(errcode, errmsg) panic exit +zsvjmp \&(jumpbuf, status) save process status +zdojmp \&(jumpbuf, status) restore process status +.TE +.sp 0.08i +.PP +Separate sets of primitives are defined for connected and detached +subprocesses. A subprocess is connected with \fBzopcpr\fR, which spawns +the subprocess and opens the IPC channels. The child is assumed +to inherit the current working directory of the parent. +Connection of the IPC channels to FIO and transmission of the environment +list to the subprocess is left to the high level code. +.PP +A connected subprocess is disconnected with \fBzclcpr\fR, which waits +(indefinitely) for the subprocess to exit and returns the exit status code, +e.g. OK. The high level code must command the subprocess to shutdown +before calling \fBzclcpr\fR or deadlock will occur. +The high level code guarantees that \fBzclcpr\fR will be called to close +any subprocess opened with \fBzopcpr\fR. The \fBzintpr\fR primitive +raises the interrupt exception X_INT in a connected subprocess. +.PP +A detached process is spawned or submitted to a batch queue with \fBzopdpr\fR. +It is up to \fBzopdpr\fR to pass the name of the background file on to the +child by some means (the background file tells the detached process what +to do). A detached process may be killed or removed from the batch queue +by a call to \fBzcldpr\fR. The high level code will call \fBzcldpr\fR if +the detached process terminates while the parent is still executing, +but there is no guarantee that a detached process will be closed. +.PP +The remaining primitives are used by all processes. The \fBzgtpid\fR primitive +returns the process id of the process which calls it; this is useful for +constructing unique temporary file names. The \fBzpanic\fR primitive is +called when error recovery fails, i.e., when an error occurs during error +recovery, causing error recursion. A panic shutdown causes immediate process +termination, posting an error message to the process standard error output +and returning an integer error code to the parent process. +.PP +The IRAF main calls \fBzsvjmp\fR to save the process control status for +error restart. The process status is saved in \fIjumpbuf\fR, +allowing several jump points to be simultaneously defined. +A subsequent call to \fBzdojmp\fR restores +the process status, causing a return from the matching \fBzsvjmp\fR call +\fIin the context of the procedure which originally called \fBzsvjmp\fR. +The \fIstatus\fR argument is input to \fBzdojmp\fR and output by \fBzsvjmp\fR, +and is zero on the first call to \fBzsvjmp\fR, making it possible for the +procedure which calls \fBzsvjmp\fR to determine how it was entered. +These extremely machine dependent routines are patterned after the UNIX +\fBsetjmp\fR and \fBlongjmp\fR primitives, but are Fortran callable. +They will almost certainly have to be written in assembler since they fiddle +with the hardware stack and registers. +.PP +On a \fBmultiple processor\fR system it should be possible to spawn both +connected and detached processes on a remote processor. For example, +if the parent process resides on a diskless node in a cluster, it may be +desirable to run subprocesses that do heavy i/o on the remote file server +processor which has a high i/o bandwidth to disk. On such a system +advice on the optimal processor type should be encoded as extra information +in the process file name passed to \fBzopcpr\fR or \fBzopdpr\fR; this will +require modification of the CL \fBtask\fR statement for such processes. + +.NH 2 +Exception Handling +.PP +An exception is an asynchronous event, i.e., an interrupt. +Typical \fBhardware exceptions\fR are an attempt to access an unmapped region +of memory, illegal instruction, integer overflow, or divide by zero. +A hardware exception occurs when the hardware detects an error condition +while executing a hardware instruction. Typical \fBsoftware exceptions\fR are +interrupt and kill. A software exception occurs when a program, e.g., +the terminal driver or an applications program like the CL, sends an +interrupt or \fBsignal\fR to a process. When an exception occurs program +execution is interrupted and control transfers to an \fBexception handler\fR, +i.e., to a previously posted system or user procedure. +.PP +The ability to post an exception handler or to send a signal to a process +is fundamental in any multiprocessing operating system, but regrettably there +are still some older systems that do not make such facilities available to +applications code. Hopefully yours is not such a system. Even if a system +provides exception handling facilities, the set of exceptions defined for +a particular computer or operating system is very system dependent. +Exception handling can be subtly machine dependent, e.g., it is not always +possible to disable an exception, and it is not always possible to resume +program execution (restart the interrupted instruction) following an exception. +.PP +The IRAF Main posts a default set of exception handlers during process startup. +All host system exceptions that might occur during the execution of an IRAF +process should be catchable by one of the default exception handlers. +If an exception is not caught and is instead handled by the host, +the process will almost certainly die without the knowledge of the CL, +leading at best to a cryptic "write to a subprocess with no reader" error +message, and at worst to deadlock. Since error recovery and process shutdown +will be skipped if an uncatchable exception occurs, disk data structures may +be corrupted. +.PP +The virtual system recognizes only four classes of exceptions; all possible +host system exceptions should either be mapped into one of these exceptions +or caught in the kernel and mapped into ERR. +.sp 0.08i +.TS +center box; +cb s s +ci | ci | ci +l | c | l. +Virtual Machine Exceptions +_ +exception code meaning += +X_ACV 501 access violation +X_ARITH 502 arithmetic error +X_INT 503 interrupt +X_IPC 504 write to IPC with no reader +.TE +.sp 0.08i +.PP +The largest class of exceptions on many systems will be the access violations. +This class includes such things as illegal memory reference, illegal +instruction, illegal system call, and so on. +Arithmetic exceptions include divide by zero, integer overflow, and the like. +Interrupt is the exception raised by \fBzintpr\fR or by the +host system terminal driver when the interrupt sequence is typed (e.g. ctrl/c). +.sp 0.08i +.TS +center; +cb s +n l. +Exception Handling Primitives +.sp +zxwhen \&(exception, handler, old_handler) post an exception +zxgmes \&(os_exception, outstr, maxch) get OS code and message +.TE +.sp 0.08i +.PP +An exception handler is posted for a virtual exception with the primitive +\fBzxwhen\fR. All host exceptions in the indicated virtual exception class +are affected. The argument \fIhandler\fR is either the entry point address +of the new user exception handler or the magic value X_IGNORE (null). +The address of the old handler or X_IGNORE is returned as the third argument, +making it possible for the high level code to chain exception handlers +or repost old exception handlers. The calling sequence for a user exception +handler is as follows: +.DS +user_handler (exception, next_handler) +.DE +.LP +The user exception handler is called with the integer code for the actual +virtual exception as the first argument. The integer code of the last +machine exception and a packed character string describing the exception +may be obtained by a subsequent call to \fBzxgmes\fR. A program which uses +a machine exception code is machine dependent, but machine exception codes +can be parameterized and some programs need to know. +The CL, for example, has to be able to recognize the machine exception for +a write to a process (an IPC channel) with no reader. +.PP +When an exception occurs control actually transfers to the \fBkernel exception +handler\fR, which maps the machine exception into a virtual exception, +looks at the kernel exception table to determine what type of action is +required, and then calls the user exception handlers. A user exception +handler is expected to handle the exception in some application dependent +way and then either change the process context by calling \fBzdojmp\fR or +\fBzrestt\fR, or return control to the kernel exception handler. +If control returns to the kernel handler the output argument \fBnext_handler\fR +will contain either the entry point address of the next exception handler +or X_IGNORE. The high level code assumes that once an exception handler +is posted it stays posted, i.e., is not reset when an exception occurs. +.PP +Few programs actually post exception handlers; most just post an error handler +with \fBonerror\fR. Such an error handler will be called by the IRAF Main +either when an exception occurs or when an error action is taken, i.e., +when a program is aborted for whatever reason. If a user exception handler +is not posted the default handler will be called, causing error restart of the +Main, calls to all \fBonerror\fR procedures, and transmission of the \fBerror\fR +statement to the CL. If the CL is interrupted while executing an external +program it passes the interrupt on to the child with \fBzintpr\fR and then +resumes normal processing. The external program retains control, and therefore +can choose to either ignore the interrupt or take some application dependent +action. + +.NH 2 +Memory Management +.PP +The IRAF system relies heavily on memory management for dynamic buffer +allocation in both system and applications software. +Both stack and heap storage are provided at the program interface level. +The \fBstack\fR is used primarily for "automatic" storage allocation, +i.e., for buffers which are allocated upon entry to a procedure and deallocated +upon exit from the procedure. Stack management incurs very little overhead +for small buffers. The \fBheap\fR is a more general storage mechanism; +buffers may be allocated and deallocated in any order, and allocation and +deallocation may occur in different procedures. A heap buffer may be +reallocated, i.e., changed in size. The stack is implemented portably in terms +of the heap, and hence need not concern us further here. +.sp 0.08i +.TS +center; +cb s +n l. +Memory Management Primitives +.sp +zmaloc \&(buffer, nbytes, status) allocate a buffer +zmfree \&(buffer, status) deallocate a buffer +zraloc \&(buffer, nbytes, status) reallocate a buffer +zlocva \&(variable, address) get address of a variable +zawset \&(bestsize, newsize, oldsize, textsize) adjust working set size +.TE +.sp 0.08i +.PP +Buffer space is allocated on the heap by the primitive \fBzmaloc\fR. +The address of a buffer at least \fInbytes\fR in size is returned +as the argument \fIbuffer\fR. Nothing is assumed about the alignment of +the buffer. The contents of the buffer are not assumed to be initialized. +.PP +The buffer address returned by \fBzmaloc\fR is in units of SPP \fBchars\fR +rather than in physical units. The \fBzlocva\fR primitive returns the +address of a \fBcsilrd\fR variable, array, or array element in the same +units. By using char address units and by doing all pointer dereferencing +by subscripting Fortran arrays, we avoid building knowledge of the memory +addressing characteristics of the host system into SPP programs. +The zero point of a char address is undefined; negative addresses are +possible depending on the implementation. It must be possible to store an +address in an integer variable, and it must be possible to perform +\fIsigned integer\fR comparisons and arithmetic on addresses. +.PP +A buffer allocated with \fBzmaloc\fR may be reallocated with \fBzraloc\fR. +In this case the \fIbuffer\fR argument is used both for input and for output. +If the input value is NULL a new buffer should be allocated, otherwise the +size of the buffer should be either increased or decreased depending on the +value of \fInbytes\fR. The buffer may be moved if necessary, provided the +contents of the buffer are preserved. This primitive may be implemented as +a call to \fBzmaloc\fR followed by an array copy and a call to \fBzmfree\fR +if desired, saving one kernel primitive, with significant loss of efficiency +in some applications. +.PP +A buffer allocated with \fBzmaloc\fR is deallocated with \fBzmfree\fR. +Deallocation need not involve physically returning memory pages to the +operating system; if the buffer is small this will not be possible. The buffer +being deallocated need not be at the end of the process address space. +.PP +The \fBzlocva\fR primitive returns the address in char units of the first +argument as the integer value of the second argument. Since Fortran is +call by reference, this is a simple matter of copying the pointer to the +first argument (as opposed to the value pointed to) to the integer location +pointed to by the second argument. +Only arguments of datatypes \fBcsilrd\fR are permitted in calls to \fBzlocva\fR. +Arguments of Fortran type COMPLEX, CHARACTER, and EXTERNAL are sometimes passed +using more than one physical argument (depending on the host compiler) +and hence cannot be used in procedures that operate upon an arbitrary datatype. +.PP +The \fBzawset\fR primitive is used both to determine and to change the amount +of physical memory in machine bytes available to a process, i.e., the +\fBworking set size\fR on a virtual memory machine. +If called with a \fIbestsize\fR of zero the current working set size and text +segment size is returned. If called with nonzero \fIbestsize\fR on a +virtual memory machine the working set size will be adjusted up or down +as indicated, returning the actual new working set size in \fInewsize\fR +and the old working set size in \fIoldsize\fR. +It is not an error if the amount of memory requested cannot be allocated; +the high level code will ask for what it wants but take what it gets. +High level routines which need lots of memory rely on this primitive to +avoid running out of memory on nonvirtual machines and to avoid thrashing +on virtual machines. +.PP +The high level code (\fBmalloc\fR) converts the address returned +by the kernel primitives into an integer valued offset (array index) into +the \fBMem\fR common. +The dynamically allocated buffer (which has nothing to do with the Mem common) +is referenced by indexing off the end of an \fBMem\fR array. +The portable MEMIO code ensures alignment between the physical buffer and +the "pointer" returned to the user (index into Mem). This technique should +work on any machine which permits referencing off the end of an array. +.PP +There is one place in the system code, however, which does something trickier +and which should be checked on a new system. +The \fBstropen\fR routine in FIO uses the +same pointer technique (for efficiency reasons) to reference a \fIstatically\fR +allocated \fBchar\fR array in the user program. If the compiler does not +guarantee that a statically allocated \fBchar\fR array will be aligned with +the array \fBMemc\fR in the Mem common this will not work, +and \fBstropen\fR will have to be modified. +.NH 2 +Procedure Call by Reference +.PP +Fortran allows an external procedure to be passed by reference to a +subprogram via the subprogram argument list. An external procedure passed +as an argument to a subprogram may be called by the subprogram but may not be +saved and called at some later time. IRAF (e.g. FIO and IMIO) requires the +capability to save a reference to a procedure in an integer variable for +execution at some later time. +.PP +The \fBzlocpr\fR primitive is used to determine the entry point address +(EPA) of an external procedure. IRAF assumes that the EPA of a procedure +may be stored in an integer variable and that two procedures with the same +EPA are identically the same procedure. No other operations are permitted +on EPA values, e.g., signed comparisons and arithmetic are not permitted. +.sp 0.08i +.TS +center; +cb +n. +Call by Reference Primitives +.sp +zlocpr \&(proc, entry_point_address) get address of a procedure +zcall1 \&(procedure, arg1) +zcall2 \&(procedure, arg1, arg2) +zcall3 \&(procedure, arg1, arg2, arg3) +zcall4 \&(procedure, arg1, arg2, arg3, arg4) +zcall5 \&(procedure, arg1, arg2, arg3, arg4, arg5) +.TE +.sp 0.08i +.PP +A \fBzcall\fIn\fR primitive is used to call an external subroutine +referenced by the integer variable \fIprocedure\fR, the entry point address +of the procedure returned in a prior call to \fBzlocpr\fR. +Only subroutines may be called by reference; there is no comparable facility +for functions. The datatypes of the arguments are unspecified but are +restricted to the SPP datatypes \fBcsilrd\fR. + +.NH 2 +Date and Time +.PP +Kernel primitives are required to read the system clock, to determine the +amount of cpu time consumed by a process (for performance measurements), +and to generate time delays. +.sp 0.08i +.TS +center; +cb s +n l. +Date and Time Primitives +.sp +zgtime \&(local_time, cpu_time) get clock time and cpu time +ztslee \&(delay) countdown timer +.TE +.sp 0.08i +.PP +The \fBzgtime\fR primitive returns two long integer arguments. The local +standard time in integer seconds since midnight on January 1, 1980 is +returned as the first argument (the "clock" time). The second argument +is the total cpu time consumed by the process since process execution, +in units of milliseconds. The countdown timer primitive \fBztslee\fR +suspends execution of the calling process for the specified number of +integer seconds. There is currently no provision for generating delays +of less than one second. + +.NH 2 +Sending a Command to the Host OS +.PP +The ability to send an explicitly machine dependent command to the host +system command interpreter is required by the CL and by some of the system +utilities. Any program which uses this command is bypassing the system +interface and is system dependent. Nonetheless it is very useful for the +\fIuser\fR to be able to send a command to the host without leaving the +IRAF environment, and certain of the system utilities are much easier to +code given the capability (e.g., \fBdiskspace\fR and \fBspy\fR). These +utilities help provide a consistent user interface on all systems, and in +many cases such a utility program can be built in a few minutes for a new +system. No science program or essential system utility bypasses the system +interface in this fashion. +.sp 0.08i +.TS +center; +cb s +n l. +Host OS Command Primitive +.sp +zoscmd \&(cmd, stdout, stderr, status) send a command to the host OS +.TE +.sp 0.08i +.PP +The command \fIcmd\fR may be any packed string acceptable to the host +system. The call does not return until the command has been executed. +The status OK or ERR is returned indicating success or failure. +If either of the filename strings \fIstdout\fR or \fIstderr\fR is nonnull +the associated output stream of the command will be directed (if possible) +to the named text file. + +.NH +Bit and Byte Primitives +.PP +The bit and byte primitives are not considered true kernel procedures since +they are purely numerical and are only potentially machine dependent. +These primitives are more properly part of the \fBprogram interface\fR than +the kernel, since they are callable from ordinary applications programs. +Both SPP or Fortran and C versions of most of the routines are supplied +with the system which will port to most modern minicomputers and some large +computers. The source directory is \fBsys$osb\fR. +The following classes of routines are required: +.sp 0.05i +.DS +\(bu bitwise boolean operations (and, or, etc.) +\(bu bitfield insertion and extraction +\(bu byte primitives (copy, swap, string pack/unpack) +\(bu type conversion for byte, unsigned short datatypes +\(bu machine independent integer format conversions +.DE +.sp 0.05i +.PP +The IRAF system uses 8 standard datatypes in compiled SPP programs, +as shown in the table below. Variables and arrays may be declared and +accessed conventionally using any of these datatypes. Data may additionally +be stored on disk, in images, and in memory in packed char or integer arrays +in the exotic datatypes \fBunsigned byte\fR, \fBunsigned short\fR, +and \fBpacked string\fR. +.sp 0.08i +.TS +center box; +cb s s +ci | ci | ci +lb | c | l. +Standard SPP Datatypes +_ +name suffix Fortran equivalent += +bool b LOGICAL +char c nonstandard +short s nonstandard +int i INTEGER +long l nonstandard +real r REAL +double d DOUBLE PRECISION +complex x COMPLEX +.TE +.sp 0.08i +.PP +The char and short datatypes are commonly implemented as INTEGER*2, +and long as INTEGER*4, but all could be implemented as the standard +INTEGER if necessary. To save space char may be implemented using a +signed byte datatype if the host Fortran compiler provides one, +provided the special datatype chosen may be equivalenced with the +standard datatypes (the minimum precision of a char is 8 bits signed). +IRAF assumes that the 7 types \fBcsilrdx\fR may be equivalenced in common +(e.g. in \fBMem\fR). The standard type suffixes \fBbcsilrdx\fR are +commonly appended to procedure names to identify the datatype or datatypes +upon which the procedure operates. +.NH 2 +Bitwise Boolean Primitives +.PP +The bitwise boolean primitives are used to set and clear bits or bitfields +in integer variables. The practice is portable provided the minimum +precision of an integer variable (16 bits) is not exceeded. Primitives +are provided for the 3 integer datatypes, i.e., short, int, and long, +denoted by the suffix \fB[sl]\fR in the table below. In other words, +the notation \fBand[sl]\fR refers to the procedures \fBands\fR and \fBandl\fR. +These quasi-primitives, unlike the true kernel primitives, are user callable +and are implemented as \fIfunctions\fR. +.sp 0.08i +.TS +center; +cb s +n n. +Bitwise Boolean Primitives +.sp +and,and[sl] \&(a, b) int = and \&(a, b) +or,or[sl] \&(a, b) int = or \&(a, b) +xor,xor[sl] \&(a, b) int = xor \&(a, b) +not,not[sl] \&(a, b) int = not \&(a, b) +.TE +.sp 0.08i +.PP +Bitwise boolean primitives are provided in many Fortran compilers +as integer intrinsic functions. If this is the case it suffices (and +is more efficient) to place a \fBdefine\fR statement in \fBiraf.h\fR to +map the IRAF name for the function to that recognized by the host Fortran +compiler. For example, +.DS +\fBdefine\fR and iand +.DE +would cause all occurrences of the identifier \fBand\fR in SPP programs to +be replaced by \fBiand\fR in the Fortran output, which the host compiler +would hopefully compile using inline code. +.NH 2 +Bitfield Primitives +.PP +A \fBbitfield\fR is an unsigned integer segment of a bit array, where the +number of bits in the segment must be less than or equal to NBITS_INT, +the number of bits in an integer. A \fBbit array\fR is a sequence of +bits stored one bit per bit in a char or integer array. The essential +thing about a bit array is that byte and word boundaries are irrelevant, +i.e., a bitfield may straddle a word boundary. +.sp 0.08i +.TS +center; +cb s +n n. +Bitfield Primitives +.sp +bitpak \&(intval, bit_array, bit_offset, nbits) integer \(-> bitfield +int = bitupk \&(bit_array, bit_offset, nbits) bitfield \(-> integer +.TE +.sp 0.08i +.PP +Bit offsets range from 1, not 0, to MAX_INT. A bitfield is zero-extended +when unpacked by \fBbitupk\fR, and unset bits are zeroed when an integer +is packed into a bitfield by \fBbitpak\fR. If the integer is too large +to fit in the bitfield it is truncated. These primitives should be +implemented in assembler on a machine like the VAX which has bitfield +instructions. +.NH 2 +Byte Primitives +.PP +The byte primitives are difficult to use portably in high level code +without building knowledge of the sizes of the SPP datatypes in bytes +into the code. Fortunately the byte primitives are rarely used; the most +common usage is in programs used to transport data between machines (e.g., +a magtape reader program). A number of machine constants are defined +in \fBiraf.h\fR to allow parameterization of programs which operate on +data in units of bytes. +.sp 0.08i +.TS +center box; +cb s +ci | ci +l | l. +Machine Parameters for Byte Data +_ +name definition += +SZB_CHAR machine bytes per char +NBITS_BYTE nbits in a machine byte +SZ_\fItype\fR size of datatype \fItype\fR (upper case) in chars +.TE +.sp 0.08i +.PP +On most machines the byte primitives can be written in Fortran by +representing a byte array as an array of CHARACTER*1. This suffices for +programs which merely move bytes around, but not for programs which +do numerical comparisons and arithmetic operations upon character data +using CHAR and ICHAR, because the collating sequence for CHARACTER data +in Fortran is not necessarily ASCII. +.PP +Nonetheless CHAR and ICHAR can be used on most machines to operate upon bytes, +i.e., upon non-CHARACTER data stored in CHARACTER*1 arrays. +Of course we are asking for trouble using CHARACTER for non-CHARACTER +operations, so routines which do so are potentially machine dependent. +Both Fortran and C versions of most of the byte primitives are supplied. +The \fBbytmov\fR primitive should be written in assembler on a machine such +as the VAX which can perform the operation in a single instruction +(it is even more important to perform this optimization for the \fBamov\fR +vector operators, which are more widely used). +.sp 0.08i +.TS +center; +cb s +n l. +Byte Primitives +.sp +bytmov \&(a, aoff, b, boff, nbytes) move an array of bytes +bswap2 \&(a, b, nbytes) swap every pair of bytes +bswap4 \&(a, b, nbytes) swap every 4 bytes +chrpak \&(a, aoff, b, boff, nchars) pack chars into bytes +chrupk \&(a, aoff, b, boff, nchars) unpack bytes into chars +strpak \&(a, b, maxchars) SPP string \(-> byte-packed string +strupk \&(a, b, maxchars) byte-packed string \(-> SPP string +.TE +.sp 0.08i +.PP +The \fBbytmov\fR primitive moves a portion of a byte array into a portion +of another byte array. The move is nondestructive, i.e., if the input +and output arrays overlap data must not be destroyed. The \fBzlocva\fR +primitive may be used to determine if the arrays will overlap. +Byte swapping is performed by the \fBbswap2\fR and \fBbswap4\fR primitives, +which swap every 2 bytes or every 4 bytes, respectively, regardless of the +number of bytes per short or long integer on the host machine. These routines +are used primarily to swap bytes in interchange data before is it unpacked into +host integers (or after packing into interchange format), hence the primitives +are defined independently of the host word size. A 2 byte swap interchanges +successive pairs of bytes; a 4 byte swap of two 4 byte integers rearranges +the bytes as 12345678 \(-> 43218765. +.PP +The \fBchrpak\fR and \fBchrupk\fR primitives pack and unpack SPP chars +into bytes, performing sign extension in the unpacking operation. +The mapping is nondestructive, i.e., the input and output arrays may +be the same, and the numeric value of a character is not changed +by the mapping (the collating sequence is not changed by the mapping). +If SZB_CHAR is 1, \fBchrpak\fR and \fBchrupk\fR are equivalent, and if +the input and output arrays are the same or do not overlap they are +equivalent to \fBbytmov\fR. +.PP +The \fBstrpak\fR and \fBstrupk\fR primitives pack and unpack SPP strings +into packed strings. A packed string is a sequence of zero or more characters, +packed one character per byte, delimited by end-of-string (EOS). +While SPP strings are always ASCII the collating sequence of a packed string +is whatever is used for character data by the host machine. +The mapping is nondestructive in the sense that the input and output arrays +may be the same. Since the collating sequence may be changed in the mapping +and the mapping need not be one-to-one, information may be lost if an +arbitrary string is packed and later unpacked. +.PP +A packed string is not the same as a Fortran CHARACTER variable or constant. +Many Fortran compilers use two physical arguments to pass a Fortran CHARACTER +argument to a subprocedure, while a packed string is always passed by reference +like an ordinary integer array. There is no machine independent way to fake +a Fortran string in an argument list. Furthermore, packed strings are heavily +used in the kernel for machine dependent filenames, and these file names +typically contain characters not permitted by the restrictive Fortran standard +character set. The packed string format is equivalent to that expected by +the C language. +.NH 2 +Vector Primitives +.PP +Nearly all of the operators in the vector operators package +(VOPS, \fBsys$vops\fR) are machine independent. The exceptions are the +\fBacht\fR primitives used to change the datatype of a vector to or from +one of the special datatypes \fBunsigned byte\fR and \fBunsigned short\fR. +An \fBacht\fR operator is provided for every possible type conversion +in the set of datatypes \fBcsilrdx\fR plus unsigned byte (\fBB\fR) and +unsigned short (\fBU\fR), for a total of 9 squared or 81 operators in all. +The \fBbool\fR datatype is not supported by VOPS. +.PP +Two type suffixes are used to specify the type conversion performed by an +operator; for example, \fBachtir\fR will convert an integer array into a +real array. In the table below the underscore stands for the set of +datatypes \fBUBcsilrdx\fR, hence each the operators shown is actually +a generic operator consisting of 9 type specific operators. +Both C and Fortran sources are provided for all primitives, the C sources +being more efficient. The Fortran operators will work on many hosts +but are potentially machine dependent and should be checked. The C versions +are more efficient since Fortran does not support the unsigned datatypes +and a masking operation must be performed to undo sign extension when +converting from unsigned to signed. +.sp 0.08i +.TS +center; +cb s +n l. +Machine Dependent Vector Primitives +.sp +acht_b \&(a, b, npix) SPP datatype \(-> unsigned byte +acht_u \&(a, b, npix) SPP datatype \(-> unsigned short +achtb_ \&(a, b, npix) unsigned byte \(-> SPP datatype +achtu_ \&(a, b, npix) unsigned short \(-> SPP datatype +.TE +.sp 0.08i +.PP +Many of the conversions do not preserve precision, i.e., double to real +or integer to unsigned byte. The imaginary part of a complex number is +discarded when converting to some other datatype, and the imaginary part +is set to zero when converting a non-complex datatype to complex. +All type conversion operators allow the conversion to be performed in +place, i.e., the input and output arrays may be the same. +.NH 2 +MII Format Conversions +.PP +The Machine Independent Integer format (MII) is used to transport binary +integer data between computers. The format is independent of the transmission +medium, and hence might be used to transport data via magnetic tape, over a +local area network, or over a modem. The MII integer format is equivalent +to that defined by the FITS image interchange format. The MII primitives are +used in the IRAF FITS reader and writer programs and will probably be used +in the GKS (Graphical Kernel System) software to implement a machine +independent VDM (Virtual Device Metafile). +.PP +MII defines 3 integer datatypes, 8 bit unsigned integer, 16 bit twos-complement +signed integer, and 32 bit twos-complement signed integer. An integer array +in MII format may be thought of as a stream of 8-bit bytes. In each 2 and +4 byte integer successive bytes are written in order of decreasing significance. +The sign bit is in the first byte of each 2 or 4 byte integer. For example, +two 16 bit integers would be represented in MII format as the following +sequence of 4 bytes. +.sp 0.08i +.TS +center; +ci ci +l l. +byte significance +.sp +1 high byte of first integer, including sign bit +2 low byte of first integer +3 high byte of second integer, including sign bit +4 low byte of second integer +.TE +.PP +The order of the bits within a byte is (must be) standardized at the +hardware level, else we would not be able to transmit character data via +cardimage tapes and modems. Hence the sign bit is bit 200B (octal) of the +first byte of an MII integer, the most significant bit is bit 100B of the +first byte, and so on. +.sp 0.08i +.TS +center; +cb s +n l. +MII Primitives +.sp +miipak \&(spp, mii, nelems, spp_type, mii_type) SPP \(-> MII +miiupk \&(mii, spp, nelems, mii_type, spp_type) MII \(-> SPP +intlen = miilen \&(n_mii_elements, mii_type) get size of array +.TE +.sp 0.08i +.PP +SPP or Fortran integer data is converted to MII format with \fBmiipak\fR, +and MII data is converted to SPP format with \fBmiiupk\fR. +The argument \fIspp\fR refers to an SPP integer array of datatype +\fIspp_type\fR, and \fImii\fR refers to an MII byte stream of MII datatype +\fImii_type\fR. The legal integer values of \fImii_type\fR are 8, 16, +and 32. MII data is stored in an SPP array of type \fBint\fR. +The length of the SPP integer array required to store \fIn_mii_elements\fR +of MII type \fImii_type\fR is returned by the primitive \fBmiilen\fR. +.PP +An SPP implementation of the MII primitives which should work for all +host machines with 16 or 32 bit twos-complement signed integer datatypes +is supplied with the system. Most modern minicomputers fall into this class. +All one need do to use the supplied MII primitives is determine whether or +not byte swapping is necessary. If the parameter BYTE_SWAP2 is defined +as YES in \fBiraf.h\fR then \fBbswap2\fR will be called to swap 2 byte +MII integers, producing an SPP \fBshort\fR as output. If the parameter +BYTE_SWAP4 is defined as YES then \fBbswap4\fR will be called to swap 4 +byte MII integers, producing an SPP \fBlong\fR as output. +.NH 2 +Machine Constants for Mathematical Libraries +.PP +The most often used machine constants are parameterized in include files +for use within SPP programs. Defined constants are the most readable +and efficient way to represent machine constants, but not the most accurate +for floating point quantities. Furthermore, SPP defined constants cannot +be used within Fortran procedures; functions returning the machine constants +are often used instead. The most widely used set of such functions appears +to be those developed at Bell Laboratories for the \fIPort\fR mathematical +subroutine library. +.sp 0.08i +.TS +center; +cb s +n l. +Mathematical Machine Constants +.sp +int = i1mach \&(parameter) get INTEGER machine constants +real = r1mach \&(parameter) get REAL machine constants +double = d1mach \&(parameter) get DOUBLE PRECISION machine constants +.TE +.sp 0.08i +.PP +These routines are used in many numerical packages, including the IEEE +signal processing routines and the NCAR graphics software. Documentation +is given in the Fortran sources; values of the constants have already been +prepared for most of the computers used at scientific centers. The integer +codes of the machine parameters are parameterized in \fBlib$mach.h\fR for +use in SPP programs. + +.NH +System Parameterization and Tuning +.PP +All machine dependent system and language parameters are defined in the two +SPP include files \fBlib$iraf.h\fR and \fBlib$config.h\fR, in the C include +file \fBcl$config.h\fR, and in the CL script file \fBlib$clpackage.cl\fR. +Additional machine dependent include files will often be used (\fIshould\fR +be used) to implement the kernel for a particular machine but these are too +host specific to be described here. +.PP +The include file \fBlib$iraf.h\fR is automatically loaded by the SPP whenever +an SPP source file is preprocessed, hence the defines in \fBiraf.h\fR are +effectively part of the SPP language. This global include file defines +the language parameters (e.g., EOF, ERR) as well as many machine constants. +The two configuration files \fBlib$config.h\fR and \fBcl$config.h\fR define +additional constants pertaining to the host OS as well as various size limiting +and heuristic parameters used to tune IRAF for optimum performance on a given +host system. The CL script file \fBlib$clpackage.cl\fR contains \fBset +environment\fR declarations for all system directories and default devices. +.PP +Documentation for the individual machine constants and system tuning parameters +is maintained directly in the source files to ensure that it is up to date. +Some of the parameters in \fBconfig.h\fR pertain only to the inner workings +of the program interface and changes can be expected in successive releases +of the IRAF system, without corresponding changes to the external +specifications of the program interface. + +.NH +Other Machine Dependencies +.PP +In the ideal world all of the machine dependence of the system would be +concentrated into the kernel and a few include files. While this has been +achieved for the scientific software some of the system utilities currently +bypass the system interface and are machine dependent. +This is not necessarily a bad thing; if a certain capability is only needed +by one or two system utilities the machine dependence of the system will often +be less if we take the simple way out and write a system dependent program +than if we further expand the formal system interface. +.PP +The machine dependent utilities are in the packages \fBsystem\fR and +\fBsoftools\fR. All machine dependent procedures are listed in the README +files in the package directories. The string MACHDEP is placed in the source +files to mark any machine dependent code segments. The machine dependent +utilities are not essential to the use of the system, but are useful for +maintaining the system. Examples of machine dependent software utilities +include \fBmake\fR, \fBxcompile\fR, and the generic preprocessor. These +utilities will all eventually be replaced by portable programs. +Machine dependent system utilities include \fBallocate\fR and \fBdeallocate\fR, +\fBedit\fR (the interface to the host editor), and \fBdiskspace\fR. +These utilities are actually just command level interfaces to the host +system command interpreter, and are easy to modify for a new host. +.NH 2 +Machine Dependencies in the CL +.PP +As noted earlier, from a structural design point of view the Command Language +is an IRAF applications program; as such it is highly portable. The CL is not +completely portable because it is written in C. Since the CL is written in +C it cannot reference the standard include files \fBiraf.h\fR and +\fBconfig.h\fR, and therefore has its own machine dependent \fBconfig.h\fR +include file instead. Since the CL requires file i/o, process control and +exception handling, formated i/o, the TTY interface, etc., it is interfaced +to the IRAF program interface (the CL has a special Main of its own but this +is portable). +.PP +The C code in the CL is purely numeric, like the Fortran in SPP based +applications programs. All communication with the host system is via the +IRAF program interface. The program interface is written in SPP, i.e., +Fortran, and Fortran procedures +cannot portably be called from C (see \(sc3.1.2). To render the bulk of the +code portable a special C callable subset of the program interface is +defined for the CL, and all CL calls to program interface routines are via +this interface. Efficiency is not a problem because the CL does little i/o; +the CL is the control center of the system, and tends to be compute bound when +it is not idle waiting for a command. +.PP +In summary, to interface the CL to a new system it is necessary to first edit +the \fBcl$config.h\fR, which parameterizes the characteristics of the host +system and which contains the system tuning parameters affecting the CL. +The CL interface to the subset program interface consists of a set of +rather simple interface procedures in the file \fBcl$machdep.h\fR. +If you are lucky these will not have to be changed to port the CL to your host +computer, but even if the routines have to be modified they are few in number +and quite simple in nature. +.NH +Specifications for the Kernel Procedures +.PP +The remainder of this document consists of a summary of the machine dependent +procedures, followed by the detailed technical specifications for each +procedure. Only the specifications for the kernel primitives are given; +the bitwise boolean primitives are part of the program interface and are +documented elsewhere. Most likely either the Fortran or C code supplied for +the bitwise primitives will be usable on a new host system without significant +modification. +.PP +While the kernel consists of quite a few procedures, this does not necessarily +mean that it is going to be harder to implement than if there were only a +quarter or a third as many procedures. Our design goal was to minimize the +complexity and size of the kernel, and we felt that it was more important to +define simple, single function primitives than to minimize the \fInumber\fR +of primitives. +.PP +A large part of the kernel consists of device driver +subroutines; on a system which provides device independent i/o these may map +to the same low level procedures and a count of the number of driver +subroutines will have little meaning. On a system which does not have device +independent i/o it will be easier to implement separate procedures for each +device than to try to make the host system look like it has device independent +i/o (FIO already does that anyhow). Furthermore, the provision for separate +drivers makes it easy to optimize i/o for a particular device, and makes it +possible to dynamically interface new devices to FIO without modifying the +basic system. +.PP +All kernel procedures are called by virtual operating system procedures +in the program interface. The kernel procedures are \fInot\fR callable +directly from applications code. Extensive error checking is performed by +the high level code before a kernel procedure is called, hence error +checking in kernel procedures is largely redundant and should be omitted +if it will significantly compromise performance. Do not get clever +with kernel procedures; \fIkeep it simple\fR. +.sp 0.08i +.TS +center box; +cb s s +ci | ci | ci +l | c | l. +Summary of Kernel Constants +_ +name value usage += +APPEND 4 write at EOF +BINARY_FILE 12 +BOF \(mi3 beginning of file +EOF \(mi2 end of file +EOS '\\\\0' end of string delimiter +ERR \(mi1 function was unsuccessful +FI_DIRECTORY 2 directory file (\fBzfinfo\fR) +FI_EXECUTABLE 3 executable file +FI_REGULAR 1 ordinary file +FI_SPECIAL 4 special file +FSTT_BLKSIZE 1 device block size +FSTT_FILSIZE 2 file size, bytes +FSTT_MAXBUFSIZE 4 maximum transfer size +FSTT_OPTBUFSIZE 3 optimum transfer size +LEN_JUMPBUF ?? integer length of \fBzsvjmp\fR buffer +NEW_FILE 5 create a new file +NO 0 no (false) +OK 0 function successfully completed +PR_CONNECTED 1 connected subprocess +PR_DETACHED 2 detached process +PR_HOST 3 process spawned by host +QUERY_PROTECTION 2 query file protection (\fBzfprot\fR) +READ_ONLY 1 file access modes +READ_WRITE 2 +REMOVE_PROTECTION 0 remove file protection (\fBzfprot\fR) +SET_PROTECTION 1 set file protection (\fBzfprot\fR) +SZ_LINE 161 default textfile line length +TEXT_FILE 11 file types +WRITE_ONLY 3 +X_ACV 501 access violation +X_ARITH 502 arithmetic error +X_INT 503 interrupt +X_IPC 504 write to IPC with no reader +YES 1 yes (true) +.TE +.sp 0.08i + + +.bp +.LG +.ce +\fBSummary of Machine Dependent Procedures\fR +.NL +.sp 2 +.TS +center; +cb s +n l. +Kernel Primitives +.sp +\fBzawset\fR \&(bestsize, newsize, oldsize, textsize) adjust working set size +\fBzcall\fIn\fR \&(procedure, arg1,...,arg\fIn\fR) call by reference +\fBzclcpr\fR \&(pid, exit_status) close connected subprocess +\fBzcldir\fR \&(chan, status) close directory +\fBzcldpr\fR \&(jobcode, killflag, exit_status) close or dequeue detached process +\fBzdojmp\fR \&(jumpbuf, status) restore process status +\fBzfacss\fR \&(osfn, mode, type, status) access file +\fBzfaloc\fR \&(osfn, nbytes, status) preallocate a binary file +\fBzfchdr\fR \&(new_directory, status) change directory +\fBzfdele\fR \&(osfn, status) delete a file +\fBzfgcwd\fR \&(osdir, maxch, status) get current working directory +\fBzfinfo\fR \&(osfn, out_struct, status) get info on a file +\fBzfmkcp\fR \&(old_osfn, new_osfn, status) make null copy of a file +\fBzfpath\fR \&(osfn, pathname, maxch, status) osfn to pathname +\fBzfprot\fR \&(osfn, prot_flag, status) file protection +\fBzfrnam\fR \&(old_osfn, new_osfn, status) rename a file +\fBzfsubd\fR \&(osdir, subdir, new_osdir, maxch, nchars) get subdirectory name +\fBzfxdir\fR \&(osfn, osdir, maxch, status) extract OS directory prefix +\fBzgfdir\fR \&(chan, osfn, maxch, status) get next OSFN from directory +\fBzgtime\fR \&(local_time, cpu_time) get clock time and cpu time +\fBzgtpid\fR \&(pid) get process id of current process +\fBzintpr\fR \&(pid, exception, status) interrupt connected subprocess +\fBzlocpr\fR \&(proc, entry_point_address) get EPA of a procedure +\fBzlocva\fR \&(variable, address) get address of a variable +\fBzmaloc\fR \&(buffer, nbytes, status) allocate a buffer +\fBzmfree\fR \&(buffer, status) deallocate a buffer +\fBzopcpr\fR \&(process, inchan, outchan, pid) open connected subprocess +\fBzopdir\fR \&(osfn, chan) open a directory +\fBzopdpr\fR \&(process, bkgfile, jobcode) open or queue detached process +\fBzoscmd\fR \&(cmd, stdout, stderr, status) send a command to the host JCL +\fBzpanic\fR \&(errcode, errmsg) panic exit +\fBzraloc\fR \&(buffer, nbytes, status) reallocate a buffer +\fBzsvjmp\fR \&(jumpbuf, status) save process status +\fBztslee\fR \&(delay_in_seconds) countdown timer +\fBzxgmes\fR \&(os_exception, errmsg, maxch) get exception code and message +\fBzxwhen\fR \&(exception, new_handler, old_handler) post an exception +.TE +.sp 2 +.TS +center box; +cb s s s s s s s s s +ci | ci | ci ci ci ci ci ci ci ci +l | c | c c c c c c c c. +Text File Device Drivers +_ +device code opn cls get put fls not sek stt += +normal (disk) tx * * * * * * * * +terminal ty * * * * * * +.TE +.sp 2 +.TS +center box; +cb s s s s s s s +ci | ci | ci ci ci ci ci ci +l | c | c c c c c c. +Binary File Device Drivers +_ +device code opn cls ard awr awt stt += +normal (disk) bf * * * * * * +line printer lp * * * * * +IPC pr * * * * +static file sf * * * * * * +magtape mt * * * * * * +.TE +.sp 2 +.TS +center; +cb s +n l. +Magtape Device Primitives +.sp +\fBzzopmt\fR \&(drive, density, mode, oldrec, oldfile, newfile, chan) open +\fBzzclmt\fR \&(chan, mode, nrecords, nfiles, status) close +\fBzzrdmt\fR \&(chan, buf, maxbytes) aread +\fBzzwrmt\fR \&(chan, buf, nbytes) awrite +\fBzzwtmt\fR \&(chan, nrecords, nfiles, status) await +\fBzzrwmt\fR \&(chan, status) arewind +.TE +.sp 2 +.TS +cb s +n l. +Bit and Byte Primitives +.sp +\fBand,and[sl]\fR \&(a, b) bitwise and +\fBor,or[sl]\fR \&(a, b) bitwise or +\fBxor,xor[sl]\fR \&(a, b) exclusive or +\fBnot,not[sl]\fR \&(a, b) complement +\fBbitpak\fR \&(intval, bit_array, bit_offset, nbits) integer \(-> bitfield +int = \fBbitupk\fR \&(bit_array, bit_offset, nbits) bitfield \(-> integer +\fBbytmov\fR \&(a, aoff, b, boff, nbytes) move an array of bytes +\fBbswap2\fR \&(a, b, nbytes) swap every pair of bytes +\fBbswap4\fR \&(a, b, nbytes) swap every 4 bytes +\fBchrpak\fR \&(a, aoff, b, boff, nchars) pack chars into bytes +\fBchrupk\fR \&(a, aoff, b, boff, nchars) unpack bytes into chars +\fBstrpak\fR \&(a, b, maxchars) SPP string \(-> byte-packed string +\fBstrupk\fR \&(a, b, maxchars) byte-packed string \(-> SPP string +\fBacht_b\fR \&(a, b, npix) SPP datatype \(-> unsigned byte +\fBacht_u\fR \&(a, b, npix) SPP datatype \(-> unsigned short +\fBachtb_\fR \&(a, b, npix) unsigned byte \(-> SPP datatype +\fBachtu_\fR \&(a, b, npix) unsigned short \(-> SPP datatype +\fBmiipak\fR \&(spp, mii, nelems, spp_type, mii_type) SPP \(-> MII +\fBmiiupk\fR \&(mii, spp, nelems, mii_type, spp_type) MII \(-> SPP +intlen = \fBmiilen\fR \&(n_mii_elements, mii_type) get length of MII array +int = \fBi1mach\fR \&(parameter) get int machine constants +real = \fBr1mach\fR \&(parameter) get real machine constants +double = \fBd1mach\fR \&(parameter) get double machine constants +.TE |