aboutsummaryrefslogtreecommitdiff
path: root/sys/ki/README
diff options
context:
space:
mode:
Diffstat (limited to 'sys/ki/README')
-rw-r--r--sys/ki/README648
1 files changed, 648 insertions, 0 deletions
diff --git a/sys/ki/README b/sys/ki/README
new file mode 100644
index 00000000..147ee538
--- /dev/null
+++ b/sys/ki/README
@@ -0,0 +1,648 @@
+ Kernel Interface Package
+ 12-May-85 dct
+ (21-Aug-85)
+
+
+1. Introduction
+
+ The kernel interface package (KI) isolates the IRAF virtual operating
+system (VOS) from the IRAF kernel, permitting access either to the local
+kernel or to an IRAF kernel resident on a remote host, via an IRAF kernel
+server (KS). The KI provides access to files, peripherals and devices,
+and subprocesses irrespective of whether the resource is located on the
+local machine or on a remote node. Since the KI is implemented on top of
+the IRAF kernel the different nodes may run different native operating systems.
+
+
+ +-----------+
+ | | VOS = virtual operating system
+ | V O S | KI = kernel interface
+ | | KS = kernel server
+ +-----------+
+ | :
+ |
+ +-----------+ : +-----------+
+ | | | |
+ | K I |=========:========| K S |==== (etc)
+ | | | |
+ +-----------+ : +-----------+
+ | |
+ | : |
+ +-----------+ +-----------+
+ | | : | |
+ | local | | remote |
+ | kernel | : | kernel |
+ | | | |
+ +-----------+ : +-----------+
+
+ :
+ (host1) (host2)
+
+
+ Architecture of the Kernel Interface
+
+
+2. Conceptual Design
+
+ The purpose of the kernel interface is to permit access to logical files,
+devices, and subprocesses regardless of where such objects physically reside
+in a network. This permits sharing of peripherals such as image displays and
+plotters, data sharing (centralized databases), load sharing (running of
+subprocesses on lightly loaded nodes), and makes such mundane operations as
+copying a file between two nodes possible without developing special purpose
+software and further complicating the user interface.
+
+By addressing these problems by implementation of an interface just above
+the IRAF kernel it becomes possible to remotely access nodes running different
+native operating systems, e.g., UNIX and VMS, with the IRAF kernel on each host
+system resolving the differences between file formats and filenames found
+on the different host systems. Full access to text files and raw peripherals
+is inherent in the scheme regardless of the architecture of the host machine.
+Full access to binary datafiles is possible provided the binary format is
+machine independent, a desirable goal in any case.
+
+The kernel interface is conceived as an optional and invisible interface
+between the IRAF virtual operating system and the IRAF kernel.
+The specifications of the kernel are not affected by the KI, except for the
+addition of a new device driver for talking to remote kernel servers. The VOS
+sees the same kernel interface whether or not the system is configured with a
+KI.
+
+A VOS call to a kernel procedure such as ZOPNBF, for example, is directed
+instead to the KI procedure KOPNBF which in turn calls either the local ZOPNBF
+or passes the call to a remote KS. The VOS still calls the procedure "zopnbf",
+but the external name of this procedure is redefined in <config.h> as "kopnbf",
+permitted substitution of the KI merely by recompilation. When the file is
+opened a channel descriptor is allocated within the KI telling whether the
+channel is local or remote, and if remote giving the KS address on the net,
+and giving the kernel channel number of the file on the node on which it
+resides. The KI channel number instead of the kernel channel number is
+returned to the VOS. Subsequent i/o requests on the channel are also processed
+by the KI, which uses the channel descriptor to determine whether the file
+is local or remote and then fulfils the request.
+
+
+3. Other Network Interfaces
+
+ The kernel interface is not intended to replace or compete with "real"
+network implmentations such as NFS or DECNET. If the local net provides such
+facilities then remote access via the host network is equivalent to a local
+file reference. Few such commercial networks, however, permit equivalent
+access when the different nodes run different operating systems.
+Often the type of access provided by the network is restricted in some way,
+e.g., NFS permits transparent access to remote files but not to remote devices.
+On VMS an IRAF kernel routine which accesses an i/o channel at the QIO level
+is bypassing DECNET hence can only be used to access a local device.
+The kernel server approach neatly sidesteps these problems without preventing
+use of the host networking facilities in cases were it is desirable to do so.
+
+
+4. Performance
+
+ The overhead of the KI for a local file access is one additional procedure
+call per i/o request, e.g., an additional 20-30 microseconds per i/o request.
+A remote file access requires encoding of the request as a KI instruction,
+transmission of the instruction to the remote host, decoding of the instruction
+by the remote host, a kernel access on the remote host, and in the case of a
+read request transmission of the data back to the requesting node where it is
+read out into the caller's buffer. Handshaking is required to return the status
+value after reads and writes to a remote node, hence large transfers will
+significantly reduce the overhead involved in remote file access. Large data
+transfers are automatically transferred in segments by the interface code,
+hence there is no builtin limit on the size of a transfer.
+
+A file open or process spawn requiring connection to a kernel server on a
+remote host is a relatively expensive procedure since it requires spawning of
+the kernel server process on the remote host. Once a process has connected to
+a kernel server on a remote host that server will remain connected for the
+life of the local process, with the single connection providing simultaneous
+access to any number of files, devices, or child processes on the remote host.
+Each local process requires its own kernel server process on the remote node.
+
+
+5. Modifications to the Existing I/O System
+
+ As noted above, implementation of the KI should require minimal
+modifications to existing software. The specific modifications or additions
+required appear to be the following:
+
+
+additions:
+
+ add package KI (kernel interface)
+ add source for the KS (kernel server) process
+ add KS device driver to the kernel (package OS)
+ add ZGHOST (get hostname) primitive to OS
+ add SPP and LIBC include files <knames.h> to map zroutine names
+
+
+modifications:
+
+ add reference to <knames.h> to VOS procedures which access kernel
+ finit.h: install kernel server device driver at process startup
+ modify filename mapping:
+ do not map filenames which begin with a node name
+ prepend CWD to relative filenames if CWD is on a remote node (?)
+
+
+6. Filenames
+
+ Files and processes anywhere in the network may be accessed by prepending
+the node name to the VFN or OSFN of the file. In essence we have extended
+the VFN filename syntax to support node names. The syntax is as follows:
+
+ node "!" filename
+
+The field delimiter "!" is used because it is not used in filenames in AOS,
+UNIX, or VMS. The delimiters ":" and "::", commonly used as node name
+delimiters in commercial networds, were intentionally not used so that network
+filenames may be specified without conflict on systems with host network
+support. Typical filenames using this notation are shown below.
+
+ 2!/dev/iis
+ 1!lib$motd
+ 3!dra0:[user.data]pic.db
+
+An important consideration in choosing the node delimiter character is that
+it not be necessary to quote filenames in common everyday usage. It turns out
+that in the CL "command mode" the character ! is only recognized as a
+metacharacter when it occurs at the beginning of a command (as the OS escape),
+hence quoting of filenames containing ! will not be necessary.
+
+The strategy for resolving such filenames is as follows. The full filename
+including the nodename is passed to the filename mapping code on the local
+machine. If the filename includes a nodename the filename is not mapped.
+The KI receives the filename and strips off the nodename, using it to
+select the node which will execute the kernel primitive. The remainder of
+the filename (minus the nodename) is passed to the KS on the addressed node,
+and the KS performs the filename mapping and passes the mapped filename on
+to the IRAF kernel. Since the filename mapping is carried out by the KS on
+the remote node the filename will be mapped in the context of the remote
+machine, i.e., the mapping will depend upon the native operating system used
+on the remote node. If the KS is itself configured with a kernel interface,
+multiple indirection is possible, permitting gateways onto other networks,
+e.g., "gateway!node!filename".
+
+There should be no penalty for using a self referential node name in a
+filename, i.e., the system should be smart enough to ignore node names
+when the node named is the local node. This permits use of the same absolute
+network filenames in configuration tables on all nodes, without having to
+maintain different sources on the different nodes.
+
+Fully functional filename mapping requires that the IRAF environment variables
+used by a process on the local machine be propagated to the kernel server on
+the remote machine. If this is not done then package directory references,
+etc., appearing in virtual filenames will not be resolved. The kernel server
+will inherit all environment variables except iraf$, since logical directory
+names of the form "node!ldir$file" must be expanded relative to the iraf
+root directory on the remote node. User defined logical directories should
+include an explicit node name to permit access from any node.
+
+
+6.1 External KI Procedures
+
+ The majority of the KI procedures are internal, since the KI interface
+is hidden behind the procedure redefinitions in <knet.h>. Other subsystems
+of the VOS do however occasionally need access to the KI, and the following
+procedures are provided for this purpose.
+
+
+ ki_extnode - Extract (or delimit) the node name field
+ ki_mapchan - Map KI channel into OS channel (or pid) and node name
+
+ len_prefix = ki_extnode (resource, nodename, maxch, nchars)
+ oschan = ki_mapchan (kichan, nodename, maxch)
+
+
+The EXTNODE function is used to extract the node name from a resource name,
+e.g., to produce a simpler name as required by the low level code, or to
+propagate the node name prefix to a second resource name. The MAPCHAN function
+is used to convert a KI channel descriptor into the corresponding OS channel
+number and node name, e.g., for output to the user (the KI channel code is
+meaningless to the user).
+
+
+7. KI Procedures
+
+ Only those procedures in the IRAF kernel which take a filename as an operand
+or which do i/o to a file (or to IPC) need be mapped by the kernel interface.
+This subset of the kernel includes all the file primitives, all the device
+drivers, and some exception handling procedures.
+
+ zfacss - determine file accessibility
+ zfaloc - allocate a file
+ zfchdr - change default directory (??)
+ zfdele - delete a file
+ zfgcwd - get default directory (??)
+ zfinfo - get directory info on a file
+ zfmkcp - make a null length copy of a file
+ zfprot - set, remove, or query file protection
+ zfrnam - rename a file
+ zopdir - open a directory
+ zgfdir - get next filename from a directory
+
+ zopdpr - open a detached process
+ zcldpr - close a detached process
+ zopcpr - open a connected subprocess
+ zclcpr - close a connected subprocess
+ zintpr - interrupt a subprocess
+
+ zfiobf - binary file driver
+ zfiolp - line printer driver
+ zfiomt - magtape driver
+ zfiopl - plotter driver
+ zfiopr - ipc driver
+ zfiosf - static file driver
+ zfiotx - text file driver
+ zfioty - terminal driver
+
+
+The basic function of a KI procedure is to determine which host is to execute
+the kernel primitive, call the local or remote kernel to execute the primitive,
+and return the results to the calling program.
+
+
+7.1 Pseudocode for a KI Procedure
+
+ Consider the common case of reading a file on a remote host. The file
+open procedure must connect to the KS on the remote host, command the remote
+kernel to open the file, and read back the status of the open. Whether the
+file is on the local or remote node, a channel descriptor must be set up
+to tell the KI i/o primitives how to access the file.
+
+
+# KOPNBF -- KI version of ZOPNBF (open binary file).
+
+procedure kopnbf (osfn, mode, status)
+
+begin
+ ks = get_kernel_server (osfn)
+ if (ks is the local node)
+ call zopnbf (osfn, mode, status)
+ else {
+ encode KI instruction for zopnbf
+ write instruction to KS
+ read reply from KS
+ if (error on channel to KS)
+ return (status=ERR)
+ }
+
+ set up channel descriptor (set channel codes of KS and file)
+end
+
+
+# GET_KERNEL_SERVER -- Return a channel to the kernel server controlling
+# a virtual filename.
+
+int procedure get_kernel_server (vfn)
+
+begin
+ extract node name from vfn
+ if (no node name given)
+ return (0)
+ else if (node is already connected)
+ return (node channel)
+ else {
+ connect to the KS on the remote node
+ save channel of node in channel table
+ return (node channel)
+ }
+end
+
+
+# KARDBF -- KI version of ZARDBF (read from a binary file). The read will not
+# be asynchronous if the file is resident on a remote node. The size of a
+# transfer is not limited by the maximum block of the channel.
+
+procedure kardbf (chan, buf, maxchars, offset)
+
+begin
+ if (ks[chan] == 0)
+ call zardbf (chan, buf, maxchars, offset)
+ else {
+ encode KI instruction for ZARDBF
+ write KI instruction to remote kernel server
+ read back the KI header of the response
+
+ if (read status ok and KI status for read ok) {
+ transfer data from channel directly into callers buffer,
+ in blocks no larger than the channel block size
+ save channel status for KAWTBF
+ }
+ }
+end
+
+
+7.2 Data Structures
+
+ The data structures required by the kernel interface are small and are
+statically allocated.
+
+
+7.2.1 Host Name Table
+
+ The host name table (HNT) lists all of the hosts in the network which the
+KI can access, giving for each node the node filename of the kernel server
+process on that node, and a list of aliases (node names) for the node.
+The host name table is the text file "dev$hosts". The format of the file
+is illustrated by the following example:
+
+
+ 1!/iraf/lib/irafks.e : 1 a vax1 aquila
+ 2!/iraf/lib/irafks.e : 2 b vax2 lyra
+ 3!usr1\:[irafx.lib]irafks.exe : 3 vax3 vela
+ 5!/iraf/lib/irafks.e : 5 c vax5 carina
+ 6!usr1\:[irafx.lib]irafks.exe : 6 vax6 draco
+ 11!/iraf/lib/irafks.e : 11 sun1 petunia
+
+
+The format of an entry in this table is "server ':' aliases", where "server"
+is a machine dependent, colon delimited string to be passed to the ZFIOKS
+driver, and where the aliases are a sequence of null delimited logical names
+by which the high level code or the user can refer to the nodes. The first
+field, e.g., "server", is limited to 80 characters, the aliases to 8 characters
+(longer names will be silently truncated, either limit may be increased if
+necessary). Up to 8 aliases may be specified. The alias "0" will be
+automatically added to the alias list of the local node by the KI; this alias
+is used by the high level code to force a file to be accessed on the local node.
+The local node should also include as an alias the name returned by ZGHOST.
+The maximum number of nodes is a <config.h> parameter. The entries may appear
+in any order.
+
+
+7.2.2 Node Descriptor Table
+
+ For each entry in the host name table there is a corresponding entry in
+the node descriptor table (NDT). The fields of the NDT are initialized
+when the first filename containing a node reference is processed. When a
+kernel server is opened on a node the N_KSCHAN field is set to the channel
+code returned by the kernel server device driver.
+
+
+ int n_nnodes # number of nodes in table
+
+ struct node_descriptor {
+ int n_kschan # KS i/o channel or NULL
+ int n_local # set to YES for the local node
+ int n_nalias # number of aliases for this node
+ char n_server[64] # netname!process
+ char n_alias[8,8] # aliases
+ } ndt[MAX_NODES]
+
+
+Runtime node references are satisfied by searching the NDT for the specified
+alias. If the referenced node is the local node a NULL channel number is
+returned. If the N_KSCHAN field of the referenced node is null a kernel
+server is spawned and the channel number of the server returned. If a kernel
+server is already connected mapping an alias into a channel number is very
+fast (compared to a file open). Once a server is spawned it remains connected
+until the local process shuts down.
+
+
+7.2.3 Channel Descriptor Table
+
+ Each open file or connected subprocess requires one channel descriptor
+for each i/o channel or process id. The K_KSCHAN field will contain NULL if
+the channel resides on the local node. Channel descriptors are allocated at
+file or device open time or process connect time and are freed at close or
+disconnect time. If the channel is connected to a kernel server, the K_OSCHAN
+and K_PID fields will contain the OS channel number or process id of the
+resource as returned by the kernel server on the remote node.
+
+
+ struct channel_descriptor {
+ int k_kschan # KS channel (NULL if local node)
+ int k_oschan # OS channel number or PID
+ int k_status # status for the ZAWAIT call
+ } cdt[MAX_CHANNELS]
+
+
+7.3 KII Instruction Format
+
+ The kernel interface instruction format (KII) is the binary encoding of
+the data blocks sent to and received from the remote kernel server to execute
+a call to a kernel procedure. This format is machine independent, i.e.,
+it is defined in a way that is independent of the byte ordering, char size,
+etc. used by a node. The KII format provides a machine independent interface
+for the execution of all kernel subroutines as well as for text file i/o.
+Binary data blocks passed via the binary file i/o procedures are not affected,
+hence if a machine independent binary format is desired it must be implemented
+at a level higher than the KI.
+
+The KII format selected is a fixed format to simplify encode/decode and
+packet transfer (if the transfer medium is stream oriented fixed size packets
+are simpler to extract from the stream). For media such as Ethernet
+performance depends more upon the number of packets transferred than upon
+the size of a packet, so there is little penalty for wasted space in a fixed
+size packet. The maximum amount of information necessary to encode a kernel
+procedure call is limited (the argument lists are never large) hence it is
+easy to set an upper bound on the packet size. The KII packet structure is
+shown below.
+
+ struct kii_packet {
+ int p_opcode # instruction opcode
+ int p_subcode # subcode (for device drivers)
+ int p_arg[13] # procedure arguments
+ int p_sbuflen # nchars in string buffer
+ char p_sbuf[255] # string buffer
+ }
+
+ sizeof (struct kii_packet) == ((16*4)+256) == 320 bytes
+
+
+The OPCODE and SUBCODE fields identify the kernel procedure to be executed.
+Each procedure argument is passed in the corresponding field of the ARG array;
+for string arguments this field contains the offset of the string value in
+the P_SBUF field. Procedure argument N is passed in field N of the ARG array.
+P_SBUFLEN is the number of chars in the string buffer, including the EOS at
+the end of each string.
+
+Before transmission of the packet over the net the packet is encoded in
+a machine independent form. The 16 integer fields are converted into MII
+32 bit signed integer format, and the 256 character string buffer is packed
+one byte per character (only the first SBUFLEN characters are packed). The
+inverse transformation is performed by the kernel server on the remote node
+before accessing the contents of the packet.
+
+Most kernel procedures return a status value and/or data. The KII packet
+structure is used both to remotely execute a kernel procedure and to return
+the status value and any scalar or string output arguments. In the case of
+text file i/o the text data is passed in the SBUF field. Procedures which
+return a data structure (e.g., ZFINFO) may pass the data structure as a
+sequence of ARG array elements, encode the structure in chararacter form
+in SBUF, or pass the structure in a separate packet. In most cases a single
+packet will be used for greater efficiency and to avoid the need to
+explicitly encode the return value.
+
+The purpose of the SUBCODE field is to exploit the fact that the text and
+binary file drivers each have the same set of driver procedures, each with
+the same set of arguments. The OPCODE field identifies the device driver
+and the SUBCODE field the driver function, e.g., OPN, ARD, AWR, AWT, etc.
+
+
+8. KS Driver
+
+ Connecting and disconnecting kernel servers, and all i/o to connected
+kernel servers, is provided by the KS driver. To the KI in the calling
+process a KS behaves like a synchronous binary file opened for readwrite
+access. The driver is patterned after a binary file driver, allowing the
+server process to be connected to FIO as a streaming binary file. Normally,
+however, the driver procedures will be called directly by the KI.
+
+
+ zopnks (server, mode, chan)
+ zawrks (chan, buf, nbytes, offset)
+ zardks (chan, buf, maxbytes, offset)
+ zawtks (chan, status)
+ zsttks (chan, what, lvalue)
+ zclsks (chan, status)
+
+
+The entry points of the KS driver are shown above. The argument SERVER is
+the network pathname of the kernel server process, e.g., "2!/iraf/lib/ks.e".
+There no requirement, however, that the named process be a kernel server
+process. The driver will attempt to execute and set up readwrite streaming
+i/o to any process, hence the KS driver may be useful for network functions
+other than the kernel interface.
+
+
+9. Kernel Server Process
+
+ The kernel server process is an IRAF process with a standard IRAF Main
+but a special ONENTRY procedure and no task dictionary (like the CL).
+The server communicates with a KI via the binary streams CLIN and CLOUT,
+both of which are connected to a single channel in the calling process.
+The server process contains an interpreter capable of calling any kernel
+procedure which does i/o. In addition, the kernel server contains enough
+of FIO to map filenames and read and write CLIN and CLOUT.
+
+
+10. Filename Mapping Details
+
+ The system interface procedures used for filename mapping must be trapped
+by the KI to deal with node pathnames. The relevant procedures are the
+following:
+
+ zfnbrk break vfn into its component parts
+ zfchdr change directory
+ zfgcwd get default directory
+ zfxdir extract directory prefix
+ zfpath convert vfn to pathname
+ zfsubd fold subdirectory into pathname
+
+The first routine, ZFNBRK, merely parses filenames into their component fields
+and it should be possible for the same routine to be used on all nodes without
+help from the KI. The remaining routines are fundamental to the action of
+the KI.
+
+
+10.1 Default Directory
+
+ Normally, either the kernel or the host system keeps track of the default
+directory. This remains the case when the KI is in use, provided the default
+directory is on the local node. If the default directory is changed to some
+different node the following actions are taken by the KI:
+
+ [1] Extract node prefix and save in a KI common for use during filename
+ mapping.
+
+ [2] Execute ZFCHDR on the remote node (minus the node prefix) to set
+ the node-relative default directory.
+
+All subsequent runtime file references are mapped as follows:
+
+ [1] The default node prefix is prepended to all filenames which do
+ not include an explicit node name prefix.
+
+ [2] Normal filename mapping is performed, i.e., filename mapping is
+ disabled on the local node (since the filename has a node prefix),
+ deferring mapping to the kernel server on the remote node.
+
+All normal runtime filename mapping starts with a call to ZFXDIR. If ZFXDIR
+returns anything the filename is assumed to be host dependent and is not mapped.
+Hence we want ZFXDIR to return the entire VFN as if it were an OSDIR name,
+if the VFN includes a node prefix and the node referenced is not the local
+node. ZFPATH and ZFSUBD must behave similarly.
+
+
+10.2 Semicode
+
+
+procedure kfchdr (osdir, status)
+
+begin
+ server = ki_connect (osdir)
+
+ if (server == NULL) {
+ # Directory is on the local node.
+
+ default node = local node
+ call zfchdr (osdir_minus_node_prefix, status)
+
+ } else {
+ # Directory is on a remote node.
+
+ pass zfchdr request to remote node
+ if (request is successful)
+ default node = remote node
+ }
+end
+
+
+procedure kfgcwd (osdir, nchars)
+
+begin
+ if (default node is the local node)
+ call zfgcwd to get default directory
+ else
+ pass zfgcwd request to remote node
+
+ return (default_node // default_directory)
+end
+
+
+procedure zfxdir (vfn, osdir, maxch, nchars)
+
+begin
+ extract node name
+
+ if (no node name specified)
+ node name = default node
+
+ if (node is the local node)
+ pass the request to the local kernel
+ else
+ return (node // vfn)
+end
+
+
+procedure zfpath (vfn, pathname, maxch, nchars)
+
+begin
+ extract node name
+
+ if (no node name specified)
+ node name = default node
+
+ if (node is the local node) {
+ call zfpath to compute local pathname
+ return (node // pathname)
+ } else
+ return (node // vfn)
+end
+
+
+procedure zfsubd (osfn, maxch, subdir, nchars)
+
+begin
+ extract node name
+
+ if (no node name specified)
+ node name = default node
+
+ if (node is the local node) {
+ call zfsubd to compute local pathname
+ return (node // pathname)
+ } else
+ return (node // vfn // "subdir/")
+end
generated by cgit (git 2.34.1) at 2025-09-20 08:24:06 -0400