.help imtool Jan87 "Image Display on the Sun Workstation" .sp 3 .ce \fBImage Display on the Sun Workstation\fR .ce Design Specification (prototype display) .ce Doug Tody .ce January 1987 .NH Introduction This document describes the prototype Sunview based SUN/IRAF image display. The standard Sun-3 color workstation provides an 1152 by 900 by 8 bit frame buffer (some models are larger) with an associated 256 entry color table and an RGB display. A keyboard and a three button mouse are provided for interactive control and command entry. In typical interactive use the user will have a number of windows open on the screen, e.g., a GTERM virtual graphics terminal with text and graphics windows, the image display window, a small console window, and several icons. Due to limitations on the amount of screen space available and due to speed limitations when refreshing a full 8 bit deep window, it is desirable to limit the size of the image display window to a fraction of the screen space most of the time. Due to the complexity of the task of designing a fully functional, device independent image display interface, and the need to get at least a basic image display capability functioning for SUN/IRAF as soon as possible, a prototype display will be implemented first in January 1987 followed by the full interface later in the year. Although the prototype will not provide full functionality or system and device independence, the basic features are expected to be provided im much the same form that they will take in the final interface. .NH User Interfaces The prototype image display will use the existing TV/DISPLAY program to load images into the image display. None of the other tasks in the TV package will be supported and the entire contents of the TV package will disappear later this year. Display interaction and cursor readback will be via the same cursor mode interface currently used for the graphics terminal interface. Additional display-level interaction will be available under mouse control, e.g., to support display dependent functions not otherwise supported in cursor mode, such as greyscale windowing, filtering (resampling), region of interest control, and the like. .NH Display Features .NH 2 Image and Graphics Planes The display size is a setup configuration parameter but is fixed while the display is in use. Typical display sizes might be 512, 800, 1024, or 2048 pixels square, or some rectangular size. Sufficient physical memory should be available so that the internal frame buffers may be locked into memory. The image display will be 7 bits deep with 1 bit of graphics overlay, leaving half of the color table entries for use by other graphics windows. The image may be windowed without affecting the color table entries of the other windows on the screen. If 7 bits of greyscale proves to be a serious limitation something on the order of 7.9 bits is possible by playing tricks with the software (clever use of the color table). This latter feature will not be available in the initial prototype. The prototype display will provide N image frames, all of which must be the same size, each with its own independent lookup table and viewport. The frame size and number of frames are arbitrary and will be a setup option. Blinking, split screen, true color representation, and other operations involving simultaneous display of multiple frames will not be implemented n the prototype display. The full display will support N image frames, each with an associated 1 bit graphics overlay, with all frame-graphics pairs the same size, and with support for multiple screen operations such as blink and split screen. Independent lookup tables will not be possible in split screen mode, but this may not be a serious problem given the autoscaling algorithm in the DISPLAY task, and the possibility of normalization of multiple image frames to use a common lookup table (by modifying the loaded pixel values), once the lookup table has been independently adjusted for each frame. .NH 2 Pseudocolor and True Color Pseudocolor will probably not be supported in the initial prototype, although it may be added to the prototype before the final interface is written. True color can probably be provided in the final interface by maintaining an independent lookup table for each of the three frames (RGB) in software, generating the compound output in software, and displaying the result in hardware using a software generated linear pseudocolor lookup table. Once the true color image is generated all of the zoom, pan, etc. functions will work with their usual speed. Since true color display is used infrequently (and fully buffered memory to memory operations are fairly fast anyhow) this technique should prove quite adequate. .NH 2 The Display Window The user views the displayed image through a window which may be resized and moved about on the screen at any time, under interactive control of the mouse. Resizing the window changes only the size of the subraster of the fixed-size image frame to be displayed. Moving the window under Sunview does not change the window into the image, merely the position of the image display window on the screen. Zoom and pan is implemented in software. Zoom is implemented by pixel replication and dezoom by pixel subsampling, operating simultaneously on both the greyscale and graphics planes. Each operation results in a reload of the portion of the image displayed in the image window from data stored in the fixed size internal frame buffer. Zoom, dezoom, and roam are controlled interactively using either the normal cursor mode keystrokes (ZXYEP,1-9M,0), the mouse, or both. Once the region of interest has been adjusted on the display via the full pixel sampling techniques, a "resample" operation may be performed to recompute the contents of the displayed subraster using block averaging and/or bilinear interpolation techniques. A possible variation on this is to compute several block averaged versions of the full image at various dezoom factors, e.g., 2, 4, and 8 times dezoom, and read from these rather than the full frame when dezoom is selected. For example, if the frame size is 2048 square, the display would simultaneously store 2048, 1024, 512, and 256 square block-averaged versions of the frame and dynamically select the version to be used depending upon the dezoom factor. The percentage of additional memory required to hold the dezoomed frames is only about 40% greater than the fully sampled frame. Timing tests need to be performed to see if this is worthwhile. .NH 2 Region of Interest Due to screen space limitations the image window will rarely exceed about 512 square pixels in size, hence one will normally view either a dezoomed version of the image or a subraster of the full frame (full screen display will of course be an option). One possible way to live with dezoom without suffering loss of resolution is to use an icon, e.g., 64 pixels square, to always display the region under the cursor at full resolution or better. The display would consist of, for example, a 512 square main window displaying the dezoomed image, and an independently movable 64 square by 8 bit icon displaying an enlarged version of the image cursor, crosshairs, graphics overlay, and all. Alternatively, the cursor could be positioned on the full window and then a mouse button pressed to display the cursor at full resolution (or better), at the position of the cursor, with the display returning to normal when the mouse button is released. Yet another possibility is for the cursor to act as a magnifier which is moved about over the dezoomed image. Most probably all these choices will be provided as options, since they are all variations on the same thing. .NH 2 Windowing The lookup tables will be readable and writable under software control for sophisticated applications such as interactive entry of a piecewise linear transfer function. Most commonly, however, greyscale windowing will be performed by the image display under control of the mouse. The most attractive way to do this is probably to display a histogram of the image in the graphics plane, overlaid by the current transfer function, which typically looks like this: .ks .nf /------------ / / ---------/ .fi .ke Moving the cursor to the left will lower the threshold (move the displayed curve to the left), and moving the cursor up will increase the contrast (increase the slope of the transfer function). Ideally window mode would be entered by pressing a mouse button (or some combination of a mouse button and a control or shift key), windowing performed with the mouse button held down, and window mode exited when the mouse button is released. .NH 2 Reading the Cursor The image display cursor may be any 64 pixel square object with a shape which is suitable for centering on two dimensional objects. The default cursor will be a pair of 64 pixel long crosshairs with a gap in the middle. When a cursor read is in progress the cursor will blink rapidly, e.g., at 4-8 Hz, signaling to the user that the program is waiting for a cursor read (this is like turning on the full crosshairs on the GTERM graphics screen). The cursor may be moved either with the mouse or with the HJKL keys in cursor mode. A cursor read will be terminated by typing a key on the keyboard, or by pressing a mouse button which has been aliased to a key (as in GTERM). All cursor input will be via the standard IRAF cursor mode cursor input facility. The result of an image cursor read will be a line of text identical to that returned for a graphics cursor read, i.e., containing the cursor position in world coordinates (image pixel coordinates in this case), the WCS number, the key typed, and so on. Semi-automatic readout of the image cursor coordinates may be obtained via the "C" function in cursor mode, perhaps aliased to a mouse button. .NH 2 Hardcopy Output While there will be no snap/crtpict support initially, it will be possible to get pseudo-greyscale output hardcopy on the laserwriter via menu selection, as in GTERM, by taking advantage of the greyscale rendering capabilities of Postscript and the 300 dpi resolution of the laserwriter. It also should be possible to use :.snap or "=" to get hardcopy output in cursor mode. .NH Features Not Supported The prototype display will provide as IRAF callable functions only image frame buffer i/o, screen clear, viewport control (zoom/pan), and read and write cursor position. Greyscale windowing, hardcopy output, and window resize and move will be handled directly by the image display and Sunview, under mouse control. Initially there will be no access to the graphics overlay, and only monochrome display will be supported (pseudocolor may be added to the prototype at some point). Full functionality will follow when the GIO imaging extensions and IDI interface specification have been completed. .NH Implementation The following subsystems need to be added or modified to implement the prototype SUN/IRAF Sunview image display. .ls .ls [1] The software image display program (new code). This will be implemented as a Sunview tool (process), to be linked into the "basetools" executable (where GTERM lives). The image display process (or processes if there are multiple displays) will communicate with other processes via sockets and/or shared memory. A special file entry will be required in /dev for each display pseudo-device. The Sunview server process will normally be set up when the user's .suntools file is interpreted, but will not be displayed until the display is first loaded. Commands and data will be passed between the server process and the client (e.g.,some IRAF process) via the conventional socket based IPC mechanism and an application defined RPC mechanism. Shared memory will be used to eliminate pixel i/o to the image frame buffers when the server resides on the same machine as the client. .le .ls [2] The low level code in the DISPLAY program must be modified to talk with the display server. For the prototype, this is probably best done using the FIO based display interface rather than GIO, since the current display program is written to use FIO. The display task will directly read and write the display server frame buffer using the Sys V shared memory facilities. The final interface will do the same (or use a socket if the server is on a remote machine), but will use the more flexible GIO interface. The DISPLAY task must also be modified to compute the WCS and output it to the cursor mode code via GIO, in order for cursor readback to function. .le .ls [3] Cursor mode must be modified to support cursor input from imaging devices. The principle modification seems to be the addition of a set device viewport instruction, to be used to pass zoom and pan instructions on to the device kernel rather than implementing them directly in the cursor mode software as is done for the STDGRAPH devices. The open / close workstation, activate / deactivate workstation, clear screen, set cursor, and get cursor functions are also needed, but need not be different than the equivalent STDGRAPH functions. .le .le Cursor mode requires implementation of some portion of a GIO device interface for the image display. Probably this will be implemented using an inline kernel, at least for the prototype display. Both the GIO kernel and the DISPLAY task will communicate directly with the display server via IPC. .NH 2 Display Control Interface Applications programs such as the DISPLAY task or the CL cursor mode code will access the display via IPC carrying a GKI command and data stream. Most of the GKI instructions used are already provided by the current GIO/GKI interface. These are summarized below for reference. .ks .nf GKI_EOF = BOI 0 L * GKI_OPENWS = BOI 1 L M N D * GKI_CLOSEWS = BOI 2 L N D * GKI_REACTIVATEWS = BOI 3 L * GKI_DEACTIVATEWS = BOI 4 L * GKI_MFTITLE = BOI 5 L N T * GKI_CLEARWS = BOI 6 L GKI_CANCEL = BOI 7 L GKI_FLUSH = BOI 8 L GKI_POLYLINE = BOI 9 L N P GKI_POLYMARKER = BOI 10 L N P GKI_TEXT = BOI 11 L P N T GKI_FILLAREA = BOI 12 L N P * GKI_PUTCELLARRAY = BOI 13 L LL UR NC NL P * GKI_SETCURSOR = BOI 14 L CN POS GKI_PLSET = BOI 15 L LT LW CI GKI_PMSET = BOI 16 L MT MW CI GKI_TXSET = BOI 17 L UP SZ SP P HJ VJ F Q CI GKI_FASET = BOI 18 L FS CI * GKI_GETCURSOR = BOI 19 L CN GKI_CURSORVALUE = BOI 19 L CN POS KEY * GKI_GETCELLARRAY = BOI 20 L LL UR NC NL GKI_CELLARRAY = BOI 20 L NP P GKI_ESCAPE = BOI 25 L FN N DC GKI_SETWCS = BOI 26 L N WCS GKI_GETWCS = BOI 27 L N .fi .ke Those instructions which are required in the prototype interface are marked with an asterisk. In addition we need a number of new instructions to perform various common image display control functions. This is the topic of the GIO imaging extensions project, which we do not wish to attempt to seriously address in the prototype display. The new GKI instructions outlined in the remainder of this section are intended only to permit the implementation of the prototype SUN image display, and will be replaced by a full and much more carefully defined set of instructions when the full interface is designed. A new instruction is needed to set the device viewport, i.e., the portion of the frame buffer to be displayed in the display window. The set viewport instructions are used to implement zoom and pan at the cursor level. .ks .nf GKI_SETVIEWPORT = BOI 28 L XC YC ZX ZY GKI_GETVIEWPORT = BOI 29 L XC YC ZX ZY GKI_VIEWPORT = BOI 29 L XC YC ZX ZY .fi .ke The device viewport is specified by the [x,y] center of the displayed area in frame buffer pixel coordinates (1-indexed), and the zoom or dezoom factor in each axis. This representation is used, rather than specifying the corner points of the intended viewport, so that the viewport may be specified independently of the size of the actual device window used, and to avoid questions about what happens near the edge of the frame. Instructions are also needed to set and inquire the index of the frame currently being displayed, and the index of the reference frame for the i/o and control functions; these need not be the same frame. .ks .nf GKI_SETFRAME = BOI 30 L RF DF GKI_GETFRAME = BOI 31 L GKI_FRAME = BOI 31 L RF DF .fi .ke Here, RF is the index of the reference frame, i.e., the frame to be operated upon or referenced in all set, get, and pixel i/o operations, and DF is the index of the frame currently being displayed or to be displayed. The frames are numbered 1, 2, ..., N; 0 indicates that the reference frame or display frame is not to be changed. The concept of the display frame is important in cursor reads. In a cursor read or write operation, a CN (cursor number) index of 0 indicates that the cursor of the display frame is to be referenced; the index of the frame actually referenced will be returned in the CN field of the cursor value structure. If a nonzero CN is given the cursor of the indicated frame is referenced, whether or not the indicated reference frame is currently displayed. Lastly, we need an inquiry function to determine the size and depth of the image frame buffer, and the number of image planes. It is assumed that the frame buffer size, etc., will be set up in advance by the user, either using the mouse and the display setup panel, or via command line arguments in the .suntools file when the display server is initially spawned. .ks .nf GKI_GETFRAMESIZE = BOI 32 L GKI_FRAMESIZE = BOI 32 L NP NX NY NZ .fi .ke Here, NP is the number of image planes, NX and NY are the size in pixels of each image plane, and NZ is the number of displayable greylevels, e.g., 256. Note that NZ need not be a power of two as the entire colortable may not be available. The image display will be a standard GIO (inline or sub) kernel down to the level where some action occurs, after decoding the i/o instruction to be executed. At that point a device dependent C subroutine will be called which will pass the command on to the display server, which will always reside in a separate process (it has to, due to the peculiar notifier based nature of the Sunview environment). .NH 2 Display Internals The primary functions of the display server are [1] to maintain a frame buffer, and respond to requests to display portions of the frame buffer in the display window at some zoom or dezoom factor, and [2] to read and execute commands from a client process. Case [1] includes both refresh of the primary window, which occurs whenever a new image is loaded, the device viewport changes, or the window is resized or uncovered, and refresh of the region of interest (cursor) window, which occurs whenever any of the above events occurs or whenever the cursor is moved. In fact it appears that both operations are really the same thing, except that the cursor window always hides the main window, has a narrower border and no label, and is refreshed more often than the main window. The refresh operation sets up a mapping between the frame buffer and a window and then either does a series of pixrect operations to refresh the window (which may be partially or fully covered), or carries out a resampling operation on a region of the frame buffer, writing the zoomed or dezoomed data to the display window. The commands to be executed in case [2] are the GKI functions described in the previous section. In the prototype display the primary commands are open / close workstation, set viewport, set cursor position, and read cursor. The set viewport function merely sets up a transformation and calls the window refresh code described in the previous paragraph.