path: root/unix/sun/Imtool.hlp

.help imtool Jan87 "Image Display on the Sun Workstation"
.sp 3
.ce
\fBImage Display on the Sun Workstation\fR
.ce
Design Specification (prototype display)
.ce
Doug Tody
.ce
January 1987

.NH
Introduction

    This document describes the prototype Sunview based SUN/IRAF image display.
The standard Sun-3 color workstation provides an 1152 by 900 by 8 bit frame
buffer (some models are larger) with an associated 256 entry color table and
an RGB display.  A keyboard and a three button mouse are provided for
interactive control and command entry.

In typical interactive use the user will have a number of windows open on the
screen, e.g., a GTERM virtual graphics terminal with text and graphics windows,
the image display window, a small console window, and several icons.
Due to limitations on the amount of screen space available and due to speed
limitations when refreshing a full 8 bit deep window, it is desirable to limit
the size of the image display window to a fraction of the screen space most
of the time.

Due to the complexity of the task of designing a fully functional, device
independent image display interface, and the need to get at least a basic
image display capability functioning for SUN/IRAF as soon as possible,
a prototype display will be implemented first in January 1987 followed by
the full interface later in the year.  Although the prototype will not provide
full functionality or system and device independence, the basic features are
expected to be provided im much the same form that they will take in the final
interface.

.NH
User Interfaces

    The prototype image display will use the existing TV/DISPLAY program to
load images into the image display.  None of the other tasks in the TV package
will be supported and the entire contents of the TV package will disappear
later this year.  Display interaction and cursor readback will be via the
same cursor mode interface currently used for the graphics terminal interface.
Additional display-level interaction will be available under mouse control,
e.g., to support display dependent functions not otherwise supported in cursor
mode, such as greyscale windowing, filtering (resampling), region of interest
control, and the like.

.NH
Display Features
.NH 2
Image and Graphics Planes

    The display size is a setup configuration parameter but is fixed while the
display is in use.  Typical display sizes might be 512, 800, 1024, or 2048
pixels square, or some rectangular size.  Sufficient physical memory should be
available so that the internal frame buffers may be locked into memory.

The image display will be 7 bits deep with 1 bit of graphics
overlay, leaving half of the color table entries for use by other
graphics windows.  The image may be windowed without affecting the
color table entries of the other windows on the screen.  If 7 bits
of greyscale proves to be a serious limitation something on the order
of 7.9 bits is possible by playing tricks with the software (clever
use of the color table).  This latter feature will not be available
in the initial prototype.

The prototype display will provide N image frames, all of which must be the
same size, each with its own independent lookup table and viewport.
The frame size and number of frames are arbitrary and will be a setup option.
Blinking, split screen, true color representation, and other operations
involving simultaneous display of multiple frames will not be implemented 
n the prototype display.

The full display will support N image frames, each with an associated 1 bit
graphics overlay, with all frame-graphics pairs the same size, and with
support for multiple screen operations such as blink and split screen.
Independent lookup tables will not be possible in split screen mode,
but this may not be a serious problem given the autoscaling algorithm in
the DISPLAY task, and the possibility of normalization of multiple image
frames to use a common lookup table (by modifying the loaded pixel values),
once the lookup table has been independently adjusted for each frame.

.NH 2
Pseudocolor and True Color

    Pseudocolor will probably not be supported in the initial prototype,
although it may be added to the prototype before the final interface is written.
True color can probably be provided in the final interface by maintaining an
independent lookup table for each of the three frames (RGB) in software,
generating the compound output in software, and displaying the result in
hardware using a software generated linear pseudocolor lookup table.  Once the
true color image is generated all of the zoom, pan, etc. functions will work
with their usual speed.  Since true color display is used infrequently (and
fully buffered memory to memory operations are fairly fast anyhow) this
technique should prove quite adequate.

.NH 2
The Display Window

    The user views the displayed image through a window which may be resized and
moved about on the screen at any time, under interactive control of the mouse.
Resizing the window changes only the size of the subraster of the fixed-size
image frame to be displayed.  Moving the window under Sunview does not change
the window into the image, merely the position of the image display window on
the screen.

Zoom and pan is implemented in software.  Zoom is implemented by pixel
replication and dezoom by pixel subsampling, operating simultaneously on both
the greyscale and graphics planes.  Each operation results in a reload of the
portion of the image displayed in the image window from data stored in the
fixed size internal frame buffer.  Zoom, dezoom, and roam are controlled
interactively using either the normal cursor mode keystrokes (ZXYEP,1-9M,0),
the mouse, or both.

Once the region of interest has been adjusted on the display via the full pixel 
sampling techniques, a "resample" operation may be performed to recompute the
contents of the displayed subraster using block averaging and/or bilinear
interpolation techniques.  A possible variation on this is to compute several
block averaged versions of the full image at various dezoom factors, e.g.,
2, 4, and 8 times dezoom, and read from these rather than the full frame when
dezoom is selected.  For example, if the frame size is 2048 square, the display
would simultaneously store 2048, 1024, 512, and 256 square block-averaged
versions of the frame and dynamically select the version to be used depending
upon the dezoom factor.  The percentage of additional memory required to hold
the dezoomed frames is only about 40% greater than the fully sampled frame.
Timing tests need to be performed to see if this is worthwhile.

.NH 2
Region of Interest

    Due to screen space limitations the image window will rarely exceed about
512 square pixels in size, hence one will normally view either a dezoomed
version of the image or a subraster of the full frame (full screen display
will of course be an option).  One possible way to live with dezoom without
suffering loss of resolution is to use an icon, e.g., 64 pixels square,
to always display the region under the cursor at full resolution or better.
The display would consist of, for example, a 512 square main window displaying
the dezoomed image, and an independently movable 64 square by 8 bit icon
displaying an enlarged version of the image cursor, crosshairs, graphics
overlay, and all.

Alternatively, the cursor could be positioned on the full window and then a
mouse button pressed to display the cursor at full resolution (or better),
at the position of the cursor, with the display returning to normal when the
mouse button is released.  Yet another possibility is for the cursor to act
as a magnifier which is moved about over the dezoomed image.  Most probably
all these choices will be provided as options, since they are all variations
on the same thing.

.NH 2
Windowing

    The lookup tables will be readable and writable under software control for
sophisticated applications such as interactive entry of a piecewise linear
transfer function.  Most commonly, however, greyscale windowing will be
performed by the image display under control of the mouse.  The most attractive
way to do this is probably to display a histogram of the image in the graphics
plane, overlaid by the current transfer function, which typically looks like
this:

.ks
.nf
                                  /------------
                                 /
                                /
                      ---------/
.fi
.ke

Moving the cursor to the left will lower the threshold (move the displayed
curve to the left), and moving the cursor up will increase the contrast
(increase the slope of the transfer function).  Ideally window mode would
be entered by pressing a mouse button (or some combination of a mouse button
and a control or shift key), windowing performed with the mouse button held
down, and window mode exited when the mouse button is released.

.NH 2
Reading the Cursor

    The image display cursor may be any 64 pixel square object with a shape
which is suitable for centering on two dimensional objects.  The default cursor
will be a pair of 64 pixel long crosshairs with a gap in the middle.  When a
cursor read is in progress the cursor will blink rapidly, e.g., at 4-8 Hz,
signaling to the user that the program is waiting for a cursor read (this is
like turning on the full crosshairs on the GTERM graphics screen).  The cursor
may be moved either with the mouse or with the HJKL keys in cursor mode.
A cursor read will be terminated by typing a key on the keyboard, or by
pressing a mouse button which has been aliased to a key (as in GTERM).

All cursor input will be via the standard IRAF cursor mode cursor input
facility.  The result of an image cursor read will be a line of text identical
to that returned for a graphics cursor read, i.e., containing the cursor
position in world coordinates (image pixel coordinates in this case), the
WCS number, the key typed, and so on.  Semi-automatic readout of the image
cursor coordinates may be obtained via the "C" function in cursor mode,
perhaps aliased to a mouse button.

.NH 2
Hardcopy Output

    While there will be no snap/crtpict support initially, it will be possible
to get pseudo-greyscale output hardcopy on the laserwriter via menu selection,
as in GTERM, by taking advantage of the greyscale rendering capabilities of
Postscript and the 300 dpi resolution of the laserwriter.  It also should be
possible to use :.snap or "=" to get hardcopy output in cursor mode.

.NH
Features Not Supported

    The prototype display will provide as IRAF callable functions only image
frame buffer i/o, screen clear, viewport control (zoom/pan), and read and write
cursor position.  Greyscale windowing, hardcopy output, and window resize and
move will be handled directly by the image display and Sunview, under mouse
control.  Initially there will be no access to the graphics overlay, and only
monochrome display will be supported (pseudocolor may be added to the prototype
at some point).  Full functionality will follow when the GIO imaging extensions
and IDI interface specification have been completed.

.NH
Implementation

    The following subsystems need to be added or modified to implement the
prototype SUN/IRAF Sunview image display.
.ls
.ls [1]
The software image display program (new code).  This will be implemented as
a Sunview tool (process), to be linked into the "basetools" executable (where
GTERM lives).  The image display process (or processes if there are multiple
displays) will communicate with other processes via sockets and/or shared
memory.  A special file entry will be required in /dev for each display
pseudo-device.  The Sunview server process will normally be set up when the
user's .suntools file is interpreted, but will not be displayed until the
display is first loaded.  Commands and data will be passed between the server
process and the client (e.g.,some IRAF process) via the conventional socket
based IPC mechanism and an application defined RPC mechanism.  Shared memory
will be used to eliminate pixel i/o to the image frame buffers when the server
resides on the same machine as the client.
.le
.ls [2]
The low level code in the DISPLAY program must be modified to talk with the
display server.  For the prototype, this is probably best done using the FIO
based display interface rather than GIO, since the current display program
is written to use FIO.  The display task will directly read and write the
display server frame buffer using the Sys V shared memory facilities.  The
final interface will do the same (or use a socket if the server is on a remote
machine), but will use the more flexible GIO interface.  The DISPLAY task
must also be modified to compute the WCS and output it to the cursor mode
code via GIO, in order for cursor readback to function.
.le
.ls [3]
Cursor mode must be modified to support cursor input from imaging devices.
The principle modification seems to be the addition of a set device viewport
instruction, to be used to pass zoom and pan instructions on to the device
kernel rather than implementing them directly in the cursor mode software as
is done for the STDGRAPH devices.  The open / close workstation, activate /
deactivate workstation, clear screen, set cursor, and get cursor functions
are also needed, but need not be different than the equivalent STDGRAPH
functions.
.le
.le

Cursor mode requires implementation of some portion of a GIO device interface
for the image display.  Probably this will be implemented using an inline
kernel, at least for the prototype display.  Both the GIO kernel and the
DISPLAY task will communicate directly with the display server via IPC.

.NH 2
Display Control Interface

    Applications programs such as the DISPLAY task or the CL cursor mode code
will access the display via IPC carrying a GKI command and data stream.  Most
of the GKI instructions used are already provided by the current GIO/GKI
interface.  These are summarized below for reference.


.ks
.nf
			GKI_EOF = BOI  0 L
*		     GKI_OPENWS = BOI  1 L M N D
*		    GKI_CLOSEWS = BOI  2 L N D
*	       GKI_REACTIVATEWS = BOI  3 L
*	       GKI_DEACTIVATEWS = BOI  4 L
*		    GKI_MFTITLE = BOI  5 L N T
*		    GKI_CLEARWS = BOI  6 L
		     GKI_CANCEL = BOI  7 L
		      GKI_FLUSH = BOI  8 L
		   GKI_POLYLINE = BOI  9 L N P
		 GKI_POLYMARKER = BOI 10 L N P
		       GKI_TEXT = BOI 11 L P N T
		   GKI_FILLAREA = BOI 12 L N P
*	       GKI_PUTCELLARRAY = BOI 13 L LL UR NC NL P
*		  GKI_SETCURSOR = BOI 14 L CN POS
		      GKI_PLSET = BOI 15 L LT LW CI
		      GKI_PMSET = BOI 16 L MT MW CI
		      GKI_TXSET = BOI 17 L UP SZ SP P HJ VJ F Q CI
		      GKI_FASET = BOI 18 L FS CI
*		  GKI_GETCURSOR = BOI 19 L CN
		GKI_CURSORVALUE = BOI 19 L CN POS KEY
*	       GKI_GETCELLARRAY = BOI 20 L LL UR NC NL
		  GKI_CELLARRAY = BOI 20 L NP P
		     GKI_ESCAPE = BOI 25 L FN N DC
		     GKI_SETWCS = BOI 26 L N WCS
		     GKI_GETWCS = BOI 27 L N
.fi
.ke


Those instructions which are required in the prototype interface are marked
with an asterisk.  In addition we need a number of new instructions to
perform various common image display control functions.  This is the topic
of the GIO imaging extensions project, which we do not wish to attempt to
seriously address in the prototype display.  The new GKI instructions outlined
in the remainder of this section are intended only to permit the implementation
of the prototype SUN image display, and will be replaced by a full and much
more carefully defined set of instructions when the full interface is designed.

A new instruction is needed to set the device viewport, i.e., the portion of
the frame buffer to be displayed in the display window.  The set viewport
instructions are used to implement zoom and pan at the cursor level.


.ks
.nf
		GKI_SETVIEWPORT = BOI 28 L XC YC ZX ZY
		GKI_GETVIEWPORT = BOI 29 L XC YC ZX ZY
		   GKI_VIEWPORT = BOI 29 L XC YC ZX ZY
.fi
.ke


The device viewport is specified by the [x,y] center of the displayed area in 
frame buffer pixel coordinates (1-indexed), and the zoom or dezoom factor in
each axis.  This representation is used, rather than specifying the corner
points of the intended viewport, so that the viewport may be specified
independently of the size of the actual device window used, and to avoid
questions about what happens near the edge of the frame.

Instructions are also needed to set and inquire the index of the frame
currently being displayed, and the index of the reference frame for the i/o
and control functions; these need not be the same frame.


.ks
.nf
		   GKI_SETFRAME = BOI 30 L RF DF
		   GKI_GETFRAME = BOI 31 L
		      GKI_FRAME = BOI 31 L RF DF
.fi
.ke


Here, RF is the index of the reference frame, i.e., the frame to be operated
upon or referenced in all set, get, and pixel i/o operations, and DF is the
index of the frame currently being displayed or to be displayed.  The frames
are numbered 1, 2, ..., N; 0 indicates that the reference frame or display
frame is not to be changed.

The concept of the display frame is important in cursor reads.  In a cursor
read or write operation, a CN (cursor number) index of 0 indicates that the
cursor of the display frame is to be referenced; the index of the frame
actually referenced will be returned in the CN field of the cursor value
structure.  If a nonzero CN is given the cursor of the indicated frame is
referenced, whether or not the indicated reference frame is currently
displayed.

Lastly, we need an inquiry function to determine the size and depth of the
image frame buffer, and the number of image planes.  It is assumed that the
frame buffer size, etc., will be set up in advance by the user, either using
the mouse and the display setup panel, or via command line arguments in the
.suntools file when the display server is initially spawned.

		
.ks
.nf
	       GKI_GETFRAMESIZE = BOI 32 L
	          GKI_FRAMESIZE = BOI 32 L NP NX NY NZ
.fi
.ke


Here, NP is the number of image planes, NX and NY are the size in pixels of
each image plane, and NZ is the number of displayable greylevels, e.g., 256.
Note that NZ need not be a power of two as the entire colortable may not be
available.

The image display will be a standard GIO (inline or sub) kernel down to the
level where some action occurs, after decoding the i/o instruction to be
executed.  At that point a device dependent C subroutine will be called which
will pass the command on to the display server, which will always reside in a
separate process (it has to, due to the peculiar notifier based nature of the
Sunview environment).

.NH 2
Display Internals

    The primary functions of the display server are [1] to maintain a frame
buffer, and respond to requests to display portions of the frame buffer in the
display window at some zoom or dezoom factor, and [2] to read and execute
commands from a client process.

Case [1] includes both refresh of the primary window, which occurs whenever a
new image is loaded, the device viewport changes, or the window is resized or
uncovered, and refresh of the region of interest (cursor) window, which occurs
whenever any of the above events occurs or whenever the cursor is moved.
In fact it appears that both operations are really the same thing, except that
the cursor window always hides the main window, has a narrower border and no
label, and is refreshed more often than the main window.  The refresh operation
sets up a mapping between the frame buffer and a window and then either does
a series of pixrect operations to refresh the window (which may be partially
or fully covered), or carries out a resampling operation on a region of the
frame buffer, writing the zoomed or dezoomed data to the display window.

The commands to be executed in case [2] are the GKI functions described in the
previous section.  In the prototype display the primary commands are open /
close workstation, set viewport, set cursor position, and read cursor.
The set viewport function merely sets up a transformation and calls the
window refresh code described in the previous paragraph.