diff options
| author | Joseph Hunkeler <jhunkeler@gmail.com> | 2015-07-08 20:46:52 -0400 |
|---|---|---|
| committer | Joseph Hunkeler <jhunkeler@gmail.com> | 2015-07-08 20:46:52 -0400 |
| commit | fa080de7afc95aa1c19a6e6fc0e0708ced2eadc4 (patch) | |
| tree | bdda434976bc09c864f2e4fa6f16ba1952b1e555 /pkg/bench/bench.hlp | |
| download | iraf-linux-fa080de7afc95aa1c19a6e6fc0e0708ced2eadc4.tar.gz | |
Initial commit
Diffstat (limited to 'pkg/bench/bench.hlp')
| -rw-r--r-- | pkg/bench/bench.hlp | 1723 |
1 files changed, 1723 insertions, 0 deletions
diff --git a/pkg/bench/bench.hlp b/pkg/bench/bench.hlp new file mode 100644 index 00000000..3b7a97b9 --- /dev/null +++ b/pkg/bench/bench.hlp @@ -0,0 +1,1723 @@ +.help bench Mar86 "IRAF Performance Tests" +.ce +\fBA Set of Benchmarks for Measuring IRAF System Performance\fR +.ce +Doug Tody +.ce +March 28, 1986 +.ce +(Revised July 1987) + +.nh +Introduction + + This set of benchmarks has been prepared with a number of purposes in mind. +Firstly, the benchmarks may be run after installing IRAF on a new system to +verify that the performance expected for that machine is actually being +achieved. In general, this cannot be taken for granted since the performance +actually achieved on a particular system can be highly dependent upon how the +system is configured and tuned. Secondly, the benchmarks may be run to compare +the performance of different IRAF hosts, or to track the system performance +over a period of time as improvements are made, both to IRAF and to the host +system. Lastly, the benchmarks provide a metric which can be used to tune +the host system. + +All too often, the only benchmarks run on a system are those which test the +execution time of optimized code generated by the host Fortran compiler. +This is primarily a hardware benchmark and secondarily a test of the Fortran +optimizer. An example of this type of test is the famous Linpack benchmark. + +The numerical execution speed test is an important benchmark but it tests only +one of the many factors contributing to the overall performance of the system +as perceived by the user. In interactive use other factors are often more +important, e.g., the time required to spawn or communicate with a subprocess, +the time required to access a file, the response of the system as the number +of users (or processes) increases, and so on. While the quality of optimized +code is a critical factor for cpu intensive batch processing, other factors +are often more important for sophisticated interactive applications. + +The benchmarks described here are designed to test, as fully as possible, +the major factors contributing to the overall performance of the IRAF system +on a particular host. A major factor in the timings of each benchmark is +of course the IRAF system itself, but comparisons of different hosts are +nonetheless possible since the code is virtually identical on all hosts. +The IRAF kernel is coded differently for each host, but the functions +performed by the kernel are identical on each host, and in most cases the +kernel operations are a negligible factor in the final timings. + +The IRAF version number, host operating system and associated version number, +and the host computer hardware configuration are all important in interpreting +the results of the benchmarks, and should always be recorded. + +.nh +What is Measured + + Each benchmark measures two quantities, the total cpu time required to +execute the benchmark, and the total (wall) clock time required to execute the +benchmark. If the clock time measurement is to be of any value the benchmarks +must be run on a single user system. Given this "best time" measurement, +it is not difficult to predict the performance to be expected on a loaded +system. + +The total cpu time required to execute a benchmark consists of the "user" time +plus the "system" time. The "user" time is the cpu time spent executing +the instructions comprising the user program. The "system" time is the cpu +time spent in kernel mode executing the system services called by the user +program. When possible we give both measurements, while in some cases only +the user time is given, or only the sum of the user and system times. +If the benchmark involves several concurrent processes no cpu time measurement +may be possible on some systems. The cpu time measurements are therefore +only reliable for the simpler benchmarks. + +The clock time measurement will of course include both the user and system +execution time, plus the time spent waiting for i/o. Any minor system daemon +processes executing while the benchmarks are being run may bias the clock +time measurement slightly, but since these are a constant part of the host +environment it is fair to include them in the timings. Major system daemons +which run infrequently (e.g., the print symbiont in VMS) should invalidate +the benchmark. + +A comparison of the cpu and clock times tells whether the benchmark was cpu +or i/o bound (assuming a single user system). Those benchmarks involving +compiled IRAF tasks do not include the process startup and pagein times +(these are measured by a different benchmark), hence the task should be run +once before running the benchmark to connect the subprocess and page in +the memory used by the task. A good procedure to follow is to run each +benchmark once to start the process, and then repeat the benchmark three times, +averaging the results. If inconsistent results are obtained further iterations +and/or monitoring of the host system are called for until a consistent result +is achieved. + +Many benchmarks depend upon disk performance as well as compute cycles. +For such a benchmark to be a meaningful measure of the i/o bandwidth of the +system it is essential that no other users (or batch jobs) be competing for +disk seeks on the disk used for the test file. There are subtle things to +watch out for in this regard, for example, if the machine is in a VMS cluster +or on a local area network, processes on other nodes may be accessing the +local disk, yet will not show up on a user login or process list on the local +node. It is always desirable to repeat each test several times or on several +different disk devices, to ensure that no outside requests were being serviced +while the benchmark was being run. If the system has disk monitoring utilities +use these to find an idle disk before running any benchmarks which do heavy i/o. + +Beware of disks which are nearly full; the maximum achievable i/o bandwidth +will fall off rapidly as a disk fills up, due to disk fragmentation (the file +must be stored in little pieces scattered all over the physical disk). +Similarly, many systems (VMS, AOS/VS) suffer from disk fragmentation problems +that gradually worsen as a files system ages, requiring that the disk +periodically be backed off onto tape and then restored. In some cases, +disk fragmentation can cause the maximum achievable i/o bandwidth to degrade +by an order of magnitude. + +.nh +The Benchmarks + + Instructions are given for running each benchmark, and the operations +performed by each benchmark are briefly described. The system characteristics +measured by the benchmark are briefly discussed. A short mnemonic name is +associated with each benchmark to identify it in the tables given in the +\fIresults\fR section. + +.nh 2 +Host Level Benchmarks + + The benchmarks discussed in this section are run at the host system level. +The examples are given for the UNIX cshell, under the assumption that a host +dependent example is better than none at all. These commands must be +translated by the user to run the benchmarks on a different system. + +.nh 3 +CL Startup/Shutdown [CLSS] + + Go to the CL login directory, mark the time (the method by which this is +done is system dependent), and startup the CL. Enter the "logout" command +while the CL is starting up so that the CL will not be idle (with the clock +running) while the command is being entered. Mark the final cpu and clock +time and compute the difference. + +.nf + % time cl + logout +.fi + +This is a complex benchmark but one which is of obvious importance to the +IRAF user. The benchmark is probably dominated by the cpu time required to +start up the CL, i.e., start up the CL process, initialize the i/o system, +initialize the environment, interpret the CL startup file, interpret the +user LOGIN.CL file, connect and disconnect the x_system.e subprocess, and so on. +Most of the remaining time is the overhead of the host operating system for +the process spawns, page faults, file accesses, and so on. + +.nh 3 +Mkpkg (verify) [MKPKGV] + + Go to the PKG directory and enter the (host system equivalent of the) +following command. The method by which the total cpu and clock times are +computed is system dependent. + +.nf + % cd $iraf/pkg + % time mkpkg -n +.fi + +This benchmark does a "no execute" make-package of the entire PKG suite of +applications and systems packages. This tests primarily the speed with which +the host system can read directories, resolve pathnames, and return directory +information for files. Since the PKG directory tree is continually growing, +this benchmark is only useful for comparing the same version of IRAF run on +different hosts, or the same version of IRAF on the same host at different +times. + +.nh 3 +Mkpkg (compile) [MKPKGC] + + Go to the directory "iraf$pkg/bench/xctest" and enter the (host system +equivalents of the) following commands. The method by which the total cpu +and clock times are computed is system dependent. Only the \fBmkpkg\fR +command should be timed. + +.nf + % cd $iraf/pkg/bench/xctest + % mkpkg clean # delete old library, etc., if present + % time mkpkg + % mkpkg clean # delete newly created binaries +.fi + +This tests the time required to compile and link a small IRAF package. +The timings reflect the time required to preprocess, compile, optimize, +and assemble each module and insert it into the package library, then link +the package executable. The host operating system overhead for the process +spawns, page faults, etc. is also a major factor. + +.nh 2 +IRAF Applications Benchmarks + + The benchmarks discussed in this section are run from within the IRAF +environment, using only standard IRAF applications tasks. The cpu and clock +execution times of any (compiled) IRAF task may be measured by prefixing +the task name with a $ when the command is entered, as shown in the examples. +The significance of the cpu time measurement is not precisely defined for +all systems. On a UNIX host, it is the "user" cpu time used by the task. +On a VMS host, there does not appear to be any distinction between the user +and system times (probably because the system services execute in the context +of the calling process), hence the cpu time given probably includes both. + +.nh 3 +Mkhelpdb [MKHDB] + + The \fBmkhelpdb\fR task is in the \fBsoftools\fR package. The function of +the task is to scan the tree of ".hd" help-directory files and compile the +binary help database. + +.nf + cl> softools + cl> $mkhelpdb +.fi + +This benchmark tests primarily the global optimization of the Fortran +compiler, since the code being executed is quite complex. It also tests the +speed with which text files can be opened and read. Since the size of the +help database varies with each version of IRAF, this benchmark is only useful +for comparing the same version of IRAF run on different hosts, or the same +version run on a single host at different times. + +.nh 3 +Sequential Image Operators [IMADDS,IMADDR,IMSTATR,IMSHIFTR] + + These benchmarks measure the time required by typical image operations. +All tests should be performed on 512 square test images created with the +\fBimdebug\fR package. The \fBimages\fR package will already have been +loaded by the \fBbench\fR package. Enter the following commands to create +the test images. + +.nf + cl> imdebug + cl> mktest pix.s s 2 "512 512" + cl> mktest pix.r r 2 "512 512" +.fi + +The following benchmarks should be run on these test images. Delete the +output images after each benchmark is run. Each benchmark should be run +several times, discarding the first timing and averaging the remaining +timings for the final result. +.ls +.ls [IMADDS] +cl> $imarith pix.s + 5 pix2.s +.le +.ls [IMADDR] +cl> $imarith pix.r + 5 pix2.r +.le +.ls [IMSTATR] +cl> $imstat pix.r +.le +.ls [IMSHIFTR] +cl> $imshift pix.r pix2.r .33 .44 interp=spline3 +.le +.le + +The IMADD benchmarks test the efficiency of the image i/o system, including +binary file i/o, and provide an indication of how long a simple disk to disk +image operation takes on the system in question. This benchmark should be +i/o bound on most systems. The IMSTATR and IMSHIFTR benchmarks are expected +to be cpu bound, and test primarily the quality of the code generated by the +host Fortran compiler. Note that the IMSHIFTR benchmark employs a true two +dimensional bicubic spline, hence the timings are a factor of 4 greater than +one would expect if a one dimensional interpolator were used to shift the two +dimensional image. + +.nh 3 +Image Load [IMLOAD,IMLOADF] + + To run the image load benchmarks, first load the \fBtv\fR package and +display something to get the x_display.e process into the process cache. +Run the following two benchmarks, displaying the test image PIX.S (this image +contains a test pattern of no interest). +.ls +.ls [IMLOAD] +cl> $display pix.s 1 +.le +.ls [IMLOADF] +cl> $display pix.s 1 zt=none +.le +.le + +The IMLOAD benchmark measures how long it takes for a normal image load on +the host system, including the automatic determination of the greyscale +mapping, and the time required to map and clip the image pixels into the +8 bits (or whatever) displayable by the image display. This benchmark +measures primarily the cpu speed and i/o bandwidth of the host system. +The IMLOADF benchmark eliminates the cpu intensive greyscale transformation, +yielding the minimum image display time for the host system. + +.nh 3 +Image Transpose [IMTRAN] + + To run this benchmark, transpose the image PIX.S, placing the output in a +new image. + + cl> $imtran pix.s pix2.s + +This benchmark tests the ability of a process to grab a large amount of +physical memory (large working set), and the speed with which the host system +can service random rather than sequential file access requests. + +.nh 2 +Specialized Benchmarks + + The next few benchmarks are implemented as tasks in the \fBbench\fR package, +located in the directory "pkg$bench". This package is not installed as a +predefined package as the standard IRAF packages are. Since this package is +used infrequently the binaries may have been deleted; if the file x_bench.e is +not present in the \fIbench\fR directory, rebuild it as follows: + +.nf + cl> cd pkg$bench + cl> mkpkg +.fi + +To load the package, enter the following commands. It is not necessary to +\fIcd\fR to the bench directory to load or run the package. + +.nf + cl> task $bench = "pkg$bench/bench.cl" + cl> bench +.fi + +This defines the following benchmark tasks. There are no manual pages for +these tasks; the only documentation is what you are reading. + +.ks +.nf + fortask - foreign task execution + getpar - get parameter; tests IPC overhead + plots - make line plots from an image + ptime - no-op task (prints the clock time) + rbin - read binary file; tests FIO bandwidth + rrbin - raw (unbuffered) binary file read + rtext - read text file; tests text file i/o speed + subproc - subprocess connect/disconnect + wbin - write binary file; tests FIO bandwidth + wipc - write to IPC; tests IPC bandwidth + wtext - write text file; tests text file i/o speed +.fi +.ke + +.nh 3 +Subprocess Connect/Disconnect [SUBPR] + + To run the SUBPR benchmark, enter the following command. +This will connect and disconnect the x_images.e subprocess 10 times. +Difference the starting and final times printed as the task output to get +the results of the benchmark. The cpu time measurement may be meaningless +(very small) on some systems. + + cl> subproc 10 + +This benchmark measures the time required to connect and disconnect an +IRAF subprocess. This includes not only the host time required to spawn +and later shutdown a process, but also the time required by the IRAF VOS +to set up the IPC channels, initialize the VOS i/o system, initialize the +environment in the subprocess, and so on. A portion of the subprocess must +be paged into memory to execute all this initialization code. The host system +overhead to spawn a subprocess and fault in a portion of its address space +is a major factor in this benchmark. + +.nh 3 +IPC Overhead [IPCO] + + The \fBgetpar\fR task is a compiled task in x_bench.e. The task will +fetch the value of a CL parameter 100 times. + + cl> $getpar 100 + +Since each parameter access consists of a request sent to the CL by the +subprocess, followed by a response from the CL process, with a negligible +amount of data being transferred in each call, this tests the IPC overhead. + +.nh 3 +IPC Bandwidth [IPCB] + + To run this benchmark enter the following command. The \fBwipc\fR task +is a compiled task in x_bench.e. + + cl> $wipc 1E6 > dev$null + +This writes approximately 1 Mb of binary data via IPC to the CL, which discards +the data (writes it to the null file via FIO). Since no actual disk file i/o is +involved, this tests the efficiency of the IRAF pseudofile i/o system and of the +host system IPC facility. + +.nh 3 +Foreign Task Execution [FORTSK] + + To run this benchmark enter the following command. The \fBfortask\fR +task is a CL script task in the \fBbench\fR package. + + cl> fortask 10 + +This benchmark executes the standard IRAF foreign task \fBrmbin\fR (one of the +bootstrap utilities) 10 times. The task is called with no arguments and does +nothing other than execute, print out its "usage" message, and shut down. +This tests the time required to execute a host system task from within the +IRAF environment. Only the clock time measurement is meaningful. + +.nh 3 +Binary File I/O [WBIN,RBIN,RRBIN] + + To run these benchmarks, load the \fBbench\fR package, and then enter the +following commands. The \fBwbin\fR, \fBrbin\fR and \fBrrbin\fR tasks are +compiled tasks in x_bench.e. A binary file named BINFILE is created in the +current directory by WBIN, and should be deleted after the benchmark has been +run. Each benchmark should be run at least twice before recording the time +and moving on to the next benchmark. Successive calls to WBIN will +automatically delete the file and write a new one. + +.nf + cl> $wbin binfile 5E6 + cl> $rbin binfile + cl> $rrbin binfile + cl> delete binfile # (not part of the benchmark) +.fi + +These benchmarks measure the time required to write and then read a binary disk +file approximately 5 Mb in size. This benchmark measures the binary file i/o +bandwidth of the FIO interface (for sequential i/o). In WBIN and RBIN the +common buffered READ and WRITE requests are used, hence some memory to memory +copying is included in the overhead measured by the benchmark. The RRBIN +benchmark uses ZARDBF to read the file in chunks of 32768 bytes, giving an +estimate of the maximum i/o bandwidth for the system. + +.nh 3 +Text File I/O [WTEXT,RTEXT] + + To run these benchmarks, load the \fBbench\fR package, and then enter the +following commands. The \fBwtext\fR and \fBrtext\fR tasks are compiled tasks +in x_bench.e. A text file named TEXTFILE is created in the current directory +by WTEXT, and should be deleted after the benchmarks have been run. +Successive calls to WTEXT will automatically delete the file and write a new +one. + +.nf + cl> $wtext textfile 1E6 + cl> $rtext textfile + cl> delete textfile # (not part of the benchmark) +.fi + +These benchmarks measure the time required to write and then read a text disk +file approximately one megabyte in size (15,625 64 character lines). +This benchmark measures the efficiency with which the system can sequentially +read and write text files. Since text file i/o requires the system to pack +and unpack records, text i/o tends to be cpu bound. + +.nh 3 +Network I/O [NWBIN,NRBIN,NWNULL,NWTEXT,NRTEXT] + + These benchmarks are equivalent to the binary and text file benchmarks +just discussed, except that the binary and text files are accessed on a +remote node via the IRAF network interface. The calling sequences are +identical except that an IRAF network filename is given instead of referencing +a file in the current directory. For example, the following commands would +be entered to run the network binary file benchmarks on node LYRA (the node +name and filename are site dependent). + +.nf + cl> $wbin lyra!/tmp3/binfile 5E6 [NWBIN] + cl> $rbin lyra!/tmp3/binfile [NRBIN] + cl> $wbin lyra!/dev/null 5E6 [NWNULL] + cl> delete lyra!/tmp3/binfile +.fi + +The text file benchmarks are equivalent with the obvious changes, i.e., +substitute "text" for "bin", "textfile" for "binfile", and omit the null +textfile benchmark. The type of network interface used (TCP/IP, DECNET, etc.), +and the characteristics of the remote node should be recorded. + +These benchmarks test the bandwidth of the IRAF network interfaces for binary +and text files, as well as the limiting speed of the network itself (NWNULL). +The binary file benchmarks should be i/o bound. NWBIN should outperform +NRBIN since a network write is a pipelined operation, whereas a network read +is (currently) a synchronous operation. Text file access may be either cpu +or i/o bound depending upon the relative speeds of the network and host cpus. +The IRAF network interface buffers textfile i/o to minimize the number of +network packets and maximize the i/o bandwidth. + +.nh 3 +Task, IMIO, GIO Overhead [PLOTS] + + The \fBplots\fR task is a CL script task which calls the \fBprow\fR task +repeatedly to plot the same line of an image. The graphics output is +discarded (directed to the null file) rather than plotted since otherwise +the results of the benchmark would be dominated by the plotting speed of the +graphics terminal. + + cl> plots pix.s 10 + +This is a complex benchmark. The benchmark measures the overhead of task +(not process) execution and the overhead of the IMIO and GIO subsystems, +as well as the speed with which IPC can be used to pass parameters to a task +and return the GIO graphics metacode to the CL. + +The \fBprow\fR task is all overhead and is not normally used to interactively +plot image lines (\fBimplot\fR is what is normally used), but it is a good +task to use for a benchmark since it exercises the subsystems most commonly +used in scientific tasks. The \fBprow\fR task has a couple dozen parameters +(mostly hidden), must open the image to read the image line to be plotted +on every call, and must open the GIO graphics device on every call as well. + +.nh 3 +System Loading [2USER,4USER] + + This benchmark attempts to measure the response of the system as the +load increases. This is done by running large \fBplots\fR jobs on several +terminals and then repeating the 10 plots \fBplots\fR benchmark. +For example, to run the 2USER benchmark, login on a second terminal and +enter the following command, and then repeat the PLOTS benchmark discussed +in the last section. Be sure to use a different login or login directory +for each "user", to avoid concurrency problems, e.g., when reading the +input image or updating parameter files. + + cl> plots pix.s 9999 + +Theoretically, the timings should be approximately .5 (2USER) and .25 (4USER) +as fast as when the PLOTS benchmark was run on a single user system, assuming +that cpu time is the limiting resource and that a single job is cpu bound. +In a case where there is more than one limiting resource, e.g., disk seeks as +well as cpu cycles, performance will fall off more rapidly. If, on the other +hand, a single user process does not keep the system busy, e.g., because +synchronous i/o is used, performance will fall off less rapidly. If the +system unexpectedly runs out of some critical system resource, e.g., physical +memory or some internal OS buffer space, performance may be much worse than +expected. + +If the multiuser performance is poorer than expected it may be possible to +improve the system performance significantly once the reason for the poor +performance is understood. If disk seeks are the problem it may be possible +to distribute the load more evenly over the available disks. If the +performance decays linearly as more users are added and then gets really bad, +it is probably because some critical system resource has run out. Use the +system monitoring tools provided with the host operating system to try to +identify the critical resource. It may be possible to modify the system +tuning parameters to fix the problem, once the critical resource has been +identified. + +.nh +Interpreting the Benchmark Results + + Many factors determine the timings obtained when the benchmarks are run +on a system. These factors include all of the following: + +.ls +.ls o +The hardware configuration, e.g., cpu used, clock speed, availability of +floating point hardware, type of floating point hardware, amount of memory, +number and type of disks, degree of fragmentation of the disks, bus bandwidth, +disk controller bandwidth, memory controller bandwidth for memory mapped DMA +transfers, and so on. +.le +.ls o +The host operating system, including the version number, tuning parameters, +user quotas, working set size, files system parameters, Fortran compiler +characteristics, level of optimization used to compile IRAF, and so on. +.le +.ls o +The version of IRAF being run. On a VMS system, are the images "installed" +to permit shared memory and reduce physical memory usage? Were the programs +compiled with the code optimizer, and if so, what compiler options were used? +Are shared libraries used if available on the host system? +.le +.ls o +Other activity in the system when the benchmarks were run. If there were no +other users on the machine at the time, how about batch jobs? If the machine +is on a cluster or network, were other nodes accessing the same disks? +How many other processes were running on the local node? Ideally, the +benchmarks should be run on an otherwise idle system, else the results may be +meaningless or next to impossible to interpret. Given some idea of how the +host system responds to loading, it is possible to estimate how a timing +will scale as the system is loaded, but the reverse operation is much more +difficult. +.le +.le + + +Because so many factors contribute to the results of a benchmark, it can be +difficult to draw firm conclusions from any benchmark, no matter how simple. +The hardware and software in modern computer systems is so complicated that +it is difficult even for an expert with a detailed knowledge and understanding +of the full system to explain in detail where the time is going, even when +running the simplest benchmark. On some recent message based multiprocessor +systems it is probably impossible to fully comprehend what is going on at any +given time, even if one fully understands how the system works, because of the +dynamic nature of such systems. + +Despite these difficulties, the benchmarks do provide a coarse measure of the +relative performance of different host systems, as well as some indication of +the efficiency of the IRAF VOS. The benchmarks are designed to measure the +performance of the \fIhost system\fR (both hardware and software) in a number +of important areas, all of which play a role in determining the suitability of +a system for scientific data processing. The benchmarks are \fInot\fR +designed to measure the efficiency of the IRAF software itself (except parts +of the VOS), e.g., there is no measure of the time taken by the CL to compile +and execute a script, no measure of the speed of the median algorithm or of +an image transpose, and so on. These timings are also important, of course, +but should be measured separately. Also, measurements of the efficiency of +individual applications programs are much less critical than the performance +criteria dealt with here, since it is relatively easy to optimize an +inefficient or poorly designed applications program, even a complex one like +the CL, but there is generally little one can do about the host system. + +The timings for the benchmarks for a number of host systems are given in the +appendices which follow. Sometimes there will be more than one set of +benchmarks for a given host system, e.g., because the system provided two or +more disks or floating point options with different levels of performance. +The notes at the end of each set of benchmarks are intended to document any +special features or problems of the host system which may have affected the +results. In general we did not bother to record things like system tuning +parameters, working set, page faults, etc., unless these were considered an +important factor in the benchmarks. In particular, few IRAF programs page +fault other than during process startup, hence this is rarely a significant +factor when running these benchmarks (except possibly in IMTRAN). + +Detailed results for each configuration of each host system are presented on +separate pages in the Appendices. A summary table showing the results of +selected benchmarks for all host systems at once is also provided. +The system characteristic or characteristics principally measured by each +benchmark is noted in the table below. This is only approximate, e.g., the +MIPS rating is a significant factor in all but the most i/o bound benchmarks. + +.ks +.nf + benchmark responsiveness mips flops i/o + + CLSS * + MKPKGV * + MKHDB * * + PLOTS * * + IMADDS * * + IMADDR * * + IMSTATR * + IMSHIFTR * + IMTRAN * + WBIN * + RBIN * +.fi +.ke + + +By \fIresponsiveness\fR we refer to the interactive response of the system +as perceived by the user. A system with a good interactive response will do +all the little things very fast, e.g., directory listings, image header +listings, plotting from an image, loading new packages, starting up a new +process, and so on. Machines which score high in this area will seem fast +to the user, whereas machines which score poorly will \fIseem\fR slow, +sometimes frustratingly slow, even though they may score high in the areas +of floating point performance, or i/o bandwidth. The interactive response +of a system obviously depends upon the MIPS rating of the system (see below), +but an often more significant factor is the design and computational complexity +of the host operating system itself, in particular the time taken by the host +operating system to execute system calls. Any system which spends a large +fraction of its time in kernel mode will probably have poor interactive +response. The response of the system to loading is also very important, +i.e., if the system has trouble with load balancing as the number of users +(or processes) increases, response will become increasingly erratic until the +interactive response is hopelessly poor. + +The MIPS column refers to the raw speed of the system when executing arbitrary +code containing a mixture of various types of instructions, but little floating +point, i/o, or system calls. A machine with a high MIPS rating will have a +fast cpu, e.g., a fast clock rate, fast memory access time, large cache memory, +and so on, as well as a good optimizing Fortran compiler. Assuming good +compilers, the MIPS rating is primarily a measure of the hardware speed of +the host machine, but all of the MIPS related benchmarks presented here also +make a significant number of system calls (MKHDB, for example, does a lot of +files accesses and text file i/o), hence it is not that simple. Perhaps a +completely cpu bound pure-MIPS benchmark should be added to our suite of +benchmarks (the MIPS rating of every machine is generally well known, however). + +The FLOPS column identifies those benchmarks which do a significant amount of +floating point computation. The IMSHIFTR and IMSTATR benchmarks in particular +are heavily into floating point. These benchmarks measure the single +precision floating point speed of the host system hardware, as well as the +effectiveness of do-loop optimization by the host Fortran compiler. +The degree of optimization provided by the Fortran compiler can affect the +timing of these benchmarks by up to a factor of two. Note that the sample is +very small, and if a compiler fails to optimize the inner loop of one of these +benchmark programs, the situation may be reversed when running some other +benchmark. Any reasonable Fortran compiler should be able to optimize the +inner loop of the IMADDR benchmark, so the CPU timing for this benchmark is +a good measure of the hardware floating point speed, if one allows for do-loop +overhead, memory i/o, and the system calls necessary to access the image on +disk. + +The I/O column identifies those benchmarks which are i/o bound and which +therefore provide some indication of the i/o bandwidth of the host system. +The i/o bandwidth actually achieved in these benchmarks depends upon +many factors, the most important of which are the host operating system +software (files system data structures and i/o software, disk drivers, etc.) +and the host system hardware, i.e., disk type, disk controller type, bus +bandwidth, and DMA memory controller bandwidth. Note that asynchronous i/o +is not currently used in these benchmarks, hence higher transfer rates are +probably possible in special cases (on a busy system all i/o is asynchronous +at the host system level anyway). Large transfers are used to minimize disk +seeks and synchronization delays, hence the benchmarks should provide a good +measure of the realistically achievable host i/o bandwidth. + +.bp + . +.sp 20 +.ce +APPENDIX 1. IRAF VERSION 2.5 BENCHMARKS +.ce +April-June 1987 + +.bp +.sh +UNIX/IRAF V2.5 4.3BSD UNIX, 8Mb memory, VAX 11/750+FPA RA81 (lyra) +.br +CPU times are given in seconds, CLK times in minutes and seconds. +.br +Wednesday, 1 April, 1987, Suzanne H. Jacoby, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes \fR + (user+sys) (m:ss) + +CLSS 7.4+2.6 0:17 CPU = user + system +MKPKGV 13.4+9.9 0:39 CPU = user + system +MKPKGC 135.1+40. 3:46 CPU = user + system +MKHDB 22.79 0:40 [1] +IMADDS 3.31 0:10 512X512X16 +IMADDR 4.28 0:17 512X512X32 +IMSTATR 10.98 0:15 512X512X32 +IMSHIFTR 114.41 2:13 512X512X32 +IMLOAD 7.62 0:15 512X512X16 +IMLOADF 2.63 0:08 512X512X16 +IMTRAN 10.19 0:17 512X512X16 +SUBPR n/a 0:20 10 conn/discon 2.0 sec/proc +IPCO 0.92 0:07 100 getpars +IPCB 2.16 0:15 1E6 bytes 66.7 Kb/sec +FORTSK n/a 0:06 10 commands 0.6 sec/cmd +WBIN 4.32 0:24 5E6 bytes 208.3 Kb/sec +RBIN 4.08 0:24 5E6 bytes 208.3 Kb/sec +RRBIN 0.12 0:22 5E6 bytes 227.3 Kb/sec +WTEXT 37.30 0:42 1E6 bytes 23.8 Kb/sec +RTEXT 26.49 0:32 1E6 bytes 31.3 Kb/sec +NWBIN 4.64 1:43 5E6 bytes 48.5 Kb/sec [2] +NRBIN 6.49 1:34 5E6 bytes 53.2 Kb/sec [2] +NWNULL 4.91 1:21 5E6 bytes 61.7 Kb/sec [2] +NWTEXT 44.03 1:02 1E6 bytes 16.1 Kb/sec [2] +NRTEXT 31.38 2:04 1E6 bytes 8.1 Kb/sec [2] +PLOTS n/a 0:29 10 plots 2.9 sec/PROW +2USER n/a 0:44 10 plots 4.4 sec/PROW +4USER n/a 1:19 10 plots 7.9 sec/PROW +.fi + + +Notes: +.ls [1] +All cpu timings from MKHDB on do not include the "system" time. +.le +.ls [2] +The remote node used for the network tests was aquila, a VAX 11/750 running +4.3 BSD UNIX. The network protocol used was TCP/IP. +.le + +.bp +.sh +UNIX/IRAF V2.5 SUN UNIX 3.3, SUN 3/160C, (tucana) +.br +16 MHz 68020, 68881 fpu, 8Mb, 2-380Mb Fujitsu Eagle disks +.br +Friday, June 12, 1987, Suzanne H. Jacoby, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes \fR + (user+sys) (m:ss) + +CLSS 2.0+0.8 0:03 CPU = user + system +MKPKGV 3.2+4.5 0:17 CPU = user + system +MKPKGC 59.1+26.2 2:13 CPU = user + system +MKHDB 5.26 0:10 [1] +IMADDS 0.62 0:03 512X512X16 +IMADDR 3.43 0:09 512X512X32 +IMSTATR 8.38 0:11 512X512X32 +IMSHIFTR 83.44 1:33 512X512X32 +IMLOAD 6.78 0:11 512X512X16 +IMLOADF 1.21 0:03 512X512X16 +IMTRAN 1.47 0:05 512X512X16 +SUBPR n/a 0:07 10 conn/discon 0.7 sec/proc +IPCO 0.16 0:02 100 getpars +IPCB 0.70 0:05 1E6 bytes 200.0 Kb/sec +FORTSK n/a 0:02 10 commands 0.2 sec/cmd +WBIN 2.88 0:08 5E6 bytes 625.0 Kb/sec +RBIN 2.58 0:11 5E6 bytes 454.5 Kb/sec +RRBIN 0.01 0:10 5E6 bytes 500.0 Kb/sec +WTEXT 9.20 0:10 1E6 bytes 100.0 Kb/sec +RTEXT 6.75 0:07 1E6 bytes 142.8 Kb/sec +NWBIN 2.65 1:04 5E6 bytes 78.1 Kb/sec [2] +NRBIN 3.42 1:16 5E6 bytes 65.8 Kb/sec [2] +NWNULL 2.64 1:01 5E6 bytes 82.0 Kb/sec [2] +NWTEXT 11.92 0:39 1E6 bytes 25.6 Kb/sec [2] +NRTEXT 7.41 1:24 1E6 bytes 11.9 Kb/sec [2] +PLOTS n/a 0:09 10 plots 0.9 sec/PROW +2USER n/a 0:16 10 plots 1.6 sec/PROW +4USER n/a 0:35 10 plots 3.5 sec/PROW +.fi + + +Notes: +.ls [1] +All timings from MKHDB on do not include the "system" time. +.le +.ls [2] +The remote node used for the network tests was aquila, a VAX 11/750 +running 4.3BSD UNIX. The network protocol used was TCP/IP. +.le + +.bp +.sh +UNIX/IRAF V2.5 SUN UNIX 3.3, SUN 3/160C + FPA (KPNO 4 meter system) +.br +16 MHz 68020, Sun-3 FPA, 8Mb, 2-380Mb Fujitsu Eagle disks +.br +Friday, June 12, 1987, Suzanne H. Jacoby, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes \fR + (user+sys) (m:ss) + +CLSS 1.9+0.7 0:04 CPU = user + system +MKPKGV 3.1+3.9 0:19 CPU = user + system +MKPKGC 66.2+20.3 2:06 CPU = user + system +MKHDB 5.30 0:11 [1] +IMADDS 0.63 0:03 512X512X16 +IMADDR 0.86 0:06 512X512X32 +IMSTATR 5.08 0:08 512X512X32 +IMSHIFTR 31.06 0:36 512X512X32 +IMLOAD 2.76 0:06 512X512X16 +IMLOADF 1.22 0:03 512X512X16 +IMTRAN 1.46 0:04 512X512X16 +SUBPR n/a 0:06 10 conn/discon 0.6 sec/proc +IPCO 0.16 0:01 100 getpars +IPCB 0.60 0:05 1E6 bytes 200.0 Kb/sec +FORTSK n/a 0:02 10 commands 0.2 sec/cmd +WBIN 2.90 0:07 5E6 bytes 714.3 Kb/sec +RBIN 2.54 0:11 5E6 bytes 454.5 Kb/sec +RRBIN 0.03 0:10 5E6 bytes 500.0 Kb/sec +WTEXT 9.20 0:11 1E6 bytes 90.9 Kb/sec +RTEXT 6.70 0:08 1E6 bytes 125.0 Kb/sec +NWBIN n/a +NRBIN n/a [3] +NWNULL n/a +NWTEXT n/a +NRTEXT n/a +PLOTS n/a 0:06 10 plots 0.6 sec/PROW +2USER n/a 0:10 10 plots 1.0 sec/PROW +4USER n/a 0:26 10 plots 2.6 sec/PROW +.fi + + +Notes: +.ls [1] +All timings from MKHDB on do not include the "system" time. +.le + +.bp +.sh +UNIX/IRAF V2.5, SUN UNIX 3.2, SUN 3/160 (taurus) +.br +16 MHz 68020, Sun-3 FPA, 16 Mb, SUN SMD disk 280 Mb +.br +7 April 1987, Skip Schaller, Steward Observatory, University of Arizona + +.nf +\fBBenchmark CPU CLK Size Notes\fR + (user+sys) (m:ss) + +CLSS 01.2+01.1 0:03 +MKPKGV 03.2+10.1 0:18 +MKPKGC 65.4+25.7 2:03 +MKHDB 5.4 0:18 +IMADDS 0.6 0:04 512x512x16 +IMADDR 0.9 0:07 512x512x32 +IMSTATR 11.4 0:13 512x512x32 +IMSHIFTR 30.1 0:34 512x512x32 +IMLOAD (not available) +IMLOADF (not available) +IMTRAN 1.4 0:04 512x512x16 +SUBPR - 0:07 10 conn/discon 0.7 sec/proc +IPCO 0.1 0:02 100 getpars +IPCB 0.8 0:05 1E6 bytes 200.0 Kb/sec +FORTSK - 0:03 10 commands 0.3 sec/cmd +WBIN 2.7 0:14 5E6 bytes 357.1 Kb/sec +RBIN 2.5 0:09 5E6 bytes 555.6 Kb/sec +RRBIN 0.1 0:06 5E6 bytes 833.3 Kb/sec +WTEXT 9.0 0:10 1E6 bytes 100.0 Kb/sec +RTEXT 6.4 0:07 1E6 bytes 142.9 Kb/sec +NWBIN 2.8 1:08 5E6 bytes 73.5 Kb/sec +NRBIN 3.1 1:25 5E6 bytes 58.8 Kb/sec +NWNULL 2.7 0:55 5E6 bytes 90.9 Kb/sec +NWTEXT 12.3 0:44 1E6 bytes 22.7 Kb/sec +NRTEXT 7.7 1:45 1E6 bytes 9.5 Kb/sec +PLOTS - 0:07 10 plots 0.7 sec/PROW +2USER - 0:13 +4USER - 0:35 +.fi + + +Notes: +.ls [1] +The remote node used for the network tests was carina, a VAX 11/750 +running 4.3 BSD UNIX. The network protocol used was TCP/IP. +.le + +.bp +.sh +Integrated Solutions (ISI), Lick Observatory +.br +16-Mhz 68020, 16-Mhz 68881 fpu, 8Mb Memory +.br +IRAF compiled with Greenhills compilers without -O optimization +.br +Thursday, 14 May, 1987, Richard Stover, Lick Observatory + +.nf +\fBBenchmark CPU CLK Size Notes\fR + (user+sys) (m:ss) + +CLSS 1.6+0.7 0:03 +MKPKGV 3.1+4.6 0:25 +MKPKGC 40.4+11.6 1:24 +MKHDB 6.00 0:17 +IMADDS 0.89 0:05 512X512X16 +IMADDR 3.82 0:10 512X512X32 +IMSTATR 7.77 0:10 512X512X32 +IMSHIFTR 81.60 1:29 512X512X32 +IMLOAD n/a +IMLOADF n/a +IMTRAN 1.62 0:06 512X512X16 +SUBPR n/a 0:05 10 donn/discon 0.5 sec/proc +IPCO 0.27 0:02 100 getpars +IPCB 1.50 0:08 1E6 bytes 125.0 Kb/sec +FORTSK n/a 0:13 10 commands 1.3 sec/cmd +WBIN 4.82 0:17 5E6 bytes 294.1 Kb/sec +RBIN 4.63 0:18 5E6 bytes 277.8 Kb/sec +RRBIN 0.03 0:13 5E6 bytes 384.6 Kb/sec +WTEXT 17.10 0:19 1E6 bytes 45.5 Kb/sec +RTEXT 7.40 0:08 1E6 bytes 111.1 Kb/sec +NWBIN n/a +NRBIN n/a +NWNULL n/a +NWTEXT n/a +NRTEXT n/a +PLOTS n/a 0:10 10 plots 1.0 sec/PROW +2USER n/a +4USER n/a +.fi + + +Notes: +.ls [1] +An initial attempt to bring IRAF up on the ISI using the ISI C and Fortran +compilers failed due to there being too many bugs in these compilers, so +the system was brought up using the Greenhills compilers. +.le + +.bp +.sh +ULTRIX/IRAF V2.5, ULTRIX 1.2, VAXStation II/GPX (gll1) +.br +5Mb memory, 150 Mb RD54 disk +.br +Thursday, 21 May, 1987, Suzanne H. Jacoby, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes \fR + (user+sys) (m:ss) + +CLSS 4.2+1.8 0:09 CPU = user + system +MKPKGV 9.8+6.1 0:37 CPU = user + system +MKPKGC 96.8+24.4 3:15 CPU = user + system +MKHDB 15.50 0:38 [1] +IMADDS 2.06 0:09 512X512X16 +IMADDR 2.98 0:17 512X512X32 +IMSTATR 10.98 0:16 512X512X32 +IMSHIFTR 95.61 1:49 512X512X32 +IMLOAD 6.90 0:17 512X512X16 [2] +IMLOADF 2.58 0:10 512X512X16 [2] +IMTRAN 4.93 0:16 512X512X16 +SUBPR n/a 0:19 10 conn/discon 1.9 sec/proc +IPCO 0.47 0:03 100 getpars +IPCB 1.21 0:07 1E6 bytes 142.9 Kb/sec +FORTSK n/a 0:08 10 commands 0.8 sec/cmd +WBIN 1.97 0:29 5E6 bytes 172.4 Kb/sec +RBIN 1.73 0:24 5E6 bytes 208.3 Kb/sec +RRBIN 0.08 0:24 5E6 bytes 208.3 Kb/sec +WTEXT 25.43 0:27 1E6 bytes 37.0 Kb/sec +RTEXT 16.65 0:18 1E6 bytes 55.5 Kb/sec +NWBIN 2.24 1:26 5E6 bytes 58.1 Kb/sec [3] +NRBIN 2.66 1:43 5E6 bytes 48.5 Kb/sec [3] +NWNULL 2.22 2:21 5E6 bytes 35.5 Kb/sec [3] +NWTEXT 27.16 2:43 1E6 bytes 6.1 Kb/sec [3] +NRTEXT 17.44 2:17 1E6 bytes 7.3 Kb/sec [3] +PLOTS n/a 0:20 10 plots 2.0 sec/PROW +2USER n/a 0:30 10 plots 3.0 sec/PROW +4USER n/a 0:51 10 plots 5.1 sec/PROW +.fi + + +Notes: +.ls [1] +All cpu timings from MKHDB on do not include the "system" time. +.le +.ls [2] +Since there is no image display on this node, the image display benchmarks +were run using the IIS display on node lyra via the network interface. +.le +.ls [3] +The remote node used for the network tests was lyra, a VAX 11/750 running +4.3 BSD UNIX. The network protocol used was TCP/IP. +.le +.ls [4] +Much of the hardware and software for this system was provided courtesy of +DEC so that we may better support IRAF on the microvax. +.le + +.bp +.sh +VMS/IRAF V2.5, VMS V4.5, 28Mb, VAX 8600 RA81/Clustered (draco) +.br +Friday, 15 May, 1987, Suzanne H. Jacoby, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes \fR + (user+sys) (m:ss) + +CLSS 2.87 0:08 +MKPKGV 33.57 1:05 +MKPKGC 3.26 1:16 +MKHDB 8.59 0:17 +IMADDS 1.56 0:05 512X512X16 +IMADDR 1.28 0:07 512X512X32 +IMSTATR 2.09 0:04 512X512X32 +IMSHIFTR 13.54 0:32 512X512X32 +IMLOAD 2.90 0:10 512X512X16 [1] +IMLOADF 1.04 0:08 512X512X16 [1] +IMTRAN 2.58 0:06 512X512X16 +SUBPR n/a 0:27 10 conn/discon 2.7 sec/proc +IPCO 0.00 0:02 100 getpars +IPCB 0.04 0:06 1E6 bytes 166.7 Kb/sec +FORTSK n/a 0:13 10 commands 1.3 sec/cmd +WBIN 1.61 0:17 5E6 bytes 294.1 Kb/sec +RBIN 1.07 0:08 5E6 bytes 625.0 Kb/sec +RRBIN 0.34 0:08 5E6 bytes 625.0 Kb/sec +WTEXT 10.62 0:17 1E6 bytes 58.8 Kb/sec +RTEXT 4.64 0:06 1E6 bytes 166.7 Kb/sec +NWBIN 2.56 2:00 5E6 bytes 41.7 Kb/sec [2] +NRBIN 5.67 1:57 5E6 bytes 42.7 Kb/sec [2] +NWNULL 2.70 1:48 5E6 bytes 46.3 Kb/sec [2] +NWTEXT 12.06 0:47 1E6 bytes 21.3 Kb/sec [2] +NRTEXT 10.10 1:41 1E6 bytes 9.9 Kb/sec [2] +PLOTS n/a 0:09 10 plots 0.9 sec/PROW +2USER n/a 0:10 10 plots 1.0 sec/PROW +4USER n/a 0:18 10 plots 1.8 sec/PROW +.fi + + +Notes: +.ls [1] +The image display was accessed via the network (IRAF TCP/IP network interface, +Wollongong TCP/IP package for VMS), with the IIS image display residing on +node lyra and accessed via a UNIX/IRAF kernel server. The binary and text +file network tests also used lyra as the remote node. +.le +.ls [2] +The remote node for network benchmarks was aquila, a VAX 11/750 running +4.3BSD UNIX. Connection made via TCP/IP. +.le +.ls [3] +The system was linked using shared libraries and the IRAF executables for +the cl and system tasks, as well as the shared library, were "installed" +using the VMS INSTALL utility. +.le +.ls [4] +The high value of the IPC bandwidth for VMS is due to the use of shared +memory. Mailboxes were considerably slower and are no longer used. +.le +.ls [5] +The foreign task interface uses mailboxes to talk to a DCL run as a +subprocess and should be considerably faster than it is. It is slow at +present due to the need to call SET MESSAGE before and after the user +command to disable pointless DCL error messages having to do with +logical names. +.le + +.bp +.sh +VMS/IRAF V2.5, VAX 11/780, VMS V4.5, 16Mb memory, RA81 disks (wfpct1) +.br +Tuesday, 19 May, 1987, Suzanne H. Jacoby, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes\fR + (user+sys) (m:ss) + +CLSS 7.94 0:15 +MKPKGV 102.49 2:09 +MKPKGC 9.50 2:22 +MKHDB 26.10 0:31 +IMADDS 3.57 0:10 512X512X16 +IMADDR 4.22 0:17 512X512X32 +IMSTATR 6.78 0:10 512X512X32 +IMSHIFTR 45.11 0:57 512X512X32 +IMLOAD n/a +IMLOADF n/a +IMTRAN 7.83 0:14 512X512X16 +SUBPR n/a 0:53 10 donn/discon 5.3 sec/proc +IPCO 0.02 0:03 100 getpars +IPCB 0.17 0:10 1E6 bytes 100.0 Kb/sec +FORTSK n/a 0:20 10 commands 2.0 sec/cmd +WBIN 4.52 0:30 5E6 bytes 166.7 Kb/sec +RBIN 3.90 0:19 5E6 bytes 263.2 Kb/sec +RRBIN 1.23 0:17 5E6 bytes 294.1 Kb/sec +WTEXT 37.99 0:50 1E6 bytes 20.0 Kb/sec +RTEXT 18.52 0:19 1E6 bytes 52.6 Kb/sec +NWBIN n/a +NRBIN n/a +NWNULL n/a +NWTEXT n/a +NRTEXT n/a +PLOTS n/a 0:19 10 plots 1.9 sec/PROW +2USER n/a 0:31 10 plots 3.1 sec/PROW +4USER n/a 1:04 10 plots 6.4 sec/PROW +.fi + + +Notes: +.ls [1] +The Unibus interface used for the RA81 disks for these benchmarks is +notoriously slow, hence the i/o bandwidth of the system as tested was +probably significantly worse than many sites would experience (using +disks on the faster Massbus interface). +.le + +.bp +.sh +VMS/IRAF V2.5, VAX 11/780, VMS V4.5 (wfpct1) +.br +16Mb memory, IRAF installed on RA81 disks, data on RM03/Massbus [1]. +.br +Tuesday, 9 June, 1987, Suzanne H. Jacoby, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes\fR + (user+sys) (m:ss) + +CLSS n/a +MKPKGV n/a +MKPKGC n/a +MKHDB n/a +IMADDS 3.38 0:08 512X512X16 +IMADDR 4.00 0:11 512X512X32 +IMSTATR 6.88 0:08 512X512X32 +IMSHIFTR 45.47 0:53 512X512X32 +IMLOAD n/a +IMLOADF n/a +IMTRAN 7.71 0:12 512X512X16 +SUBPR n/a +IPCO n/a +IPCB n/a +FORTSK n/a +WBIN 4.22 0:22 5E6 bytes 227.3 Kb/sec +RBIN 3.81 0:12 5E6 bytes 416.7 Kb/sec +RRBIN 0.98 0:09 5E6 bytes 555.6 Kb/sec +WTEXT 37.20 0:47 1E6 bytes 21.3 Kb/sec +RTEXT 17.95 0:18 1E6 bytes 55.6 Kb/sec +NWBIN n/a +NRBIN n/a +NWNULL n/a +NWTEXT n/a +NRTEXT n/a +PLOTS n/a 0:16 10 plots 1.6 sec/PROW +2USER +4USER +.fi + +Notes: +.ls [1] +The data files were stored on an RM03 with 23 free Mb and a Massbus interface +for these benchmarks. Only those benchmarks which access the RM03 are given. +.le + +.bp +.sh +VMS/IRAF V2.5, MicroVMS 4.5, VAXStation II/GPX (gll1) +.br +5Mb memory, 70Mb RD53 plus 300 Mb Maxstor with Emulex controller. +.br +Wednesday, 13 May, 1987, Suzanne H. Jacoby, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes\fR + (user+sys) (m:ss) + +CLSS 9.66 0:17 +MKPKGV 109.26 2:16 +MKPKGC 9.25 2:53 +MKHDB 27.58 0:39 +IMADDS 3.51 0:07 512X512X16 +IMADDR 4.31 0:10 512X512X32 +IMSTATR 9.31 0:11 512X512X32 +IMSHIFTR 74.54 1:21 512X512X32 +IMLOAD n/a +IMLOADF n/a +IMTRAN 10.81 0:27 512X512X16 +SUBPR n/a 0:53 10 conn/discon 5.3 sec/proc +IPCO 0.03 0:03 100 getpars +IPCB 0.13 0:07 1E6 bytes 142.8 Kb/sec +FORTSK n/a 0:29 10 commands 2.9 sec/cmd +WBIN 3.29 0:16 5E6 bytes 312.5 Kb/sec +RBIN 2.38 0:10 5E6 bytes 500.0 Kb/sec +RRBIN 0.98 0:09 5E6 bytes 555.5 Kb/sec +WTEXT 41.00 0:53 1E6 bytes 18.9 Kb/sec +RTEXT 28.74 0:29 1E6 bytes 34.5 Kb/sec +NWBIN 8.28 0:46 5E6 bytes 108.7 Kb/sec [1] +NRBIN 5.66 0:50 5E6 bytes 100.0 Kb/sec [1] +NWNULL 8.39 0:42 5E6 bytes 119.0 Kb/sec [1] +NWTEXT 30.21 0:33 1E6 bytes 30.3 Kb/sec [1] +NRTEXT 20.05 0:38 1E6 bytes 26.3 Kb/sec [1] +PLOTS 0:16 10 plots 1.6 sec/plot +2USER 0:26 10 plots 2.6 sec/plot +4USER +.fi + +Notes: +.ls [1] +The remote node for the network tests was draco, a VAX 8600 running +V4.5 VMS. The network protocol used was DECNET. +.le +.ls [2] +Much of the hardware and software for this system was provided courtesy of +DEC so that we may better support IRAF on the microvax. +.le + +.bp +.sh +VMS/IRAF V2.5, MicroVMS 4.5, VAXStation II/GPX (gll1) +.br +5 Mb memory, IRAF on 300 Mb Maxstor/Emulex, data on 70 Mb RD53 [1]. +.br +Sunday, 31 May, 1987, Suzanne H. Jacoby, NOAO/Tucson. + +.nf +\fBBenchmark CPU CLK Size Notes\fR + (user+sys) (m:ss) + +CLSS n/a n/a +MKPKGV n/a n/a +MKPKGC n/a n/a +MKHDB n/a n/a +IMADDS 3.44 0:07 512X512X16 +IMADDR 4.31 0:15 512X512X32 +IMSTATR 9.32 0:12 512X512X32 +IMSHIFTR 74.72 1:26 512X512X32 +IMLOAD n/a +IMLOADF n/a +IMTRAN 10.83 0:35 512X512X16 +SUBPR n/a +IPCO n/a +IPCB n/a +FORTSK n/a +WBIN 3.33 0:26 5E6 bytes 192.3 Kb/sec +RBIN 2.30 0:17 5E6 bytes 294.1 Kb/sec +RRBIN 0.97 0:11 5E6 bytes 294.1 Kb/sec +WTEXT 40.84 0:54 1E6 bytes 18.2 Kb/sec +RTEXT 27.99 0:28 1E6 bytes 35.7 Kb/sec +NWBIN n/a +NRBIN n/a +NWNULL n/a +NWTEXT n/a +NRTEXT n/a +PLOTS 0:17 10 plots 1.7 sec/plot +2USER n/a +4USER n/a +.fi + + +Notes: +.ls [1] +IRAF installed on a 300 Mb Maxstor with Emulax controller; data files on a +70Mb RD53. Only those benchmarks which access the RD53 disk are included +below. +.le + +.bp +.sh +VMS/IRAF V2.5, VMS V4.5, VAX 11/750+FPA RA81/Clustered, 7.25 Mb (vela) +.br +Friday, 15 May 1987, Suzanne H. Jacoby, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes \fR + (user+sys) (m:ss) + +CLSS 14.11 0:27 +MKPKGV 189.67 4:17 +MKPKGC 18.08 3:44 +MKHDB 46.54 1:11 +IMADDS 5.90 0:11 512X512X16 +IMADDR 6.48 0:14 512X512X32 +IMSTATR 10.65 0:14 512X512X32 +IMSHIFTR 69.62 1:33 512X512X32 +IMLOAD 15.83 0:23 512X512X16 +IMLOADF 6.08 0:13 512X512X16 +IMTRAN 14.85 0:20 512X512X16 +SUBPR n/a 1:54 10 conn/discon 11.4 sec/proc +IPCO 1.16 0:06 100 getpars +IPCB 2.92 0:09 1E6 bytes 111.1 Kb/sec +FORTSK n/a 0:33 10 commands 3.3 sec/cmd +WBIN 6.96 0:21 5E6 bytes 238.1 Kb/sec +RBIN 5.37 0:13 5E6 bytes 384.6 Kb/sec +RRBIN 1.86 0:10 5E6 bytes 500.0 Kb/sec +WTEXT 66.12 1:24 1E6 bytes 11.9 Kb/sec +RTEXT 32.06 0:36 1E6 bytes 27.7 Kb/sec +NWBIN 13.53 1:49 5E6 bytes 45.9 Kb/sec [1] +NRBIN 19.52 2:06 5E6 bytes 39.7 Kb/sec [1] +NWNULL 13.40 1:44 5E6 bytes 48.1 Kb/sec [1] +NWTEXT 82.35 1:42 1E6 bytes 9.8 Kb/sec [1] +NRTEXT 63.00 2:39 1E6 bytes 6.3 Kb/sec [1] +PLOTS n/a 0:25 10 plots 2.5 sec/PROW +2USER n/a 0:53 10 plots 5.3 sec/PROW +4USER n/a 1:59 10 plots 11.9 sec/PROW +.fi + + +Notes: +.ls [1] +The remote node for network benchmarks was aquila, a VAX 11/750 running +4.3BSD UNIX. Connection made via TCP/IP. +.le +.ls [2] +The interactive response of this system seemed to decrease markedly when it +was converted to 4.X VMS and is currently pretty marginal, even on a single +user 11/750. In interactive applications which make frequent system calls the +system tends to spend much of the available cpu time in kernel mode even if +there are only a few active users. +.le +.ls [2] +Compare the 2USER and 4USER timings with those for the UNIX 11/750. This +benchmark is characteristic of the two systems. No page faulting was evident +on the VMS 11/750 during the multiuser benchmarks. It took much longer to +run the 4USER benchmark on the VMS 750, as the set up time was much longer +once one or two other PLOTS jobs were running. The UNIX machine, on the other +hand, seemed almost as fast (or as slow) as usual, even with the PLOTS jobs +running on the other terminals. +.le +.ls [4] +The high value of the IPC bandwidth for VMS is due to the use of shared +memory. Mailboxes were considerably slower and are no longer used. +.le +.ls [5] +The foreign task interface uses mailboxes to talk to a DCL run as a subprocess +and should be considerably faster than it is. It is slow at present due to +the need to call SET MESSAGE before and after the user command to disable +pointless DCL error messages having to do with logical names. +.le + +.bp +.sh +AOSVS/IRAF V2.5, AOSVS 7.54, Data General MV 10000 (solpl) +.br +24Mb, 2-600 Mb ARGUS disks and 2-600 Mb KISMET disks +.br +17 April 1987, Skip Schaller, Steward Observatory, University of Arizona + +.nf +\fBBenchmark CPU CLK Size Notes\fR + (sec) (m:ss) +CLSS 2.1 0:14 [1] +MKPKGV 9.6 0:29 +MKPKGC n/a 3:43 +MKHDB 6.4 0:25 +IMADDS 1.5 0:06 512x512x16 +IMADDR 1.6 0:08 512x512x32 +IMSTATR 4.8 0:07 512x512x32 +IMSHIFTR 39.3 0:47 512x512x32 +IMLOAD 3.1 0:08 512x512x16 [2] +IMLOADF 0.8 0:06 512x512x16 [2] +IMTRAN 2.9 0:06 512x512x16 +SUBPR n/a 0:36 10 conn/discon 3.6 sec/proc +IPCO 0.4 0:03 100 getpars +IPCB 0.9 0:07 1E6 bytes 142.9 Kb/sec +FORTSK n/a 0:17 10 commands 1.7 sec/cmd +WBIN 1.7 0:56 5E6 bytes 89.3 Kb/sec [3] +RBIN 1.7 0:25 5E6 bytes 200.0 Kb/sec [3] +RRBIN 0.5 0:27 5E6 bytes 185.2 Kb/sec [3] +WTEXT 12.7 0:25 1E6 bytes 40.0 Kb/sec [3] +RTEXT 8.4 0:13 1E6 bytes 76.9 Kb/sec [3] +CSTC 0.0 0:00 5E6 bytes [4] +WSTC 1.9 0:11 5E6 bytes 454.5 Kb/sec +RSTC 1.5 0:11 5E6 bytes 454.5 Kb/sec +RRSTC 0.1 0:10 5E6 bytes 500.0 Kb/sec +NWBIN 2.0 1:17 5E6 bytes 64.9 Kb/sec [5] +NRBIN 2.1 2:34 5E6 bytes 32.5 Kb/sec +NWNULL 2.0 1:15 5E6 bytes 66.7 Kb/sec +NWTEXT 15.1 0:41 1E6 bytes 24.4 Kb/sec +NRTEXT 8.7 0:55 1E6 bytes 18.2 Kb/sec +PLOTS n/a 0:09 10 plots 0.9 sec/PROW +2USER n/a 0:12 +4USER n/a 0:20 +.fi + + +Notes: +.ls [1] +The CLSS given is for a single user on the system. With one user already +logged into IRAF, the CLSS was 0:10. +.le +.ls [2] +These benchmarks were measured on the CTI system, an almost identically +configured MV/10000, with an IIS Model 75. +.le +.ls [3] +I/O throughput depends heavily on the element size of an AOSVS file. For +small element sizes, the throughput is roughly proportional to the element +size. I/O throughput in general could improve when IRAF file i/o starts +using double buffering and starts taking advantage of the asynchronous +definition of the kernel i/o drivers. +.le +.ls [4] +These static file benchmarks are not yet official IRAF benchmarks, but are +analogous to the binary file benchmarks. Since they use the supposedly +more efficient static file driver, they should give a better representation +of the true I/O throughput of the system. Since these are the drivers used +for image I/O, they represent the I/O throughput for the bulk image files. +.le +.ls [5] +The remote node used for the network tests was taurus, a SUN 3-160 +running SUN/UNIX 3.2. The network protocol used was TCP/IP. +.le + +.bp +.sh +AOSVS/IRAF V2.5, Data General MV 8000 (CTIO La Serena system) +.br +5Mb memory (?), 2 large DG disks plus 2 small Winchesters [1] +.br +17 April 1987, Doug Tody, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes\fR + (sec) (m:ss) +CLSS n/a 0:28 [2] +MKPKGV n/a 2:17 +MKPKGC n/a 6:38 +MKHDB 13.1 0:57 +IMADDS 2.9 0:12 512x512x16 +IMADDR 3.1 0:17 512x512x32 +IMSTATR 9.9 0:13 512x512x32 +IMSHIFTR 77.7 1:31 512x512x32 +IMLOAD n/a +IMLOADF n/a +IMTRAN 5.69 0:12 512x512x16 +SUBPR n/a 1:01 10 conn/discon 6.1 sec/proc +IPCO 0.6 0:04 100 getpars +IPCB 2.1 0:13 1E6 bytes 76.9 Kb/sec +FORTSK n/a 0:31 10 commands 3.1 sec/cmd +WBIN 5.0 2:41 5E6 bytes 31.1 Kb/sec +RBIN 2.4 0:25 5E6 bytes 200.0 Kb/sec +RRBIN 0.8 0:28 5E6 bytes 178.6 Kb/sec +WTEXT 24.75 0:57 1E6 bytes 17.5 Kb/sec +RTEXT 23.92 0:30 1E6 bytes 33.3 Kb/sec +NWBIN n/a +NRBIN n/a +NWNULL n/a +NWTEXT n/a +NRTEXT n/a +PLOTS n/a 0:16 10 plots 1.6 sec/PROW +2USER n/a 0:24 10 plots 2.4 sec/PROW +4USER +.fi + + +Notes: +.ls [1] +These benchmarks were run with the disks very nearly full and badly +fragmented, hence the i/o performance of the system was much worse than it +might otherwise be. +.le +.ls [2] +The CLSS given is for a single user on the system. With one user already +logged into IRAF, the CLSS was 0:18. +.le + +.bp + . +.sp 20 +.ce +APPENDIX 2. IRAF VERSION 2.2 BENCHMARKS +.ce +March 1986 + +.bp +.sh +UNIX/IRAF V2.2 4.2BSD UNIX, VAX 11/750+FPA RA81 (lyra) +.br +CPU times are given in seconds, CLK times in minutes and seconds. +.br +Saturday, 22 March, D. Tody, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes \fR + (user+sys) (m:ss) + +CLSS 06.8+04.0 0:13 +MKPKGV 24.5+26.0 1:11 +MKPKGC 160.5+67.4 4:33 +MKHDB 25.1+? 0:41 +IMADDS 3.3+? 0:08 512x512x16 +IMADDR 4.4 0:15 512x512x32 +IMSTATR 23.6 0:29 512x512x32 +IMSHIFTR 116.3 2:14 512x512x32 +IMLOAD 9.6 0:15 512x512x16 +IMLOADF 3.9 0:08 512x512x16 +IMTRAN 9.8 0:16 512x512x16 +SUBPR - 0:28 10 conn/discon 2.8 sec/proc +IPCO 1.3 0:08 100 getpars +IPCB 2.5 0:16 1E6 bytes 62.5 Kb/sec +FORTSK 4.4 0:22 10 commands 2.2 sec/cmd +WBIN 4.8 0:23 5E6 bytes 217.4 Kb/sec +RBIN 4.4 0:22 5E6 bytes 227.3 Kb/sec +RRBIN 0.2 0:20 5E6 bytes 250.0 Kb/sec +WTEXT 37.2 0:43 1E6 bytes 23.2 Kb/sec +RTEXT 32.2 0:37 1E6 bytes 27.2 Kb/sec +NWBIN 5.1 2:01 5E6 bytes 41.3 Kb/sec +NRBIN 8.3 2:13 5E6 bytes 37.6 Kb/sec +NWNULL 5.1 1:55 5E6 bytes 43.5 Kb/sec +NWTEXT 40.5 1:15 1E6 bytes 13.3 Kb/sec +NRTEXT 24.8 2:15 1E6 bytes 7.4 Kb/sec +PLOTS - 0:25 10 plots 2.5 clk/PROW +2USER - 0:43 +4USER - 1:24 +.fi + + +Notes: +.ls [1] +All cpu timings from MKHDB on do not include the "system" time. +.le +.ls [2] +4.3BSD UNIX, due out shortly, reportedly differs from 4.2 mostly in that +a number of efficiency improvements have been made. These benchmarks will +be rerun as soon as 4.3BSD becomes available. +.le +.ls [3] +In UNIX/IRAF V2.2, IPC communications are implemented with pipes which +are really sockets (a much more sophisticated mechanism than we need), +which accounts for the relatively low IPC bandwidth. +.le +.ls [4] +The remote node used for the network tests was aquila, a VAX 11/750 running +4.2 BSD UNIX. The network protocol used was TCP/IP. +.le +.ls [5] +The i/o bandwidth to disk should be improved dramatically when we implement +the planned "static file driver" for UNIX. This will provide direct, +asynchronous i/o for large preallocated binary files which do not change +in size after creation. The use of the global buffer cache by the UNIX +read and write system services is the one major shortcoming of the UNIX +system for image processing applications. +.le + +.bp +.sh +VMS/IRAF V2.2, VMS V4.3, VAX 11/750+FPA RA81/Clustered (vela) +.br +Wednesday, 26 March, D. Tody, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes \fR + (user+sys) (m:ss) + +CLSS 14.4 0:40 +MKPKGV 260.0 6:05 +MKPKGC - 4:51 +MKHDB 40.9 1:05 +IMADDS 6.4 0:10 512x512x16 +IMADDR 6.5 0:13 512x512x32 +IMSTATR 15.8 0:18 512x512x32 +IMSHIFTR 68.2 1:17 512x512x32 +IMLOAD 10.6 0:15 512x512x16 +IMLOADF 4.1 0:07 512x512x16 +IMTRAN 14.4 0:20 512x512x16 +SUBPR - 1:03 10 conn/discon 6 sec/subpr +IPCO 1.4 0:06 100 getpars +IPCB 2.8 0:07 1E6 bytes 143 Kb/sec +FORTSK - 0:35 10 commands 3.5 sec/cmd +WBIN (ra81)Cl 6.7 0:20 5E6 bytes 250 Kb/sec +RBIN (ra81)Cl 5.1 0:12 5E6 bytes 417 Kb/sec +RRBIN (ra81)Cl 1.8 0:10 5E6 bytes 500 Kb/sec +WBIN (rm80) 6.8 0:17 5E6 bytes 294 Kb/sec +RBIN (rm80) 5.1 0:13 5E6 bytes 385 Kb/sec +RRBIN (rm80) 1.8 0:09 5E6 bytes 556 Kb/sec +WTEXT 65.6 1:19 1E6 bytes 13 Kb/sec +RTEXT 32.5 0:34 1E6 bytes 29 Kb/sec +NWBIN (not available) +NRBIN (not available) +NWNULL (not available) +NWTEXT (not available) +NRTEXT (not available) +PLOTS - 0:24 10 plots +2USER - 0:43 +4USER - 2:13 response was somewhat erratic +.fi + + +Notes: + +.ls [1] +The interactive response of this system seemed to decrease markedly either +when it was converted to 4.x VMS or when it was clustered with our 8600. +In interactive applications which involve a lot of process spawns and other +system calls, the system tends to spend about half of the available cpu time +in kernel mode even if there are only a few active users. These problems +are much less noticeable on an 8600 or even on a 780, hence one wonders if +VMS has perhaps become too large and complicated for the relatively slow 11/750, +at least when used in a VAX-cluster configuration. +.le +.ls [2] +Compare the 2USER and 4USER timings with those for the UNIX 11/750. This +benchmark is characteristic of the two systems. No page faulting was evident +on the VMS 11/750 during the multiuser benchmarks. It took much longer to +run the 4USER benchmark on the VMS 750, as the set up time was much longer +once one or two other PLOTS jobs were running. The UNIX machine, on the other +hand, seemed almost as fast (or as slow) as usual, even with the PLOTS jobs +running on the other terminals. +.le +.ls [3] +The RA81 was clustered with the 8600, whereas the RM80 was directly connected +to the 11/750. +.le +.ls [4] +The high value of the IPC bandwidth for VMS is due to the use of shared +memory. Mailboxes were considerably slower and are no longer used. +.le +.ls [5] +The foreign task interface uses mailboxes to talk to a DCL run as a subprocess +and should be considerably faster than it is. It is slow at present due to +the need to call SET MESSAGE before and after the user command to disable +pointless DCL error messages having to do with logical names. +.le + +.bp +.sh +VMS/IRAF V2.2, VMS V4.3, VAX 8600 RA81/Clustered (draco) +.br +Saturday, 22 March, D. Tody, NOAO/Tucson + +.nf +\fBBenchmark CPU CLK Size Notes \fR + (user+sys) (m:ss) + +CLSS 2.4 0:08 +MKPKGV 48.0 1:55 +MKPKGC - 1:30 +MKHDB 7.1 0:21 +IMADDS 1.2 0:04 512x512x16 +IMADDR 1.5 0:08 512x512x32 +IMSTATR 3.0 0:05 512x512x32 +IMSHIFTR 13.6 0:20 512x512x32 +IMLOAD 2.8 0:07 512x512x16 via TCP/IP to lyra +IMLOADF 1.3 0:07 512x512x16 via TCP/IP to lyra +IMTRAN 3.2 0:07 512x512x16 +SUBPR - 0:26 10 conn/discon 2.6 sec/proc +IPCO 0.0 0:02 100 getpars +IPCB 0.3 0:07 1E6 bytes 142.9 Kb/sec +FORTSK - 0:13 10 commands 1.3 sec/cmd +WBIN (RA81)Cl 1.3 0:13 5E6 bytes 384.6 Kb/sec +RBIN (RA81)Cl 1.1 0:08 5E6 bytes 625.0 Kb/sec +RRBIN (RA81)Cl 0.3 0:07 5E6 bytes 714.0 Kb/sec +WTEXT 10.7 0:20 1E6 bytes 50.0 Kb/sec +RTEXT 5.2 0:05 1E6 bytes 200.0 Kb/sec +NWBIN 1.8 1:36 5E6 bytes 52.1 Kb/sec +NRBIN 8.0 2:06 5E6 bytes 39.7 Kb/sec +NWNULL 2.5 1:20 5E6 bytes 62.5 Kb/sec +NWTEXT 6.5 0:43 1E6 bytes 23.3 Kb/sec +NRTEXT 5.9 1:39 1E6 bytes 10.1 Kb/sec +PLOTS - 0:06 10 plots 0.6 sec/PROW +2USER - 0:08 +4USER - 0:14 +.fi + + +Notes: + +.ls [1] +Installed images were not used for these benchmarks; the CLSS timing +should be slightly improved if the CL image is installed. +.le +.ls [2] +The image display was accessed via the network (IRAF TCP/IP network interface, +Wollongong TCP/IP package for VMS), with the IIS image display residing on +node lyra and accessed via a UNIX/IRAF kernel server. The binary and text +file network tests also used lyra as the remote node. +.le +.ls [3] +The high value of the IPC bandwidth for VMS is due to the use of shared +memory. Mailboxes were considerably slower and are no longer used. +.le +.ls [4] +The foreign task interface uses mailboxes to talk to a DCL run as a +subprocess and should be considerably faster than it is. It is slow at +present due to the need to call SET MESSAGE before and after the user +command to disable pointless DCL error messages having to do with +logical names. +.le +.ls [5] +The cpu on the 8600 is so fast, compared to the fairly standard VAX i/o +channels, that most tasks are i/o bound. The system can therefore easily +support several heavy users before much degradation in performance is seen +(provided they access data stored on different disks to avoid a disk seek +bottleneck). This is borne out in the 2USER and 4USER benchmarks shown above. +The cpu did not become saturated until the fourth user was added in this +particular benchmark. +.le |