path: root/vendor/voclient/doc/Test.iraf

								9/1/11

			IRAF/VO Integration Testing


Test Machine, Sample Data Files/URLs
====================================

    The test machine to be used is vaotest4.tuc.noao.edu (140.252.6.84)
which is a 32-bit Ubunto 10.10 Linux system.  IRAF is installed for use
by all users, any needed support programs (STILTS, Topcat, etc) are all
available from the /usr/local/lib directory.  Logins for testing should
be done using the 'iraftest' account on the machine (contact me for the
passwd).

    A directory of sample data files is inlcude in the ~iraftest/data
directory and may also be accessed at 

	http://iraf.noao.edu/votest/

Sample data includes:

    sia.xml		Sample SIA service result from MAST
    sia2.xml		    Same result but with URLs pointing to NOAO
    h_n5194-*.fits	    FITS files in the SIA file

    LGA_I_m32.xml	Another SIA service result from IPAC

    sif.fits		Standard dev$pix test image as a Simple FITS
    mef.fits		Standard dev$pix test image as a Multi-Extn FITS

    usno-b.xml		Cone search table result from USNO-B
    usno-b.fits		Same file but as a FITS bintable

Prefixing the above URL to these files should produce results the same
as if the local file were accessed.  Using the e.g. 'sia2.xml' file as
an SIA result will have faster download times from the vaotest4 machine
but is otherwise the same.



VOTable Support in Tasks
========================

    The first phase of development adds support for VOTable documents to
any task expecting tabular data in one of the currently supported formats
(i.e. FITS tables, ascii tables, or ".tab" files).  Typically these are
tasks in the NTTOOLS package (formerly the TABLES.TTOOLS package) but also
includes tasks in the DIGIPHOT package.  The goal is that any task example
will work if the input file is a VOTable, for instance the commands

	cl> tlcol usno-b.xml
	cl> tlcol usno-b.fits

should produce identical results.

    Output tables cannot be written in VOTable format (yet), their use
is strictly input at this point.  In the final VO package a task will
be provided that can be used to write tables out to VOTable.


Known Issues:

    - Req 1.1 cannot yet be satisfied due to the current implementation of
	the internal conversion from XML to FITS.  This now uses STILTS
	as a stopgap, the code will be replaced with a native CFITSIO
	conversion that will carry the PARAM/INFO fields into the PHU.
	This also explains the brief delay when first opening a votable
	(and imposes a dependency on STILTS that must be removed).

    - Req 1.2.1 refers to selection of a table column in the image list
	enhancements.  Addressing a column by a variety of XML attributes 
	is possible with some enhancements to the nttools$lib/tctexp.x
	column template code but can't be done until the new conversion
	code is in place.


URL Support in Tasks
====================

    URL support is not a specific deliverable of the VO integration, but is
needed for the "seamless" aspects of the project (e.g. SAMP messages tend
to pass around URLs, data access references are URLs, etc).  Thus,
modifications were made to each of the 'open' procedures used for image,
table and general file I/O to permit the argument to be an HTTP url.  The
first time a URL is accessed it will be downloaded and put into the general
file cache, thereafter it will be the cache file that is opened.  As with
VOTable support, URLs may be used for input arguments only (i.e.  a URL
used as an output parameter name will not POST the file to the URL, an
error will be printed instead).

     The caching mechanism means that a URL such as an access reference
(or service query) may be used repeatedly without hitting the server
multiple times, but it also means there is no clear method of determining
when a URL would return new a different result, and that modifying the
cache file on the desktop means it no longer exactly represents the URL
source.  In some cases this problem can be mitigated by setting a short
expiration time on the cache to force a refresh of the URL, otherwise we
put the burden on the user/developer to copy the URL file to a local name
for persistent use.

    Typical example command that would use a URL are things like

	cl> type http://iraf.noao.edu/index.html	    # files
	cl> imstat http://iraf.noao.edu/votest/dpix.fits    # images
	cl> tlcol http://iraf.noao.edu/vo/testusno-b.xml    # tables

Note that in some output what gets printed in the cached filename and not
the URL.


File Caching
------------

    A new FCACHE task is available in the main SYSTEM package for listing
and managing the file cache.  Similarly, a 'cache' logical variable defines
the location of the cache directory itself.  A few handy commands:

    cl> fcache init		# initialize the cache
    cl> fcache purge age=7	# remove files older than a week
    cl> fcache list		# list contents of cache$ and the src

See the help page for details.  Source for the FCACHE task is in 
system$fcache.x and the task itself can/should be tested to make sure
it manipulates the files properly.  Otherwise, the cache interface is
used by both the VOTable and URL support code to hide the details of 
how the foreign files/formats are managed by the system.



Image List Expansion
--------------------

    The image list interface in IRAF was enhanced to allow for greater
selection capabilities and to expand the concept of what represents and
"image list" to include MEF files (the file itself is a list of the image
extensions it contains) and VOTables (the accessReference column is
inherently a list of URLs representing FITS files).  Selectability is a
key new component since it is at the heart of the DAL philosophy of
query-then-access.

    Not all features of the image list expansion have yet been implemented,
in particular, selection-by-column-value when using VOTables is also 
awaiting a proper VOTable conversion code.

    See iraf$sys/imio/imt/README for details.



Known Issues:

    - The URL mechanism must also be made to support file:// prefixes
    - The FCACHE task help page is not yet completed .....



SAMP Integration
================

    SAMP interoperability is one of the promised deliverables and is meant
to allow CL scripts/tasks to send SAMP messages to other SAMP-enabled apps,
and for the CL to receive messages that make it useful in a SAMP workflow.
This requires an extension to the currently recognized list of SAMP mtypes
to include the following:

    Mtype	      Args		Meaning
    -----	      ----		-------
    client.env.get    <var>         	get an environment variable
    client.env.set    <var> <value>     set an environment variable
    client.param.get  <param>       	get a parameter value
    client.param.set  <param> <value>   set a parameter value
    client.cmd.exec   <cmd>  		execute a command

These mtypes are supported by the SAMP library used by the system and so 
the test application in the library can be used to exercise the IRAF
implementation.  In this case, a <param> argument is expected to be in 
the format "<task>.<param>', e.g. as "implot.image" to refer to the 'image'
parameter of the 'implot' task.  Similarly, the <cmd> argument is a string
expected to contain a fully valid iraf command string.

    A valid SAMP Hub must be running for messages to be sent or received.
We rely on either an external standalone Hub being available or that some
other SAMP app has a builtin Hub running.  Demonstrating the requirements
have been met for SAMP interoperability should be easy to do.


Sending Messages from IRAF
--------------------------

    Sending messages is done using the following builtin CL functions:

	  sampLoadImage  (file|url [, recip [, name [, tag ]]])
	   sampLoadFITS  (file|url [, recip [, name [, tag ]]])
	sampLoadVOTable  (file|url [, recip [, name [, tag ]]])

These functions map directly to the most commonly used SAMP mtypes,
additional (or generalized) functions could be added but it isn't clear
these are necessary for an IRAF application.  Examples would include:

    1) Broadcast the message to all subscribed clients:

      cl> = sampLoadImage ("http://iraf.noao.edu/votest/sif.fits")

    2) Send the message to a named client:

      cl> = sampLoadImage ("data$foo.fits", "aladin")

    3) Load the image with a given name:

      cl> = sampLoadFITS ("/data/image001.fits", "aladin", "image1")

Each function returns either an "ok" string or an error message.



Receiving Messages in IRAF
--------------------------

    The VOCL will connect to a SAMP Hub on startup and is already 
configured to handle the above 5 mtypes.  The env and parameter
set/get methods don't cause any output to the terminal window, however
a client.cmd.exec message will echo the command string to the prompt
line just as if the user had typed it.

    The intent of receiving messages like this is that an IRAF command
can be invoked from SAMP, however no attempt is currently made to use
SAMP messaging to drive the application itself (e.g to allow subsequent
SAMP messages to provide cursor input).

    Dummy handlers are currently in place to accept other mtypes but 
just silently consumes the message.  The last step is to implement a
function such as

	sampHandler (mtype, cmd_string)

to the CL the allow a particular command string to be called when a
message of the given mtype is received.  This would allow users to post
commands they wish to call in response to e.g. an image.load.fits message.
Expect this functionality very soon.


Sending Test Messages
---------------------

The SAMP library test apps can be used to send specific messages to IRAF
for testing.  In particular, the 'send' command is installed in
/usr/local/bin and can be used as e.g.

	% send -r iraf client.cmd.exec imstat dev\$pix

where the "-r iraf" says the recipient is 'iraf' (the name broadcast by
the VOCL when it connects to the Hub), the 'client.cmd.exec' mtype should
be sent, and the remaining args make up the command itself (in this case,
a "imstat dev$pix" command -- note the '$' must be escaped from the unix
shell to be passed in this way).

    The '-h' flag will print a help summary for the 'send' command.  Source
code for the SAMP library used is in the 'libsamp' subdirectory of the 
iraftest account for inspection, see the libsamp/examples directory for
the send.c source.


Starting the VOCL
-----------------

    The VOCL (VO-enabled CL) is not yet configured to be the default CL
for the v2.16 release and so must be started using the alternative command

	% cl -vo

For the 'iraftest' account this has been alias to just the plain 'cl' 
command for convenience, a decision will be made following testing as
to whether this version is stable enough to be made the distribution
default.


Starting the SAMP Hub
---------------------

    In order to test either the sending/receiving of messages, a Hub
must be running on the machine.  This can be a builtin hub such as that
found in Topcat (installed on vaotest4), or using the standalone JSamp
Hub.  To start a JSAmp hub, in a separate window on vaotest4 type:

	% /usr/local/bin/hub
or	% /usr/local/bin/topcat

Note that in either case you'll need to have an X11 DISPLAY set.



Known Issues:

    - Discovery of a Hub needs to be automated, i.e. if there is no Hub
	running when a VOCL session starts, it is never searched for again.
	What we'd like is for the VOCL to watch for a Hub to appear and
	then connect automatically so subsequent commands in the same VOCL
	session can begin messaging.
    - Must still implement
   		sampShowRow () 	sampSelectRowList ()
   		sampPointAt () 	sampSpecLoad ()




-------------------------------------------------------------------------------

Automated Tests
===============

    The following is taken directly from the /iraf/iraf/vo/votest/README
file and is intended to explain how the automated test system works.  The
idea is that once the VOCL is started, the VOTEST package can be loaded
to run a series of regression tests.  These tests are comprised of whatever
scripts are available in the votest$tests directory (which has been made
publically writeable on the vaotest4 machine).  We expect that as the
VO/IRAF testing progresses, additional tests will be added to this directory.



			VO Test Package

    The VOTEST package allows users and developers to create simple scripts
that can be used to run a regression test on the package after changes to
system interfaces, test for specific problems to be sure they done recur,
and the ensure that the help examples continue to function as specified.

This directory contains the following:

	README			This file

	data/			Sample data directory
	tests/			Test script directory

	run_test.cl		Utility task to execute a single test
	test.cl			Main test script

	votest.cl		Standard package files
	votest.hd
	votest.men
	votest.par
	doc/			Task documentation directory
	

Test Script Usage:
------------------

    The package defines a single (visible) task call TEST with the following
usage:

	cl> test [<module>] [list] [verbose]

where
	module		Either 'all' or the name of a specific test module
	list		If set, list available tests only
	verbose		Print verbose output

If no parameters are specified, all tests are run.


To Add A New Test:
------------------

    Test scripts are placed in the 'tests' subdirectory and are expected to
follow the following conventions:

    1) A description of tests in a "module" is contained in a file 'module.men'
       and should contain something like

	    #  Module Tests -- Tests of the stuff needing testing

	           module_001 -- test 1 description
	           module_002 -- test 2 description
		       :	      :   :     :
	           module_NNN -- test N description

	Lines beginning with a '#' will be used to comment the tests being run.
 
    2) A module test script contains commands to be executes.  It *should*
       also set the "votest.descr" parameter to be some short description
       of the test, this text is printed when the test is run.  Test scripts
       should be named using the convention 'module_NNN.cl' where "module" is
       the common module name, "NNN" is some running number for the test, and
       the script is expected to have a ".cl" filename extension.

    3) A module test script that produces any form of output should have the
       expected output in a file called "module_NNN.out".  A test is deemed to
       have passed when a diff of this file with the output of the test script
       produces no output (i.e. no differences).

    4) A "test module" is comprised of all the above ".men", ".cl" and ".out" 
       files.  Typically tests will be grouped by some common these, e.g. a
       particular task or feature of the system to be tested.

The TEST task will automatically execute any scripts in the 'tests' directory,
minimizing the amount of changes required to add a new test.





-------------------------------------------------------------------------------


Requirements Summary:
=====================

VOTable Support:
  1.    Users shall be able to use a VOTable in places where tasks accept
        tabular data in other formats (ASCII files, FITS bintables, .tab 
	files, etc) for input.
  1.1   Users shall be able to access the <PARAM> and <INFO> elements of a
	VOTable as standard table header information.
  1.2   Users shall be able to select specific rows and/or columns of a
	VOTable using the existing task functionality.
  1.2.1 Users shall be able to identify a column in a VOTable by the 'id',
	'name' or 'ucd' attribute of a <FIELD> or by column number.
  1.3 	Users shall be able to select a column in a VOTable for use in tasks
	that expect a list of values.
  1.4 	VOTable Interface code will support the IVOA Standard specification
	of the VOTable format at time of release.


SAMP Interoperability	
  2.	Users shall be able to interoperate with other SAMP (or WebSAMP)
	enabled applications.
  2.1	Users shall be able to send messages to specific clients or
	broadcast to all available clients.
  2.2	Users shall be able to execute IRAF tasks and set/retrieve
	information in/from the IRAF environment via SAMP messaging from 
	clients that implement the required message types.