diff options
Diffstat (limited to 'vendor/voclient/voapps/README')
| -rw-r--r-- | vendor/voclient/voapps/README | 324 |
1 files changed, 324 insertions, 0 deletions
diff --git a/vendor/voclient/voapps/README b/vendor/voclient/voapps/README new file mode 100644 index 00000000..aea037b0 --- /dev/null +++ b/vendor/voclient/voapps/README @@ -0,0 +1,324 @@ + +============================= +Last Modified: Aug 10, 2012 +============================= + + + This directory contains the VOClient command-line applications. +Key files in this directory include: + + voApps.c The generic unix main() linked to all tasks + generic.c A template code for building VOClient apps + + voatlas.c The VOATLAS multi-wavelength image atlas task + vodata.c The VODATA general data query retrieval task + + voregistry.c The VOREGISTRY task to query/resolve Registry resources + + vosesame.c The VOSESAME name resolver + + vosamp.c The VOSAMP command-line SAMP tool + + votcnv.c The VOTCNV votable conversion tool + votget.a The VOTGET task to retrieve acrefs from a votable + votinfo.c The VOTINFO task to print information about a votable + votpos.c The VOTPOS task to extract positional cols from votables + votsort.c The VOTSORT task to sort a votable based on a column + votstat.c The VOTSTAT task to print colum statistics + + + Planned tasks Not Yet Implemented: + + votcat.c // VOTable Resource concatenator + votjoin.c // VOTable inner joins + votselect.c // Select rows/cols by index/expr + votsplit.c // Split multi-resource votables + + VOClientd // C-based minimal implementation of VOClient Daeomon + hub // C implementation of SAMP Hub + + voxmatch // Cross-compare local table and VO data + + vosput // Put files to a VOSpace + vosget // Get files from a VOSpace + vosmove // Move files/nodes between VOSpaces + voslist // List files/nodes in a VOSpace + vosdelete // Delete files/nodes in a VOSpace + + voskybot // List known moving objects in an image + + + +The 'lib' subdirectory contains support code for the applications. Of +particular interest is the 'lib/voTask.c' code that defines the tasking +interface used by the applications. + + +Task Interface +-------------- + + All VOClient apps are written as C procedures sharing a common signature: + + <task> (int argc, char **argv, size_t *len, void **result); + +where 'argc' and 'argv' have the usual meaning for the CLI argument vector, +'len' is the length of any returned object (or 0 if there is no return), +and 'result' is a pointer to the returned object in memory. The task() +procedure returns 0 for success, and a positive value to indicate an error +return (this may include a signal number for a crashed child process as +well as task-specific error return codes). + + Tasks share a common main() responsible only for passing in the +parameter options, executing the task as a connected subprocess (or by +calling the procedure directly if the VOAPP_CONNECTED environment variable +is defined, this is to aid in debugging) and for retrieving any returned +result object. This architecture ensures that any memory allocated or +threads spawned by the VOClient app do not interfere with an app using +the language API. + + Host Process App Sub-Process + +----------+ +----------------------+ + | C | parameters | | + | main() |--------------->| VOClient | + | or |<---------------| application code | + | Lang API | result | | + +----------+ +----------------------+ + +The application is spawned as a child process by the main() and the CLI +parameters are sent over IPC. Likewise, any return value from the app is +sent back via IPC, a host CLI tool would simply discard this result however +a language binding or other API would then have a pointer to the result +object it could pass up to the calling interface. The details of what can +be returned by a task is task-specific and the caller must be aware of the +options to be of use, for this discussion the returned 'result' is a memory +object of size 'len' given the above calling signature. + + The 'voApps.c' code contains the common main() function which normally +calls the vo_runTask() procedure to execute the task based on the task name +derived from a lookup table. (See lib/voTask.c for this procedure). + + + +Parameter Interface +------------------- + + VOClient apps are written to accept CLI/parameter options in a number +of formats suited for use as Unix tasks or from an API that may be better +suited to pass in "<param>=<value>" strings. The following formats for +option strings are supported by all tasks: + + -p param[+-] --param --param=value param=<value> + +For CLI tasks the following would be equivalent: + + % task -p # to set the 'p' option + % task --param # long-form of 'p' option + % task --param+ # if 'param' is a boolean option + % task --param=value # to set a specific value for param + % task --param value # to set a specific value for param + % task param=value # to set a specific value for param + +From an API one could imagine these same parameters being passed in to form +the required argument/parameter 'argv' vector as e.g. + + from VOClient import tasks as voc + status = voc.task ("param=value", ....) + +Accomodating a return value from the task would affect this sort of interface +(e.g. if the task were to return an in-memory FITS file instead of a status +code) but the idea is to support both standard CLI options as well as a p=v +format more appropriate for a tasking API interface. + + +Common Options/Parameters: + + All tasks(*) support a core set of options, namely + + -h,--help Print a task usage summary and examples + -%,--test <input> Run unit tests (some tasks require <input>) + -r,--return <opt> Return optional result object + +All other options are task-specific however an effort has been made to make +these consistent across tasks (e.g. a "--input" for input files, "--fmt" for +output VOTable formats, "--samp" for SAMP functionality, etc). + + The "--help" (or "-h") option to each task should provide enough runtime +documentation to get started with using a task, in particular the examples +given in the help are meant to always be valid. + + The "--test" (or "-%") option is a way to run unit tests on a particular +task, primarily based on the examples given in "--help". Depending on the +task an input file/arguments of some form must be provided for the tests to +work. For example, unit tests of VOTable tools can be run using votables +returned from various data services to ensure proper behavior for all VO +resources or to test operation against various (non)compliant files. This +same mechanism can be used to build a regression test suite against a static +list of VO Resources or standardized data files. + + The "--return" option is task-specific and its use will depend largely +on the API implementing it. A detailed description of the return objects +will be provided in the task/API documentation. + + + (*) Exceptions as of this writing are the original VO-CLI tasks + VODATA/VOSESAME/VODIRECTORY, these will be implemented before + release. + + + +SAMP Interoperability +--------------------- + + The VOSAMP task a CLI tool for sending messages to other SAMP-enabled +applications, either as a one-off CLI execution or from within a scripted +environment. To hide the SAMP details, the CLI interface is written to be +more user-friendly, e.g. the options(*) on the command line are (in part): + + + load <url> load the named image/table file + loadImage <url> load the named image + loadVOTable <url> load the named VOTable + loadFITS <url> load the named FITS bintable + showRow [<url>] <row> [tblId] highlight specified row + selectRows [<url>] <rows> [tblId] select specified rows + bibcode <bibcode> load the named bibcode + + exec <cmd> execute a client command + pointAt <ra> <dec> point at given coords + setenv <name> <value> set an environment value + getenv <name> get an environment value + setparam <name> <value> set a parameter value + getparam <name> get a parameter value + + send <mtype> [<args> ...] generalized <mtype> message send + + +where "<url>" might be an actual HTTP reference, a local file path/name, or +a 'file://' URI. The commands themselves are cases insensitive and the +VOSAMP help page should be consulted for details on sending non-standard +messages using the 'send' option for general messages (i.e. those messages +that don't conform to an IVOA-supported 'mtype' but which might be used in +a custom workflow). When VOSAMP is called using an API these commands are +passed through the argument string an provide a trivial high-level method +of sending SAMP messages without requiring the app to explicty connect to +the Hub (receiving messages will still normally require the app to register +itself in order to handle messages using the application's callbacks). + + A "normal" SAMP-enabled application will establish a connection to the +local desktop apps once and then process subsequent messages for the life +of the application. In a CLI tool, this model introduces the overhead of +the initial connection (several seconds) on each command, an effect which +compounds when a CLI tool is used from within a script that might be +processing many tens-to-thousands of messages as it is used in some +user-defined workflow. To avoid this overhead, the VOSAMP task will +default to become a bacakground 'proxy' service that lingers after the +initial command is executed, i.e. in the same way a 'sudo' command won't +require a password for each command for some time after the initial call, +VOSAMP will background itself after the first command to maintain its +connection to SAMP Hub, subsequent calls to VOSAMP will simply forward the +command from the CLI to this proxy to make use of the existing SAMP +connection and execute as quickly as any other persistant SAMP task. + +(*) SSA message data are not yet implemented. + + +Inter-Desktop Messaging: + + VOSAMP reads its command either from stdin, a named command file (e.g. +as from a VOSAMP-shell interpreter), or from a socket created when the proxy +process is created. This socket is created on an inet port that may be +visible to the whole internet, or on a private port restricted by local +system administrators to trusted clients (e.g. port 8080 if that is a +general service provided by a site, or the default port 4000 for sites +willing to open a firewall hole). + + An example sequence of SAMP commands would be something like + + % vosamp listClients + % vosamp loadVOTable foo.xml + +where the first command establishes a SAMP connection, and the second would +forward the CLI command to the (already running) proxy client without +establishing a new-application context. + + In this model, a proxy VOSAMP app reading from an inet socket opens +the possibility of using this proxy from a remote host that can send a +command from a trusted machine. For example, + + On host A: + % vosamp start # start proxy client on host A (140.252.1.86) + + On host B: + % vosamp --proxy=140.252.1.86:4000 loadVOTable foo.xml + +where the '140.252.1.86' is the machine running the intial VOSAMP task, +'4000' is the inet port that proxy is reading commands, and the remainder +of the commandline are args to be passed thru as it they were issued from +the local host. Local data (e.g. the 'foo.xml' file) is sent to the remote +before executing the command, file:// URL's are rewritten so the file +reference remains valid on the remote machine (http:// URLs are unchanged). + + +SAMP Session Manager: (In development) + + By using the proxy client, we can send commands to a VOSAMP application +from a remote machine, but this is not quite the same as federating two (or +more) desktops in a shared session. We are also limited by the ability to +bypass firewalls in order to connect to the proxy client. The solution then +is to have a public "session manager" service that serves as an alternate +input source to the proxy client and will acto to forward message from one +machine to the others. For example, + + +-------------+ + | Session Mgr | + +-------------+ + ^ + / \ + / \ + +-------------------------+ +-------------------------+ + | Topcat \ | | / Topcat | + | Hub - VOSAMP | | VOSAMP - Hub | + | Aladin / | | \ IRAF | + +-------------------------+ +-------------------------+ + Host A Host B + +Since the session manager is on a public host, the VOSAMP on each machine +makes an outgoing client connection and sets up that socket to receive +commands and subscribes to all message types so that SAMP messages received +on the local machine can be forwarded back to the session manager and then +on to other machines in the session. For example, the Aladin on Host A +broadcasts an image.load.FITS message, the VOSAMP on Host forwards the +message to the session manager than then passes it on to the VOSAMP on +Host B for rebroadcast. In this way the message is seen on both desktops +transparently. When local data is being used, this is uploaded to the +session manager for storage in a web-accessible area, the forwarded message +is then rewritten to use the URL and is accessed only when needed. + + Sessions are created/joined with no special setup required, e.g. + + % vosamp --session=foo e.g. commandline tool +or + voc.vosamp ("session=foo") e.g. from language API + +The VOSAMP in this case would contact the session manager to join the list +of machines in session 'foo', this session would be created if it did not +already exist and sessions quietly end when the VOSAMP proxies timeout due +to inactivity or are shut down explicitly. There is no need for formal +security since it is up to the parties in the session to agree on and share +the session name of their choosing. Applications written using the language +bindings can participate in sessions by simply calling the VOSAMP task to +create the proxy regardless of whether it is used explicitly for messaging. + + If the VAO were to operate a public session manager service, it would +trivial to log its use for reporting to the funding agencies, and would +meet the mandate of VAO providing tools for community use. Details of +the communication protocol could be written as an IVOA note to allow other +implementations to make use of the service, or it could be formalized into +a next version of the SAMP protocol itself. + +[Status: Proxy clients for use on the desktop and between trusted machines + is implemented, development of the session manager and extensions to + VOSAMP will be ready for demonstration at the Seattle meeting. (8/11/12)] + + + |