VoClient Expanded Tasking Interface ------------------------------------ Current Interface -------------------------- There is a single VOClient "VO" package consisting of a set of tasks implemented as library functions indexed as an "apps" array of function pointers. vo_runTask forks a subprocess to execute each task. Parameters are passed Unix-style as strings (argc,argv[]). A single result object is returned as a binary byte-counted blob. Client: status = vo_runTask (method, apps[], argc, argv[], *len, **result) Task: status = vo_setResultFromInt (value, *len, **data) status = vo_setResultFromReal (value, *len, **data) status = vo_setResultFromString (value, *len, **data) This is ok, a straightforward mapping of the original host CLI task interface into a programmatic API. However with a little more work we can generalize this to make the interface self-describing, allowing generic tasking code to be implemented (all the way up the client language layer eg Python) and allow new packages/tasks to be addded. Expanded Interface -------------------------- This is an attempt to define the simplest possible interface which is still general enough to support the package/task paradigm, providing a self describing capability to allow generic/reusable software to be written to manipulate and run tasks. The main simplification is that a "package" is implemented as an executable (a single self contained file) which may contain multiple tasks and which can be queried to determine its capabilities. This limits "packages" to implementations which can be self contained enough to be runnable from a single runtime file, which is adequate for now to support the VOClient CLI tasks and which can be generalized later if necessary. A "package" is a precompiled executable with a defined API which can be queried with standard CLI arguments to get package metadata as well as to run the individual tasks. Metadata queries include user help as in "-help", plus programatic information such as the package name and description, list of tasks, task interfaces, builtin help, etc. "Package" executables are installed in a standard directory which can be examined by the tasking layer to determine what packages are available. The file names of package executables follow a simple naming convention so that they can be easily referenced at runtime. Client Interface: xml = vo_pkgList () # Available packages xml = vo_taskList (pkgName) # Tasks in package xml = vo_taskParams (pkgName, taskName) # Input param set defs tp = vo_taskInit (pkgName, taskName) # Prepare to run a task vo_taskSetStringParam (tp, sval) # Set input parameters vo_taskSetBoolParam (tp, sval) vo_taskSetIntParam (tp, ival) vo_taskSetRealParam (tp, dval) vo_taskSetCallback (tp, &func) # Set callback for output param rp = vo_taskExecuteSync (tp) # Execute synchronous; returns # output pset pointer vo_taskExecuteAsync (tp) # Execute async rp = vo_taskWait (tp) # Wait for task, get RP status = vo_taskCheckError (tp, *msg) # Check for any error posted # to TP or RP int = vo_taskOutputParamsCount (rp) # Access output params vo_taskGetOutputParamInfo (rp, pnum, *name, *type, *encoding, *description) # pnum used if paramName=null *void = vo_taskGetOutputParamValue (rp, paramName, pnum) *char = vo_taskGetStringOutputParam (rp, paramName, pnum) ival = vo_taskGetBoolOutputParam (rp, paramName, pnum) ival = vo_taskGetIntOutputParam (rp, paramName, pnum) dval = vo_taskGetRealOutputParam (rp, paramName, pnum) vo_taskClose (tp) # Free task resources Notes: 1. vo_pkglist essentially just lists the valid package files in the package directory. Simple since each package is a single file in this scheme. 2. vo_taskInit sets a task context and creates a default/empty param list. The client sets the params it cares about. When the task is executed the params are serialized in whatever way the tasking layer wants to do it internally, e.g. argc/argv. 3. Tasks can execute either sync or async. In async mode the client may post a callback handler for output parameters. This is called during task execution whenever any output parameter (of any name or type) is set. We can have standard system params stdout (dump text to stdout), status (running, done), warning (like stderr), error (task aborted), etc. It is up to the task what parameters to output. Custom parameters, normally of a standard type/encoding, are used to return data. 4. In sync task mode the client waits for completion and gets a pointer to an output param set which it can walk to process the output params. Output params can also be retrieved by name if the client knows what params it expects back. 5. The client should call vo_taskClose when done to free up task resources such as output parameters. Otherwise they are retained indefinitely. 6. We should define standard values for the param TYPE and ENCODING e.g., "bool", "int", "image", "fits", "votable", etc.), however he task may return custom object types or encodings as well which he client may or may not recognize. Task Interface: # Called by the task itself at runtime tp = vo_taskStart () char *vo_taskGetPackageName (tp) char *vo_taskGetTaskName (tp) *char = vo_taskGetStringParam (tp, paramName) ival = vo_taskGetBoolParam (tp, paramName) ival = vo_taskGetIntParam (tp, paramName) dval = vo_taskGetBoolParam (tp, paramName) status = vo_setIntOutputParam (tp, paramName, ival) status = vo_setRealOutputParam (tp, paramName, dval) status = vo_setStringOutputParam (tp, paramName, sval) status = vo_setOutputParam (tp, paramName, paramType, paramEncoding, *len, **void) tp = vo_taskEnd (tp) Notes: 1. Task should call vo_taskStart initially to get a runtime context, and vo_taskEnd when done to free resources, flush any output, and so forth. 2. When a task is run in a "connected" (remote) fashion it is passed a keyword table of parameter values, via argc/argv or whatever. True host level, CLI (argc/argv) mode can be provided as well in which case all the -help etc. generics can be provided by the task main (container). Defaulting can be performed task-side if provided. 3. Input parameters can be retrieved by name (do we need to walk a list as well? probably not in this case). 4. Output parameters are flushed through to the client in each setOutputParam call, allowing interactive output at runtime. A single output parameter may be set multiple times, e.g., for stdout, warning, status, etc. this could be normal. The client tasking code accumulates output parameters in an output param set during execution. Whether or not parameters are passed on to a client callback is up to the client. Whether or not a task executes directly in the client process or in a separate process is transparent to the client; normally tasks will execute in a subprocess. The main limitation here is the assumption that a "package" is input to the system as a file. This is simple enough for C code where tasks are just functions linked into a single executable with a common main. In the simplest case there is no support for things like task dependencies. However the single file paradigm can be extended in the future should we need to do so.