path: root/noao/digiphot/photcal/doc/obsfile.hlp

.help obsfile Apr94 noao.digiphot.photcal
.ih
NAME
obsfile -- prepare an observations file from a list of user created 
photometry files containing observations of objects in one or more fields
.ih
USAGE
obsfile photfiles incolumns idfilters imsets observations
.ih
PARAMETERS
.ls photfiles
A list of text files containing the image names or an image id, x-y positions,
magnitudes, magnitude errors, airmasses, filter ids, exposure times, and time
of observation of all the measured objects. \fIPhotfiles\fR are assumed to be
the output of the user's own digital photometry program. All the files in the
list must have the format specified by \fIincolumns\fR.
.le
.ls incolumns
A list of ten numbers separated by commas or whitespace specifying which
columns in \fIphotfiles\fR contain the following fields: the image name or id,
x coordinate, y coordinate, filter id, exposure time, airmass, time of
observation, instrumental magnitude, magnitude error, and object id
respectively.  
.le
.ls idfilters
The list of filter ids separated by whitespace or commas which define a
complete observation. The filter ids must correspond to those in
\fIphotfiles\fR.
.le
.ls imsets
The name of the text file which lists the observations of each field, assigns a
name to each field, and tells OBSFILE which images belong to the same
observation of that field. Only observations corresponding to the images
specified in \fIimsets\fR will be extracted from \fIphotfiles\fR. Observations
are listed in \fIimsets\fR, 1 observation per line with the field name in
column 1, a colon in column 2, followed by, in filter order and separated by
whitespace, the names of the images of that field belonging to that
observation. The format of \fIimsets\fR is described in detail below.
.le
.ls observations
The output observations file suitable for input to FITPARAMS or
EVALFIT/INVERTFIT.
.le
.ls wrap = yes
Format the output observations file for easy reading ? If wrap = yes then the
observation in each filter are written on a separate line and the * character
in column 1 is interpreted as a continuation character. Otherwise all the
observations for a single filter are written on a single line where the maximum
length of a line is SZ_LINE characters.
.le
.ls obsparams = ""
The name of an optional text file containing the correct filter ids, exposure
times, airmasses, and time of observations for each image whose values are
either undefined or incorrectly stored in \fIphotfiles\fR. The observing
parameters for each image are listed in the file \fIobsparams\fR, 1 image per
line with the image name in column 1 and the filter ids, exposure times,
airmasses, and times of observation listed in columns \fIobscolumns\fR. The
image names must match those in \fIimsets\fR. Images which have no entries in
\fIobsparams\fR are assigned the values stored in \fIphotfiles\fR.
.le
.ls obscolumns = "2 3 4 5"
The list of numbers separated by commas or whitespace specifying which columns
in the text file \fIobsparams\fR contain the correct filter ids, exposure
times, airmasses, and times of observation respectively. The number 0 can be
used as a place holder in the \fIobscolumns\fR string. For example, to correct
only  the \fIphotfiles\fR airmass values, \fIobscolumns\fR should be set to
"0 0 column 0", where column is the airmass column number. The default value of
\fIobscolumns\fR corresponds to the format of the default \fIobsparams\fR file
produced by MKIMSETS.
.le
.ls minmagerr = 0.001
The error that will be assigned to a non-INDEF valued magnitude measurement
whose recorded error is less than \fIminmagerr\fR.
.le
.ls shifts = ""
The name of the text file specifying the x and y shifts to be ADDED to the x-y
positions of all objects in an image before position matching (the original x's
and y's are retained in the output). Shifts are listed for each image, 1 image
per line with the name of the image in column 1, followed by the x and y shifts
in columns 2 and 3 respectively. Image names must match those in \fIimsets\fR.
Images for which no shift is supplied are assigned x and y shifts of zero.
.le
.ls apercors = ""
The name of the text file specifying the aperture corrections to be ADDED to
the extracted magnitudes. Aperture corrections are listed for each image, 1
image per line with the name of the image in column 1, followed by the aperture
correction in magnitudes in column 2.  The image names must match those in
\fIimsets\fR. Images for which no aperture correction is supplied are assigned
a default value of zero.
.le
.ls normtime = no
Normalize the magnitudes to an exposure time of one time unit using the
exposure times in \fIphotfiles\fR.
.le
.ls tolerance = 5.0
The tolerance in pixels for matching objects in the same observation, but
different images.  OBSFILE extracts the x and y coordinates of each object
in each image of a given observation from \fIphotfiles\fR, adds the shift for
that image in \fIshifts\fR to the extracted x-y coordinates, and matches the
objects to within \fItolerance\fR pixels. Missing objects are assigned INDEF
entries in \fIobservations\fR. If \fItolerance\fR is less than or equal to 0
no coordinate matching is done, and objects are matched in order of occurrence
with missing objects being assigned INDEF values.
.le
.ls allfilters = no
Output only objects which are successfully matched in all the filters specified
by \fIidfilters\fR?
.le
.ls verify = no
Verify interactive user input? This option is used only if any of \fIimsets\fR,
\fIobsparams\fR, \fIshifts\fR, or \fI apercors\fR are set to the standard input
"STDIN".
.le
.ls verbose = yes
Print messages about actions taken by the task or any warnings or errors
encountered?
.le

.ih
DESCRIPTION

OBSFILE takes a list of user generated text files \fIphotfiles\fR, where each
file contains  observations of one or more objects taken through one or more
filters, and the image set file \fIimsets\fR, and prepares a single
observations file \fIobservations\fR. OBSFILE is intended for use with any
user digital stellar photometry program which writes its output in simple text
files format.

OBSFILE performs the following functions: 1) extracts the quantities
image name or image id, x and y position, filter id, exposure time, airmass,
time of observation, magnitude, and magnitude error from
\fIphotfiles\fR, 2) corrects any erroneous or missing values of filter id,
exposure time, airmass, or time of observation in \fIphotfiles\fR,  3) associates each 
field with one or more sets of images of that
field taken through different filters 4) matches individual objects within
a given observation by order of occurrence or x-y position, and
5) assigns a unique name to each object in each field.

The parameter \fIincolumns\fR describes the format of \fIphotfiles\fR.
\fIIncolumns\fR is a list of ten numbers separated by commas or whitespace
which specify the columns containing the following fields: the
image name or id,
the x coordinate, the y coordinate, the filter id, the exposure time, 
the airmass, the time of observation the instrumental magnitude, the
magnitude error, and the object id.
For example
if \fIincolumns\fR is "10 2 3 6 8 7 9 4 5 1", the object id is assumed to
be in column 1, the image id in column 10, the x and y positions in columns 2 and 3, the filter id,
exposure time, airmass, and time of observation in columns 6, 8, 7 and 9,
and the instrumental
magnitude and magnitude error in columns 4 and 5. The image names must
match those in \fIimsets\fR or the corresponding input data is skipped.
The columns image name, x coordinate, y coordinate, and magnitude
are mandatory and must be present in \fIphotfiles\fR. 
Other missing columns in the data may be represented by a "0" in the
appropriate place in \fIincolumns\fR.
For example, if there is no magnitude error
column in \fIphotfiles\fR a value of INDEF will be written in the appropriate
column in \fIobservations\fR. 
If there is no airmass column in \fIphotfiles\fR the value in
\fIobspararms\fR if any, or the value INDEF will be written to the appropriate
column in \fIobservations\fR. 
If there is no filter id column in \fIphotfiles\fR the value in
\fIobspararms\fR if any, or one of the values in \fIidfilters\fR
will be written to the appropriate column in \fIobservations\fR. 
If there is no exposure time column in \fIphotfiles\fR the value in
\fIobspararms\fR if any, or a value of one will be assumed.
If there is no time of observation time column in \fIphotfiles\fR the value in
\fIobspararms\fR if any, or a value of INDEF will be assumed.

The image set file \fIimsets\fR assigns a name to each field.
For fields containing only a single standard star this name should
match the name of the standard star in the standard star catalog.
For fields containing more than one star, OBSFILE constructs a unique
name for each object in the field by adding a sequence number to the 
field name in \fIimsets\fR, which if the star is a standard star, the
user must later edit. For example the fourth star in the field "M92"
will be assigned the name "M92-4" in \fIobservations\fR.
If this star is a standard star and its true name is "IX-10" in the
standard star catalog, then the user must change "M92-4" to "IX-10"
in \fIobservations\fR.
\fIImsets\fR also tells OBSFILE which images
in \fIphotfiles\fR are images of the same region of the sky belonging
to the same observation.
The format of \fIimsets\fR is described in detail below.
If the number of observations is small the user may wish to simply type
in \fIimsets\fR by hand. If the number of observations is large, a 
separate task MKIMSETS is available to assist users in preparing
\fIimsets\fR.

Values of the filter ids, exposure times, airmasses, and times of observation,
which are undefined or incorrect in \fIphotfiles\fR,
can be corrected by reading values listed in the columns \fIobscolumns\fR
in the file \fIobsparams\fR. The format of \fIobsparams\fR is described
in detail below.

OBSFILE matches the objects in different images within the same observation
either
by order of occurrence if \fItolerance\fR is less than or equal to 0.0,
or by x-y position. Matching by position is done by identifying which objects
in each of the
images of a given field and observation set are within \fItolerance\fR
pixels of each other.  The user may supply an optional file of x and y
shifts \fIshifts\fR to be added to the object positions prior to
matching. The format of \fIshifts\fR is described in detail below.
If the parameter \fIallfilters\fR is "yes", only objects which are matched
in all the filters \fIidfilters\fR are output to \fIobservations\fR.

OBSFILE permits the user to supply 
an optional file of aperture corrections \fIapercors\fR containing
magnitude corrections which are added to the instrumental
magnitudes in \fIphotfiles\fR.
The format of \fIapercors\fR is described in detail below.

Each new observations file created by OBSFILE has an associated format
description file listing the column names and numbers in \fIobservations\fR,
of the fields extracted from \fIphotfiles\fR. This file, referenced 
by its parent observations file name, can be used as input to the
MKCONFIG task. The actual name of the format description file on disk is
constructed by prepending the string "f" and appending the string ".dat"
to \fIobservations\fR.
For example if a new observations file called "nite1" is created by
OBSFILE, a format description file called "fnite1.dat" will also be
created. Any pre-existing format description file of that name, which does
not have an associated observations file, will be deleted.


THE IMSETS FILE

The \fIimsets\fR file lists the 
the observations of each field, assigns a name to each
field, and informs OBSFILE which images belong to the same
observation of that field.
Observations are listed in \fIimsets\fR, 1 observation
per line with the field name in column 1, a colon in column 2,
followed by the names of the
images of that field for that observation separated by whitespace.
Only data for image names in \fIimsets\fR which match those in
\fIphotfiles\fR will be extracted.

The field name is used to generate the output object name in \fIobservations\fR.
If there is only a single measured object in the field, then the name
of that object in \fIobservations\fR will be the name of the field. If
the single object is a standard star, the user should edit \fIimsets\fR
so that the field name is the same as the name of the standard star in
the standard star catalog. If a stellar field contains more than one
measured object, OBSFILE generates names of the form "field-#" where
"field" is the field name and "#" is a sequence number. For example the
fourth star in the field "M92" will be assigned the name "M92-4" in
\fIobservations\fR. If the star is a standard star, the user must edit
the object names in \fIobservations\fR to match those in the standard
star catalog.

Any number of observations may have the same field name. This normally occurs,
for example, when multiple observations of a single standard star of
standard star field are made at several airmasses.

If there
are no filter ids in \fIphotfiles\fR or \fIobsparams\fR then the images in
each image set are assigned the filter ids in \fIidfilters\fR in order
of occurrence.

The \fIimsets\fR file for a  set of 50 UBV frames of fifteen standard star
fields is listed below. There is only a single bright star per field.
The name of star field in column 1 has been edited to be identical
to the name of the standard in the standard star catalog. Column 2 contains
a ':'. The U, B and V
images for each field are listed in columns 3, 4 and 5 respectively.
The missing U image for field "STD7" is represented by the name "INDEF".
Standard stars "STD1" and "STD2" were observed twice in the same night
at different airmasses.

.nf
	STD1 :	nite001   nite002  nite003
	STD1 :  nite045   nite046  nite047
	STD2 :	nite004   nite005  nite006
	STD2 :	nite048   nite049  nite050
	...
	STD7 :  INDEF     nite019  nite020
	...
	STD14 : nite039   nite040  nite041
	STD15 : nite042   nite043  nite044
.fi

THE OBSPARAMS FILE

A sample corrections file \fIobsparams\fR for the previous set of
UBV standards observations is shown below.
The filter ids, exposure times, airmasses, and times of observation for all the images were
correctly read
from the image headers with the exception of the filter id, exposure time,
and airmass for the first  "STD2" V frame.
The correct filter id, exposure time, airmass, and time of observation, is supplied
in \fIobsparams\fR  and \fIobscolumns\fR is set to "2 3 4 5"

.nf
	nite006    3 8 1.256 14:30:02.3
.fi

Zero can be used as a place holder in \fIobscolumns\fR,
as in the following example where
the user only wants to correct the exposure time and the airmass and
leave the filter id alone. In this case \fIobscolumns\fR is "0 2 3 0"
and \fIobsparams\fR looks as follows.

.nf
	nite006    8 1.256
.fi

Only images listed in \fIimsets\fR can have their observing parameters
modified by \fIobsparams\fR.

THE SHIFTS FILE

The file \fIshifts\fR lists the shifts for each image, 1 shift per line,
with the image name in column 1 and the x and y shifts in columns 2 and
3 respectively.
The image names in \fIshifts\fR must match those in \fIimsets\fR.

A sample shifts file for the previous set of UBV standards
observations is shown below. All the standards except for "STD14" are assumed
to have no significant shifts from filter to filter. The B and V frames
for "STD14" are shifted -10 pixels in x and -5 pixels
in y with respect to the U frame. Therefore +10 and +5 pixels should be
added to the "STD14" B and V frame positions respectively before
position matching.

.nf
	nite040   10.0   5.0
	nite041   10.0   5.0
.fi

An alternate way of listing the same observations would be the following.

.nf
	nite039   -10.0 -5.0
.fi

THE APERCORS FILE

The file \fIapercors\fR lists the aperture corrections for each image,
1 aperture correction per line,
with the image name in column 1 and the aperture correction in magnitudes
in column 2 respectively.
The image names in \fIapercors\fR must match those in \fIimsets\fR.

The \fIapercors\fR file for the previous set of UBV observations is shown
below.
The aperture corrections for all the standard stars are assumed to be
zero except for "STD14".

.nf
	nite039    -0.150
	nite040    -0.100
	nite041    -0.090
.fi

.ih
OUTPUT
For the previous set of UBV observations the output file
\fIobservations\fR produced by OBSFILE will look like the following.
The filter ids for the U,B,V filters are assumed to be 1,2,3.
Note that the exposure times are assumed to have been normalized either
prior to running OBSFILE or by OBSFILE itself,
and so are not included in \fIobservations\fR.

.nf
	# FIELD   FILTER   OTIME  AIRMASS  X     Y     MAG   MERR

	  STD1    1        .      .        .     .     .     .
	  *       2        .      .        .     .     .     .
	  *       3        .      .        .     .     .     .
	  STD1    1        .      .        .     .     .     .
	  *       2        .      .        .     .     .     .   
	  *       3        .      .        .     .     .     .
	  STD2    1        .      .        .     .     .     .
	  *       2        .      .        .     .     .     .
	  *       3        .      .        .     .     .     .
	  STD2    1        .      .        .     .     .     .
	  *       2        .      .        .     .     .     .
	  *       3        .      .        .     .     .     .
	  ........................................................
	  STD7    INDEF    INDEF  INDEF    INDEF INDEF INDEF INDEF
	  *       2        .      .        .     .     .     .
	  *       3        .      .        .     .     .     .
	  .......................................................
	  STD14   1        .      .        .     .     .     .
	  *       2        .      .        .     .     .     .
	  *       3        .      .        .     .     .     .
	  STD15   1        .      .        .     .     .     .
	  *       2        .      .        .     .     .     .
	  *       3        .      .        .     .     .     .
.fi

The accompanying format description file has the following form.

.nf
# Declare the observations file variables

observations

X1            3              # airmass in filter 1
T1            4              # time of observation in filter 1
x1            5              # x coordinate in filter 1
y1            6              # y coordinate in filter 1
m1            7              # instrumental magnitude in filter 1
error(m1)     8              # magnitude error in filter 1

X2            10             # airmass in filter 2
T2            11             # time of observation in filter 2
x2            12             # x coordinate in filter 2
y2            13             # y coordinate in filter 2
m2            14             # instrumental magnitude in filter 2
error(m2)     15             # magnitude error in filter 2

X3            16             # airmass in filter 3
T3            17             # time of observation in filter 3
x3            18             # x coordinate in filter 3
y3            19             # y coordinate in filter 3
m3            20             # instrumental magnitude in filter 3
error(m3)     21             # magnitude error in filter 3
.fi

.ih
EXAMPLES

1. Prepare an observations file, from a set of standard star observations
in a file output by the user's own digital stellar photometry program,
for input to FITPARAMS. A sample of the file illustrating the format
is shown below.
Since there is only one star per field, position matching is not necessary.
The magnitudes have already been normalized to unit exposure time by the
user's program, and the filter ids and airmasses are correct. However the
observing time column is missing and represented by a zero in the incolumns
parameters.

.nf
	ph> head magsfile

	    ... print out the first few lines of the photometry file

	    std1u   40.4   50.3   18.059   0.043   U   1.030   1.0
	    std1b   42.5   53.1   17.089   0.023   B   1.032   1.0
	    std1v   43.8   56.9   16.023   0.020   V   1.034   1.0
	    std2u   39.4   55.3   17.029   0.040   U   1.135   1.0
	    std2b   41.5   57.3   15.905   0.020   B   1.140   1.0
	    std2v   42.6   58.9   14.899   0.018   V   1.144   1.0
	    .....   ....   ....   ......   .....   .   .....   ...
	    .....   ....   ....   ......   .....   .   .....   ...

	ph> type fields

	    ... print out the corresponding image set file

	    std1 : std1u  std1b  std1v
	    std2 : std2u  std2b  std2v
	    ..... .....  .....  .....
	    ..... .....  .....  .....

	ph> obsfile magsfile "1 2 3 6 8 7 0 4 5" "U,B,V" fields standards.obs\
	    tol=0.0

	    ... create the observations file

	ph> edit standards.obs

	    ... edit the observations file so that the object names
		match those in the standard star catalog
.fi

2. Prepare an observations file from a set of program star observations
of a crowded field in the globular cluster M92 computed by the same
digital photometry
program as above, for input to FITPARAMS.  The 3 input files contain UBV
measurements of over 2000 stars in the M92 field. Since the same stars
were not measured in all filters position matching is necessary.

.nf
	ph> head m92umags,m92bmags,m92vmags

	    ... print the first few lines of the input files on the
	        standard output

	    m92u    80.4   42.3   17.046   0.046   U   1.056   1.0
	    m92u    ....   ....   ......   .....   U   1.056   1.0

	    m92b    62.6   81.1   18.071   0.041   B   1.030   1.0
	    m92b    ....   ....   ......   .....   B   1.030   1.0

	    m92v    33.8   26.9   16.023   0.022   V   1.034   1.0
	    m92v    ....   ....   ......   .....   V   1.034   1.0

	ph> type fields

	    ... print out the image set file

	    m92 : m92u  m92b  m92v

	ph> obsfile m92umags,m92bmags,m92vmags "1 2 3 6 8 7 0 4 5" "U,B,V"\
	    fields standards.obs tolerance=8.0

.fi

.ih
TIME REQUIREMENTS
.ih
BUGS
.ih
SEE ALSO
mkimsets,mknobsfile,mkobsfile
.endhelp