1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
|
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
Detailed Specifications for the Multi-Spectra Extraction Package
F. Valdes
December 8, 1983
1. Introduction
The multi-spectra extraction package (MULTISPEC) provides the
basic tools for modeling, cleaning, and extracting spectra from images
containing multiple aperture spectra running roughly parallel. These
tools will generally be combined in reduction script tasks but may
also be used directly for non-standard analysis.
This design presents the requirements and specifications for the
MULTISPEC package. Details concerning the algorithms are given in a
separate document, Algorithms for the Multi-Spectra Extraction Package.
2. Input Data Requirements
The input data for the MULTISPEC package consists of image files
containing one or more aperture spectra. The spectra are required to
run roughly parallel to each other and parallel to the second
digitization axis. The latter requirement may require a general
rotation and interpolation image operator. The images are assumed to
be corrected to linear relative intensity. Thus, the steps of
correcting digital detector images for dark current, bias, and
pixel-to-pixel sensitivity variations must be performed before using
the MULTISPEC tasks.
Because the the MULTISPEC package is being developed concurrently
with the IRAF standard image processing tools this document specifies
the requirements for the preliminary image processing needed to
prepare digital detector images for the MULTISPEC package.
2.1 Basic Digital Detector Reduction Tasks
The prelimary reduction of multi-spectra images uses CL scripts
based on general image operators. Some of the scripts are for
specific instruments or specific reduction applications and some are
generally useful image processing tasks. The scripts allow the
specification of many images for which the operations will be
repetitively applied.
The following CL scripts are required to reduce multi-spectra
images from digital detectors.
debias multispec_flat flat_divide
-1-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
debias
The files in a list of filenames are automatically debiased and
trimmed. This routine will be instrument specific but used by
other reduction tasks beyond MULTISPEC.
multispec_flat
The files in a list of quartz multi-spectra filenames are added,
the result is smoothed along the dispersion dimension, and then
the original image is divided by the smoothed image to produce a
flat field image. The unsmoothed to smoothed ratio is computed
only if the value of the smoothed pixel is greater than a
specified amount. Otherwise, the ratio is set to unity. This
routine is not instrument specific but is used only for MULTISPEC
reductions.
flat_divide
The files in a list of filenames are each divided by a specified
flat field image. This routine is not instrument or application
specific.
The required general image processing programs needed to implement
these scripts are specified below.
(1) A routine to compute the average value from a specified area of the
image. Used to determine the average bias value from a bias strip.
(2) A routine to trim a specified portion of an image. Used to trim
the bias strip.
(3) Routines to multiply and subtract images by a constant. Used to
scale images such as dark exposures and to remove the average bias
value as obtained by (1) above.
(4) Routines to subtract, add, and divide images. Used to subtract
dark current and bias exposures, to add several exposures to
increase the signal-to-noise, and to divide by a flat field image.
The divide routine must give the user the option to substitute a
constant or ignore any divisions in which the denominator is less
than a specified value.
(5) A routine to rotate or transpose an image. Used to align the
spectra along lines or columns.
(6) A routine to apply a filter to lines of the image. For
multi-spectra images a smooth quartz is produced by using a
running quadratic filter along each line of the dispersion
dimension. The filter must be able to recognize bad pixels
(specified by a user defined threshold) and remove them from the
filtering operation.
-2-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
3. Requirements for the MULTSPEC Package
The MULTISPEC package shall satisfy the following requirements.
(1) The component programs shall be CL callable.
(2) The programs shall interact only through image files and MULTISPEC
data files.
(3) It shall be possible to extract spectra without modeling.
(4) The entire image shall be extracted and not limited by failures in
the algorithms.
(5) It shall be possible to specify specific lines or swaths in the
image on which to operate.
(6) CL scripts shall be provided for the common data sources. These
scripts will work automatically.
The follow functions shall be provided:
o Make an initial rough but automated identification of the spectra
locations.
o Provide for a user identification list for the spectra locations.
This list shall be of the standard image cursor type to allow
generation of the list with the standard image cursor programs.
o Determine and correct for a slowly varying background.
o Reliably and accurately trace spectra in the presence of geometric
distortions (pincushion, s, shear, etc.).
o Extract spectra by one of:
a. Strips of constant width about the located spectra. The width
may be specified to fractions of a pixel and the extraction
will use fractional pixel interpolation. l
b. Strips of width proportional to a Gaussian width parameter.
c. Modeling to obtain estimates of the total luminosity. The
estimate will be the integral of the model.
d. Summation of the data pixel values with fractional
contributions of the pixel value to the spectra based on
modeling.
o An option shall be available to specify whether to ignore blank
pixels or use interpolated values.
o Programs shall be provided to produce data files which can be
-3-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
accessed by one dimensional spectroscopic reduction routines.
At a minimum these formats shall include:
a. Reduction to an image file consisting of one line per
extracted spectrum
b. The standard IIDS format available with the CYBER
Multi-Aperture Plate programs
3.2 Modeling Requirements
The modeling of multi-spectra images, particularly in the case of
blended spectra, shall:
(1) Model blended spectra with sufficient reliability and robustness
that a reasonable solution is always obtained, though of possibly
limited usefulness.
(2) The modeling shall provide estimates for the uncertainties in the
fitted parameters as a function of position along the spectrum.
(3) Remove cosmic rays and other defective pixels by reference to the
model.
(4) Allow the transfer of a model solution for one image to another
image.
(5) Display numerically and graphically the data, the fitted model, and
the residuals.
4. Program Specifications
4.1 Basic Programs
The basic programs of the package are general purpose tools which
initialize a MULTISPEC data file and perform a single fundamental
operation on the data in the MULTISPEC data file. There is one data
file associated with each image. The data file is hidden from the
user and so the user need not be aware of the data file. The data
files are referenced only the image filename specified in the program
parameters. The data files contain such information as a processing
history, the spectra positions and extracted luminosities, the model
parameters (one set for each spectra for each modelled image line (or
swath), etc. The programs generally are allowed to specify specific
lines, columns, and/or spectra on which to operate. The line, column
and spectra specifications are given as strings which contain numbers
separated by whitespace, commas, and the range indicator "-". The
script tasks of section 4.2 will combine these basic programs to
perform a general multi-spectra extraction.
-4-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
ap_plate copy_params find_spectra convolve
fit_bckgrnd find_bckgrnd line_list model_extrac
model_fit model_image model_list sigma_extract
strip_extract to_iids to_image to_onedspec
ap_plate
The information from an on-line data file containing descriptions
of all the aperture plates prepared at Kitt Peak is read to find a
specified aperture plate. The drilled aperture positions are
correlated with the spectra in the image to deduce relative
wavelength offsets. The identifications for the spectra as well
as other auxiliary information is recorded in the data file. If
no image file is specified then only the aperture plate
information is printed. This program is used in the
MULTIAP_EXTRACT program. This program is not essential to the
operation of the MULTISPEC package.
Multi-Spectra image image =
Aperture plate plate =
(mode = ql)
-5-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
The Background
The are two possibilities for dealing with the background. In the
first case, FIT_BCKGRND, the background will be fitted by
polynomials and the coefficients stored in the MULTISPEC data
file. These coefficients are then used by the other programs to
estimate the background at the spectra. The second option,
FIND_BCKGRND, generates a background image in which the spectra
and other selected areas are set to blank pixels. Then a general
image interpolator is used fill in the blank pixels with background
estimates. The other MULTISPEC programs will then access this
background frame. The background frame image name will be stored
in the MULTISPEC data file and the image header.
fit_bckgrnd
Fit a background in a MULTISPEC image by a polynomial using
pixels not near the spectra and in the user specified swaths
and columns. The buffer distance is in pixels and refers to a
minimum distance from the center of any spectrum beyond which
the background pixels are found. Blank pixels are ignored in
the background fit. Deviant pixels will be rejected.
Multi-Spectra image image =
Buffer from spectra buffer = 12
Polynomial order order = 3
Lines per swath (lines_per_swath = 32)
Swaths to fit (swaths = 1-1000)
Columns to fit (columns = 1-1000)
Rejection threshold (threshold = 5)
Print general diagnostics (verbose = no)
(mode = ql)
find_bckgrnd
The spectra within a buffer distance and specified areas are
set to blank pixels and the remaining pixels copied to a
background image file.
Multi-Spectra image image =
Background image background =
Buffer from spectra buffer = 12
Lines to ignore (lines = )
Columns to ignore (columns = )
(mode = ql)
convolve
A program will be provided to reduce either the extracted spectrum
or the modeled image to a common point-spread function.
-6-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
copy_params
Create a MULTISPEC data file for a new image using appropriate
MULTISPEC parameters from an old image. The old image must have
been processed to find the spectra using FIND_SPECTRA and possibly
model fit.
Old Multi-Spectra image old_image =
New Multi-Spectra image new_image =
(mode = ql)
find_spectra
Initially locate the spectra in a MULTISPEC image. The positions
of the spectra within the range of columns are determined for the
starting line and then the spectra are tracked within the range of
lines. The minimum separation and minimum width would generally
be set for a particular instrument. If the automatic search is
not used then a list of cursor positions is read from the standard
input.
Multi-Spectra image image =
Automatic search auto = yes
Starting line start_line =
Minimum separation (min_sep = 1)
Minimum width (min_width = 1)
Averaging width (average = 32)
Lines to search (lines = 1-1000)
Columns to search (columns = 1-1000)
Print general diagnostics (verbose = no)
(mode = ql)
line_list
For the specified lines in the image print the image column
number, data value (possibly as a swath average), the model value
at that point (i.e. the sum of the model contributions from all
the spectra), the background value, and the residual. Plotting
scripts may be written using this routine to show the quality of a
model fit.
Multi-Spectra image image =
Lines to list (lines = 1-1000)
(mode = ql)
-7-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
model_extract
A previously fitted model is used to extract the spectra total
luminosity by apportioning the data values to spectra in the ratio
indicated by the model. If the clean option is specified then the
model is used to detect pixels which deviate from the model by a
specified amount. The model value replaces the deviant pixel in
the extraction and, if specified, also in the image file.
Multi-Spectra image image =
Lines to extract (lines = 1-1000)
Clean spectra (clean = yes)
Cleaning threshold (threshold = 5)
Modify image (modify = yes)
Print general diagnostics (verbose = no)
(mode = ql)
model_fit
A specified model is iteratively fitted to the data in each of the
specified lines (or swaths) until the RMS residual fails to
decrease. The models are selected by a string. The possible
values are
(null string) - initialize the model
i - fit only the intensity scale
ip - fit the intensity scale and the position
ips1 - fit the intensity scale, position, and one parameter shape
ips2 - fit the intensity scale, position, and two parameter shape
ips3 - fit the intensity scale, position, and three parameter shape
ips4 - fit the intensity scale, position, and four parameter shape
These models will be combined in a script to search for the best
fit.
The initial shape parameters will generally be set by scripts for a
particular data reduction.
Multi-Spectra image image =
Model type model =
Lines per swath (lines_per_swath = 32)
Swaths to model (swaths = 1-1000)
Initial shape1 (shape1 = .1 )
Initial shape2 (shape2 = 0 )
Initial shape3 (shape3 = 0 )
Initial shape4 (shape4 = 5 )
Print general diagnostics (verbose = no)
(mode = ql)
-8-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
model_image
An image file of the fitted model is created. This image may then
be displayed or a residual image may be calculated and displayed.
Multi-Spectra image image =
Model image model =
(mode = ql)
.nf
.le
.ls model_list
For the specified lines and spectra the model is listed.
The listing gives, for each spectra,
the spectrum number, the line number, the fitted position,
the estimated wavelength, the
extracted luminosity, the intensity scale, model width parameters, and
the background polynomial coefficients. This routine can be used in scripts
to plot the extracted spectra, the trend of width with wavelength, and so
forth.
.nf
Multi-Spectra image image =
Lines to list (lines = 1-1000)
Spectra to list (spectra = 1-1000)
(mode = ql)
sigma_extract
A previously fitted model is used to extract the spectra luminosity
within a specified sigma of the peak. Because the model is not
necessarily a Gaussian the sigma is used to compute the intensity
ratio of the cutoff to the peak assumining a Gaussian profile and
then the data is extracted to the point the model intensity falls
below that cutoff. If the clean option is specified then the
model is used to detect pixels which deviate from the model by a
specified amount. The model value replaces the deviant pixel in
the extraction and, if specified, also in the image file.
Multi-Spectra image image =
Sigma extraction width width = 1.
Lines to extract (lines = 1-1000)
Clean spectra (clean = yes)
Cleaning threshold (threshold = 5)
Modify image (modify = yes)
Print general diagnostics (verbose = no)
(mode = ql)
-9-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
strip_extract
A strip of constant width about the spectra positions is extracted.
If cleanning is desired a smoothed estimate of the profile is
obtained by averaging a number of lines about the line to be
cleaned. After fitting for the intensity scale pixels are found
which deviate from the profile by a specified amount. The profile
value replaces the deviant pixel in the extraction and, if
specified, also in the image file. No prior modeling is required
to use this extraction routine.
Multi-Spectra image image =
Strip extraction width width = 1.
Lines to extract (lines = 1-1000)
Clean spectra (clean = yes)
Cleaning threshold (threshold = 5)
Lines per profile average (averge_lines = 32)
Modify image (modify = yes)
Print general diagnostics (verbose = no)
(mode = ql)
to_iids
For a specified prefix, files of the form prefix.nn, where nn is a
specified spectra number, are created containing the extracted
spectra for all the specified image files. The format of the
files is the IIDS format developed for the CYBER Multi-Aperture
Plate Extractions.
Multi-Spectra image images =
IIDS filename prefix iids_file =
Spectra to format (spectra = 1-1000)
(mode = ql)
to_image
An image file containing one line of the extracted luminosities
for each specified spectra in the specified MULTISPEC image.
Multi-Spectra image in_image =
Extracted spectra image out_image =
Spectra (spectra = 1-1000)
(mode = ql)
to_onedspec
The extractions are converted to an as yet to be specified format
for use in the ONEDSPEC reduction package.
Multi-Spectra images images =
ONEDSPEC data file onedspec_file =
Spectra (spectra = 1-1000)
(mode = ql)
-10-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
4.2 General MULTISPEC CL Scripts
The general MULTISPEC CL scripts perform a series of steps needed
to extract the spectra from a specified list of image files. These
steps have been found to generally perform the desired extraction task
fully.
multiap_extract echelle_extract
multiap_extract
The specified multi-aperture plate images are extracted. If no
starting solution image, one which has previously been extracted,
is specified then the script performs an automatic search for the
specified number of spectra. Otherwise the solution from the
starting image is used as the initial model. The background is
then determined. This is followed by a series of fitting steps on
swaths of data. (For further details on the fitting steps see the
Algorithms paper). A MODEL_EXTRACT and cleaning follows.
Finally, the extraction is correlated with the specified aperture
plate using AP_PLATE. If there was no starting image then this
extraction becomes the initial solution image. Subsequent images
are extracted starting from the initial solution image.
Multi-Aperture images images =
Initial solution image initial =
Aperture plate number plate =
Number of spectra nspectra =
(mode = ql)
echelle_extract
The specified echelle images are extracted. If no starting
solution image, one which has previously been extracted, is
specified then the script performs an automatic search for the
specified number of orders. Otherwise the solution from the
starting image is used as the initial starting point. The
background is then determined. Finally a STRIP_EXTRACT and
cleaning is performed. If there was no starting image then this
extraction becomes the initial solution image. Subsequent images
are extracted starting from the initial solution image.
Echelle images images =
Initial solution image initial =
Number of orders norders =
Extraction width width =
(mode = ql)
5. Outline of a MULTISPEC Reduction
The following outline is for the reduction of a cryogenic camera
multi-aperture plate. All the programmer supplied default values are
used.
-11-
MULTISPEC (Oct83) Multi-Spectra Extraction Package MULTISPEC (Oct83)
(1) rcamera mtb, "ap165.", "s", "3-9"
(2) debias "ap165.*"
(3) multispec_flat "ap165.[36]", "ap165.flat"
(4) flat_divide "ap165.*", "ap165.flat"
(5) multiap_extract "ap165.*", "", 165, 50
(6) to_onedspec "ap165.*", oned165
(1) The data is read from the observing tape(s) using RCAMERA. The
image files created are ap165.3, ap165.4, ..., ap165.9. This is
easily accomplished by using the filename prefix "ap165." in the
RCAMERA program. The raw images may be examined at this point on
a display.
(2) The images are debiased using DEBIAS with all the "ap165." files
specified. The debias program knows about the location of the
bias strip for the cryogenic camera.
(3) A a flat field is created using MULTISPEC_FLAT in which the
desired quartz frames are specified and a flat field image
filename is defined. The created flat field image may be examined
on an image display if desired.
(4) All the debiased images are divided by the flat field using
FLAT_DIVIDE.
(5) The script MULTIAP_EXTRACT is run in which the aperture plate
number, the number of spectra, and the image files to be extracted
are specified. The number of spectra is found by examining an
image on an image display or by plotting a cut across the spectra
using a general image profile program.
(6) Finally, the extracted spectra are formatted for the ONEDSPEC
package using TO_ONEDSPEC with the extracted images specified.
|
