OMC consolidate the steps required to build, test, and deploy calibration pipelines and other software suites maintained by engineers at the Space Telescope Science Institute.

Requirements

• Linux, or MacOS (Darwin)
• cmake
• libcurl
• libxml2
• rsync

Installation

Download the source code

git clone https://github.com/jhunkeler/ohmycal.git
cd ohmycal

Create and enter the build directory

mkdir build
cd build

Run cmake

cmake .. -DCMAKE_INSTALL_PREFIX="$HOME/programs/ohmycal"

Compile and install ohmycal

make
make install
export PATH="$HOME/programs/ohmycal/bin:$PATH"

Quickstart

Step 1: Create a mission

Missions definitions live in OMC's etc/mission directory. Let's create a new one specifically geared toward generic data analysis tools. You may override the path to the etc directory by setting the OMC_SYSCONFDIR environment variable to different location.

mkdir $HOME/programs/ohmycal/etc/missions/mymission
touch $HOME/programs/ohmycal/etc/missions/mymission/mymission.ini

Now populate the new data analysis mission configuration. Refer to the release formatters section to see a list of what each % formatter does.

[meta]
release_fmt = %n-%v-%r-py%p-%o-%a
; e.g. mydelivery-1.2.3-1-py312-linux-x86_64

build_name_fmt = %n-%v
; e.g. mydelivery-1.2.3

build_number_fmt = %v.%r
; e.g. 1.2.3.1

Text files containing OMC template strings can be stored at the same level as the mission configuration, and rendered to anywhere inside the output directory. This will can you time if you plan to release for multiple platforms and architectures.

[template:readme.md.in]
destination = {{ storage.delivery_dir }}/README-{{ info.release_name }}.md
; e.g [..]/output/delivery/README-mydelivery-1.2.3-1-py312-linux-x86_64.md

[template:Dockerfile.in]
destination = {{ storage.build_docker_dir }}/Dockerfile
; e.g [..]/build/docker/Dockerfile

Step 2: Create a delivery configuration

OMC's configuration parser does not distinguish input files by extension, so mydelivery.ini, mydelivery.cfg, mydelivery.txt, and abc123.zyx987 are all perfectly valid file names.

All deliveries require a [meta] section. Here global metadata such as the delivery's name, version, and any programs/dependencies that make up the deliverable.

[meta]
mission = mymission
name = mydelivery
version = 2024.3.1
rc = 1
python = 3.12

The [conda] section instructs OMC how to obtain the conda installer of your choice, and defines the packages to be installed into the delivery's release environment.

[conda]
; e.g. Download Miniforge3-23.11.0-0 for the current system platform and architecture
installer_name = Miniforge3
installer_version = 23.11.0-0
installer_platform = {{env:OMC_CONDA_PLATFORM}}
installer_arch = {{env:OMC_CONDA_ARCH}}
installer_baseurl = https://github.com/conda-forge/miniforge/releases/download/{{conda.installer_version}}

conda_packages =
    qt>=5.0

pip_packages =
    our_cool_program
    our_other_cool_program

Create some test cases for packages.

[test:our_cool_program]
version = 1.2.3
repository = https://github.com/org/our_cool_program
script =
    pytest -v \
        --basetemp="{{ storage.results_dir }}/truth-$(basename $(pwd))-{{ info.release_name }}" \
        --junit-xml="{{ storage.results_dir }}/results-$(basename $(pwd))-{{ info.release_name }}.xml" \
        tests/

[test:our_other_cool_program]
version = 4.5.6
repository = https://github.com/org/our_other_cool_program
script =
    pytest -v \
        --basetemp="{{ storage.results_dir }}/truth-$(basename $(pwd))-{{ info.release_name }}" \
        --junit-xml="{{ storage.results_dir }}/results-$(basename $(pwd))-{{ info.release_name }}.xml" \
        tests/

Step 3: Run OMC

omc mydelivery.ini

Configuration

Environment variables

Variable expansion

Template strings

Template strings can be accessed using the {{ subject.key }} notation in any Ohmycal configuration file.

The template engine also provides an interface to environment variables using the {{ env:VARIABLE_NAME }} notation.

[meta]
name = {{ env:MY_DYNAMIC_DELIVERY_NAME }}
version = {{ env:MY_DYNAMIC_DELIVERY_VERSION }}
python = {{ env:MY_DYNAMIC_PYTHON_VERSION }}

Delivery files

Sections

All configuration section names and keys are case-sensitive.

meta

conda

runtime

Environment variables exported are global to all programs executed by ohmycal. There is no limit to the number of environment variables that can be set.

test:name

Sections starting with test: will be used during the testing phase of the ohmycal pipeline. Where the value of name following the colon is an arbitrary value, and only used for reporting which test-run is executing. Section names must be unique.

deploy:artifactory:name

Sections starting with deploy:artifactory: will define the upload behavior of build and test artifacts to Artifactory. Where the value of name is an arbitrary value, and only used for reporting. Section names must be unique.

deploy:docker

The deploy:docker section controls how Docker images are created, when a Dockerfile is present in the build_docker_dir.

Mission files

Mission rules are defined in the OMC_SYCONFDIR/mission directory. Each mission configuration file shares the same name as the directory. To create a new mission, example, the directory structure will be as follows:

OMC_SYSCONFDIR/
    mission/
        example/
            example.ini
            supporting_text.md.in

Sections

meta

Release formatters

template:filename

Sections starting with template: will make use of the built-in template engine, replacing any strings in filename, relative to the mission directory, with expanded data values. The output will be written to destination.

# example.ini
[template:supporting_text.md.in]
destination = {{ storage.delivery_dir }}/output_filename.md