blob: e62576b068e363718596573f3d2e2e342cf43b62 (
plain) (
blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
|
# dm
a.k.a. Delivery Merge
## What does it do?
1. Install miniconda3 in the current working directory
2. Create a new environment based on an explicit dump file
3. Transpose packages from configuration file into the new environment
4. Generate a YAML, explicit, and freeze dump of the new environment
5. [TODO] Scan packages installed via configuration file and execute their tests
## Where should I run this?
Inside of a CI/CD pipeline.
## Usage
```
Create reproducible pipeline deliveries
--config Required: dm yaml configuration
-o --output-dir store delivery-related results in dir
-p --install-prefix path to install miniconda
--install-variant miniconda Python variant
-i --install-version version of miniconda installer
-h --help This help information.
```
## Configuration File
```yaml
# Define your delivery
delivery_name: hstdp
delivery_version: 2019.3
delivery_rev: 0
delivery_python: 3.6.8
# Create base environment using a previous delivery
# Note 1: This may be defined as a file path or URL
# Note 2: /dev/null ignores this feature
base_spec: "/dev/null"
# "exit" or "continue" on error
# TODO: NOT IMPLEMENTED
on_error: "exit"
# environment variables passed to all child processes
runtime:
TEST_BIGDATA: "https://bytesalad.stsci.edu/artifactory"
# Channel order is preserved
conda_channels:
- http://ssb.stsci.edu/astroconda
- defaults
- http://ssb.stsci.edu/astroconda-dev
# Deliver the following Conda packages
# Note: When using base_spec, these packages will replace existing ones
conda_requirements:
- drizzlepac=3.0.2rc4
- fitsblender=0.3.4.dev2
- photutils=0.6.dev121
- stsci-hst
- stsci.tools=3.5.2rc1.dev0
- stsci.skypac=1.0.4
- stwcs=1.5.1rc3.dev0
# Deliver the following pypi packages
# Note: Packages here will only be recorded for Conda's *.yml dumps
pip_requirements:
- git+https://github/user/repo.git@1.2.3#egg=repo
# Define additional pypi repositories
# TODO: NOT IMPLEMENTED
pip_index:
- https://bytesalad.stsci.edu/artifactory/api/pypi/datb-pypi-virtual
# Enable/disable integration testing (true/false)
run_tests: true
# Test runner
test_program: "pytest"
# Arguments to pass to test_program
test_args: "-v --bigdata"
# Conda packages to use while testing
# Note: Packages are not incorporated into the delivery
test_conda_requirements:
- pytest
# pypi packages to use while testing
# Note: Packages are not incorporated into the delivery
test_pip_requirements:
- ci-watson
```
## Execution example
```sh
$ ./build.sh # < for CentOS/RHEL 6
$ dub build # < for modern Linux/Darwin
$ ./dm --config special_delivery.yml \
--installer-version=4.5.12 \
# >>> Actual output here <<<
```
|
