From 317c6ef3c32e73e4927da242859e4dd1ec8e67e8 Mon Sep 17 00:00:00 2001 From: Joseph Hunkeler Date: Wed, 15 Feb 2017 14:32:24 -0500 Subject: Clarity5000 (#48) * We are not conda, but we do use it. * More elaborate explanations * Spelling and wording * Wording * Wording * Line break * Yes you need Conda. * To the point, nothing more * Reword intro * Replace old URLs with conda.io * Wording. * Start implementing a better intro * Renamed installation (remaining text) to further_reading * Re-add installation * Consolidated misc text * Moved system requirements section and added reference link * You have to start somewhere * Add jumpoff reference * Rewording and beef-up clarity * Remove system requirements section * Remove misc sections * Fix bad newline * Add warning - not sure we need it. We already tell them they need BASH. * No longer use the word metapackage. They are just packages after all. * Add removing an environment section * Reworded stack definition * This sentence will work, so help me... --- source/compat.rst | 2 +- source/contributing.rst | 9 ++-- source/faq.rst | 33 +++++++----- source/further_reading.rst | 125 +++++++++++++++++++++++++++++++++++++++++++ source/getting_started.rst | 83 +++++++++++++++++++++++++++++ source/index.rst | 30 ++++++++--- source/installation.rst | 128 +++++++++++++-------------------------------- source/releases.rst | 5 +- source/resources.rst | 8 +-- source/updating.rst | 81 +++++++--------------------- 10 files changed, 322 insertions(+), 182 deletions(-) create mode 100644 source/further_reading.rst create mode 100644 source/getting_started.rst diff --git a/source/compat.rst b/source/compat.rst index 07942fe..e16bb2b 100644 --- a/source/compat.rst +++ b/source/compat.rst @@ -14,7 +14,7 @@ If you spot a compatibility problem not listed here please let us know by sendin 12/23/2016 ========== -AstroPy v1.3 fully deprecated calls to ``astropy.io.fits.new_table``. The following packages are known to be incompatiable with this release: +AstroPy v1.3 fully deprecated calls to ``astropy.io.fits.new_table``. The following packages are known to be incompatible with this release: * ``calcos =< 3.1.8`` - Bugfix pending * ``costools <= 1.2.1`` - Bugfix pending diff --git a/source/contributing.rst b/source/contributing.rst index e0930c0..aeead5f 100644 --- a/source/contributing.rst +++ b/source/contributing.rst @@ -71,7 +71,7 @@ If you have taken the liberty of looking around the astroconda-contrib directory .. note:: - This is not an full Conda packaging tutorial. For more information about creating recipes from scratch, please refer to the `conda-build documentation `_. + This is not an full Conda packaging tutorial. For more information about creating recipes from scratch, please refer to the `conda-build documentation `_. **Hint:** Investigate the contents of the recipes in astroconda-contrib. For most cases, copying an existing recipe and changing its values will suffice. @@ -86,8 +86,7 @@ Go ahead and open ``meta.yaml`` with your favorite plain-text editor: .. caution:: - It is *highly recommended* that you enable "tabs to spaces" for your editor. Tab widths are unpredictable and may - cause Conda's YAML parser to fail. + It is *highly recommended* that you enable "tabs to spaces" for your editor. Tab widths are unpredictable and may cause Conda's YAML parser to fail. .. code-block:: sh @@ -190,7 +189,7 @@ As we can see below, ``sympy`` requires ``mpmath``, ``setuptools`` and ``python` - mpmath - python x.x -What does the ``x.x`` imply exactly? This instructs ``conda build`` *not* to proceed unless ``python=[version]`` has been issued as an argument on the commandline. If ``x.x`` is omitted here, the recipe will choose the version of Python currently active in your environment. In most cases it is best to be explicit rather than implicit when it comes to defining version requirements in Conda. +What does the ``x.x`` imply exactly? This instructs ``conda build`` *not* to proceed unless ``python=[version]`` has been issued as an argument on the command-line. If ``x.x`` is omitted here, the recipe will choose the version of Python currently active in your environment. In most cases it is best to be explicit rather than implicit when it comes to defining version requirements in Conda. The ``test`` section defines few useful lists, ``imports``, ``commands``, and ``requires``. While these are not *required* to be used in any given recipe, we do use them in AstroConda. The ``imports`` section is a list of Python module imports, the ``commands`` are executed in a basic shell environment, and finally ``requires`` defines any extraneous packages to be installed into the environment before running the tests. @@ -207,7 +206,7 @@ are executed in a basic shell environment, and finally ``requires`` defines any #requires: # - does not require any extra testing-related packages -If ``sympy`` provided a commandline utility named ``sympy-show``, you would use the ``commands`` section to verify the utility's functionality. A simple example of this would be to output a usage statement. +If ``sympy`` provided a command-line utility named ``sympy-show``, you would use the ``commands`` section to verify the utility's functionality. A simple example of this would be to output a usage statement. .. code-block:: sh diff --git a/source/faq.rst b/source/faq.rst index c5f4bdc..bc4f5d6 100644 --- a/source/faq.rst +++ b/source/faq.rst @@ -5,13 +5,15 @@ F.A.Q. What is the difference between Conda, Miniconda, Anaconda, and AstroConda? ========================================================================== -- **Conda** is a userland environment and package management utility maintained by Continuum Analytics, Inc. +- **Conda** is a user-land environment and package management utility maintained by Continuum Analytics, Inc. - **Miniconda** is distributed by Continuum Analytics, Inc. as a standalone installer, and provides a minimal conda-based environment (i.e. ``conda`` and ``python``) - **Anaconda** is distributed by Continuum Analytics, Inc. as a standalone installer, and provides a fully featured conda-based environment (i.e. ``conda``, ``python``, and commonly used utilities, libraries, and Python packages). -- **AstroConda** is a collection of Conda packages, also known as a "Conda Channel", to be used with existing Miniconda and Anaconda installations via ``conda create``, ``conda install``, and ``conda update``. Package recipes are freely available on `GitHub `_ and are maintained by the `Science Software Branch (SSB) `_ of the `Space Telescope Science Institute (STScI) `_. +- **AstroConda** is a collection of Conda packages, also known as a "Conda Channel", to be used with existing Miniconda and Anaconda installations via ``conda create``, ``conda install``, and ``conda update``. + +Note: Package recipes are freely available on `GitHub `_ and are maintained by the `Science Software Branch (SSB) `_ of the `Space Telescope Science Institute (STScI) `_. Why do you recommend installing Miniconda instead of Anaconda? ============================================================== @@ -38,14 +40,14 @@ This error may occur if... - STScI is experiencing a network or power outage - The AstroConda package repository crashed or rebooted due to routine patching -If you have verified your internet connection is functioning properly but our parent site (http://www.stsci.edu) is unresonsive, then feel free to try our mirror. +If you have verified your internet connection is functioning properly but our parent site (http://www.stsci.edu) is unresponsive, then feel free to try our mirror. Currently there are two ways to access it: Using conda directly ^^^^^^^^^^^^^^^^^^^^ -The basic ``conda`` installation commands accept ``-c`` or ``--channel`` argument, which allows you to select a different resository to install packages from. +The basic ``conda`` installation commands accept ``-c`` or ``--channel`` argument, which allows you to select a different repository to install packages from. .. code-block:: yaml @@ -80,6 +82,13 @@ Conda does not support CSH (C-Shell). Please switch to a POSIX-compatible shell STScI will not maintain a separate codebase of ``conda`` in order to implement CSH support. Feel free to create an issue with the `developers `_. +What do you mean by Conda "root" environment? +============================================= + +The root environment refers to the top-level directory structure created by the Miniconda or Anaconda installer. The package manager, Conda, resides under ``/home/username/*conda/bin``. So for example, if you execute ``source deactivate`` your ``PATH`` environment variable is reset to point to the root environment. Thus, installing software at this level will be installed in the root ``bin`` directory too. + +Please refrain from installing any software into the root environment with either ``pip`` or ``conda``. It is recommended to always create a new environment with ``conda create`` and install new software into it. It is far easier to delete a broken environment with ``conda env remove`` than it is to reinstall Miniconda or Anaconda in its entirety. Custom environments are safely abstracted away from the root and live under ``/home/username/*conda/envs/``. + I installed AstroConda packages into my [Mini/Ana]conda 'root' environment. What now? ===================================================================================== @@ -88,7 +97,7 @@ Please reinstall Miniconda (or Anaconda) from scratch. AstroConda uses ``source To clarify, installing AstroConda packages directly into the 'root' may cause Miniconda (or Anaconda) itself to be come unstable. In addition to this, removing packages from this environment is tedious and will likely break your Anaconda installation if you are not careful. *Reinstalling from scratch is the safest option.* How do I reinstall Miniconda? -============================= +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assumes Miniconda3 has been installed to the default location (``/home/username/miniconda3``). We will refer to your home directory as ``~/`` from here on. @@ -97,18 +106,18 @@ This assumes Miniconda3 has been installed to the default location (``/home/user # Make a backup of your existing installation if desired: $ tar cfz /safe/place/miniconda3_OLD.tar.gz ~/miniconda3 - # Remove miniconda3 + # Remove Miniconda3 $ rm -rf ~/miniconda3 - # Download the Miniconda3 installer for your platform from: http://conda.pydata.org/miniconda.html + # Download the Miniconda3 installer for your platform from: http://conda.io/miniconda.html # Execute the installer (where PLATFORM is one of Linux or MacOSX) $ bash /path/to/Miniconda3-latest-PLATFORM-x86_64.sh -The installer may offer to automatically edit ``~/.bash_profile`` and prepend ``~/miniconda3/bin`` to your ``PATH``. If you already have this entry in your ``PATH``, simply reponse with 'no'. Responding with 'yes' will add another entry even if it exists resuling in a ``PATH`` that looks much like: ``~/miniconda3/bin:~/miniconda3/bin:[...]``. +The installer may offer to automatically edit ``~/.bash_profile`` and prepend ``~/miniconda3/bin`` to your ``PATH``. If you already have this entry in your ``PATH``, simply respond with 'no'. Responding with 'yes' will add another entry even if it exists resulting in a ``PATH`` that looks much like: ``~/miniconda3/bin:~/miniconda3/bin:[...]``. How do I reinstall Anaconda? -============================= +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assumes Anaconda3 has been installed to the default location (``/home/username/anaconda3``). We will refer to your home directory as ``~/`` from here on. @@ -126,7 +135,7 @@ This assumes Anaconda3 has been installed to the default location (``/home/usern # Execute the installer (where VERSION is the version you downloaded, and PLATFORM is one of Linux or MacOSX) $ bash /path/to/Anaconda3-VERSION-PLATFORM-x86_64.sh -The installer may offer to automatically edit ``~/.bash_profile`` and prepend ``~/anaconda3/bin`` to your ``PATH``. If you already have this entry in your ``PATH``, simply reponse with 'no'. Responding with 'yes' will add another entry even if it exists resuling in a ``PATH`` that looks much like: ``~/anaconda3/bin:~/anaconda3/bin:[...]``. +The installer may offer to automatically edit ``~/.bash_profile`` and prepend ``~/anaconda3/bin`` to your ``PATH``. If you already have this entry in your ``PATH``, simply respond with 'no'. Responding with 'yes' will add another entry even if it exists resulting in a ``PATH`` that looks much like: ``~/anaconda3/bin:~/anaconda3/bin:[...]``. Why am I being prompted by NumPy/SciPy with a MKL 30-day trial warning? ======================================================================= @@ -167,7 +176,7 @@ defined in the environment. How often are updates released? =============================== -Updates to (STScI) software will be released as bugs are identified and squashed. The ``stsci-*`` metapackages, for example, provide "releases" (i.e. a set of software used by our internal pipelines). After installing a release it is then possible to upgrade to the latest out-of-band packages by simply running: +Updates to (STScI) software will be released as bugs are identified and squashed. The ``stsci-*`` packages, for example, provide "releases" (i.e. a set of software used by our internal pipelines). After installing a release it is then possible to upgrade to the latest out-of-band packages by simply running: ``conda update -n astroconda --all`` @@ -201,7 +210,7 @@ If you are already using AstroConda under a Python 2 environment, you may simply Why is IRAF/PyRAF less functional under Python 3? ================================================= -The Python code in ``stsdas``, for example, is targeted specifically for Python 2.7 and earlier. If the demand for Python3 support under IRAF is great enough we may be able to pull our resources to accommodate the community. It is recommended to install IRAF into its own environment under Python 2.7: +The Python code in ``stsdas``, for example, is targeted specifically for Python 2.7 and earlier. If the demand for Python 3.x support under IRAF is great enough we may be able to pull our resources to accommodate the community. It is recommended to install IRAF into its own environment under Python 2.7: ``conda create -n iraf27 python=2.7 iraf pyraf stsci && source activate iraf27`` diff --git a/source/further_reading.rst b/source/further_reading.rst new file mode 100644 index 0000000..82b616b --- /dev/null +++ b/source/further_reading.rst @@ -0,0 +1,125 @@ +############### +Further Reading +############### + +.. hmmmmmmmmmmmmmmmmmm what to do? + +Fine-tuning the Software +======================== + +If you are short on hard drive space, have a slow internet connection, or are simply not interested in installing *everything but the kitchen sink*; take a quick look at the package `manifest `_ and select a custom mix of packages tailored to your needs. + +.. code-block:: sh + + $ conda create -n [package [package ...]] + $ source activate + +For example, if the work you intend to perform requires ``drizzlepac`` and nothing else, you can simply create a custom environment that contains *only* ``drizzlepac`` and its dependencies. + +.. code-block:: sh + + $ conda create -n mydriz drizzlepac + $ source activate mydriz + + +Additional Software Considerations +================================== + +While our channel provides a suite of scientific software packages that are known to work well together and are supported by engineers from STScI, by default, Conda already has access to a hundreds of packages provided directly by Continuum Analytics, Inc. The AstroConda channel itself relies heavily on Continuum for its own dependencies, so you may find it beneficial to explore what is available to you. + +Conda's full documentation set covers topics ranging from installation to maintaining environments and is available from its creators and maintainers: http://conda.io/docs/using/index.html. + +Installing additional software into your AstroConda environment is as simple as: + +.. code-block:: sh + + $ source activate astroconda + $ conda install + +Often, the fastest way to discover if a package exists in the ``conda`` ecosystem is to try searching for it with ``conda search ``. A comprehensive list of software available directly from Continuum's default channel can be found here: http://repo.continuum.io/pkgs/. + +In addition to ``conda install`` the Python-standard tool ``pip`` is also available to install packages distributed through the Python Package Index (PyPI): + +.. code-block:: sh + + $ source activate astroconda + $ pip install + +Downgrading Packages +==================== + +Did a recent update break your code? For example, if a bug is introduced into ``stsci.tools``, you can, for example, easily downgrade it to a known-good version: + +.. code-block:: sh + + $ conda search stsci.tools + . 3.4.0 py35_6 http://ssb.stsci.edu/astroconda/linux-64 + * 3.4.1 py35_0 http://ssb.stsci.edu/astroconda/linux-64 + +The ``*`` denotes the current version installed locally, while the ``.`` indicates Conda has cached that version of the package on your hard drive for quick-use. No prefix implies the package is neither cached, nor installed. + +Now the only thing left to do, is to tell Conda to install the previous release of the package: + +.. code-block:: sh + + $ conda install stsci.tools=3.4.0 + +At this point you will prompted to downgrade the package and hopefully be back in business. However, if you find yourself in this predicament please send an email to help@stsci.edu describing the situtation in detail so that we may either begin working on a bug fix release to accommodate you, or offer alternative solutions the problem. + +(`ref `__) + + +Pinning Packages +================ + +.. caution:: Pinning packages has the potential to break Conda. Only pin packages as a last resort. + +Let's take the previous example one step further... Imagine ``stsci.tools`` is broken, and the hotfix release of ``3.4.2`` only partially solved the original issue. Now what? You still need to receive updates to other packages, but ``stsci.tools`` keeps trying to update back to ``3.4.2`` every time you touch ``conda update``. + + +.. code-block:: sh + + $ echo "stsci.tools <=3.4.0" > ${CONDA_PREFIX}/conda-meta/pinned + +From now on, future calls to ``conda update`` will omit ``stsci.tools`` while performing dependency resolution. However, a clear side-effect of this will also be losing the ability to update packages that depend strictly on version ``3.4.2``. Although this is not a permanent solution it can prove useful in a bad situation. + +(`ref `__) + +Removing a Conda Environment +============================ + +It is possible to remove a Conda environment by running: + +.. code-block:: sh + + $ conda env remove -n + +Let's assume you've created a small test environment, "simple", with only a few packages you are interested in, such as ``drizzlepac`` and ``hstcal``. + +.. code-block:: sh + + $ conda create -n simple drizzlepac hstcal + $ source activate simple + +You've played around for a bit, maybe calibrated some data or checked out a new feature, but at this point you've decided you no longer want the "simple" environment anymore. So you delete it: + +.. code-block:: sh + + $ source deactivate + $ conda env remove -n simple + +And then you quickly verify your "simple" environment no longer exists using ``conda env list``: + +.. code-block:: sh + + $ conda env list + # ["simple" is not listed] + astroconda /home/username/miniconda3/envs/astroconda + iraf27 /home/username/miniconda3/envs/iraf27 + 2016.2 /home/username/miniconda3/envs/2016.2 + root * /home/username/miniconda3 + + +.. rubric:: Footnotes + +.. [#archnote] Intel x86_64 architecture diff --git a/source/getting_started.rst b/source/getting_started.rst new file mode 100644 index 0000000..347c53d --- /dev/null +++ b/source/getting_started.rst @@ -0,0 +1,83 @@ +############### +Getting Started +############### + +.. _getting_started_jump: + +Installing Conda - The choice is yours +====================================== + +AstroConda is a third-party addon channel designed for use with the Conda package management system, so therefore in order to install software provided by our channel, you must first install a basic Conda environment on your system. This can be achieved in multiple ways (described below). Our channel's software is compatible with both of the ``2`` and ``3`` variants of Continuum Analytics, Inc.'s Miniconda and Anaconda distributions (i.e. Miniconda2, Miniconda3, Anaconda2, and Anaconda3). + +Miniconda2 and Miniconda3 provide a bare-minimum Conda root environment with Python 2.7 or Python 3.x respectfully. (*Recommended*) + +Anaconda2 and Anaconda3 are Continuum Analytics Inc.'s flagship products, and provide a full-featured Conda root environment as well as hundreds of useful tools, libraries, and utilities by default. + +Both of Continuum's official distributions support a variety of operating systems and architectures, however the AstroConda channel specifically provides packages for Linux and Apple OS X running on x86_64 Intel/AMD processors. It is important to note Microsoft Windows is not supported at this time. + +Now head over to **one** of the following sites and download a copy of the installer of your choice: + +- Download `Miniconda `_ +- Download `Anaconda `_ (OS X users should choose the command-line installer) + +The installation method used for Miniconda and Anaconda are identical, however keep in mind the scripts are written in BASH (not SH), so therefore you *must* execute the installer using ``bash``: + +.. code-block:: sh + + $ cd + $ bash + +After the installation is complete double-check the bottom of ``~/.bash_profile`` to ensure Miniconda or Anaconda has been added to your ``PATH``. Otherwise, you will be unable to successfully complete this guide. + +.. warning:: + + Conda requires BASH, or a BASH-compatible shell in order to function correctly. If your default shell environment is not BASH (see also, :ref:`system_requirements`), please execute ``bash -l`` before proceeding. + + From this point forward any time you wish to use Conda's environment activation script (i.e. ``source activate ``), you will need to execute ``bash -l`` prior to doing so. + + +Verifying your Conda Environment +================================ + +Execute the command: ``which conda`` + +If the path to ``conda`` (i.e. ``/home/username/miniconda3/bin/conda``), is not returned, continue reading, otherwise skip ahead to :ref:`configure_astroconda_channel`. + +If you answered ``Y`` or ``Yes`` when prompted during installation to place Miniconda or Anaconda in your ``PATH``, and ``which conda`` still does not return a path leading back to ``conda``, go ahead and execute ``source ~/.bash_profile``, then re-execute ``which conda``. If the path to ``conda`` appears, skip ahead to :ref:`configure_astroconda_channel`. + +However, if you answered ``N`` or ``No`` when prompted, you will need to fix your ``PATH`` manually. If you installed Miniconda or Anaconda using the defaults selected by the installer, but are not sure what the directory is named, use the following command to find out: + +.. code-block:: sh + + $ ls -d ~/*conda? + #[example output] + /home/username/miniconda3 + +Now append **one of the following** ``export`` commands that best matches the output of `ls -d` above to the bottom of ``~/.bash_profile`` using a plain-text editor: + +.. code-block:: sh + + export PATH="~/miniconda/bin:$PATH" + export PATH="~/miniconda3/bin:$PATH" + export PATH="~/anaconda/bin:$PATH" + export PATH="~/anaconda3/bin:$PATH" + +At this point, to assume the new environment with ``conda`` in your ``PATH``, open a new terminal or execute ``source ~/.bash_profile`` and continue on to :ref:`configure_astroconda_channel`. + +.. _configure_astroconda_channel: + +Configure Conda to use the Astroconda Channel +============================================= + +In order to install packages directly from the AstroConda channel you will need to append our URL to Conda's channel search path. + +.. code-block:: sh + + $ conda config --add channels http://ssb.stsci.edu/astroconda + # Writes changes to ~/.condarc + +Be aware that indiscriminately adding channels to your configuration, be it from `anaconda.org `_ or via direct-URL can effect the stability of software packages in your run-time environment. + +For example, if you add a channel found on anaconda.org because it contains a software package you're interested in, but it too provides the same software found in AstroConda, it's possible you may lose track of where packages are coming from. Or worse, the software you installed from the other channel was built incorrectly or did not account for a special case, so now the packages in your environment relying on this as a dependency could very well cease to function correctly. + +If you decide to have multiple channels defined in your configuration and bugs begin to appear, it may be best to check their origin before issuing a support ticket to help@stsci.edu. ``conda list`` can be used to display such information about the packages installed in your environment. diff --git a/source/index.rst b/source/index.rst index 52bc8da..125ac1c 100644 --- a/source/index.rst +++ b/source/index.rst @@ -1,19 +1,37 @@ -Welcome to AstroConda's documentation! -====================================== +Welcome to AstroConda Channel's documentation! +============================================== -AstroConda is a free software repository maintained by the `Space Telescope Science Institute `_ (STScI) in Baltimore, Maryland. This repository provides tools and utilities commonly used by the astronomical community. +AstroConda is a free Conda channel maintained by the `Space Telescope Science Institute `_ (STScI) in Baltimore, Maryland. This channel provides tools and utilities required to process and analyze data from the Hubble Space Telescope (HST), James Webb Space Telescope (JWST), and others. -The components provided through the `astroconda `_ channel on `ssb.stsci.edu `_ include most of the tools the user needs to work with Hubble Space Telescope (HST) data and data from other telescopes in a single Anaconda Python environment. Each of the components contains sets of packages which have also been built and tested to ensure that they work with each other in the same environment. +.. _system_requirements: -To receive AstroConda announcements, or to engage in general discussion, feel free to subscribe to our `mailing list `_. +System Requirements +------------------- + + - 64-bit Intel/AMD processor (x86_64) + - 64-bit Linux (glibc ≥ 2.12) or Mac OS X (≥ 10.7) + - BASH or ZSH as your default shell environment (T/CSH is NOT supported) + +Powered by Conda +---------------- + +Conda is an open-source software package management system provided and maintained by `Continuum Analytics `_. Many software packages, provided both by Continuum and through third parties, are able to be quickly and easily installed using the Conda utility. AstroConda serves as a third-party add-on channel to provide easy access to STScI's software packages. + +- **New to conda**: :ref:`getting_started_jump` +- **Familiar with conda**: :ref:`configure_astroconda_channel` + + +To receive AstroConda announcements, or engage in general discussion, feel free to subscribe to our `mailing list `_. .. toctree:: :maxdepth: 2 + getting_started installation - releases updating + further_reading + releases compat faq contributing diff --git a/source/installation.rst b/source/installation.rst index 13a9534..d000661 100644 --- a/source/installation.rst +++ b/source/installation.rst @@ -1,136 +1,82 @@ -############ -Installation -############ +########################## +Selecting a Software Stack +########################## -Before you begin, the list below contains known requirements and limitations of AstroConda: +A "stack" is a collection of software designed to target the various use cases of our end-users. The three officially supported stacks are as follows: - - This documentation targets Miniconda3 (i.e. Python 3), but the installation instructions work equally well with any of the Miniconda distributions. - - AstroConda supports Linux (glibc ≥ 2.12) and Mac OS X (≥ 10.7; 10.6 is NOT supported) - - AstroConda contains packages for 64-bit [#archnote]_ Python 2.7 and 3.5. - - Conda only supports BASH and ZSH environments. If you are a native CSH user, execute ``bash -l`` prior to performing the procedures detailed in this guide. - - IRAF users: After configuring Miniconda for use with AstroConda, refer to the :ref:`iraf_install` section of this guide to continue your installation. +- :ref:`standard_install` provides: + - The full compliment of STScI software and utilities + - Python 2.7 or 3.x +- :ref:`iraf_install` provides: + - The full compliment of STScI software and utilities + - A IRAF/PyRAF environment + - Python 2.7 only +- :ref:`pipeline_install_jump` provides: + - The data processing environment used by STScI operations and instrument teams + - Python 3.x only -Obtain Miniconda -================ -.. note:: +.. _standard_install: - Installing Miniconda3 will not prevent you from using Python 2. +Standard Software Stack (without IRAF) +====================================== - ``conda`` allows you to deploy multiple independent Python environments, at-will, under a single Miniconda installation. You may have trouble following along with this guide if you choose to install Miniconda2. +The package management system, Conda, is now configured to pull from our repository, so you may go ahead and install the ``stsci`` package. This package installs nearly all of the software provided by STScI in one shot. - -Go grab a copy of `Miniconda3 `_ from Continuum Analytics, Inc. Be sure to select the installation medium appropriate for your operating system (Linux or OS X) and architecture (64-bit). Follow the installation instructions for your platform provided on the download page. Before moving on to the next step, open a new terminal window, and run ``conda`` to verify your terminal session can find it. - - -Configure Conda -=============== - -In order to install packages directly from the AstroConda repository, you will need to configure Conda to do so. - -.. code-block:: sh - - $ conda config --add channels http://ssb.stsci.edu/astroconda - # Writes changes to ~/.condarc - -This command will append the AstroConda channel URL to Conda's channel search path. Be aware that adding additional `anaconda.org `_ or direct-url repositories can potentially effect thestability of AstroConda's run-time environment. - -For example, if you add a channel found on anaconda.org because it contains a software package you're interested in, but also provides the same software as AstroConda, it's possible you may lose track of where packages are coming from. If you decide to have multiple channels defined in your configuration and bugs begin to appear, it may be best to check their origin before issuing a support ticket to help@stsci.edu. ``conda list`` can be used to display such information about the packages installed in your environment. - - -Install AstroConda -================== - -.. attention:: - - If you are on an instrument team or need to calibrate data using the same environment as STScI operations, please `click here `_. - -Standard Installation (without IRAF) ------------------------------------- - -Now that Conda is configured to pull from our repository, you may go ahead and install the ``stsci`` metapackage. The example below generates a new conda environment named "astroconda", however, this naming convention is merely a suggestion. Feel free to use a name that works best for you. +The following example generates a new conda environment named "astroconda", however this naming convention is merely a suggestion. Feel free to use a name that works best for you. .. code-block:: sh $ conda create -n astroconda stsci -This will prompt you to confirm the installation of all the STScI packages available in AstroConda. After they are installed, the following command "activates" the astroconda environment, making these packages available to you. +After the installation is complete go ahead and activate the "astroconda" environment. This command only needs to be executed one time per terminal session. .. code-block:: sh $ source activate astroconda -.. note:: +To deactivate the "astroconda" environment, close your terminal window or run: - Though it is repeated a lot in examples, ``source activate astroconda`` only needs to be run once per terminal session. Running this command ensures subsequent commands will take effect only in the ``astroconda`` environment, keeping your system organized. +.. code-block:: sh - To deactivate the ``astroconda`` environment, close your terminal window or run ``source deactivate``. + $ source deactivate .. _iraf_install: -Legacy Installation (with IRAF) -------------------------------------- +Legacy Software Stack (with IRAF) +================================= +The maintainers of the AstroConda channel have limited resources to support :abbr:`IRAF (Image Reduction and Analysis Facility)`, but users that require the ability to run IRAF and PyRAF tasks may want to install it via AstroConda. For help with any issues that come up during installation or use, please visit the `PyRAF FAQ `_. **Linux users** please be sure to visit `this FAQ entry `_ for a quick guide to installing IRAF's 32-bit dependencies. -The developers of AstroConda have limited resources to support :abbr:`IRAF (Image Reduction and Analysis Facility)`, but users that require the ability to run IRAF and PyRAF tasks may want to install it through AstroConda. For help with many issues that come up during installation or use, please visit the `PyRAF FAQ `_. If you are running Linux be sure to visit `this FAQ entry `_ for a quick guide to installing IRAF's 32-bit dependencies. -.. attention:: +The package management system, Conda, is now configured to pull from our repository, so you may go ahead and install the ``stsci`` package, as well as ``pyraf``, and finally ``iraf``. The ``stsci`` package installs nearly all of the software provided by STScI in one shot, however if you prefer a slimmed down IRAF/PyRAF experience, feel free to omit it. - Usage of IRAF currently requires running in a Python 2.7 environment. - The instructions below will install IRAF into a separate, Python 2.7, - environment regardless of your default Python version or which environments - you've created previously. +Due to Python 3.x incompatibilities present in several tasks, it is recommended to install IRAF alongside Python 2.7. - This will keep your IRAF environment separate from your other day-to-day - environments, which will facilitate updating only one or the other, and allow - easier transition off in the event of deprecation. Simply ``source activate iraf27`` - for iraf work and ``source activate astroconda`` for day-to-day use. +The following example generates a new conda environment named "iraf27", however this naming convention is merely a suggestion, so please feel free to apply a name that works best for you. .. code-block:: sh - $ conda create -n iraf27 python=2.7 iraf pyraf stsci + $ conda create -n iraf27 python=2.7 stsci pyraf iraf -Then, just as with the default installation, it is necessary to activate the environment to make its commands and packages available. +After the installation is complete go ahead and activate the "iraf27" environment. This command only needs to be executed one time per terminal session. .. code-block:: sh $ source activate iraf27 - -Fine-tuning the Installation -============================ - -If you are short on hard drive space, have a slow internet connection, or are simply not interested in installing *everything but the kitchen sink*; please feel free to peruse the `manifest `_ and install a custom mix of packages tailored to your needs. +To deactivate the "iraf27" environment, close your terminal window or run: .. code-block:: sh - $ conda create -n astroconda [package [package ...]] - $ source activate astroconda - -Installing Other Packages -========================= - -AstroConda provides a suite of packages that are known to work well together and are supported by engineers from STScI. However, being built on top of the widely-used ``conda`` tools for managing Python environments, AstroConda also supports the installation of additional 3rd-party packages. + $ source deactivate -Full documentation of the ``conda`` tool is available from Continuum Analytics, Inc., its creators and maintainers: http://conda.pydata.org/docs/using/index.html. However, we have provided a brief explanation of 3rd-party package installation below for quick reference. -For scientific packages available through Miniconda, installation is as simple as: - -.. code-block:: sh +.. _pipeline_install_jump: - $ source activate astroconda - $ conda install name_of_pkg - -Often, the easiest way to see if the package is available through Miniconda is to try installing it. The full list of available packages is here: http://repo.continuum.io/pkgs/. - -The Python-standard tool ``pip`` is also available to install packages distributed through the Python Package Index (PyPI): - -.. code-block:: sh - - $ source activate astroconda - $ pip install name_of_pkg +Pipeline Software Stack +======================= -.. rubric:: Footnotes +Due to the nature of the pipeline software stack, the installation instructions have been consolidated under a separate section, :ref:`pipeline_install`. -.. [#archnote] Intel x86_64 architecture diff --git a/source/releases.rst b/source/releases.rst index 2c71c13..ab53a9a 100644 --- a/source/releases.rst +++ b/source/releases.rst @@ -1,12 +1,15 @@ +.. _pipeline_install: + Pipeline Releases ################# .. note:: + - A working installation of Miniconda or Anaconda is required. - Python 2.x.x is not supported (unless noted otherwise). - 32-bit operating systems are not supported. -Pipeline releases differ from the standard AstroConda distribution and serve a different purpose. The release files described below are immutable snapshots of STScI operational software, and can be used to replicate the environment used by STScI to perform mission-specific data processing. Be aware that upgrading packages with ``conda update [pkg]`` is not recommended as it will likely introduce unwanted bugs and/or break the environment all together. +Pipeline releases differ from the standard software stack and serve a different purpose. The release files described below are immutable snapshots of STScI operational software, and can be used to replicate the environment used by STScI to perform mission-specific data processing. Be aware that upgrading packages with ``conda update [pkg]`` or ``conda update --all`` is not recommended as it will likely introduce unwanted bugs and/or break the environment all together. If you have any questions, comments, or concerns related to pipeline releases please feel free to contact help@stsci.edu diff --git a/source/resources.rst b/source/resources.rst index 72aa243..2db2350 100644 --- a/source/resources.rst +++ b/source/resources.rst @@ -58,7 +58,7 @@ Miniconda & Anaconda (Continuum Analytics, Inc.) **Miniconda** -- http://conda.pydata.org/miniconda.html +- http://conda.io/miniconda.html **Anaconda** @@ -77,15 +77,15 @@ Conda **General Documentation** -- http://conda.pydata.org/docs/ +- http://conda.io/docs/ **Build Documentation** -- http://conda.pydata.org/docs/building/build.html +- http://conda.io/docs/building/build.html **Environment Management Documentation** -- http://conda.pydata.org/docs/using/envs.html +- http://conda.io/docs/using/envs.html **Support** diff --git a/source/updating.rst b/source/updating.rst index eedebfd..206f02c 100644 --- a/source/updating.rst +++ b/source/updating.rst @@ -1,98 +1,55 @@ -******** -Updating -******** +######################### +Updating a Software Stack +######################### -Anaconda's package manager, Conda, will not automatically update unless a newer version of a package is detected during a routine package installation. Suffice to say, unless you keep your packages up to date with ``conda update``, the packages installed in your Anaconda distribution will remain relatively static. +Conda, will not automatically update unless a newer version of a package is detected during a routine package installation. Suffice to say, unless you keep your packages up to date with ``conda update``, the packages installed in your environment will remain static. -Updating AstroConda -=================== There are few simple ways to update packages obtained from AstroConda: -Updating --------- +Updating via Metapackage +======================== .. code-block:: sh $ conda update -n astroconda stsci -This is best used by individuals that favor software stability over receiving the "bleeding edge". Remember, updating the ``stsci`` metapackage only effects packages part of the **official public release** of our software. Packages that are provided by the AstroConda repository, but are not controlled by the ``stsci`` metapackage **will not receive updates**. This holds true for all other metapackages as well (e.g. stsci-hst, stsci-jwst, stsci-data-analysis, etc). +This is best used by individuals that favor software stability over receiving the "bleeding edge". Remember, updating the ``stsci`` package only effects packages part of the **official release** of our software. Packages provided by the AstroConda channel, but are not controlled by the ``stsci`` package **will not receive updates**. This is true for other packages as well (e.g. stsci-hst, stsci-data-analysis, etc). + +To clarify what this does, if the ``stsci`` package (used to create your environment) has not been updated by STScI to accommodate recently added packages, or newer versions of those packages, nothing will be updated in your environment on behalf of ``stsci``. In general, to receive interim bug fix releases please consider updating all packages, or individual packages of interest. Updating All Packages ---------------------- +===================== .. code-block:: sh $ conda update -n astroconda --all -This will update the ``stsci`` metapackage, as well all other packages installed in your enviroment. [1]_ - -This updates **all packages** regardless if they were installed from AstroConda, Continuum, Inc., or other third party repositories defined in ``$HOME/.condarc``. +This will apply updates to all packages installed in your ``astroconda`` environment [1]_ regardless if they were installed via AstroConda, Continuum Analytics, Inc., or other third party channels defined within your ``$HOME/.condarc``. -(`ref `__) +(`ref `__) -Updating Packages Individually ------------------------------- - +Updating Individual Packages +============================ .. code-block:: sh - $ conda update -n astroconda stsci.tools + $ conda update -n astroconda If you are interested in receiving updates for a particular package, then this method is for you. Be aware that packages may depend on other packages, so the total list of package updates returned by this command will vary. -General Maintenance -=================== +Updating Conda +============== .. code-block:: sh $ source deactivate $ conda update --all -Keeping AstroConda up to date is important, but not nearly as important as keeping your 'root' (i.e. the base installation) updated as well. ``conda`` is like any other software project and it requires periodic refreshing to stay current with the latest changes. Failing to do this can (over time) cause side-effects such as, the inability to upgrade, install, remove, or search for packages. - -However, updating extremely old releases of ``conda`` to the latest version have been known to break Miniconda due to a variety of API changes in the code. If this happens to you, reinstalling Miniconda and regenerating your AstroConda environment is the fastest way to resolve the problem. Refer to the `FAQ `_ for more details. - -Downgrading Packages -==================== - -Did a recent update break your code? Don't wait around for a bugfix... Keep working. For example, if a bug is introduced into ``stsci.tools``, you can easily downgrade it to a known-good version: - -.. code-block:: sh - - $ conda search stsci.tools - . 3.4.0 py35_6 http://ssb.stsci.edu/astroconda/linux-64 - * 3.4.1 py35_0 http://ssb.stsci.edu/astroconda/linux-64 - -The ``*`` denotes the current version installed locally. - -Now the only thing left to do, is to tell Conda to install the previous release of the package: - -.. code-block:: sh - - $ conda install stsci.tools=3.4.0 - -At this point you should be back in business. - -(`ref `__) - - -Pinning Packages -================ - -.. caution:: Pinning packages has the potential to break Conda. Only pin packages as a last resort. - -Let's take the previous example one step further... Imagine ``stsci.tools`` is broken, and the hotfix release of ``3.4.2`` only partially solved the original issue. Now what? You still need to receive updates to other packages, but ``stsci.tools`` keeps trying to update back to ``3.4.2`` every time you touch ``conda update``. - - -.. code-block:: sh - - $ echo "stsci.tools <=3.4.0" > ${CONDA_PREFIX}/conda-meta/pinned - -From now on, future calls to ``conda update`` will omit ``stsci.tools`` while performing dependency resolution. However, a clear side-effect of this will also be losing the ability to update packages that depend strictly on version ``3.4.2``. Although this is not a permanent solution it can prove useful in a bad situation. +Keeping AstroConda packages up to date is important, but not nearly as important as keeping your 'root' (i.e. the base installation) updated as well. Conda is like any other software project and it requires periodic refreshing to stay current with the latest changes. Failing to do this can, over time, cause side-effects such as, the inability to upgrade, install, remove, or search for packages. -(`ref `__) +Updating extremely old releases of Conda to the latest version have been known to break the 'root' environment due to a variety of API changes in the code. There is not much STScI can do about this as Conda itself is not our product, however if this happens to you, reinstalling the latest release of Miniconda or Anaconda, then regenerating your AstroConda software environment is the fastest way to resolve the problem. Refer to the `FAQ `_ for more details. .. rubric:: Footnotes -- cgit