diff --git a/README.rst b/README.rst index f581c68..04f2519 100644 --- a/README.rst +++ b/README.rst @@ -1,3 +1,6 @@ +.. image:: https://img.shields.io/pypi/pyversions/viresclient + :target: https://pypi.org/project/viresclient/ + :alt: Python Version .. image:: https://img.shields.io/pypi/v/viresclient :target: https://pypi.org/project/viresclient/ @@ -14,27 +17,36 @@ .. image:: https://zenodo.org/badge/138034133.svg :target: https://zenodo.org/badge/latestdoi/138034133 -:: - - pip install viresclient - -:: - - conda install -c conda-forge viresclient - -viresclient_ is a Python package which connects to a VirES server, of which there are two: *VirES for Swarm* (https://vires.services) and *VirES for Aeolus* (https://aeolus.services), through the WPS_ interface. This package handles product requests and downloads, enabling easy access to data and models from ESA's Earth Explorer missions, Swarm_ and Aeolus_. This service is provided for ESA by EOX_. For enquiries about the service and problems with accessing your account, please email info@vires.services. For help with usage, please email ashley.smith@ed.ac.uk (for Swarm data) or `raise an issue on GitHub`_. +``viresclient`` is a Python package which provides access to products (including on-demand processing) from two of ESA's Earth Explorer missions: `Swarm`_ and `Aeolus`_. This service is provided for ESA by EOX_. For enquiries about the service and problems with accessing your account, please email info@vires.services. For help with usage, please email ashley.smith@ed.ac.uk or `raise an issue on GitHub`_. -.. _viresclient: https://github.com/ESA-VirES/VirES-Python-Client .. _WPS: http://www.opengeospatial.org/standards/wps .. _Swarm: https://earth.esa.int/eogateway/missions/swarm .. _Aeolus: https://earth.esa.int/eogateway/missions/aeolus -.. _EOX: https://eox.at/category/vires/ +.. _EOX: https://eox.at/tag/vires/ .. _`raise an issue on GitHub`: https://github.com/ESA-VirES/VirES-Python-Client/issues -For code recipes and more, see `Swarm Notebooks`_ & `Aeolus Notebooks`_. To start experimenting right away, *viresclient* is installed on the "Virtual Research Environment" (VRE), which is a managed Jupyter-based system provided for ESA by EOX. The service is free and open to all, accessible through your VirES account - check the notebooks to read more and get started. +There are two VirES services (Virtual environments for Earth Scientists) which *viresclient* can communicate with: -.. _`Swarm Notebooks`: https://notebooks.vires.services -.. _`Aeolus Notebooks`: https://notebooks.aeolus.services +- *VirES for Swarm:* + + - Interact with the `VirES for Swarm graphical interface (web client) `_ + - Browse code recipes: `Swarm Notebooks `_ + - JupyterHub: `Swarm VRE (Virtual Research Environment) `_ + - Swarm data documentation: `Swarm handbook `_ + - *Note that this service is not only for Swarm*: + + - Multi-mission products including magnetometry from CHAMP, CryoSat-2, and more + - INTERMAGNET ground magnetometers via the ``AUX_OBS`` collection + - Custom geomagnetic model evaluation + + - Read more about the ecosystem: `Python tools for ESA's Swarm mission: VirES for Swarm and surrounding ecosystem `_ + +- *VirES for Aeolus:* + + - Interact with the `VirES for Aeolus graphical interface (web client) `_ + - Browse code recipes: `Aeolus Notebooks `_ + - JupterHub: `Aeolus VRE `_ + - `Aeolus data documentation `_ Data and models are processed on demand on the VirES server - a combination of measurements from any time interval can be accessed. These are the same data that can be accessed by the VirES GUI. *viresclient* handles the returned data to allow direct loading as a single pandas.DataFrame_, or xarray.Dataset_. @@ -109,3 +121,7 @@ You can reference *viresclient* directly using the DOI of our zenodo_ record. Vi | "We use the Python package, viresclient [1], to access [...] from ESA's VirES for Swarm service [2]" | [1] https://doi.org/10.5281/zenodo.2554162 | [2] https://vires.services + +You can also cite this paper: + + | Smith A.R.A., Pačes M. and Swarm DISC (2022) Python tools for ESA’s Swarm mission: VirES for Swarm and surrounding ecosystem. Front. Astron. Space Sci. 9:1002697. doi: 10.3389/fspas.2022.1002697 diff --git a/docs/available_parameters.rst b/docs/available_parameters.rst index 0ac9df7..8c6d046 100644 --- a/docs/available_parameters.rst +++ b/docs/available_parameters.rst @@ -361,6 +361,8 @@ NB: When using model names containing a hyphen (``-``) then extra single (``'``) ---- +.. _Swarm auxiliaries: + ``auxiliaries`` --------------- diff --git a/docs/capabilities.rst b/docs/capabilities.rst index 754d902..93194d2 100644 --- a/docs/capabilities.rst +++ b/docs/capabilities.rst @@ -22,11 +22,22 @@ VirES provides more than just *access* to data. Some operations can be peformed | :py:meth:`viresclient.SwarmRequest.get_orbit_number` | :py:meth:`viresclient.SwarmRequest.get_times_for_orbits` | :py:meth:`viresclient.SwarmRequest.get_collection_info` + **Auxiliary information** + | e.g. geomagnetic coordinates (``QDLat``), orbit ascending/descending (``OrbitDirection``), geomagnetic activity (``Kp, Dst, dDst``), solar position (``SunZenithAngle``), ... + | See :ref:`Swarm auxiliaries` **Geomagnetic model evaluation** - | Forwards evaluation of magnetic field models when a magnetic dataset is selected (e.g. ``MAGx_LR``). For more detail, see :ref:`Geomagnetic model handling`. - | :py:meth:`viresclient.SwarmRequest.available_models` - | :py:meth:`viresclient.SwarmRequest.get_model_info` - | `models` option in :py:meth:`viresclient.SwarmRequest.set_products` + + - Forward evaluation of magnetic field models when a magnetic dataset is selected (e.g. ``MAGx_LR``) and using the `models` option in :py:meth:`viresclient.SwarmRequest.set_products` + - Model evaluation at user-provided coordinates. See: + + - :py:meth:`viresclient.SwarmRequest.eval_model` + - :py:meth:`viresclient.SwarmRequest.eval_model_for_cdf_file` + + - For more detail, see :ref:`Geomagnetic model handling`, and: + + - :py:meth:`viresclient.SwarmRequest.available_models` + - :py:meth:`viresclient.SwarmRequest.get_model_info` + **Identifying conjunctions between spacecraft** | :py:meth:`viresclient.SwarmRequest.get_conjunctions` **Synchronous and asynchronous processing** diff --git a/docs/conf.py b/docs/conf.py index 2696c79..550b59f 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -16,7 +16,7 @@ # -- Project information ----------------------------------------------------- project = "viresclient" -copyright = "2018, Ashley Smith" +copyright = "2025, Ashley Smith" author = "Ashley Smith" diff --git a/docs/geomagnetic_models.rst b/docs/geomagnetic_models.rst index 036202e..ee1ebb6 100644 --- a/docs/geomagnetic_models.rst +++ b/docs/geomagnetic_models.rst @@ -8,9 +8,11 @@ The geomagnetic models provided by VirES are all based on spherical harmonics, t In VirES, we provide model evaluations calculated at the same sample points as the data products. This means that the spherical harmonic expansion is made at the time and location of each datapoint (e.g. in the case of ``MAGx_LR``, at every second). The main purpose is to provide the data-model residuals, or magnetic perturbations, useful for studying the external magnetic field, typically ionospheric in origin. -*Pending:* In a future VirES release, some interpolation will be used to provide the magnetic model predictions along the 50Hz ``MAGx_HR`` products (to improve speed). The predictions at the 1Hz (``MAGx_LR``) locations will be used and a cubic interpolation performed to provide the predictions at the 50Hz locations. +The magnetic data products provide the magnetic field vector as the parameter ``B_NEC`` in the geocentric NEC (North, East, Centre) frame, as well as the magnetic field intensity/magnitude (scalar), ``F``. When requesting models from VirES (by supplying the ``models`` kwarg to :py:meth:`viresclient.SwarmRequest.set_products`), the corresponding model predictions will be returned in parameters named ``B_NEC_`` or ``F_``. Alternatively, the data-model residual alone, named ``B_NEC_res_`` or ``F_res_`` can be returned directly by also supplying the kwarg ``residuals=True``. Models should be provided as a list, like ``models=["CHAOS", "IGRF"]``. -The magnetic data products provide the magnetic field vector as the parameter ``B_NEC`` in the NEC (North, East, Centre) frame, as well as the magnetic field intensity/magnitude (scalar), ``F``. When also requesting models from VirES (by supplying the ``models`` kwarg to :py:meth:`viresclient.SwarmRequest.set_products`), the corresponding model predictions will be returned in parameters named ``B_NEC_`` or ``F_``. Alternatively, the data-model residual alone, named ``B_NEC_res_`` or ``F_res_`` can be returned directly by also supplying the kwarg ``residuals=True``. Models should be provided as a list, like ``models=["CHAOS", "IGRF"]``. +When using high rate 50Hz ``MAGx_HR`` products, interpolatation is used to improve speed. The predictions at the 1Hz (``MAGx_LR``) locations are used and a cubic interpolation performed to provide the predictions at the 50Hz locations. This should be accurate enough for applications that we are aware of. To disable this interpolation, use ``do_not_interpolate_models=True`` in :py:meth:`viresclient.SwarmRequest.set_products`. + +To evaluate models at arbitrary times and locations, see: :py:meth:`viresclient.SwarmRequest.eval_model` and :py:meth:`viresclient.SwarmRequest.eval_model_for_cdf_file`. Available models ---------------- @@ -88,7 +90,8 @@ The predictions for these models are cached only at the positions and times defi The logic describing when the cache is used is as follows: -.. image:: https://mermaid.ink/img/pako:eNqFkk1v2zAMhv8KoVMLND20PRlDiyYxugKxW6wekCAeBkZiEgGyFEhUt6Dpf5_8kSw7zQdDfvnyISnzQ0inSGRibdwvuUXPUE1rC-l5vChSyIB0dq030SNrZx8uhyCMRveHN0ar0KsDjM_lSQzsmgMsenF8UemGjLaUWO_kScFqDxLllh6-rPz9pD21ZRi1DWCQKTA0bfFjuXHHXVA4wPxcKd2pSv-eL78H6tk_emWxzN_RxATtkUkerC0B5jfD183fLpWjANYx0G-UbPbQIMvtWcd0vblOBjsqHp9-zr7BzjsVJUMgQ5JJXR6ZXZMvvCV_9KQBbv-JtoivJ0RLf83L6XP5lEH5UsEir-C5eJ3lRV5W-TSl3w3pt8tZahY9aMvkd850P6gFxKDtBvrehluY3y0ncaUlhF034v9yxJVoyDeoVdqNj5ZRizRGQ7XI0lHRGqPhWtT2M1njTqX7zZVm50W2RhPoSmBk97a3UmTsIx1NU40bj83JRV1S0S9ht4uffwB28dYw?type=png +.. image:: https://mermaid.ink/img/pako:eNqFUstu2zAQ_BWCpwSQDT0jWwgSxE7QHuIGSFLAjlUUNLmWCEikwUca1_a_l3ql7qk6EOTM7sysyAOmkgHO8LaSv2hJlEGv97lA7ru7WDiqQlSKLS-sIoZLcXvZk2g0ujm-GCIYUeyIZufw3Goj6yNadeDs4pXXUHEBTusdFDC02SNKaAm31xt1M292jY0hXGhUEQPaoLoxH-xmre4K9BEtz5Fv8tOlW5fr7xo67R8dslo_vJPKOtFO0sF9aaOAlmF_Cv-mZBI0EtIg-CDUVHtUE0PLs8QwLsauQIwWd19-Pj6jnZLMUoM0VEANsMtBsw35ZEpQQ40bIPqHbSS-Pp_RcU9H60cXhijEhQG1k1V7AY291VwUqPPup1zG67ndcIr0rh3hfz3YwzWomnDm7v7QaOTYxawhx5nbMtgSW5kc5-LkSok18mUvKM6MsuBhJW1R4mxLKu1Odsfc373npFCk_kR3RLxJWQ8thWqs-nYQDNRcWmFwduW3tTg74A-cBUE6TtMoTdJpFCepHwUe3js49cdX08T30yiMw2CSJicP_27lHRH7gR8nwXQyjf0wnngYGDdSLbqX3T7wIeVDy_QhT38AUx3zUQ?type=png + https://mermaid.live/edit#pako: :alt: Flowchart showing the cache usage logic *Custom* configured models, e.g. ``CHAOS-Static(max_degree=80)``, are not cached and must be evaluated directly. diff --git a/docs/installation.rst b/docs/installation.rst index 77bde0f..336352c 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -3,41 +3,123 @@ Installation and First Usage .. note:: For VRE users (it's free! read more: `Swarm `_, `Aeolus `_), viresclient is already installed and configured so skip these steps -1. Installation ---------------- +1. Installation and updating +---------------------------- -Python ≥ 3.6 is required. Testing is primarily on Linux, but macOS and Windows should also work. Available through both pip and conda (conda-forge). +The package is available through both pip (PyPI) and conda (conda-forge). To add viresclient to an existing environment, use your preferred tool: .. tabs:: .. group-tab:: pip + To install: + + .. code-block:: sh + + python -m pip install viresclient + + To update: + + .. code-block:: sh + + python -m pip install --upgrade viresclient + + .. tip:: + + If you are using an online notebook (e.g. Jupyter, Google Colab), you can run (in a notebook cell): + + .. code-block:: sh + + %pip install --upgrade viresclient + + .. group-tab:: uv + + To install: + + (Assuming you use a `uv project `_) + .. code-block:: sh - pip install viresclient + uv add viresclient + + To update: + + .. code-block:: sh + + uv sync --upgrade-package viresclient .. group-tab:: conda + To install: + .. code-block:: sh conda install --channel conda-forge viresclient + To update: + + .. code-block:: sh + + conda update --channel conda-forge viresclient + .. group-tab:: mamba + To install: + .. code-block:: sh mamba install --channel conda-forge viresclient + To update: + + .. code-block:: sh + + mamba update --channel conda-forge viresclient + + Recommended setup if starting without Python already ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -There are many ways to work with Python. We recommend using conda/mamba to manage your programming environment because of the availability of many data science packages through conda. +There are many ways to work with Python. Depending on your computing environment, we recommend using either: + +- `uv `_: if your project relies specifically on Python packages (available via `PyPI `_) and you can use a system package manager (e.g. apt) to manage non-Python dependencies +- `Mamba `_ or `conda `_: which also manage non-Python dependencies (from `conda-forge `_) + + - Use mamba preferably. It is a drop-in replacement for conda and is separate from Anaconda Inc. + - Use conda if you have an existing Anaconda or Miniconda installation +- `Pixi `_: a newer tool that provides reproducible environments with lock files; works with both conda and pip packages (integrates with uv) + +This guidance is appropriate for unix-like platforms (macOS & Linux & WSL). If you are using Windows, we recommend using `Windows Subsystem for Linux (WSL) `_. While viresclient is compatible with Windows, you will get a more consistent experience with the scientific Python ecosystem by using WSL. .. tabs:: + .. group-tab:: uv + + 1. `Install uv `_ + 2. Create a new `project `_:: + + uv init --python 3.12 my-science-project + + 3. Enter the project and add some packages (including viresclient):: + + cd my-science-project + uv add viresclient scipy matplotlib cartopy + + 4. Run code within the uv-managed environment. Either: + + - Use `uv run `_ to run scripts + - Activate the environment manually (``source .venv/bin/activate``) + - Configure your IDE to use this environment + - `Launch JupyterLab together with your project's dependencies `_:: + + cd my-science-project + uv run --with jupyter jupyter lab + + You should avoid using pip to install or update packages. By instead using ``uv add ``, or ``uv sync --upgrade-package ``, this lets uv track your project's dependencies (via the ``pyproject.toml`` file) and the exact versions used (via the ``uv.lock`` file) to help with safer updates and additions to the environment, and to ensure reproducibility. + .. group-tab:: conda - 1. Install Miniconda: https://docs.conda.io/en/latest/miniconda.html + 1. (If you do not have conda already) Install Miniconda: https://docs.conda.io/en/latest/miniconda.html 2. Set the conda-forge channel as the priority to install packages from:: @@ -46,28 +128,55 @@ There are many ways to work with Python. We recommend using conda/mamba to manag You should do this to avoid mixing packages from the anaconda channel (which can result in broken environments), and try to get all packages from conda-forge where available for consistency. - 3. Create a new conda environment with some recommended packages, including viresclient:: + 3. Create a new environment with some packages (including viresclient):: - conda create --name myenv python=3.10 jupyterlab scipy matplotlib pandas xarray cartopy h5py netCDF4 pytables ipywidgets viresclient + conda create --name myenv python=3.12 jupyterlab scipy matplotlib cartopy viresclient 4. Activate the new environment (you do this each time you want to use it):: conda activate myenv - .. group-tab:: mamba + 5. Or run a command directly from within the environment (without needing to activate it), e.g.:: + + conda run -n myenv jupyter lab - `Mamba `_ is a drop-in replacement for conda. You can install it into an existing (base) conda environment (``conda install -c conda-forge mamba``) and then just use ``mamba`` in place of ``conda`` in any commands - mamba is significantly faster. You can also install *mambaforge* directly to get mamba and conda-forge immediately configured in the base environment. + .. group-tab:: mamba - 1. Download and install the `mambaforge installer `_ or check the `mamba documentation `_ + 1. Download and install `Miniforge `_ - 2. Create a new environment for your development work:: + 2. Create a new environment with some packages (including viresclient):: - mamba create --name myenv python=3.10 jupyterlab scipy matplotlib pandas xarray cartopy h5py netCDF4 pytables ipywidgets viresclient + mamba create --name myenv python=3.12 jupyterlab scipy matplotlib cartopy viresclient - 3. Activate it to use it:: + 3. Activate the new environment (you do this each time you want to use it):: mamba activate myenv + 4. Or run a command directly from within the environment (without needing to activate it), e.g.:: + + mamba run -n myenv jupyter lab + + + .. group-tab:: docker + + viresclient is available in several publicly available (experimental) Docker images: + + - `Swarm DISC SwarmPAL processor (swarmpal-processor) `_ + - `Python in Heliophysics Community environment (pyhc-environment) `_ + + .. group-tab:: cloud / online + + Online services provide access to a range of scientific software within a Jupyter environment. viresclient may be pre-installed or can be added as above. + + - `ESA Swarm VRE `_ + + - Run by us in connection with VirES (free self-signup, included with your VirES account) + - `ESA Datalabs `_ + + - Requires an authorised ESA Cosmos account + - `HelioCloud `_ + + - In development 2. First usage / Configuration ------------------------------ diff --git a/docs/notebook_intro.rst b/docs/notebook_intro.rst index 2455b5d..fe564f6 100644 --- a/docs/notebook_intro.rst +++ b/docs/notebook_intro.rst @@ -12,12 +12,12 @@ The VRE is a whole analysis suite, including many other libraries from the Pytho - View - Interact * - `Swarm-DISC/Swarm_notebooks `_ - - .. image:: https://jupyterbook.org/badge.svg + - .. image:: https://jupyterbook.org/v1/_images/badge.svg :target: https://notebooks.vires.services - .. image:: https://img.shields.io/badge/nbgitpuller-VRE--Swarm-blue :target: https://vre.vires.services/hub/user-redirect/git-pull?repo=https://github.com/Swarm-DISC/Swarm_notebooks&urlpath=lab/tree/Swarm_notebooks/notebooks/ * - `ESA-VirES/Aeolus-notebooks `_ - - .. image:: https://jupyterbook.org/badge.svg + - .. image:: https://jupyterbook.org/v1/_images/badge.svg :target: https://notebooks.aeolus.services - .. image:: https://img.shields.io/badge/nbgitpuller-VRE--Aeolus-blue :target: https://vre.aeolus.services/hub/user-redirect/git-pull?repo=https://github.com/ESA-VirES/Aeolus-notebooks&urlpath=lab/tree/Aeolus-notebooks/notebooks&branch=main