diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 0fdc4c0..300ad2c 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -17,7 +17,7 @@ repos: hooks: - id: pycln args: [--config=pyproject.toml] - - repo: https://github.com/psf/black + - repo: https://github.com/psf/black-pre-commit-mirror rev: 24.1.1 hooks: - id: black diff --git a/README.md b/README.md index 8e69794..2558798 100644 --- a/README.md +++ b/README.md @@ -15,19 +15,8 @@ pinefarm is available via pip install pinefarm ``` -### Dev - -For development you need the following tools: - -- `poetry`, follow [installation - instructions](https://python-poetry.org/docs/#installation) -- `poetry-dynamic-versioning`, used to manage the version (see - [repo](https://github.com/mtkennerly/poetry-dynamic-versioning)) -- `pre-commit`, to run maintenance hooks before commits (see - [instructions](https://pre-commit.com/#install)) - -See [below](.github/CONTRIBUTING.md#non-python-dependencies) for a few more -dependencies (already available on most systems). +Please check the [online documentation](https://pinefarm.readthedocs.io/en/latest/install.html) for more detailed +instructions. ## Documentation - The documentation is available here: Docs diff --git a/docs/source/cli.rst b/docs/source/cli.rst index de0fc98..02f8cea 100644 --- a/docs/source/cli.rst +++ b/docs/source/cli.rst @@ -1,5 +1,5 @@ -``pinefarm`` command line interface -=================================== +Command line interface +====================== You can get the full update help of the command line interface (CLI) with: @@ -17,18 +17,18 @@ Installs various programs, used to run pinecards. ``run`` ------- -It is the main command provided, and it runs the specified pinecard in the +This is the main command provided, and it runs the specified pinecard in the context of the selected theory. The output will be stored in a directory in the current path, with the name -``-``, where: +``-``, where: - the first part is the name of the selected pinecard (that is also the name of the folder in which all the files are stored) - the second part is the timestamp of the moment in which the command is issued ``update`` ---------- +---------- Update the metadata of the specified grid, with the content of the ``metadata.txt`` file in the current version of the pinecard. diff --git a/docs/source/external/index.rst b/docs/source/external/index.rst index e17682b..8eb149a 100644 --- a/docs/source/external/index.rst +++ b/docs/source/external/index.rst @@ -1,41 +1,9 @@ -External runners -================ +Overview +======== -The ``pinefarm run`` is mainly a uniform interface to some Monte Carlo (and non) -generator that are able to produce PineAPPL grids. +`pinefarm` itself is mostly physics agnostic, but the external programs +contain the actual physics. -Internally the runners are managed through a class system, with a base class -:class:`~pinefarm.external.interface.External`, that defines the basic -steps and attributes, while implementing the common actions. - -Attributes: - -- ``name``: name of the dataset -- ``theory``: identifier of the theory -- ``pdf``: PDF used for the comparison -- ``timestamp``, *optional*: the timestamp of the previous run, if rerunning an - already present grid - -Computed attributes: - -- ``dest``: folder used for all the output -- ``source``: folder containing pinecard -- ``grid``: path of the computed grid -- ``gridtmp``: path used for auxiliary grid (removed at the end of the run) - -Steps: - -- :meth:`~pinefarm.external.interface.External.install`: further install - steps, needed for the runner (not needed if the runner available as a python - package on PyPI) -- :meth:`~pinefarm.external.interface.External.run`: compute the actual - predictions -- :meth:`~pinefarm.external.interface.External.generate_pineappl`: collect - predictions into a |pineappl| grid -- :meth:`~pinefarm.external.interface.External.results`: provide runner - results on chosen PDF, for comparison with |pineappl| ``convolute`` ones -- :meth:`~pinefarm.external.interface.External.annotate_versions`: collect - versions of all the program used to compute the results (for reproducibility) -- :meth:`~pinefarm.external.interface.External.postprocess`: apply any - further step specified in :doc:`postprocess file <../pinecards/postrun>`, save - :doc:`metadata <../pinecards/metadata>`, and compress the final grid +If you want to add a new program take a look to the interface class +:class:`~pinefarm.external.interface.External` +from which you should derive your new interface. diff --git a/docs/source/external/int.rst b/docs/source/external/int.rst new file mode 100644 index 0000000..addcd14 --- /dev/null +++ b/docs/source/external/int.rst @@ -0,0 +1,21 @@ +PDF integrability +================= + +These pseudo-observables can be used to enforce the integrability of PDFs in a PDF fit +- see e.g. :cite:`NNPDF:2021njg`. +This is a built-in interface. + +Pinecard structure +------------------ + +- The ``integrability.yaml`` file (compulsory). This file contains the |pid| and kinematics of the pseudo-observable. + +Additional metadata +------------------- + +No additional metadata is written. + +Output +------ + +No additional output files are created. diff --git a/docs/source/external/matrix.jpeg b/docs/source/external/matrix.jpeg new file mode 100644 index 0000000..a31cd3c Binary files /dev/null and b/docs/source/external/matrix.jpeg differ diff --git a/docs/source/external/mg5.png b/docs/source/external/mg5.png new file mode 100644 index 0000000..eeafd56 Binary files /dev/null and b/docs/source/external/mg5.png differ diff --git a/docs/source/external/mg5.rst b/docs/source/external/mg5.rst index 99fcc88..7161e44 100644 --- a/docs/source/external/mg5.rst +++ b/docs/source/external/mg5.rst @@ -1,39 +1,66 @@ -Mg5aMC\@NLO -=========== +MadGraph5_aMC\@NLO +================== -.. toctree:: - :maxdepth: 1 - :caption: Contents: +|mg5| :cite:`Alwall:2014hca` :cite:`Frederix:2018nkq` is a general-purpose event generator +that is mainly used to compute observables in double hadronic environments such as the LHC. - mg5_launch - mg5_cuts - mg5_patches - - -Runcard structure ------------------ +Pinecard structure +------------------ - The ``output.txt`` file (compulsory). This file contains the instructions to generate the source code for the relevant process. For details, please see - `arXiv:1804.10017 `_ and - `arXiv:1405.0301 `_. The variable + :cite:`Alwall:2014hca` and :cite:`Frederix:2018nkq`. The variable ``@OUTPUT@`` must be used to generate the directory containing the source files. -- The ``launch.txt`` file (compulsory). This file contains the instructions to - run the relevant process, including the relevant physical parameters and cuts, - more info in :doc:`mg5_launch`. - - The ``analysis.f`` file (compulsory). This Fortran file must fill the - histograms from which the ``HwU`` files (histograms with uncertainties) and - the PineAPPL grids are generated. Note that a single histogram must not + histograms from which the |hwu| files and + the |pineappl| grids are generated. Note that a single histogram must not contain more than 100 bins, otherwise |mg5| will crash. However, big histograms can be split up into multiple histograms, for which the runner - will merge the PineAPPL grids together. + will merge the |pineappl| grids together. - The ``*.patch`` file(s) (optional). These are one or more ``.patch`` files that are applied after |mg5| has generated the sources. +launch.txt (compulsory) +^^^^^^^^^^^^^^^^^^^^^^^ +This file contains the instructions to +run the relevant process, including the relevant physical parameters and cuts. + +Theory parameters +################# + +To insert the actual values of the theory parameters coming from the theorycard +we provide a special syntax. You can use the names +``@GF``, ``@MH@``, ``@MT@``, ``@MW@``, ``@MZ@``, ``@WH@``, ``@WT@``, ``@WW@``, +and ``@WZ@``, which will be replaced with their numerical values upon generation. +The names are the same as chosen by ``mg5_aMC``, but written in +uppercase and surrounded with ``@``. For details about more parameters, please +see the ``Template/NLO/Cards/run_card.dat`` file in |mg5|. + +Cuts +#### + +They are implemented in two steps: + +1. cuts relevant *variables* are defined +2. cuts *code* is implemented + +A list of available codes and variables can be obtained from the +`repository `_. + +Patches +####### + +For instance, to use a dynamical scale, a patch modifying ``setscales.f`` file +should be included in the directory. To create patches use the command ``diff +-Naurb original new > patch.patch``. The patches are applied in an unspecified +order, using ``patch -p1 ...``. + +A list of available patches can be obtained from the +`repository `_. + Additional metadata ------------------- @@ -42,10 +69,13 @@ Additional metadata - ``launch.txt``: contains the generated ``launch.txt`` script (after all substitutions have been done) - ``patch``: a list of patches' names, one per row (corresponding to those - described in :doc:`mg5_patches`) + described in Patches) - ``tau_min``: the minimum :math:`\tau` value set by the user - ``user_cuts``: user defined cuts and cuts values, one per row in the format - ``cut=value`` (cuts are those defined in :doc:`mg5_cuts`) + ``cut=value`` (cuts are those defined in Cuts) +- ``mg5amc_repo`` and ``mg5amc_revno``: The + repository and revision number of the |mg5| version that was + used to generate the grid. .. note:: @@ -58,7 +88,7 @@ Output - ``DATASET``: The directory created by ``mg5_aMC``. A few interesting files in this subdirectory are: - - ``Events/-/MADatNLO.HwU``: histograms with uncertainties (HwU) + - ``Events/-/MADatNLO.HwU``: |hwu| - ``Events/-/amcblast_obs_-.pineappl``: grids created by ``mg5_aMC``, not yet merged together @@ -68,6 +98,11 @@ Output - ``launch.txt``: Run card for the 'launch' phase, with all variables substituted to their final values - ``launch.log``: Output of the external runner during the 'launch' phase -- ``pineappl.convolute``: Output of ``pineappl convolute`` -- ``pineappl.orders``: Output of ``pineappl orders`` -- ``pineappl.pdf_uncertainty``: Output of ``pineappl pdf_uncertainty`` +- ``results.log``: The numerical results of the run, comparing the results of the + grid against the results from ``mg5_aMC``. The first column (PineAPPL) are the + interpolated results, which should be similar to the Monte Carlo (MC) results + in the second column. The third column gives the relative MC uncertainty + (sigma). The next column gives the differences in terms of multiples of sigma. + The final three columns give the per mille differences of the central, minimum, and + maximum scale varied results. Ideally the first two columns are the same and + the remaining columns are zero. diff --git a/docs/source/external/mg5_cuts.rst b/docs/source/external/mg5_cuts.rst deleted file mode 100644 index e7ddd42..0000000 --- a/docs/source/external/mg5_cuts.rst +++ /dev/null @@ -1,55 +0,0 @@ -Cuts -==== - -They are implemented in two steps: - -1. cuts relevant *variables* are defined -2. cuts *code* is implemented - -List of cuts ------------- - -- ``abscoscsmax`` -- ``abscoscsmin`` -- ``atlas_1jet_8tev_r06`` -- ``atlas_2jet_7tev_r06_0005`` -- ``atlas_2jet_7tev_r06_0510`` -- ``atlas_2jet_7tev_r06_1015`` -- ``atlas_2jet_7tev_r06_1520`` -- ``atlas_2jet_7tev_r06_2025`` -- ``atlas_2jet_7tev_r06_2530`` -- ``atlas_dy3d_8tev`` -- ``atlas_wzrap11_cf`` -- ``cms_2jet_3d_8tev`` -- ``cms_2jets_7tev_0005`` -- ``cms_2jets_7tev_0510`` -- ``cms_2jets_7tev_1015`` -- ``cms_2jets_7tev_1520`` -- ``cms_2jets_7tev_2025`` -- ``dyjj`` -- ``minetal`` -- ``mjj`` -- ``mmllmax`` -- ``mtw`` -- ``ptj1min`` -- ``ptl1min`` -- ``ptmiss`` -- ``ptzmax`` -- ``ptzmin`` -- ``yh`` -- ``yll`` -- ``yt`` -- ``yz`` -- ``yzmin`` - - -Cuts variables --------------- - -- ``abscoscs`` -- ``atlas_1jet_8tev_r06`` -- ``atlas_dy3d_8tev`` -- ``atlas_wzrap11_cf`` -- ``cms_2jet_3d_8tev`` -- ``mtw`` -- ``ptmiss`` diff --git a/docs/source/external/mg5_launch.rst b/docs/source/external/mg5_launch.rst deleted file mode 100644 index 2a850db..0000000 --- a/docs/source/external/mg5_launch.rst +++ /dev/null @@ -1,16 +0,0 @@ -Launch -====== - -Here is briefly recapped |mg5|'s '``launch.txt`` -special syntax that is allowed for (and introduced by) pinecards. - - -Special syntax --------------- - -Since the parameter values are inserted by ``run.sh``, do not insert the -numerical values into the text file but rather the run variables. Supported are -``@GF``, ``@MH@``, ``@MT@``, ``@MW@``, ``@MZ@``, ``@WH@``, ``@WT@``, ``@WW@``, -and ``@WZ@``. The names are the same as chosen by ``mg5_aMC``, but written in -uppercase and surrounded with ``@``. For details about more parameters, please -see the ``Template/NLO/Cards/run_card.dat`` file in |mg5|. diff --git a/docs/source/external/mg5_patches.rst b/docs/source/external/mg5_patches.rst deleted file mode 100644 index e166aa7..0000000 --- a/docs/source/external/mg5_patches.rst +++ /dev/null @@ -1,19 +0,0 @@ -Patches -======= - -For instance, to use a dynamical scale, a patch modifying ``setscales.f`` file -should be included in the directory. To create patches use the command ``diff --Naurb original new > patch.patch``. The patches are applied in an unspecified -order, using ``patch -p1 ...``. - -List of available patches -------------------------- - -``change_etaj_to_rapj`` -~~~~~~~~~~~~~~~~~~~~~~~ - -``no_pole_cancellation_checks`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -``set_tau_min`` -~~~~~~~~~~~~~~~ diff --git a/docs/source/external/pos.rst b/docs/source/external/pos.rst new file mode 100644 index 0000000..60af6e6 --- /dev/null +++ b/docs/source/external/pos.rst @@ -0,0 +1,21 @@ +PDF positivity +============== + +These pseudo-observables can be used to enforce the positivity of PDFs in a PDF fit +:cite:`Candido:2020yat,Collins:2021vke,Candido:2023ujx`. +This is a built-in interface. + +Pinecard structure +------------------ + +- The ``positivity.yaml`` file (compulsory). This file contains the |pid| and kinematics of the pseudo-observable. + +Additional metadata +------------------- + +No additional metadata is written. + +Output +------ + +No additional output files are created. diff --git a/docs/source/external/vrap.rst b/docs/source/external/vrap.rst new file mode 100644 index 0000000..d8ab84c --- /dev/null +++ b/docs/source/external/vrap.rst @@ -0,0 +1,20 @@ +Hawaiian Vrap +============= + +Hawaiian Vrap :cite:`Barontini:2023vmr` is a modified version of the original Vrap :cite:`Anastasiou:2003ds`. +It is used to compute observables in fixed-target Drell-Yan experiments. + +Pinecard structure +------------------ + +- The ``vrap.yaml`` file (compulsory). This file contains the kinematics of the calculation. + +Additional metadata +------------------- + +- ``vrap_version``: The Vrap version used to generate the grid + +Output +------ + +No additional output files are created. diff --git a/docs/source/external/yadism.png b/docs/source/external/yadism.png new file mode 100644 index 0000000..97960a8 Binary files /dev/null and b/docs/source/external/yadism.png differ diff --git a/docs/source/external/yadism.rst b/docs/source/external/yadism.rst index 6d5867f..76aaebb 100644 --- a/docs/source/external/yadism.rst +++ b/docs/source/external/yadism.rst @@ -1,15 +1,23 @@ Yadism ====== -Runcard structure ------------------ +|yadism| :cite:`yadism` is a coefficient functions library that is used to compute +fully inclusive structure functions and cross sections in deep-inelastic scattering (DIS) experiments. + +Pinecard structure +------------------ - The ``observable.yaml`` file (compulsory). This file contains the description of the observable requested (kind and kinematics), together with further parameters specifying the process, and the details of the |yadism| calculation +Additional metadata +------------------- + +- ``yadism_version``: The |yadism| version used to generate the grids + Output ------ -- ``DATASET.yaml``: is the other |yadism| output format, fully human readable +- ``.yaml``: is the other |yadism| output format, fully human readable (but a bit verbose) diff --git a/docs/source/index.rst b/docs/source/index.rst index 66aad39..4e1aceb 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -2,59 +2,45 @@ Welcome to pinefarm's documentation! #################################### -This documentation is about the python package used to generate the PineAPPL -grids out of the input pinecards. -This program is called `pinefarm` (and its CLI ``pinefarm``). +`pinefarm` serves as common interface to several other external programs to compute |pineappl| grids :cite:`Carrazza:2020gss`. +It is part of the Pineline framework :cite:`Barontini:2023vmr`. -Install ``pinefarm`` -==================== +We currently support: -There are two ways of installing ``pinefarm``, that are: +- |mg5i| |mg5| :cite:`Alwall:2014hca` :cite:`Frederix:2018nkq` +- |yadismi| |yadism| :cite:`yadism` +- Hawaiian Vrap :cite:`Barontini:2023vmr` (a modified version of Vrap :cite:`Anastasiou:2003ds`) +- PDF positivity observables :cite:`Candido:2020yat,Collins:2021vke,Candido:2023ujx` +- PDF integrability observables -- **production**: used by *final user*, simply run +.. |mg5i| image:: /external/mg5.png + :align: middle + :width: 150 - .. code-block:: sh +.. |yadismi| image:: /external/yadism.png + :align: middle + :width: 150 - pip install pinefarm +To run `pinefarm` you need two specify to sets of inputs: - and then use ``pinefarm`` as a command available in ``PATH`` +1. a theory runcard, as is used by `nnpdf `_. + The theory runcard defines the general parameters of the QCD framework, such as perturbative + orders, coupling strength or heavy quark masses. + A list of example theory runcards is also available + `in the repository `_ +2. a pinecard, as is described :doc:`here `. + The pinecard describes the actual measurement that is performed, e.g. observable definitions, + kinematic bins, or cuts. The pinecard will also determine which external program is executed. + A list of already available observables can be inspected in the + `pinecards repository `_. -- **develop**: used by the tools developer +Given those two things you can run - .. code-block:: sh +.. code-block:: sh - poetry install + pinefarm run - Then run with: - - .. code-block:: sh - - poetry run pinefarm - - -Non Python dependencies ------------------------ - -Even if the bootstrap script and the installation management try to reduce as -much as possible the amount of dependencies, still a few ingredients have to be -available on the system. - -To run the CLI: - -- ``python`` itself -- ``pip`` available as a module of the ``python`` that is running ``pinefarm`` - (as usually is) -- ``curl`` - -To install ``pineappl``: - -- ``pkg-config`` -- ``openssl.pc`` (e.g. on Debian available in the ``libssl-dev`` package) - -To install ``mg5amc@nlo`` and its dependencies: - -- ``gfortran`` -- ``wget`` +and the program will, if necessary, install the required external program and launch it's execution. .. toctree:: @@ -62,27 +48,23 @@ To install ``mg5amc@nlo`` and its dependencies: :hidden: :caption: Usage + install + pinecards + cli run output - cli - -.. toctree:: - :maxdepth: 1 - :hidden: - :caption: Runcards - - pinecards/index - pinecards/metadata - pinecards/postrun .. toctree:: :maxdepth: 1 :hidden: - :caption: Externals + :caption: Interfaces external/index external/mg5 external/yadism + external/vrap + external/pos + external/int .. toctree:: :maxdepth: 1 @@ -91,3 +73,4 @@ To install ``mg5amc@nlo`` and its dependencies: API indices + zzz-refs diff --git a/docs/source/install.rst b/docs/source/install.rst new file mode 100644 index 0000000..8ed9fee --- /dev/null +++ b/docs/source/install.rst @@ -0,0 +1,81 @@ + +Installation +============ + +Eventually create a new virtual environment and then just run + +.. code-block:: sh + + pip install pinefarm + + +Non Python dependencies +----------------------- + +Even if the installation management tries to reduce as +much as possible the amount of dependencies, still a few ingredients have to be +available on the system. + +To install `MadGraph5_aMC@NLO `_ and its dependencies: + +- ``gfortran`` +- ``wget`` + +and that will also require a development installation of ``pineappl`` which requires: + +- ``pkg-config`` +- ``openssl`` (e.g. on Debian available in the ``libssl-dev`` package) + +Note that if ``openssl`` is not present in your system, you can either try to install it with conda (if you are in a conda environment), or you can install it from source as following + +.. code-block:: sh + + + wget https://www.openssl.org/source/openssl-X.Y.Zk.tar.gz + tar -zxvf openssl-X.Y.Zk.tar.gz + cd openssl-X.Y.Zk + + ./config --prefix=$HOME/local --openssldir=$HOME/local/openssl + make + make install + +after which you will need to add the folder containing `openssl.pc` (`libssl.pc`) to your `PKG_CONFIG_PATH` environment variable. + + +Configure paths +--------------- + +Pinefarm is acting as an interface to other external programs, which it will try to install +and manage on its own (the additional dependencies above are the exception). +However, in some situtations it might be advantages to control path and executables by hand: +this can be configured via the ``pinefarm.toml`` file. + +Please take a look to `the template configuration `_ +provided in the repository for a list of available options. + +``pinefarm`` can still run without the configuration file present, by assuming some default values. + + +Install in development mode +--------------------------- + +For development you need in addition the following tools: + +- `poetry`, follow `installation instructions `_ +- `poetry-dynamic-versioning`, used to manage the version (see + `repo `_) +- `pre-commit`, to run maintenance hooks before commits (see + `instructions `_) + +Then you can run + +.. code-block:: sh + + poetry install + + +To access the CLI you can run + +.. code-block:: sh + + poetry run pinefarm diff --git a/docs/source/output.rst b/docs/source/output.rst index d40e992..894802c 100644 --- a/docs/source/output.rst +++ b/docs/source/output.rst @@ -1,62 +1,51 @@ What is all the output? ======================= -After having run ``pinefarm run DATASET THEORY`` (see :doc:`cli`), the script -prints a table, which is useful to quickly validate the MC uncertainty and the -interpolation error of PineAPPL. The last line shows the directory where all -results are stored, which has the form ``DATASET-DATE``, where ``DATASET`` is +After having run ``pinefarm run `` (see :doc:`cli`), the script +prints a table, which is useful to quickly validate the native output and the +interpolation error of |pineappl|. The last line shows the directory where all +results are stored, which has the form ``PINECARD-DATE``, where ``PINECARD`` is the value given to the run script and ``DATE`` is a numerical date when the generation was started. The date is added so runs for the same dataset do not overwrite each other's output. The most important file in the output directory is - ``DATASET-DATE/DATASET.pineappl.lz4`` + ``PINECARD-DATE/PINECARD.pineappl.lz4`` -which is the final PineAPPL grid. +which is the final |pineappl| grid. The remaining contents of this directory are useful for testing and debugging: - ``results.log``: The numerical results of the run, comparing the results of the - grid against the results from ``mg5_aMC``. The first column (PineAPPL) are the - interpolated results, which should be similar to the Monte Carlo (MC) results - in the second column. The third column gives the relative MC uncertainty - (sigma). The next column gives the differences in terms of multiples of sigma. - The final three columns give the per mille differences of the central, minimum, and - maximum scale varied results. Ideally the first two columns are the same and - the remaining columns are zero. -- ``time.log``: Total ``time`` needed for the run - -.. note:: - - The resulting PineAPPL grid will contain the metadata written in the - ``metadata.txt`` file. - - In addition, the script ``run.sh`` and PineAPPL will automatically add the - following metadata: - - - ``initial_state_{1,2}``: The hadronic initial states of the grid, given as - PDG ids, typically ``2212`` for protons, ``-2212`` for anti-protons, and so on. - - ``lumi_id_types``: The meaning of the luminosities IDs in the definition of - the luminosity function of a PineAPPL grid. This is set to ``pdg_mc_ids`` to - signal they are PDG ids (with a possible exception of the gluon, for which - ``0`` may be used). - - ``mg5amc_repo`` and ``mg5amc_revno`` (only :doc:`external/mg5` grids): The - repository and revision number of the Madgraph5_aMC\@NLO version that was - used to generate the grid. - - ``pineappl_gitversion``: The PineAPPL version that was used to generate the - grid. - - ``results``: The comparison of the HwU results against a convolution of the - PineAPPL grid with the PDF selected in ``launch.txt``. This is the same table - printed at the end by ``run.sh``, and is used to verify the contents of each - grid. It also stores the MC uncertainties. - - ``pinecard``: Madgraph5_aMC\@NLO's pinecard that was used to generate the grid. - Here all parameters are documented. - - ``pinecard_gitversion``: The git version of this repository that was used to - generate the grid. - - ``yadism_version`` (only :doc:`external/yadism` grids): The |yadism| version - used to generate the grid (if not a released version have been used it - includes also git details). + grid against the native results from the runner. + + +Metadata +-------- + +The resulting |pineappl| grid will contain the metadata written in the +``metadata.txt`` file. + +In addition, ``pinefarm`` will automatically add the following metadata: + +- ``initial_state_{1,2}``: The hadronic initial states of the grid, given as + |pid|, typically ``2212`` for protons, ``-2212`` for anti-protons, and so on. +- ``lumi_id_types``: The meaning of the luminosities IDs in the definition of + the luminosity function of a |pineappl| grid. This is set to ``pdg_mc_ids`` to + signal they are |pid| (with a possible exception of the gluon, for which + ``0`` may be used). +- ``pineappl``: The |pineappl| version that was used to generate the grid. +- ``pinefarm``: The ``pinefarm`` version that was used to generate the grid. +- ``pinecard``: `Base64 `_ encoded ``.tar.gz`` version + of the generating pinecard. To re-extract it on a UNIX system run + ``pineappl read --get pinecard PINECARD.pineappl.lz4 | base64 -d > PINECARD.tar.gz`` where ``PINECARD`` + is to be replaced with the actual file name. +- ``results``: The comparison of the raw generator results against a convolution of the + |pineappl| grid with the selected PDF. This is the same table + printed at the end by ``pinefarm run``, and is used to verify the contents of each + grid. +- ``results_pdf``: PDF used for the comparison table Runner dependent output ----------------------- diff --git a/docs/source/pinecards/metadata.rst b/docs/source/pinecards.rst similarity index 57% rename from docs/source/pinecards/metadata.rst rename to docs/source/pinecards.rst index 4dd4eb8..823f575 100644 --- a/docs/source/pinecards/metadata.rst +++ b/docs/source/pinecards.rst @@ -1,7 +1,15 @@ -Metadata -======== +Pinecards +========= -This file collects all metadata, which is written into the grid after +Pinecards are a directory with several files that specify the configuration of the +external program and thus which may differ largely. +While the specific details are discussed :doc:`on the dedicated pages `, +there are two common files, which we discuss here. + +metadata.txt +------------ + +This optional file collects all metadata, which is written into the grid after generation. Arbitrary ``key=value`` pairs are supported, the most common are: @@ -25,11 +33,23 @@ Arbitrary ``key=value`` pairs are supported, the most common are: - ``y_unit``: The unit for the cross section (typically ``pb`` for dimensionless observables, or ``pb/GeV`` or ``pb/GeV^2``). -This key-value pairs are written into the final PineAPPL, to allow the user +This key-value pairs are written into the final |pineappl|, to allow the user to easily identify what is stored in the grid and how it was generated. It also allows for easily plotting the contents of the grids. .. note:: Further metadata are specific for each external, you can find them in the - respective external page, in the section "Additional metadata". + respective :doc:`external page `, in the section "Additional metadata". + Also the CLI itself will add more metadata, as is described :ref:`here ` + +postrun.sh +---------- + +This is an optional BASH script which must be executable and which +is run after the successful generation of the +|pineappl| grid and can be used to perform additional operations, such as +rescaling. The environment variable ``$GRID`` contains the relative path the +|pineappl| grid. Typically, this file contains instructions to remap the +one-dimensional histograms generated by |mg5| into higher +dimensional ones with the proper limits. diff --git a/docs/source/pinecards/index.rst b/docs/source/pinecards/index.rst deleted file mode 100644 index f24e4e3..0000000 --- a/docs/source/pinecards/index.rst +++ /dev/null @@ -1,14 +0,0 @@ -Pinecards preparation -===================== - -Part of the pinecards is specific by external and documented in dedicated page - -Common structure ----------------- - -The following files are important for each data set; they must be in the folder -``pinecards/DATASET``, where ``DATASET`` is the NNPDF identifier for the -dataset. - -- The :doc:`metadata.txt ` file (optional). -- The :doc:`postrun.sh ` file (optional, must be executable). diff --git a/docs/source/pinecards/postrun.rst b/docs/source/pinecards/postrun.rst deleted file mode 100644 index 3c0d95d..0000000 --- a/docs/source/pinecards/postrun.rst +++ /dev/null @@ -1,9 +0,0 @@ -Postrun -======= - -This is a BASH script which is run after the successful generation of the -PineAPPL grid and can be used to perform additional operations, such as -rescaling. The environment variable ``$GRID`` contains the relative path the -PineAPPL grid. Typically, this file contains instructions to remap the -one-dimensional histograms generated by Madgraph5_aMC\@NLO into higher -dimensional ones with the proper limits. diff --git a/docs/source/refs.bib b/docs/source/refs.bib index e69de29..e2fd61e 100644 --- a/docs/source/refs.bib +++ b/docs/source/refs.bib @@ -0,0 +1,119 @@ +@article{Frederix:2018nkq, + author = "Frederix, R. and Frixione, S. and Hirschi, V. and Pagani, D. and Shao, H. -S. and Zaro, M.", + title = "{The automation of next-to-leading order electroweak calculations}", + eprint = "1804.10017", + archivePrefix = "arXiv", + primaryClass = "hep-ph", + reportNumber = "Nikhef/2018-015, TUM-HEP-1138/18, NIKHEF-2018-015, TUM-HEP-1138-18", + doi = "10.1007/JHEP11(2021)085", + journal = "JHEP", + volume = "07", + pages = "185", + year = "2018", + note = "[Erratum: JHEP 11, 085 (2021)]" +} +@article{Alwall:2014hca, + author = "Alwall, J. and Frederix, R. and Frixione, S. and Hirschi, V. and Maltoni, F. and Mattelaer, O. and Shao, H. -S. and Stelzer, T. and Torrielli, P. and Zaro, M.", + title = "{The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations}", + eprint = "1405.0301", + archivePrefix = "arXiv", + primaryClass = "hep-ph", + reportNumber = "CERN-PH-TH-2014-064, CP3-14-18, LPN14-066, MCNET-14-09, ZU-TH-14-14", + doi = "10.1007/JHEP07(2014)079", + journal = "JHEP", + volume = "07", + pages = "079", + year = "2014" +} +@misc{yadism, + author = "Candido, A. and Hekhorn, F. and Magni, G. and Rabemananjara, T. and Stegeman, R.", + title = "{YADISM: Yet Another DIS Module}", + note = "{in preparation}", + year = "2023" +} +@article{Carrazza:2020gss, + author = "Carrazza, S. and Nocera, E. R. and Schwan, C. and Zaro, M.", + title = "{PineAPPL: combining EW and QCD corrections for fast evaluation of LHC processes}", + eprint = "2008.12789", + archivePrefix = "arXiv", + primaryClass = "hep-ph", + doi = "10.1007/JHEP12(2020)108", + journal = "JHEP", + volume = "12", + pages = "108", + year = "2020" +} +@misc{Barontini:2023vmr, + author = "Barontini, Andrea and Candido, Alessandro and Cruz-Martinez, Juan M. and Hekhorn, Felix and Schwan, Christopher", + title = "{Pineline: Industrialization of High-Energy Theory Predictions}", + eprint = "2302.12124", + archivePrefix = "arXiv", + primaryClass = "hep-ph", + reportNumber = "TIF-UNIMI-2023-4, CERN-TH-2023-021", + month = "2", + year = "2023" +} +@article{Anastasiou:2003ds, + author = "Anastasiou, Charalampos and Dixon, Lance J. and Melnikov, Kirill and Petriello, Frank", + title = "{High precision QCD at hadron colliders: Electroweak gauge boson rapidity distributions at NNLO}", + eprint = "hep-ph/0312266", + archivePrefix = "arXiv", + reportNumber = "SLAC-PUB-10288, UH-511-1042-03", + doi = "10.1103/PhysRevD.69.094008", + journal = "Phys. Rev. D", + volume = "69", + pages = "094008", + year = "2004" +} +@misc{Candido:2023ujx, + author = "Candido, Alessandro and Forte, Stefano and Giani, Tommaso and Hekhorn, Felix", + title = "{On the positivity of MSbar parton distributions}", + eprint = "2308.00025", + archivePrefix = "arXiv", + primaryClass = "hep-ph", + reportNumber = "TIF-UNIMI-2023-21", + month = "7", + year = "2023" +} +@article{Candido:2020yat, + author = "Candido, Alessandro and Forte, Stefano and Hekhorn, Felix", + title = "{Can $ \overline{\mathrm{MS}} $ parton distributions be negative?}", + eprint = "2006.07377", + archivePrefix = "arXiv", + primaryClass = "hep-ph", + reportNumber = "TIF-UNIMI-2020-9", + doi = "10.1007/JHEP11(2020)129", + journal = "JHEP", + volume = "11", + pages = "129", + year = "2020" +} +@article{Collins:2021vke, + author = "Collins, John and Rogers, Ted C. and Sato, Nobuo", + title = "{Positivity and renormalization of parton densities}", + eprint = "2111.01170", + archivePrefix = "arXiv", + primaryClass = "hep-ph", + reportNumber = "JLAB-THY-21-3507", + doi = "10.1103/PhysRevD.105.076010", + journal = "Phys. Rev. D", + volume = "105", + number = "7", + pages = "076010", + year = "2022" +} +@article{NNPDF:2021njg, + author = "Ball, Richard D. and others", + collaboration = "NNPDF", + title = "{The path to proton structure at 1\% accuracy}", + eprint = "2109.02653", + archivePrefix = "arXiv", + primaryClass = "hep-ph", + reportNumber = "Edinburgh 2021/12, Nikhef-2021-013, TIF-UNIMI-2021-11", + doi = "10.1140/epjc/s10052-022-10328-7", + journal = "Eur. Phys. J. C", + volume = "82", + number = "5", + pages = "428", + year = "2022" +} diff --git a/docs/source/run.rst b/docs/source/run.rst index b5746d5..b659a63 100644 --- a/docs/source/run.rst +++ b/docs/source/run.rst @@ -1,11 +1,11 @@ -Generate a PineAPPL grid -======================== +Generating a PineAPPL grid +========================== -To generate a PineAPPL grid run: +To generate a |pineappl| grid run: .. code-block:: sh - pinefarm run + pinefarm run In order to get a list of available `pinecards `_ run: @@ -13,10 +13,7 @@ In order to get a list of available `pinecards ` Analogously for theories: @@ -24,11 +21,4 @@ Analogously for theories: pinefarm list theories -If any software is missing, it will be installed on the fly, including: - -- `Madgraph5_aMC@NLO `_ -- `PineAPPL `_ - -.. note:: - - Only the code relevant to run the selected pinecard will be installed. +Recall to set the ``theories`` parameter in :ref:`pinefarm.toml ` diff --git a/docs/source/shared/abbreviations.rst b/docs/source/shared/abbreviations.rst index ab1976e..7a9cb16 100644 --- a/docs/source/shared/abbreviations.rst +++ b/docs/source/shared/abbreviations.rst @@ -43,38 +43,26 @@ .. |OME| replace:: :abbr:`OME (operator matrix element)` +.. |hwu| replace:: + :abbr:`HwU (histograms with uncertainties)` + .. external .. |mg5| raw:: html - MadGraph5_aMC@NLO + MadGraph5_aMC@NLO .. |yadism| replace:: :yadism:`\ ` -.. |banana| replace:: - :banana:`\ ` - .. |pineappl| raw:: html - PineAPPL - -.. |APFEL| raw:: html - - APFEL - -.. |Pegasus| raw:: html - - Pegasus + PineAPPL .. |lhapdf| raw:: html lhapdf -.. |QCDNUM| raw:: html - - QCDNUM - .. |T| raw:: html ✓ diff --git a/docs/source/zzz-refs.rst b/docs/source/zzz-refs.rst new file mode 100644 index 0000000..392bb95 --- /dev/null +++ b/docs/source/zzz-refs.rst @@ -0,0 +1,7 @@ +References +---------- + +.. bibliography:: refs.bib + :all: + +.. see also https://sphinxcontrib-bibtex.readthedocs.io/en/latest/usage.html#unresolved-citations-across-documents diff --git a/pyproject.toml b/pyproject.toml index 5c8fc5b..08ba40a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -83,6 +83,7 @@ ipy = "ipython" test-mg5 = "pinefarm run TEST_RUN_SH theories/theory_213.yaml" test-yad = "pinefarm run HERA_CC_318GEV_EM_SIGMARED theories/theory_213.yaml" docs = { "shell" = "cd docs; make html" } +docs-server = { "shell" = "cd docs; make server" } docs-view = { "shell" = "cd docs; make view" } docs-clean = { "shell" = "cd docs; make clean" } docs-cleanall = { "shell" = "cd docs; make cleanall" } diff --git a/src/pinefarm/external/vrap.py b/src/pinefarm/external/vrap.py index f176f13..2e094ee 100644 --- a/src/pinefarm/external/vrap.py +++ b/src/pinefarm/external/vrap.py @@ -10,7 +10,7 @@ Vrap datatasets can also include ``cfactors`` which need to match the name of the kinematic files and will be applied to the corresponding run ex: if the kinematic file is call "906_bin0.dat" the corresponding cfactors - are "ACC_906_bin0.dat" and "QCD_906_bin0.dat" +are "ACC_906_bin0.dat" and "QCD_906_bin0.dat" """ import subprocess as sp