diff --git a/docs/source/architecture.rst b/docs/source/architecture.rst index 9233366c..37edc926 100644 --- a/docs/source/architecture.rst +++ b/docs/source/architecture.rst @@ -121,6 +121,8 @@ content names of which one or more must be present. XML processing ---------------- +.. note:: All XML elements are described further in :doc:`opgee-xml`. + AttrDef ~~~~~~~~~~ The XML element ```` defines metadata for an attribute of an OPGEE XML element. @@ -134,13 +136,36 @@ is used for both validation and to generate the interactive user interface. ProcessChoice and ProcessGroup ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -A ```` describes a set of ```` and ```` elements that +The ```` and ```` elements enable ``Fields`` to be used +as parameterized templates. A ```` encloses a set of ```` +and ```` elements that can be enabled or disabled as a set. The ```` element encloses multiple -```` elements and selects among them based on the value of an attribute +```` elements and selects among them based on the value of a ``Field`` attribute named in the ````, whose value must match the name of one of the enclosed ```` elements. -All XML elements are described further in :doc:`opgee-xml`. +When the model is built, all the ``Streams`` and ``Processes`` identified by +``StreamRef`` and ``ProcessRef`` elements within the enclosed ``>`` +elements are disabled, then the ``Streams`` and ``Processes`` within the selected +``ProcessGroup`` are enabled. In this way, groups +of related ``Processes`` can be turned on and off. + +The file ``etc/opgee.xml``, included in the OPGEE distribution, +defines a ``Field`` named ``template`` that provides the basis for common +oil and gas field configurations. The subcommand ``csv2xml`` allows a +user to define ``Fields`` using a small number of attributes by generating +XML from the columns in the CSV file and combining these with the +``template`` field in ``etc/opgee.xml``. This system is used to generate XML +definitions for the 9,000 Fields used in testing OPGEE. See the schematic +figure below for an illustration of one such field. + +.. figure:: images/gas_lifting.* + :figclass: align-center + + Schematic diagram of a "gas lifting field" from the "template" field + in ``etc/opgee.xml``, generated using OPGEE’s built-in “graph” command. + Rectangles indicate ``Process`` subclasses, and each arrow represents a + ``Stream``. Labels on arrows show the declared ``Stream`` contents. Assorted features diff --git a/docs/source/attributes.rst b/docs/source/attributes.rst index ae1f72c0..f7f16952 100644 --- a/docs/source/attributes.rst +++ b/docs/source/attributes.rst @@ -3,12 +3,16 @@ Attributes In OPGEE the term "attributes" refers to a class of model parameters that are pre-defined with metadata (attribute definitions) and are class-specific, i.e., they related to -Analysis, Field, Process, or subclasses of Process. +:py:class:`~opgee.analysis.Analysis`, +:py:class:`~opgee.field.Field`, +:py:class:`~opgee.process.Process`, or subclasses of ``Process``. Attributes are central to the operation of both the "Smart Defaults" and Monte Carlo simulation (MCS) subsystems. This page provides a brief overview of how attributes are used in these subsystems. See the pages linked below for more information. +Attributes are defined by XML element, in the file ``etc/attributes.xml``. + See :obj:`opgee.attributes` for the Python API and more detailed documentation. Attribute Definitions diff --git a/docs/source/calculation.rst b/docs/source/calculation.rst index 5f782dc4..8190aa10 100644 --- a/docs/source/calculation.rst +++ b/docs/source/calculation.rst @@ -25,15 +25,21 @@ The GHGs tracked by OPGEE processes are: * N\ :sub:`2`\ O * VOC (non-methane volatile organic compounds) -Per-process emissions are grouped into the following categories for reporting: - -* Combustion -* Land-use -* Venting -* Flaring -* Fugitives +Emissions of carbon monoxide (CO), carbon dioxide (CO\ :sub:`2`), methane (CH\ :sub:`4`), +volatile organic compounds (VOC), and nitrous oxide (N\ :sub:`2`\ O) from each ``Process`` +are tracked in units of Mg/day, subdivided into the following categories: + +* Combustion – emissions resulting from fuel combustion for energy use +* Land-use – emissions resulting from land-use changes at the Field +* Venting – intentionally released gases +* Flaring – combustion of gases without energy use +* Fugitives – unintentional release of gases, e.g., from leaking valves or pipes * Other + +Emissions are stored in a pandas ``DataFrame`` in the :py:class:`Emissions ` +class in the file emissions.py. + Global Warming Potentials ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -107,8 +113,8 @@ The values associated with these choices are shown below. Energy use ------------------ - -OPGEE tracks the consumption of energy by each process, including: +Each Process in OPGEE tracks its own energy consumption in units of million BTUs (mmbtu) +per day, subdivided into the following categories: * Natural gas * Upgrader proc. gas @@ -116,9 +122,11 @@ OPGEE tracks the consumption of energy by each process, including: * Crude oil * Diesel * Residual fuel -* Petroleum coke +* Petroleum coke (petcoke) * Electricity +The structure is implemented as a pandas ``Series`` in the :py:class:`Energy ` class in +the file energy.py. System Boundaries ------------------- diff --git a/docs/source/conf.py b/docs/source/conf.py index 8b788e16..9457e76b 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -23,7 +23,7 @@ # -- Project information ----------------------------------------------------- project = 'OPGEE' -copyright = '2022-2024, The Board of Trustees of the Leland Stanford Junior University' +copyright = '2022-2025, The Board of Trustees of the Leland Stanford Junior University' author = 'Adam Brandt, Richard Plevin, Wennan Long' # The short X.Y version. @@ -78,5 +78,7 @@ # -- Options for intersphinx extension --------------------------------------- -# Example configuration for intersphinx: refer to the Python standard library. -intersphinx_mapping = {'python' : ('https://docs.python.org/3/', None)} +intersphinx_mapping = { + 'python' : ('https://docs.python.org/3/', None), + # 'pint' : ('https://pint.readthedocs.io/', None), +} diff --git a/docs/source/images/gas_lifting.jpg b/docs/source/images/gas_lifting.jpg new file mode 100644 index 00000000..a656fb4b Binary files /dev/null and b/docs/source/images/gas_lifting.jpg differ diff --git a/docs/source/opgee-xml.rst b/docs/source/opgee-xml.rst index 9240b57e..4cd9ec12 100644 --- a/docs/source/opgee-xml.rst +++ b/docs/source/opgee-xml.rst @@ -1,11 +1,16 @@ XML File Format ==================== +.. note:: OPGEE uses the term "attribute" to refer to model variables settable + in OPGEE XML files. To avoid ambiguity, the named arguments to XML + elements (often called "attributes" in XML descriptions) are here + referred to as "keywords". + The `opgee` package includes two built-in XML files: -* `opgee/etc/opgee.xml` describes the default model, and +* ``etc/opgee.xml`` describes the default model, and -* `opgee/etc/attributes.xml` describes attributes of various model components. +* ``etc/attributes.xml`` describes OPGEE attributes of various model components. Users can provide their own XML files which can be merged with or used in place of, the built-in definitions. The elements of the two files are described below, @@ -15,8 +20,8 @@ XML file merging ----------------- The user can specify multiple XML filenames to the `opg run` sub-command. The purpose of this feature is to allow users to use the built-in model specification as much as possible, by -specifying only modifications changes to that model. The built-in files etc/opgee.xml and -etc/attributes.xml are first loaded unless the user specifies `--no-default-model` on the command-line +specifying only modifications changes to that model. The built-in files ``etc/opgee.xml`` and +``etc/attributes.xml`` are first loaded unless the user specifies `--no-default-model` on the command-line to the `opg run` sub-command. Files are merged in the order they are given on the command-line. How merging works @@ -31,19 +36,55 @@ it needs to specify only For an elements to "match" a previously defined one, it must be in the position in the XML hierarchy where each element in the hierarchy of the new and old models have the same tag -and attributes (attribute order is irrelevant). +and keyword (keyword order is irrelevant). An element is inserted into the XML in the position where a non-matching element first appears -in the structure. **Give example** +in the structure. If a match is found and the attribute ``delete=true`` is specified in the new XML, that element and everything below it in original XML is deleted, and any XML below this element is inserted -in its place. **Give example** +in its place. + +In the example below, Fragment 2 is merged into Fragment 1, resulting in the final XML block, +with the element ``lmnop`` deleted and the element +``abc`` added after the elements forming its structural siblings. + +.. code-block:: xml + :caption: Fragment 1 + + + + abcdef + xyz + lmnop + + + +.. code-block:: xml + :caption: Fragment 2 + + + + abc + + + + +.. code-block:: xml + :caption: Merge of fragments 1 and 2 + + + + abcdef + xyz + abc + + If a match is found and the attribute ``delete=true`` is *not* specified in the new XML, the matching process proceeds recursively to the next lower level of the XML structure. If a final XML element (i.e., one with no children) matches, its text is copied in place of any that appeared -in the currently merged XML. **Give example.** +in the currently merged XML. More examples are available in the ``tests/test_merge_xml.py``. --------------------------------------------------------------------------------------------------------- @@ -56,7 +97,18 @@ The elements that comprise the ``opgee.xml`` file are described below. ^^^^^^^^^^ The top-most element, ````, encloses one or more ````, -````, and ```` elements. The ```` element takes no attributes. +````, and ```` elements. The ```` element takes no XML keywords. +The ```` element is represented by the class :py:class:`~opgee.model.Model`. + +OPGEE attributes supported by ```` include: + +* ``maximum_iterations`` – the maximum number of times a Process cycle is run + before raising an exception that terminates the model run. + +* ``maximum_change`` – the threshold for change variables to end process cycles. + This requires that a Process identify a change variable, whose value is tested + to determine if the cycle has converged, i.e., the change between successive + iterations is less than the value of the ``maximum_change`` attribute. .. [Saved for later] @@ -68,11 +120,12 @@ The top-most element, ````, encloses one or more ````, ^^^^^^^^^^^^^^^ -The ```` element is used to set values for attributes in a ````, ````, +The ```` element is used to set values for OPGEE attributes in a ````, ````, ```` or ```` element. -The XML schema allows model elements to have zero or more attributes. The XML schema ensures that the + +The XML schema allows model elements to have zero or more OPGEE attributes. The XML schema ensures that the ```` elements are syntactically correct. The semantics of these elements is defined by a corresponding -```` element (see attributes.xml) in the named class. For example, +```` element (see ``etc/attributes.xml``) in the named class. For example, .. code-block:: xml @@ -81,9 +134,9 @@ The XML schema allows model elements to have zero or more attributes. The XML sc 2342 -provides values for two attributes, `weight` and `distance`, for the `SurveyShip` class. These have a -corresponding (**required**) ```` definition in attributes.xml that provides the units, description, type, and -default value: +provides values for two OPGEE attributes, `weight` and `distance`, for the `SurveyShip` class. +These have a corresponding (**required**) ```` definition in attributes.xml that +provides the units, description, type, and default value: .. code-block:: xml @@ -94,15 +147,15 @@ default value: The ```` element contains no further elements, but it contains an optional value for the attribute, -which if absent is assigned the default for this attribute. The `name` attribute must refer to -the name of an ```` defined in the built-in `attributes.xml` or a file loaded by +which if absent is assigned the default for this attribute. The ``name`` attribute must refer to +the name of an ```` defined in the built-in ``attributes.xml`` or a file loaded by the user. -.. list-table:: Attributes +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -114,15 +167,17 @@ the user. ^^^^^^^^^^^^^ This element contains one or more ```` or ```` elements and accepts one -required attribute, `name`. The ```` elements identify fields to include in the +required attribute, ``name``. The ```` elements identify fields to include in the analysis by field name, whereas ```` elements allow matching of group names indicated in ```` definitions, by direct string match or by regular expression match. -.. list-table:: Attributes +The ```` element is represented by the class :py:class:`~opgee.analysis.Analysis`. + +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -131,17 +186,32 @@ indicated in ```` definitions, by direct string match or by regular expre - (none) - text +The ``Analysis`` class defines the following OPGEE attributes in the file ``etc/attributes.xml``: + +* ``GWP_horizon`` – Selects the time horizon for global warming potential values. + Allowable values are 20 or 100 years. + +* ``GWP_version`` – Selects the IPCC Assessment Report whose GWP values should be used. + Allowable values are “AR4”, “AR5”, “AR5_CCF” (AR5 values that include with carbon-cycle + feedbacks), or “AR6”. + +* ``functional_unit`` – Can be set to either “oil” or “gas”. + +* ``boundary`` – Defines the system boundary to use. Allowable values are “Production”, + “Transportation”, “Distribution”, and “Refinery”. + + ^^^^^^^^^ The ```` element provides a system of keyword matching by which ```` elements can declare themselves members of a group, and ```` elements can reference members of the group. -.. list-table:: Attributes +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -167,11 +237,15 @@ This element describes an oil or gas field and its processes. ```` can contain more or more ````, ````, ````, ````, or ```` elements. -.. list-table:: Attributes +The ```` element is represented by the class :py:class:`~opgee.field.Field`. +The ``Field`` element supports a large number of OPGEE attributes. +See the file ``etc/attributes.xml`` for details. + +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -191,12 +265,13 @@ This element describes an oil or gas field and its processes. ^^^^^^^^^^^^^^^ This element contains one or more ````, ````, or ```` elements. +The ```` element is represented by the class :py:class:`~opgee.process.Aggregator`. -.. list-table:: Attributes +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -216,12 +291,15 @@ It must include a `class` attribute which identifies the Python class that implements the process. The identified class must be a subclass of `Process`. ``>`` elements may contain one or more ```` elements. +The ```` element is represented by subclasses of the class +:py:class:`~opgee.process.Process`. -.. list-table:: Attributes + +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -292,12 +370,14 @@ Example: ^^^^^^^^^^^^^^^ This element contains one or more ````, ```` or ```` elements. +The ```` element is represented by the class :py:class:`~opgee.stream.Stream`. + -.. list-table:: Attributes +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -322,25 +402,33 @@ This element contains one or more ````, ```` or ```` ele - 1 - bool -The `src` and `dst` attributes must be set to the names of Process subclasses that are the +The `src` and `dst` keywords must be set to the names of Process subclasses that are the source and destination, respectively, for the `Stream`. If no `name` is provided, the name becomes "{src} => {dst}", with the names of the source and destination processes substituted -for `{src}` and `{dst}`. The `impute` attribute defaults to "1" (true); if set to "0" (or +for `{src}` and `{dst}`. The `impute` keyword defaults to "1" (true); if set to "0" (or "false" or "no") the `Stream` will not be traversed during the `impute()` processing phase, which works backwards (upstream) from the `Streams` with exogenously-defined flow rates. +The Stream class accepts the following OPGEE attributes: + +* ``temperature`` – the Stream’s temperature. + +* ``pressure`` – the Stream’s pressure. + +* ``API`` – the API gravity of oil (if any) in the Stream. + ^^^^^^^^^^^^^^^^ This element must occur within a ```` definition. Component encloses a numerical value defining an exogenous component flow rate, -expressed in mmbtu/day for all components other than electricity, expressed in kWh/day. +expressed in ``mmbtu/day`` for all components other than electricity, expressed in ``kWh/day``. (See :obj:`opgee.stream.Stream` for a list of component names.) -.. list-table:: Attributes +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -350,26 +438,29 @@ expressed in mmbtu/day for all components other than electricity, expressed in k - text * - phase - yes - - "solid", "liquid" or "gas" - - str + - (none) + - "solid", "liquid", "gas" ^^^^^^^^^^^^^^^^ This element must occur within a ```` definition. The ```` element holds a string indicating a generic name for the substance found in the stream. This allows processes to find different input streams without reference to any -specific process name. The ```` element takes no XML attributes. +specific process name. The ```` element takes no XML keywords. ^^^^^^^^^^^^^^^^^^^ This element can contain multiple ```` elements. -.. list-table:: Attributes +The ```` element is represented by the class :py:class:`~opgee.process_groups.ProcessChoice`. + + +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -387,11 +478,13 @@ This element can contain multiple ```` elements. This element can contain multiple ````, ````, and ```` elements. That is, ```` elements can nest recursively, so there can be choices within choices. -.. list-table:: Attributes +The ```` element is represented by the class :py:class:`~opgee.process_groups.ProcessGroup`. + +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -404,11 +497,11 @@ That is, ```` elements can nest recursively, so there can be choi ^^^^^^^^^^^^^^^^^^^ This element identifies a ``Process`` by name for inclusion in a ````. -.. list-table:: Attributes +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -421,11 +514,11 @@ This element identifies a ``Process`` by name for inclusion in a ````. -.. list-table:: Attributes +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -441,11 +534,11 @@ This element defines one or more updates to a built-in CSV data file. The ``name`` attribute must be the basename of a built-in table. A ``TableUpdate`` element must contain one or more ```` elements. -.. list-table:: Attributes +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -457,14 +550,14 @@ A ``TableUpdate`` element must contain one or more ```` elements. ^^^^^^^^^^^^^^^^^^^ This element defines a single replacement value for a value in a built-in CSV data -file. The ``row`` and ``col`` attributes (both required) define the cell whose +file. The ``row`` and ``col`` keywords (both required) define the cell whose value is replaced by the content -.. list-table:: Attributes +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -501,20 +594,20 @@ attributes.xml .. the :ref:`query sub-command `, or by setting a value for .. the config variable ``GCAM.RewriteSetsFile``. -This is the top-level element in the `attributes.xml` file. It accepts -no attributes and contains only ```` elements. +This is the top-level element in the ``attributes.xml`` file. It accepts +no keywords and contains only ```` elements. ^^^^^^^^^^^^^^^^^ -This element describes attributes associated with an OPGEE class, whose -name is provide by the `name` attribute. ```` elements contain +This element describes OPGEE attributes associated with an OPGEE class, whose +name is provide by the `name` keyword. ```` elements contain any number of ```` and ```` elements. -.. list-table:: Attributes +.. list-table:: Keywords :widths: 10 10 10 10 :header-rows: 1 - * - Attribute + * - Keyword - Required - Default - Values @@ -527,14 +620,14 @@ any number of ```` and ```` elements. ^^^^^^^^^^^^ This element defines a named set of legal values. Both the `name` and -`default` attributes are required. The ```` element contains +`default` keywords are required. The ```` element contains one or more (more usefully, two or more) ``