Updated rust coverage tools evaluation

docs/score_tools/tools_static_analysis_code_quality/symbol_report_blanket.rst

@@ -0,0 +1,185 @@
+..
+   # *******************************************************************************
+   # Copyright (c) 2025 Contributors to the Eclipse Foundation
+   #
+   # See the NOTICE file(s) distributed with this work for additional
+   # information regarding copyright ownership.
+   #
+   # This program and the accompanying materials are made available under the
+   # terms of the Apache License Version 2.0 which is available at
+   # https://www.apache.org/licenses/LICENSE-2.0
+   #
+   # SPDX-License-Identifier: Apache-2.0
+   # *******************************************************************************
+
+.. doc_tool:: symbol report and blanket
+   :id: doc_tool__symbol_report_blanket
+   :status: evaluated
+   :version: 1.90.0 (see [1])
+   :tcl: LOW
+   :safety_affected: YES
+   :security_affected: YES
+   :realizes: wp__tool_verification_report
+   :tags: tool_management, tools_static_analysis_code_quality
+
+Symbol Report and Blanket Verification Report
+=============================================
+
+Introduction
+------------
+Scope and purpose
+~~~~~~~~~~~~~~~~~
+`Symbol report` and `blanket` are tools for post-processing of code coverage data generated by Rust compiler.
+They are used to calculate code coverage based on the raw data generated by Rust compiler and provide a report
+with calculated coverage.
+
+Inputs and outputs
+~~~~~~~~~~~~~~~~~~
+| Inputs: Software sources (Rust), Coverage information (.profraw files)
+| Outputs: Coverage report with numbers
+
+.. figure:: _assets/symbol_report.drawio.svg
+  :width: 100%
+  :align: center
+  :alt: Symbol report and blanket overview
+
+  Symbol report and blanket overview
+
+Available information
+~~~~~~~~~~~~~~~~~~~~~
+- Version: >= 1.90.0 [1]_
+- Official repository: `ferrocene/blanket <https://github.com/ferrocene/ferrocene/tree/main/ferrocene/tools/blanket>`_, `ferrocene/symbol-report <https://github.com/ferrocene/ferrocene/tree/main/ferrocene/tools/symbol-report>`_
+- Additional information for usage in other safety projects: `ferrocene/code-coverage <https://public-docs.ferrocene.dev/main/certification/core/safety-plan/tools.html#code-coverage>`_
+
+
+Installation and integration
+----------------------------
+Installation
+~~~~~~~~~~~~
+To add the Code coverage to your project or module follow guidelines in `here <https://github.com/eclipse-score/tooling/blob/main/coverage/README.md>`_.
+
+Integration
+~~~~~~~~~~~
+Integrated in bazel.
+
+Environment
+~~~~~~~~~~~
+Requires Rust toolchain and Bazel build environment.
+
+Safety evaluation
+-----------------
+This section outlines the safety evaluation of `symbol report` and `blanket` for its use within the S-CORE project. This evaluation assumes that the Rust compiler is
+qualified and output of coverage data in `.profraw` format is correct. Due to that, we solely focus on post processing that is done by symbol report and blanket only.
+
+
+.. list-table:: Safety evaluation
+   :header-rows: 1
+   :widths: 1 2 8 2 6 4 2 2
+
+   * - Malfunction identification
+     - Use case description
+     - Malfunctions
+     - Impact on safety?
+     - Impact safety measures available?
+     - Impact safety detection sufficient?
+     - Further additional safety measure required?
+     - Confidence (automatic calculation)
+   * - 1
+     - Report statement, branch and function coverage
+     - | Reported lower statement/branch coverage than in coverage data
+       |
+       | The tool outputs statement or branch coverage values that are less than those found in the raw coverage data.
+     - yes
+     - | S-CORE project requires 100% code coverage for safety-related software development.
+       | If the tool underreports coverage, it will be investigated manually and in worst case may lead to unnecessary additional analysis and testing efforts.
+     - yes
+     - no
+     - high
+   * - 2
+     - Report statement, branch and function coverage
+     - | Reported higher statement/branch coverage than in coverage data
+       |
+       | The tool outputs statement or branch coverage values that are greater than those found in the raw coverage data.
+     - yes
+     - no
+     - no
+     - yes
+     - low
+   * - 3
+     - Report statement, branch and function coverage
+     - | Number of function calls wrong
+       |
+       | The tool reports an incorrect number of function calls, either higher or lower than what is present in the coverage data.
+     - yes
+     - | S-CORE project requires 100% statement, branch and function coverage for safety-related software development.
+       | If the tool underreports number of function calls, this is not critical if branch and statement coverage of corresponding functions are correct and expected (100%) [*]_.
+     - yes
+     - no
+     - high
+   * - 4
+     - Report statement, branch and function coverage
+     - | File not reported
+       |
+       | The tool does not include one or more files in the generated coverage report, even though these files are present in the input data.
+     - yes
+     - Manual review of coverage report.
+     - yes
+     - no
+     - high
+   * - 5
+     - Report statement, branch and function coverage
+     - | Report corrupted (e.g., unreadable, incomplete, or inconsistent data)
+       |
+       | The generated report is unreadable, incomplete, or contains inconsistent or mismatched data.
+     - yes
+     - Manual review of coverage report.
+     - yes
+     - no
+     - high
+
+.. [*] Expected that correctness of function calls are tested via mocking (e.g. gtest macro EXPECT_CALL()).
+
+
+Security evaluation
+-------------------
+This section outlines the security evaluation of Rustfmt for its use within the S-CORE project.
+
+
+.. list-table:: Security evaluation
+   :header-rows: 1
+
+   * - Threat identification
+     - Use case description
+     - Threats
+     - Impact on security?
+     - Impact security measures available?
+     - Impact security detection sufficient?
+   * - 1
+     - TBD
+     - TBD
+     - TBD
+     - TBD
+     - TBD
+
+Result
+~~~~~~
+
+Safety evaluation of rust coverage tools (`symbol report` and `blanket`) are availabele on the website of the tool provider `code-coverage <https://public-docs.ferrocene.dev/main/certification/core/safety-plan/tools.html#code-coverage>`_.
+However, in context of S-CORE project, we assume that the Rust coverage tools have low confidence level (compated to evaluation of tool provider).
+Thus, rust coverage tools (`symbol report` and `blanket`) require qualification for use in safety-related software development in context of S-CORE project according to ISO 26262.
+
+**Tool Qualification**
+-------------------------------------------
+Based on method: validation of the software tool.
+
+
+Requirements and testing aspects
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Requirements for testing and qualification should be derived from both the specific use cases
+of the project and the requirements or recommendations provided by the tool provider.
+Validation and qualification activities should then confirm that the tool meets these
+combined requirements in the project context.
+
+
+.. [1] The tool version mentioned in this document is preliminary.
+       Exact version shall be derived from qualified Rust compiler used in S-CORE project.