halnasri-Resolve_TT_CONFIDENCE_Feedback by halnasri · Pull Request #132 · score-json/json

halnasri · 2025-11-13T15:33:26Z

This PR is for resolving review comments from aschemmel in the PR Review TT-CONFIDENCE #118

TSF/trustable/statements/JLS-08.md

LucaFgr · 2025-11-13T15:58:44Z

TSF/trustable/statements/JLS-09.md

+references:
+    - type: web_content
+      url: "https://score-json.github.io/json/main/concept.html#scoring"
+      description: "Description of the algorithm how scores are accumulated, reviewed."


what exactly is reviewed here? is the description reviewed or how scores are accumulated?

The description here is just copied from Mr. Schemmel's PR. I guess it is referring to the fact that we updated the concept section of the report.

We should remove the "reviewed" and just have "Description of the algorithm on how scores are calculated and accumulated"

coveralls · 2025-11-13T16:02:46Z

coverage: 99.186%. remained the same
when pulling a6da396 on halnasri-Resolve_TT_CONFIDANCE_Feedback
into 9cb5a39 on main.

LucaFgr · 2025-11-13T16:03:06Z

TSF/trustable/statements/JLS-13.md

 level: 1.1
 normative: true
 references:
+        - type: project_website


Can you explain why these are 3 different types of references here? especially the first and third link look really similar to me

LucaFgr · 2025-11-13T16:05:10Z

TSF/trustable/statements/JLS-37.md

+      description: "Trustable Compliance Report showing scores for different TA items."
+---
+
+The confidence in the nlohmann/json library also incorporates confidence scores derived from other TA items.


I dont really understand this statement or the idea behind it. Can you explain?

I understand that you are trying to target this

However, I think the combination of JLS-08 and JLS-09 are already indirectly stating the same thing as this. Instead, I would reformulate it to something like
"High-level statements are broken down into smaller, recursive and definite expressions that can be proved to be either true or false."

LucaFgr · 2025-11-13T16:06:14Z

TSF/trustable/statements/JLS-40.md

+level: 1.1
+normative: true
+references:
+  - type: website_content


either website_content or web_content (line 8) is correct, but not both at the same time :)

LucaFgr · 2025-11-13T16:09:46Z

TSF/trustable/statements/JLS-41.md

+    description: "TSF documentation describing responsibilities, verification processes, and change control for score-json"
+---
+
+Responsibilities for manual verification and review follow documented, competence-based processes and guidelines, and the associated processes and checks are regularly reviewed and updated under defined change control.


Here we need the information to which repos/libraries this statement should apply. In the references you name score-json and nlohmann/json - is the statement meant for both?

Again, also needs to be more explicit or broken down into smaller statements.

What are the "responsibilities" and "competence based processes and guidelines"?

LucaFgr · 2025-11-13T16:10:39Z

TSF/trustable/statements/JLS-40.md

+    description: "TSF-related description of analysis, verification processes, and update concepts for score-json"
+---
+
+Manual verification activities that complement automated analysis for the nlohmann/json library are documented, reviewed against defined criteria, and considered for their impact on identifying and addressing misbehaviours.


What exactly is a "misbehavior"?

Misbehaviour is an established term in TSF (defined as 'any behaviour of the software that deviates from the defined expectations').

However, I think this statement is still on a too high level. It needs to be more explicit (or broken down into further supporting statements). What are the manual activities? what are the defined criteria?

For example:
"The manual activity of reviewing and addressing vulnerability/bug reports, is well-documented.. whatever"

Then reference to e.g., this https://github.com/nlohmann/json/security/advisories/new

Maybe even better examples:

"The manual activity of issuing a vulnerability or bug report for the nlohmann/json library is a well-defined process."

Reference: https://github.com/nlohmann/json/security/advisories/new

"All manual pull requests to the nlohmann/json repository are expected to describe the rationale behind any non-trivial changes, and link to an existing issue"

Reference: https://github.com/nlohmann/json/blob/49026f799983840d7cf1a8109ffffe7eb4b1012c/.github/CONTRIBUTING.md?plain=1#L70 and https://github.com/nlohmann/json/blob/49026f799983840d7cf1a8109ffffe7eb4b1012c/.github/CONTRIBUTING.md?plain=1#L74

"Feature requests for the nlohmann/json library are actively investigated by Niels Lohmann"

Reference: https://github.com/nlohmann/json/discussions/categories/ideas (showing that almost all feature requests are answered by Niels Lohmann)

LucaFgr · 2025-11-13T16:12:41Z

TSF/trustable/statements/JLS-36.md

+      description: "release management and update process description"    
+---
+
+All S-CORE methodologies governing releases and updates are defined and consistently followed.


I think the reference proves the "defined" part, but maybe we need another reference for the "consistently followed" part. Only because there is a readme describing the process it may not be ultimately followed.

TSF/trustable/statements/JLS-08.md

Erikhu1 · 2025-11-14T09:28:09Z

TSF/trustable/statements/JLS-08.md

+references:
+    - type: web_content
+      url: "https://score-json.github.io/json/main/generated/trustable_report_for_Software.html#compliance-for-trustable"
+      description: "Trustable Compliance Report showing the overall trustability score for nlohmann/json."


Please link to the report of the target repo instead:
https://eclipse-score.github.io/inc_nlohmann_json/main/generated/trustable_report_for_Software.html#compliance-for-trustable

Actually I'm not convinced if this reference should be included at all.

yes good point, the statement is just about that each statement is scored and says nothing about the overall score. (I will delete this reference)

yes good point. I had a look at it again and noticed that the statement is only about the fact, that each statement is scored and doesnt say anything about the overall score. (so I will delete this reference)

Erikhu1 · 2025-11-14T09:50:55Z

TSF/trustable/statements/JLS-08.md

+      description: "Trustable Compliance Report showing the overall trustability score for nlohmann/json."
+    - type: web_content
+      url: "https://score-json.github.io/json/main/generated/trustable_report_for_Software.html"
+      description: "Trustable Compliance Report showing scores for evidences."


I don't really think these references are sufficient for proving that each statement is indeed scored. Maybe you can use this link instead:
https://eclipse-score.github.io/inc_nlohmann_json/main/generated/dashboard.html#summary

Here you can see the number of reviewed/unreviewed items.

Erikhu1 · 2025-11-14T10:22:29Z

TSF/trustable/statements/JLS-09.md

+references:
+    - type: web_content
+      url: "https://score-json.github.io/json/main/concept.html#scoring"
+      description: "Description of the algorithm how scores are accumulated, reviewed."


The description here is just copied from Mr. Schemmel's PR. I guess it is referring to the fact that we updated the concept section of the report.

We should remove the "reviewed" and just have "Description of the algorithm on how scores are calculated and accumulated"

Erikhu1 · 2025-11-14T13:00:40Z

TSF/trustable/statements/JLS-13.md

 ---

-The S-Core methodologies are followed.
+All contributions to the nlohmann/json repository are reviewed according to the project’s documented contribution and review process.


This sounds similar to JLS-06 and JLS-25.

I think we need to understand TA-METHODOLOGIES better

Erikhu1 · 2025-11-14T13:02:36Z

TSF/trustable/statements/JLS-36.md

+normative: true
+references:
+    - type: verbose_file
+      path: "/workspaces/json/TSF/README.md"


please use relative path instead (i.e., without the "workspaces/json")

Erikhu1 · 2025-11-14T13:16:31Z

TSF/trustable/statements/JLS-37.md

+      description: "Trustable Compliance Report showing scores for different TA items."
+---
+
+The confidence in the nlohmann/json library also incorporates confidence scores derived from other TA items.


I understand that you are trying to target this

However, I think the combination of JLS-08 and JLS-09 are already indirectly stating the same thing as this. Instead, I would reformulate it to something like
"High-level statements are broken down into smaller, recursive and definite expressions that can be proved to be either true or false."

Erikhu1 · 2025-11-14T13:20:56Z

TSF/trustable/statements/JLS-40.md

+    description: "TSF-related description of analysis, verification processes, and update concepts for score-json"
+---
+
+Manual verification activities that complement automated analysis for the nlohmann/json library are documented, reviewed against defined criteria, and considered for their impact on identifying and addressing misbehaviours.


Misbehaviour is an established term in TSF (defined as 'any behaviour of the software that deviates from the defined expectations').

However, I think this statement is still on a too high level. It needs to be more explicit (or broken down into further supporting statements). What are the manual activities? what are the defined criteria?

Erikhu1 · 2025-11-14T13:23:02Z

TSF/trustable/statements/JLS-41.md

+    description: "TSF documentation describing responsibilities, verification processes, and change control for score-json"
+---
+
+Responsibilities for manual verification and review follow documented, competence-based processes and guidelines, and the associated processes and checks are regularly reviewed and updated under defined change control.


Again, also needs to be more explicit or broken down into smaller statements.

What are the "responsibilities" and "competence based processes and guidelines"?

Erikhu1 · 2025-11-14T14:41:49Z

TSF/trustable/statements/JLS-37.md

+normative: true
+references:
+    - type: web_content
+      url: "https://score-json.github.io/json/main/generated/trustable_report_for_Software.html#compliance-for-ta"


Please always use the report of the inc_nlohmann_json repo

halnasri added 3 commits November 13, 2025 07:53

Resolve some of the TT-CONFIDENCE review comments

e0c2f91

added more statments to TA-METHODOLOGIES

571e525

changed references type

f23dbc7

halnasri requested review from Erikhu1 and LucaFgr November 13, 2025 15:33

github-actions bot added the M label Nov 13, 2025

halnasri changed the title ~~halnasri-Resolve-TT-CONFIDENCE-Feedback~~ halnasri-Resolve_TT_CONFIDENCE_Feedback Nov 13, 2025

LucaFgr reviewed Nov 13, 2025

View reviewed changes

TSF/trustable/statements/JLS-08.md Outdated Show resolved Hide resolved

LucaFgr reviewed Nov 13, 2025

View reviewed changes

github-actions bot removed the M label Nov 14, 2025

edited the type of references and added the https validator

427a6ab

github-actions bot added the L label Nov 14, 2025

Erikhu1 reviewed Nov 14, 2025

View reviewed changes

more clarified statments

dcc7e50

Erikhu1 reviewed Nov 14, 2025

View reviewed changes

halnasri added 2 commits November 14, 2025 13:10

corrected JLS-08 reference

b9cfa22

modified the statement of JLS-8

a6da396

Erikhu1 reviewed Nov 14, 2025

View reviewed changes

github-actions bot removed the L label Nov 15, 2025

halnasri mentioned this pull request Nov 18, 2025

halnasri-Resolve_TT_CONFIDENCE_Feedback score-json/nlohmann_json#3

Merged

Conversation

halnasri commented Nov 13, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Erikhu1 Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

coveralls commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Erikhu1 Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Erikhu1 Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Erikhu1 Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Erikhu1 Nov 14, 2025 •

edited

Loading

coveralls commented Nov 13, 2025 •

edited

Loading

Erikhu1 Nov 14, 2025 •

edited

Loading

Erikhu1 Nov 14, 2025 •

edited

Loading

Erikhu1 Nov 14, 2025 •

edited

Loading