Arm enhancement

[pendingintent]

Added arm enhancements

• Drop down selection for arm type and arm dataOriginType
• Full audit trail for arm
• Arm Audit report included on edit.html

The vulnerability arises from directly embedding soa_id, a user-provided parameter (though currently validated as an int), into JavaScript code returned to the client. To fully mitigate any XSS risk, soa_id should be safely encoded before embedding it in HTML or JavaScript sent to the user.

The best fix in this case is to use html.escape() (from the standard library) or fastapi.escape() to ensure that if soa_id ever contains unexpected characters, they are properly escaped. However, since soa_id is expected to be an integer, you can safely convert it using str() before interpolation, and/or escape it for extra safety. Since only shown code snippets may be changed, import html if not present and use html.escape(str(soa_id)) when writing to HTML/JavaScript.

Changes to make:

• In src/soa_builder/web/app.py, import html near other imports.
• In the HTMLResponse containing window.location='/ui/soa/{soa_id}/edit', use html.escape(str(soa_id)) in the f-string to escape soa_id before using it.

To fix this issue, user-controlled input used inside an HTML (and specifically, JavaScript) context must be escaped to prevent arbitrary script execution. The most robust solution is to ensure arm_type_submission is properly escaped before insertion. Since the alert message is delivered inside a script block (not as HTML element content), you must encode the value so that quotes and special characters do not terminate the string or break out of the alert. The safest approach is to serialize the message argument to JavaScript using json.dumps, which will handle escaping of all special characters and quotes, making it safe for embedding in a JS string.

Additionally, other interpolated variables in the script (like soa_id) should ideally be validated or at least stringified/escaped, though if they only ever come from path parameters coerced as int, risk is limited.

You'll need to:

• Import json at the top if not already available (it already is).
• Replace the string interpolation for the alert argument with a call to json.dumps().
• For maximum safety, interpolate variables for the JS location assignment either outside the script or encode with urllib.parse.quote()—but since soa_id is always an int (from the route declaration), direct string interpolation does not present an XSS risk.

Replace the vulnerable line:

return HTMLResponse(
    f"<script>alert('Unknown Arm Type selection: {arm_type_submission}');window.location='/ui/soa/{soa_id}/edit';</script>",
    status_code=400,
)

With:

return HTMLResponse(
    f"<script>alert({json.dumps('Unknown Arm Type selection: ' + arm_type_submission)});window.location='/ui/soa/{soa_id}/edit';</script>",
    status_code=400,
)

This approach ensures the injected string is safely quoted and escaped in the rendered JS, preventing XSS.

---

To fix this reflected XSS vulnerability, we need to ensure that all user-provided data written into HTML/JavaScript contexts is properly escaped. For HTML responses (especially when embedding data inside inline <script> blocks), use html.escape() or the framework's equivalent (fastapi.escape is not available; use Python's standard library). For this case, both data_origin_type_submission and soa_id should be safely converted/escaped before interpolating them into the response.

• For data_origin_type_submission, use html.escape() to sanitize any user input before insertion.
• For soa_id, since it's an integer type, casting to str() and escaping is safe and recommended.
• Ensure html from the standard library is imported.

Required changes:

• In the src/soa_builder/web/app.py file, at the alert location:
  • Escape data_origin_type_submission before including it in the alert JavaScript.
  • (Optionally) Escape/cast soa_id when used in the URL in the same response string.
• Add import html to the imports at the top.

---

To remediate this XSS vulnerability, all user input rendered into HTML or JavaScript must be properly escaped or sanitized. In this case, since we're injecting user input inside a JavaScript string (inside <script>alert('...')</script>), it's necessary to escape or sanitize arm_type_submission so that special characters can't break out of the JavaScript string or introduce extra code.

The best approach is:

• Escape dangerous characters in arm_type_submission that could terminate the string or escape into further script execution. This means escaping characters such as quotes, backslashes, angle brackets, line breaks, etc.
• Use a helper function to safely encode such strings for JavaScript string contexts. Alternatively, escape the string for HTML context, which will protect against HTML-based injection but not necessarily for JavaScript contexts.
• For Python/FastAPI, you can use html.escape() for HTML, and for JavaScript, ensure everything including apostrophes and backslashes is escaped. One robust and readable way is to use json.dumps to serialize the string, which correctly escapes characters for use in JavaScript strings, as: json.dumps(arm_type_submission). (This produces a valid JS string literal.)

Files/regions to change:

• In src/soa_builder/web/app.py, within the ui_update_arm function, specifically the block returning the HTMLResponse containing the unescaped arm_type_submission at line 3760.

Required imports/methods:

• Import json (already present).
• Replace the use of f-string interpolation with proper JavaScript string escaping, ideally using json.dumps.

---

To fix the problem, user-supplied data (data_origin_type_submission and soa_id) must be escaped before it is interpolated into the JavaScript string literal in the response. Specifically, we should escape both for HTML (if they're interpolated outside tags) and for safe inclusion in JavaScript string literals, which is stricter (must escape single quotes, double quotes, newlines, backslashes, and closing script tag sequences). The best fix is to use the standard library or framework escaping functions to sanitize values both for HTML and JavaScript context.

• The recommended approach is to use the html.escape() function from the standard library for HTML, and additionally to escape single quotes and prevent closing </script> tag injection within the interpolated values.
• For the alert string (JS context, inside '...'), escape any single quote (') with a backslash, and for general XSS protection, also encode dangerous characters, which might be most easily done with a helper function.
• For soa_id used in the URL within JavaScript, ensure it's safe for both the HTML attribute and JS string – typically restrict to digits (it's an int, which is safe), but sanitize just in case.

Implementation plan:

• Define a helper function, e.g., js_string_escape, that escapes single quotes, backslashes, and </script> sequences.
• In the affected line, use this function for both data_origin_type_submission and soa_id before formatting them into the response string.
• Import the required standard library (html), and define the helper function within the file src/soa_builder/web/app.py.

Files/regions/lines to change:

• src/soa_builder/web/app.py:
  • Add import html at top if not present.
  • Add a helper js_string_escape() function near the top or above its use.
  • Replace line 3820 to escape data_origin_type_submission and soa_id.

---

[Copilot]

Pull request overview

This PR enhances arm management by adding dropdown selections for arm type and data origin type, implementing a full audit trail for arm operations, and displaying arm audit history in the edit interface. The changes also introduce caching mechanisms for biomedical concept categories to improve performance.

Key Changes:

• Added code mapping table and arm audit tracking infrastructure
• Implemented dropdowns for arm type (C174222) and data origin type (C188727) selection
• Added cache mechanisms with force refresh capability for concept categories

Reviewed changes

Copilot reviewed 11 out of 13 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/soa_builder/web/initialize_database.py	Creates new code table for storing terminology mappings
src/soa_builder/web/app.py	Adds caching for concept categories, enriches arm data with type/data_origin_type mappings, implements audit tracking for arm updates
src/soa_builder/web/templates/edit.html	Adds arm type and data origin type dropdown selectors, displays arm audit report, reorganizes layout
src/soa_builder/web/templates/concept_categories.html	Adds force refresh link and reorganizes table columns
src/soa_builder/web/templates/concept_category_detail.html	Adds force refresh link with cache bypass indicator
src/soa_builder/web/static/style.css	Updates font family
tests/test_concept_categories.py	Updates tests to use force=True parameter
tests/test_categories_cache.py	New test file for category caching behavior
tests/test_categories_ui_force.py	New test file for UI force refresh functionality
tests/test_concept_category_force_refresh.py	New test file for concept category force refresh
tests/test_concepts_by_category_ui_force.py	New test file for concepts by category UI force refresh

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

[nitpick] The first column header 'Concepts' is ambiguous. Since the column contains a 'View Concepts' link, consider renaming to 'Actions' or 'View' for clarity.

Suggested change

	<th style="text-align:left;">Concepts</th>
	<th style="text-align:left;">View</th>

[Copilot]

The bare except clause catches all exceptions without logging or context. Consider logging the exception or being more specific about which exceptions to catch (e.g., ValueError, IndexError) to aid debugging.

Suggested change

	except Exception:
	except Exception as e:
	logger.exception("Failed to parse code_uid for existing arm types; falling back to len(existing) + 1")

[Copilot]

This code block for computing the next Code_N identifier is duplicated in multiple places (lines 3582-3591, 3629-3640, 3784-3795, 3852-3863). Extract this logic into a helper function like _get_next_code_uid(cur, soa_id) to reduce duplication and improve maintainability.

Suggested change

	cur.execute(
	"SELECT code_uid FROM code WHERE soa_id=? AND code_uid LIKE 'Code_%'",
	(soa_id,),
	)
	existing = [x[0] for x in cur.fetchall() if x[0]]
	n = 1
	if existing:
	try:
	n = max(int(x.split("_")[1]) for x in existing) + 1
	except Exception:
	n = len(existing) + 1
	new_data_origin_uid = f"Code_{n}"
	cur.execute(
	new_data_origin_uid = _get_next_code_uid(cur, soa_id)
	cur.execute(

[Copilot]

Pull request overview

Copilot reviewed 11 out of 13 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The updated tests now always use force=True, which bypasses cache behavior. Consider adding tests that verify normal cache hit behavior (force=False) to ensure the caching mechanism works correctly in production scenarios.

[pendingintent]

@copilot open a new pull request to apply changes based on this feedback

[Copilot]

Commented-out debug logging statement should be removed. If this logging is needed for troubleshooting, consider uncommenting it or removing it entirely to keep the codebase clean.

Suggested change

	# Build mapping code_uid -> submission value (Arm dataOriginType C188727)
	conn_ddf_map = _connect()

[Copilot]

Commented-out debug logging statement should be removed. If this logging is needed for troubleshooting, consider uncommenting it or removing it entirely to keep the codebase clean.

Suggested change

	arms_enriched = []