From 454e571b06b3fa84196614481b4b4a1ba3ca0bb0 Mon Sep 17 00:00:00 2001 From: Alexander Green Date: Thu, 24 Jul 2025 14:43:00 +0200 Subject: [PATCH 01/22] Add rule --- sections/designRules.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/sections/designRules.md b/sections/designRules.md index 4b50037..fdf4f5b 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -580,6 +580,34 @@ An API is as good as the accompanying documentation. The documentation has to be +
+

Include problem details with error responses

+
+
Statement
+
+ Error responses with HTTP status codes `4xx` or `5xx` MUST use either `application/problem+json` or `application/problem+xml` as the `Content-Type` header as described in [rfc9457]. +
+
Rationale
+
+

Providing problem details in a machine readable format aids automation and debugging. By using a common error format, APIs do not need to define their own or misuse existing HTTP status codes.

+
+ 
HTTP/1.1 404 Not Found
+Content-Type: application/problem+json
+
+{
+  "type": "https://example.org/probs/not-found",
+  "title": "Resource Not Found",
+  "status": 404,
+  "detail": "No building found with id 12345.",
+  "instance": "/gebouwen/12345"
+}
+
+
+
+
How to test
+
Verify all responses with status code `4xx` or `5xx` status codes have `Content-Type` marked as either `application/problem+json` or `application/problem+xml`.
+
+ ## Versioning Changes in APIs are inevitable. APIs should therefore always be versioned, facilitating the transition between changes. From 239a7f8632a96b7298e10104c284806debf335fb Mon Sep 17 00:00:00 2001 From: Alexander Green Date: Thu, 24 Jul 2025 14:57:00 +0200 Subject: [PATCH 02/22] Formatting --- sections/designRules.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/sections/designRules.md b/sections/designRules.md index fdf4f5b..e3bb34f 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -581,20 +581,21 @@ An API is as good as the accompanying documentation. The documentation has to be
-

Include problem details with error responses

+

Include problem details with error responses

Statement
- Error responses with HTTP status codes `4xx` or `5xx` MUST use either `application/problem+json` or `application/problem+xml` as the `Content-Type` header as described in [rfc9457]. + Error responses with HTTP status codes 4xx or 5xx MUST use either application/problem+json or application/problem+xml as the Content-Type header, and the response body MUST conform to the structure defined in [[rfc9457]].
Rationale
-

Providing problem details in a machine readable format aids automation and debugging. By using a common error format, APIs do not need to define their own or misuse existing HTTP status codes.

+

Providing problem details in a machine-readable format aids automation and debugging. By using a common error format, APIs do not need to define their own or misuse existing HTTP status codes.

- 
HTTP/1.1 404 Not Found
+            The following example shows the head and body of a detailed error response.
+            
HTTP/1.1 404 Not Found
 Content-Type: application/problem+json
-
-{
+            

+            
{
   "type": "https://example.org/probs/not-found",
   "title": "Resource Not Found",
   "status": 404,
@@ -605,9 +606,13 @@ Content-Type: application/problem+json
How to test
-
Verify all responses with status code `4xx` or `5xx` status codes have `Content-Type` marked as either `application/problem+json` or `application/problem+xml`.
+
+ Verify all responses with status code 4xx or 5xx have Content-Type set to application/problem+json or application/problem+xml. +
+
+ ## Versioning Changes in APIs are inevitable. APIs should therefore always be versioned, facilitating the transition between changes. From 015633b7d8918a894cd4b04fb5902a1c46b38ee6 Mon Sep 17 00:00:00 2001 From: Alexander Green Date: Thu, 24 Jul 2025 15:23:44 +0200 Subject: [PATCH 03/22] markdownlint --- sections/designRules.md | 1 - 1 file changed, 1 deletion(-) diff --git a/sections/designRules.md b/sections/designRules.md index e3bb34f..7427386 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -612,7 +612,6 @@ Content-Type: application/problem+json - ## Versioning Changes in APIs are inevitable. APIs should therefore always be versioned, facilitating the transition between changes. From ee44da4b1bd77655ab24e3aa358bd8f57924eaa3 Mon Sep 17 00:00:00 2001 From: Alexander Green Date: Wed, 13 Aug 2025 15:46:46 +0200 Subject: [PATCH 04/22] Apply suggestions from code review Co-authored-by: Tim van der Lippe --- sections/designRules.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/sections/designRules.md b/sections/designRules.md index 7427386..ccddb80 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -585,24 +585,23 @@ An API is as good as the accompanying documentation. The documentation has to be
Statement
- Error responses with HTTP status codes 4xx or 5xx MUST use either application/problem+json or application/problem+xml as the Content-Type header, and the response body MUST conform to the structure defined in [[rfc9457]]. + Error responses with HTTP status codes 4xx or 5xx MUST use either application/problem+json or application/problem+xml as the Content-Type header, and the response body MUST conform to the structure defined in [[rfc9457]].
Rationale

Providing problem details in a machine-readable format aids automation and debugging. By using a common error format, APIs do not need to define their own or misuse existing HTTP status codes.

The following example shows the head and body of a detailed error response. - 
HTTP/1.1 404 Not Found
-Content-Type: application/problem+json
-
- 
{
+            
HTTP/1.1 404 Not Found
+Content-Type: application/problem+json
+{
   "type": "https://example.org/probs/not-found",
   "title": "Resource Not Found",
   "status": 404,
   "detail": "No building found with id 12345.",
   "instance": "/gebouwen/12345"
 }
-            

+
How to test
From 3f348134b7a828426543448add9f05082d43d07a Mon Sep 17 00:00:00 2001 From: Alexander Green Date: Wed, 13 Aug 2025 15:52:38 +0200 Subject: [PATCH 05/22] Added rule tag --- linter/spectral.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/linter/spectral.yml b/linter/spectral.yml index 7cca73a..468974d 100644 --- a/linter/spectral.yml +++ b/linter/spectral.yml @@ -170,6 +170,7 @@ rules: functionOptions: match: ^https://.* + #/core/problem-details nlgov:use-problem-schema: severity: warn message: "The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457." From 3d4f08c72962b98d878ce860734ff808340a93b8 Mon Sep 17 00:00:00 2001 From: Alexander Green Date: Fri, 22 Aug 2025 11:40:53 +0200 Subject: [PATCH 06/22] Update sections/designRules.md Co-authored-by: Tim van der Lippe --- sections/designRules.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/sections/designRules.md b/sections/designRules.md index ccddb80..3ccd2d8 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -593,8 +593,7 @@ An API is as good as the accompanying documentation. The documentation has to be
The following example shows the head and body of a detailed error response. 
HTTP/1.1 404 Not Found
-Content-Type: application/problem+json
-{
+Content-Type: application/problem+json{
   "type": "https://example.org/probs/not-found",
   "title": "Resource Not Found",
   "status": 404,

From 7dbe3ad3fda911343a038d1db84b15f657d48515 Mon Sep 17 00:00:00 2001
From: Alexander Green 
Date: Fri, 22 Aug 2025 13:52:29 +0200
Subject: [PATCH 07/22] problem schema members

---
 linter/spectral.yml     | 22 ++++++++++++++++++++++
 sections/designRules.md |  3 ++-
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/linter/spectral.yml b/linter/spectral.yml
index 468974d..572beab 100644
--- a/linter/spectral.yml
+++ b/linter/spectral.yml
@@ -183,6 +183,28 @@ rules:
             - required: ["application/problem+json"]
             - required: ["application/problem+xml"]
 
+  nlgov:problem-schema-members:
+    severity: warn
+    message: "These fields are required: status, title and detail. Additionally, only type and instance are allowed."
+    given: $..[responses][?(@property && @property.match(/(4|5)\d\d/))].content
+      [?(@property=="application/problem+json" || @property=="application/problem+xml")]..schema
+    then:
+      function: schema
+      functionOptions:
+        schema:
+          type: object
+          properties:
+            status: {}
+            title: {}
+            detail: {}
+            type: {}
+            instance: {}
+          required:
+            - status
+            - title
+            - detail
+          additionalProperties: false
+
   nlgov:property-casing:
     severity: warn
     given:
diff --git a/sections/designRules.md b/sections/designRules.md
index 3ccd2d8..091f43b 100644
--- a/sections/designRules.md
+++ b/sections/designRules.md
@@ -585,7 +585,8 @@ An API is as good as the accompanying documentation. The documentation has to be
    

       
Statement

       

-         Error responses with HTTP status codes 4xx or 5xx MUST use either application/problem+json or application/problem+xml as the Content-Type header, and the response body MUST conform to the structure defined in [[rfc9457]].
+         
Error responses with HTTP status codes 4xx or 5xx MUST use either application/problem+json or application/problem+xml as the Content-Type header, and the response body MUST conform to the structure defined in [[rfc9457]].

+         
The following fields MUST be present: status, title, and detail. Additionally, only these fields MAY be present: type and instance.

       

       
Rationale

       


From 5789d9e9ef4712d0567e0de7e053eecca53e3e8d Mon Sep 17 00:00:00 2001
From: Alexander Green 
Date: Fri, 22 Aug 2025 15:12:09 +0200
Subject: [PATCH 08/22] test cases

---
 linter/spectral.yml                           |  3 +-
 .../expected-output.txt                       |  5 +
 .../error-type-illegal-field/openapi.json     | 98 +++++++++++++++++++
 .../expected-output.txt                       |  5 +
 .../error-type-missing-required/openapi.json  | 96 ++++++++++++++++++
 5 files changed, 205 insertions(+), 2 deletions(-)
 create mode 100644 linter/testcases/error-type-illegal-field/expected-output.txt
 create mode 100644 linter/testcases/error-type-illegal-field/openapi.json
 create mode 100644 linter/testcases/error-type-missing-required/expected-output.txt
 create mode 100644 linter/testcases/error-type-missing-required/openapi.json

diff --git a/linter/spectral.yml b/linter/spectral.yml
index 572beab..688ce12 100644
--- a/linter/spectral.yml
+++ b/linter/spectral.yml
@@ -186,8 +186,7 @@ rules:
   nlgov:problem-schema-members:
     severity: warn
     message: "These fields are required: status, title and detail. Additionally, only type and instance are allowed."
-    given: $..[responses][?(@property && @property.match(/(4|5)\d\d/))].content
-      [?(@property=="application/problem+json" || @property=="application/problem+xml")]..schema
+    given: $..[responses][?(@property && @property.match(/(4|5)\d\d/))].content[?(@property=="application/problem+json" || @property=="application/problem+xml")]..schema
     then:
       function: schema
       functionOptions:
diff --git a/linter/testcases/error-type-illegal-field/expected-output.txt b/linter/testcases/error-type-illegal-field/expected-output.txt
new file mode 100644
index 0000000..d37352a
--- /dev/null
+++ b/linter/testcases/error-type-illegal-field/expected-output.txt
@@ -0,0 +1,5 @@
+
+/testcases/error-type-illegal-field/openapi.json
+ 60:42  warning  nlgov:problem-schema-members  These fields are required: status, title and detail. Additionally, only type and instance are allowed.  paths./openapi.json.get.responses[400].content.application/problem+json.schema
+
+✖ 1 problem (0 errors, 1 warning, 0 infos, 0 hints)
diff --git a/linter/testcases/error-type-illegal-field/openapi.json b/linter/testcases/error-type-illegal-field/openapi.json
new file mode 100644
index 0000000..661c96c
--- /dev/null
+++ b/linter/testcases/error-type-illegal-field/openapi.json
@@ -0,0 +1,98 @@
+{
+    "openapi": "3.0.3",
+    "info": {
+        "title": "Baseline",
+        "description": "Deze OpenAPI specification bevat het minimale om aan alle regels te voldoen.",
+        "contact": {
+            "name": "Beheerder",
+            "url": "https://www.example.com",
+            "email": "mail@example.com"
+        },
+        "version": "1.0.0"
+    },
+    "servers": [
+        {
+            "url": "https://example.com/api/v1"
+        }
+    ],
+    "security": [
+        {
+            "default": []
+        }
+    ],
+    "tags": [
+        {
+            "name": "openapi"
+        }
+    ],
+    "paths": {
+        "/openapi.json": {
+            "get": {
+                "tags": [
+                    "openapi"
+                ],
+                "description": "OpenAPI document",
+                "operationId": "getOpenapiJSON",
+                "parameters": [],
+                "responses": {
+                    "200": {
+                        "description": "OK",
+                        "headers": {
+                            "API-Version": {
+                                "description": "De huidige versie van de applicatie",
+                                "style": "simple",
+                                "schema": {
+                                    "type": "string"
+                                }
+                            },
+                            "access-control-allow-origin": {
+                                "description": "Alle origins mogen bij deze resource",
+                                "schema": {
+                                    "type": "string"
+                                }
+                            }
+                        }
+                    },
+                    "400": {
+                        "description": "NOK",
+                        "content": {
+                            "application/problem+json": {
+                                "schema": {
+                                    "type": "object",
+                                    "properties": {
+                                        "status": { "type": "integer" },
+                                        "title": { "type": "string" },
+                                        "detail": {"type": "string" },
+                                        "extra": {"type": "string" }
+                                    },
+                                    "required": ["status", "title", "detail"],
+                                    "additionalProperties": false
+                                }
+                            }
+                        }
+                    }
+                },
+                "security": [
+                    {
+                        "default": []
+                    }
+                ]
+            }
+        }
+    },
+    "components": {
+        "schemas": {
+        },
+        "securitySchemes": {
+            "default": {
+                "type": "oauth2",
+                "flows": {
+                    "implicit": {
+                        "authorizationUrl": "https://test.com",
+                        "scopes": {}
+                    }
+                }
+            }
+        }
+    }
+}
\ No newline at end of file
diff --git a/linter/testcases/error-type-missing-required/expected-output.txt b/linter/testcases/error-type-missing-required/expected-output.txt
new file mode 100644
index 0000000..6454fee
--- /dev/null
+++ b/linter/testcases/error-type-missing-required/expected-output.txt
@@ -0,0 +1,5 @@
+
+/testcases/error-type-missing-required/openapi.json
+ 60:42  warning  nlgov:problem-schema-members  These fields are required: status, title and detail. Additionally, only type and instance are allowed.  paths./openapi.json.get.responses[400].content.application/problem+json.schema
+
+✖ 1 problem (0 errors, 1 warning, 0 infos, 0 hints)
diff --git a/linter/testcases/error-type-missing-required/openapi.json b/linter/testcases/error-type-missing-required/openapi.json
new file mode 100644
index 0000000..3cb69cc
--- /dev/null
+++ b/linter/testcases/error-type-missing-required/openapi.json
@@ -0,0 +1,96 @@
+{
+    "openapi": "3.0.3",
+    "info": {
+        "title": "Baseline",
+        "description": "Deze OpenAPI specification bevat het minimale om aan alle regels te voldoen.",
+        "contact": {
+            "name": "Beheerder",
+            "url": "https://www.example.com",
+            "email": "mail@example.com"
+        },
+        "version": "1.0.0"
+    },
+    "servers": [
+        {
+            "url": "https://example.com/api/v1"
+        }
+    ],
+    "security": [
+        {
+            "default": []
+        }
+    ],
+    "tags": [
+        {
+            "name": "openapi"
+        }
+    ],
+    "paths": {
+        "/openapi.json": {
+            "get": {
+                "tags": [
+                    "openapi"
+                ],
+                "description": "OpenAPI document",
+                "operationId": "getOpenapiJSON",
+                "parameters": [],
+                "responses": {
+                    "200": {
+                        "description": "OK",
+                        "headers": {
+                            "API-Version": {
+                                "description": "De huidige versie van de applicatie",
+                                "style": "simple",
+                                "schema": {
+                                    "type": "string"
+                                }
+                            },
+                            "access-control-allow-origin": {
+                                "description": "Alle origins mogen bij deze resource",
+                                "schema": {
+                                    "type": "string"
+                                }
+                            }
+                        }
+                    },
+                    "400": {
+                        "description": "NOK",
+                        "content": {
+                            "application/problem+json": {
+                                "schema": {
+                                    "type": "object",
+                                    "properties": {
+                                        "status": { "type": "integer" },
+                                        "title": { "type": "string" }
+                                    },
+                                    "required": ["status", "title"],
+                                    "additionalProperties": false
+                                }
+                            }
+                        }
+                    }
+                },
+                "security": [
+                    {
+                        "default": []
+                    }
+                ]
+            }
+        }
+    },
+    "components": {
+        "schemas": {
+        },
+        "securitySchemes": {
+            "default": {
+                "type": "oauth2",
+                "flows": {
+                    "implicit": {
+                        "authorizationUrl": "https://test.com",
+                        "scopes": {}
+                    }
+                }
+            }
+        }
+    }
+}
\ No newline at end of file

From 6ad883de49558ed2eb615282a648bd059f0bfecb Mon Sep 17 00:00:00 2001
From: Alexander Green 
Date: Fri, 29 Aug 2025 11:47:13 +0200
Subject: [PATCH 09/22] conform feedback

---
 linter/spectral.yml     | 4 ++--
 sections/designRules.md | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/linter/spectral.yml b/linter/spectral.yml
index 688ce12..d6b2950 100644
--- a/linter/spectral.yml
+++ b/linter/spectral.yml
@@ -172,7 +172,7 @@ rules:
 
   #/core/problem-details
   nlgov:use-problem-schema:
-    severity: warn
+    severity: error
     message: "The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457."
     given: $..[responses][?(@property && @property.match(/(4|5)\d\d/))].content
     then:
@@ -184,7 +184,7 @@ rules:
             - required: ["application/problem+xml"]
 
   nlgov:problem-schema-members:
-    severity: warn
+    severity: error
     message: "These fields are required: status, title and detail. Additionally, only type and instance are allowed."
     given: $..[responses][?(@property && @property.match(/(4|5)\d\d/))].content[?(@property=="application/problem+json" || @property=="application/problem+xml")]..schema
     then:
diff --git a/sections/designRules.md b/sections/designRules.md
index 091f43b..43af740 100644
--- a/sections/designRules.md
+++ b/sections/designRules.md
@@ -606,7 +606,7 @@ Content-Type: application/problem+json{
       

       
How to test

       

-         Verify all responses with status code 4xx or 5xx have Content-Type set to application/problem+json or application/problem+xml.
+         Verify all responses with status code 4xx or 5xx have Content-Type set to application/problem+json or application/problem+xml and contain the fields status, title, and detail. Optional fields are type and instance. Verify no additional fields are present.
From 4fd0ce8210148a8cab82e6106448ac4c123c6465 Mon Sep 17 00:00:00 2001 From: Alexander Green Date: Fri, 29 Aug 2025 11:47:31 +0200 Subject: [PATCH 10/22] Update linter/testcases/error-type-illegal-field/openapi.json Co-authored-by: Tim van der Lippe --- linter/testcases/error-type-illegal-field/openapi.json | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/linter/testcases/error-type-illegal-field/openapi.json b/linter/testcases/error-type-illegal-field/openapi.json index 661c96c..c7ce4f5 100644 --- a/linter/testcases/error-type-illegal-field/openapi.json +++ b/linter/testcases/error-type-illegal-field/openapi.json @@ -64,9 +64,7 @@ "title": { "type": "string" }, "detail": {"type": "string" }, "extra": {"type": "string" } - }, - "required": ["status", "title", "detail"], - "additionalProperties": false + } } } } From 97a05119f0840e12c9aacfde4e618753bbfdafc5 Mon Sep 17 00:00:00 2001 From: Alexander Green Date: Fri, 29 Aug 2025 11:51:05 +0200 Subject: [PATCH 11/22] updated expected outputs --- linter/testcases/cor-api/expected-output.txt | 48 +++++++++---------- .../expected-output.txt | 4 +- .../expected-output.txt | 4 +- .../testcases/error-type/expected-output.txt | 4 +- 4 files changed, 30 insertions(+), 30 deletions(-) diff --git a/linter/testcases/cor-api/expected-output.txt b/linter/testcases/cor-api/expected-output.txt index 5c46649..dea62f8 100644 --- a/linter/testcases/cor-api/expected-output.txt +++ b/linter/testcases/cor-api/expected-output.txt @@ -1,28 +1,28 @@ /testcases/cor-api/openapi.json - 70:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./heartbeat.get.responses[429].content - 80:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./heartbeat.get.responses[503].content + 70:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./heartbeat.get.responses[429].content + 80:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./heartbeat.get.responses[503].content 181:29 warning nlgov:paths-kebab-case /laatsteWijziging is not kebab-case. paths./laatsteWijziging - 211:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[400].content - 221:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[404].content - 231:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[405].content - 241:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[406].content - 251:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[429].content - 261:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[500].content - 271:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[503].content - 506:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[400].content - 516:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[404].content - 526:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[405].content - 536:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[406].content - 546:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[429].content - 556:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[500].content - 566:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[503].content - 684:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[400].content - 694:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[404].content - 704:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[405].content - 714:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[406].content - 724:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[429].content - 734:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[500].content - 744:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[503].content + 211:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[400].content + 221:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[404].content + 231:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[405].content + 241:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[406].content + 251:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[429].content + 261:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[500].content + 271:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./laatsteWijziging.get.responses[503].content + 506:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[400].content + 516:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[404].content + 526:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[405].content + 536:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[406].content + 546:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[429].content + 556:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[500].content + 566:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties.get.responses[503].content + 684:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[400].content + 694:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[404].content + 704:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[405].content + 714:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[406].content + 724:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[429].content + 734:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[500].content + 744:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./organisaties/{oin}.get.responses[503].content -✖ 24 problems (0 errors, 24 warnings, 0 infos, 0 hints) +✖ 24 problems (23 errors, 1 warning, 0 infos, 0 hints) diff --git a/linter/testcases/error-type-illegal-field/expected-output.txt b/linter/testcases/error-type-illegal-field/expected-output.txt index d37352a..d6cfb4b 100644 --- a/linter/testcases/error-type-illegal-field/expected-output.txt +++ b/linter/testcases/error-type-illegal-field/expected-output.txt @@ -1,5 +1,5 @@ /testcases/error-type-illegal-field/openapi.json - 60:42 warning nlgov:problem-schema-members These fields are required: status, title and detail. Additionally, only type and instance are allowed. paths./openapi.json.get.responses[400].content.application/problem+json.schema + 60:42 error nlgov:problem-schema-members These fields are required: status, title and detail. Additionally, only type and instance are allowed. paths./openapi.json.get.responses[400].content.application/problem+json.schema -✖ 1 problem (0 errors, 1 warning, 0 infos, 0 hints) +✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints) diff --git a/linter/testcases/error-type-missing-required/expected-output.txt b/linter/testcases/error-type-missing-required/expected-output.txt index 6454fee..563bec1 100644 --- a/linter/testcases/error-type-missing-required/expected-output.txt +++ b/linter/testcases/error-type-missing-required/expected-output.txt @@ -1,5 +1,5 @@ /testcases/error-type-missing-required/openapi.json - 60:42 warning nlgov:problem-schema-members These fields are required: status, title and detail. Additionally, only type and instance are allowed. paths./openapi.json.get.responses[400].content.application/problem+json.schema + 60:42 error nlgov:problem-schema-members These fields are required: status, title and detail. Additionally, only type and instance are allowed. paths./openapi.json.get.responses[400].content.application/problem+json.schema -✖ 1 problem (0 errors, 1 warning, 0 infos, 0 hints) +✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints) diff --git a/linter/testcases/error-type/expected-output.txt b/linter/testcases/error-type/expected-output.txt index f2893cd..11b48f6 100644 --- a/linter/testcases/error-type/expected-output.txt +++ b/linter/testcases/error-type/expected-output.txt @@ -1,5 +1,5 @@ /testcases/error-type/openapi.json - 58:35 warning nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./openapi.json.get.responses[400].content + 58:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./openapi.json.get.responses[400].content -✖ 1 problem (0 errors, 1 warning, 0 infos, 0 hints) +✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints) From 04b7be5a72e20cecf34f5b0e4cdb222f55cf3ac7 Mon Sep 17 00:00:00 2001 From: Alexander Green Date: Fri, 29 Aug 2025 13:18:57 +0200 Subject: [PATCH 12/22] fixed `nlgov:problem-schema-members` --- linter/spectral.yml | 25 +++++++++++-------- .../expected-output.txt | 2 +- .../expected-output.txt | 2 +- 3 files changed, 16 insertions(+), 13 deletions(-) diff --git a/linter/spectral.yml b/linter/spectral.yml index d6b2950..03af7ff 100644 --- a/linter/spectral.yml +++ b/linter/spectral.yml @@ -185,7 +185,7 @@ rules: nlgov:problem-schema-members: severity: error - message: "These fields are required: status, title and detail. Additionally, only type and instance are allowed." + message: "{{error}}. These fields are required: status, title and detail. Additionally, only type and instance are allowed." given: $..[responses][?(@property && @property.match(/(4|5)\d\d/))].content[?(@property=="application/problem+json" || @property=="application/problem+xml")]..schema then: function: schema @@ -193,16 +193,19 @@ rules: schema: type: object properties: - status: {} - title: {} - detail: {} - type: {} - instance: {} - required: - - status - - title - - detail - additionalProperties: false + properties: + type: object + properties: + status: {} + title: {} + detail: {} + type: {} + instance: {} + required: + - status + - title + - detail + additionalProperties: false nlgov:property-casing: severity: warn diff --git a/linter/testcases/error-type-illegal-field/expected-output.txt b/linter/testcases/error-type-illegal-field/expected-output.txt index d6cfb4b..6f21bc9 100644 --- a/linter/testcases/error-type-illegal-field/expected-output.txt +++ b/linter/testcases/error-type-illegal-field/expected-output.txt @@ -1,5 +1,5 @@ /testcases/error-type-illegal-field/openapi.json - 60:42 error nlgov:problem-schema-members These fields are required: status, title and detail. Additionally, only type and instance are allowed. paths./openapi.json.get.responses[400].content.application/problem+json.schema + 62:50 error nlgov:problem-schema-members Property "extra" is not expected to be here. These fields are required: status, title and detail. Additionally, only type and instance are allowed. paths./openapi.json.get.responses[400].content.application/problem+json.schema.properties ✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints) diff --git a/linter/testcases/error-type-missing-required/expected-output.txt b/linter/testcases/error-type-missing-required/expected-output.txt index 563bec1..0dc5cf0 100644 --- a/linter/testcases/error-type-missing-required/expected-output.txt +++ b/linter/testcases/error-type-missing-required/expected-output.txt @@ -1,5 +1,5 @@ /testcases/error-type-missing-required/openapi.json - 60:42 error nlgov:problem-schema-members These fields are required: status, title and detail. Additionally, only type and instance are allowed. paths./openapi.json.get.responses[400].content.application/problem+json.schema + 62:50 error nlgov:problem-schema-members "properties" property must have required property "detail". These fields are required: status, title and detail. Additionally, only type and instance are allowed. paths./openapi.json.get.responses[400].content.application/problem+json.schema.properties ✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints) From 326aca6b866c87aab0b0cde7f636364069f7cd76 Mon Sep 17 00:00:00 2001 From: Alexander Green Date: Thu, 4 Sep 2025 16:50:53 +0200 Subject: [PATCH 13/22] Correctie voorbeeld "instance" https://github.com/Logius-standaarden/API-Design-Rules/pull/251#discussion_r2315391262 --- sections/designRules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sections/designRules.md b/sections/designRules.md index 43af740..9bbc041 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -599,7 +599,7 @@ Content-Type: application/problem+json{ "title": "Resource Not Found", "status": 404, "detail": "No building found with id 12345.", - "instance": "/gebouwen/12345" + "instance": "/errors/abc" } From df6964c1167f6d74294de0ed076f9eb11b25dad3 Mon Sep 17 00:00:00 2001 From: Tim van der Lippe Date: Thu, 30 Oct 2025 10:24:09 +0100 Subject: [PATCH 14/22] Verwerk feedback uit werksessie --- linter/spectral.yml | 9 +-------- .../testcases/error-type-extra-field/expected-output.txt | 1 + .../openapi.json | 0 .../error-type-illegal-field/expected-output.txt | 5 ----- .../error-type-missing-required/expected-output.txt | 2 +- sections/designRules.md | 8 +++----- 6 files changed, 6 insertions(+), 19 deletions(-) create mode 100644 linter/testcases/error-type-extra-field/expected-output.txt rename linter/testcases/{error-type-illegal-field => error-type-extra-field}/openapi.json (100%) delete mode 100644 linter/testcases/error-type-illegal-field/expected-output.txt diff --git a/linter/spectral.yml b/linter/spectral.yml index 03af7ff..ede765f 100644 --- a/linter/spectral.yml +++ b/linter/spectral.yml @@ -185,7 +185,7 @@ rules: nlgov:problem-schema-members: severity: error - message: "{{error}}. These fields are required: status, title and detail. Additionally, only type and instance are allowed." + message: "{{error}}. These fields are required: status, title and detail." given: $..[responses][?(@property && @property.match(/(4|5)\d\d/))].content[?(@property=="application/problem+json" || @property=="application/problem+xml")]..schema then: function: schema @@ -195,17 +195,10 @@ rules: properties: properties: type: object - properties: - status: {} - title: {} - detail: {} - type: {} - instance: {} required: - status - title - detail - additionalProperties: false nlgov:property-casing: severity: warn diff --git a/linter/testcases/error-type-extra-field/expected-output.txt b/linter/testcases/error-type-extra-field/expected-output.txt new file mode 100644 index 0000000..95cc954 --- /dev/null +++ b/linter/testcases/error-type-extra-field/expected-output.txt @@ -0,0 +1 @@ +No results with a severity of 'error' found! diff --git a/linter/testcases/error-type-illegal-field/openapi.json b/linter/testcases/error-type-extra-field/openapi.json similarity index 100% rename from linter/testcases/error-type-illegal-field/openapi.json rename to linter/testcases/error-type-extra-field/openapi.json diff --git a/linter/testcases/error-type-illegal-field/expected-output.txt b/linter/testcases/error-type-illegal-field/expected-output.txt deleted file mode 100644 index 6f21bc9..0000000 --- a/linter/testcases/error-type-illegal-field/expected-output.txt +++ /dev/null @@ -1,5 +0,0 @@ - -/testcases/error-type-illegal-field/openapi.json - 62:50 error nlgov:problem-schema-members Property "extra" is not expected to be here. These fields are required: status, title and detail. Additionally, only type and instance are allowed. paths./openapi.json.get.responses[400].content.application/problem+json.schema.properties - -✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints) diff --git a/linter/testcases/error-type-missing-required/expected-output.txt b/linter/testcases/error-type-missing-required/expected-output.txt index 0dc5cf0..f23f2f9 100644 --- a/linter/testcases/error-type-missing-required/expected-output.txt +++ b/linter/testcases/error-type-missing-required/expected-output.txt @@ -1,5 +1,5 @@ /testcases/error-type-missing-required/openapi.json - 62:50 error nlgov:problem-schema-members "properties" property must have required property "detail". These fields are required: status, title and detail. Additionally, only type and instance are allowed. paths./openapi.json.get.responses[400].content.application/problem+json.schema.properties + 62:50 error nlgov:problem-schema-members "properties" property must have required property "detail". These fields are required: status, title and detail. paths./openapi.json.get.responses[400].content.application/problem+json.schema.properties ✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints) diff --git a/sections/designRules.md b/sections/designRules.md index 9bbc041..7af57a8 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -585,8 +585,8 @@ An API is as good as the accompanying documentation. The documentation has to be
Statement
-

Error responses with HTTP status codes 4xx or 5xx MUST use either application/problem+json or application/problem+xml as the Content-Type header, and the response body MUST conform to the structure defined in [[rfc9457]].

-

The following fields MUST be present: status, title, and detail. Additionally, only these fields MAY be present: type and instance.

+

Error responses with HTTP status codes 4xx or 5xx MUST use either application/problem+json or application/problem+xml as the Content-Type header, and the response body MUST conform to the structure defined in [[rfc9457]]. +

The following fields MUST be present: status, title, and detail.

Rationale
@@ -595,11 +595,9 @@ An API is as good as the accompanying documentation. The documentation has to be The following example shows the head and body of a detailed error response. 
HTTP/1.1 404 Not Found
 Content-Type: application/problem+json{
-  "type": "https://example.org/probs/not-found",
   "title": "Resource Not Found",
   "status": 404,
-  "detail": "No building found with id 12345.",
-  "instance": "/errors/abc"
+  "detail": "No building found with id 12345."
 }
From 8db1b25a9e5c2c73ca2c9b209f6c995c8ee6baff Mon Sep 17 00:00:00 2001 From: Joost Farla Date: Thu, 30 Oct 2025 14:48:24 +0100 Subject: [PATCH 15/22] Error handling (#275) * Cleanup * Update sections/designRules.md Co-authored-by: Tim van der Lippe * Update description of 'code' property in design rules * Clarify error object requirements in documentation * Documentation of `code` values * Update wording for 'code' URI dereferencing --------- Co-authored-by: Tim van der Lippe --- sections/designRules.md | 158 ++++++++++++++++++++++++++++++++-------- 1 file changed, 129 insertions(+), 29 deletions(-) diff --git a/sections/designRules.md b/sections/designRules.md index 7af57a8..13fdfb2 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -483,6 +483,135 @@ https://api.example.org/v1/comments/456
+## Error handling + +
+

Use problem details for error responses

+
+
Statement
+
+

Error responses with HTTP status codes 4xx or 5xx MUST use either application/problem+json or application/problem+xml as the Content-Type header, and the response body MUST conform to the structure defined in [[rfc9457]]. +

The following fields MUST be present: status, title, and detail. +

+
Rationale
+
+

Providing problem details in a machine-readable format aids automation and debugging. By using a common error format, APIs do not need to define their own or misuse existing HTTP status codes.

+
+ The following example shows the head and body of a detailed error response. + 
HTTP/1.1 404 Not Found
+Content-Type: application/problem+json{
+  "status": 404,
+  "title": "Resource Not Found",
+  "detail": "No building found with id 12345."
+}
+
+
+
+
How to test
+
+ Verify all responses with status code 4xx or 5xx have Content-Type set to application/problem+json or application/problem+xml and contain the fields status, title, and detail. Optional fields are type and instance. Verify no additional fields are present. +
+
+
+ +
+

Use status code 400 for invalid input

+
+
Statement
+
+

API requests containing invalid input MUST result in HTTP status code 400 Bad Request. Invalid input includes syntax errors, missing or invalid query parameters and schema violations for the request payload.

+
+
Rationale
+
+

The semantics of status code 400 ("the server cannot or will not process the request due to something that is perceived to be a client error") match validation failures more closely than status code 422, which historically originates from WebDAV and introduces no added interoperability benefit.

+
+
How to test
+
+ Verify that operations accepting query parameters and/or a request body contain a response with status code 400. +
+
+
+ +
+

Add specific errors for bad-request responses

+
+
Statement
+
+

Problem details with status code 400 (Bad Request) MUST include an additional member errors containing an ordered list of validation error objects, as specified below.

+

Each error object MUST contain in and detail members, and MAY optionally contain location and code members.

+
    +
  • in - where the error occurs: body or query.
    • +
  • location (optional) - a structured locator for the offending value: +
      +
    • For JSON request bodies: an object with a pointer property containing a JSON Pointer [[rfc6901]] expression pointing to the value.
      • +
    • For XML request bodies: an element with a path attribute containing an absolute XPath v3.1 [[xpath-31]] expression pointing to the value.
      • +
    • For query parameters: an object with a name property (the parameter name). When the same name appears multiple times, include an index property, containing a zero-based index position.
      • +
    + For body errors, the location member may be omitted, in case the error refers to the body as a whole (e.g. syntax errors). +
    • +
  • code (optional) - a short, stable machine-readable code as a rule identifier (e.g. date.format). If a type URI is provided on the message-level, dereferencing this URI SHOULD result in a page describing all possible code values including a description for each value.
    • +
  • detail - a human-readable message describing the violation.
    • +
+
+

JSON example:

+ 
HTTP/1.1 400 Bad Request
+Content-Type: application/problem+json{
+    "status": 400,
+    "title": "Request validation failed",
+    "errors": [
+      {
+        "in": "body",
+        "location": {
+          "pointer": "#/foo[0]/bar"
+        },
+        "code": "date.format",
+        "detail": "must be ISO 8601"
+      },
+      {
+        "in": "query",
+        "location": {
+          "name": "foo",
+          "index": 1
+        },
+        "code": "date.format",
+        "detail": "must be ISO 8601"
+      }
+    ]
+  }
+
+
+

XML example:

+ 
HTTP/1.1 400 Bad Request
+Content-Type: application/problem+xml<?xml version="1.0" encoding="UTF-8"?>
+<problem xmlns="urn:ietf:rfc:7807">
+  <status>400</status>
+  <title>Request validation failed</title>
+  <errors>
+    <error in="body">
+      <location path="/foo[1]/bar/text()" />
+      <code>date.format</code>
+      <detail>must be ISO 8601</detail>
+    </error>
+    <error in="query">
+      <location name="foo" index="1" />
+      <code>date.format</code>
+      <detail>must be ISO 8601</detail>
+    </error>
+  </errors>
+</problem>
+
+
+
Rationale
+
+

Having a single, consistent errors structure makes validation issues predictable for clients, while relying on established locators using universal standards (JSON Pointer, XPath).

+
+
How to test
+
+ Verify that content schemas for media types application/problem+json or application/problem+xml as part of responses with status 400, contain a required errors member conforming to the requirements above. +
+
+
+ ## Documentation An API is as good as the accompanying documentation. The documentation has to be easily findable, searchable and publicly accessible. Most developers will first read the documentation before they start implementing. Hiding the technical documentation in PDF documents and/or behind a login creates a barrier for both developers and search engines. @@ -580,35 +709,6 @@ An API is as good as the accompanying documentation. The documentation has to be
-
-

Include problem details with error responses

-
-
Statement
-
-

Error responses with HTTP status codes 4xx or 5xx MUST use either application/problem+json or application/problem+xml as the Content-Type header, and the response body MUST conform to the structure defined in [[rfc9457]]. -

The following fields MUST be present: status, title, and detail. -

-
Rationale
-
-

Providing problem details in a machine-readable format aids automation and debugging. By using a common error format, APIs do not need to define their own or misuse existing HTTP status codes.

-
- The following example shows the head and body of a detailed error response. - 
HTTP/1.1 404 Not Found
-Content-Type: application/problem+json{
-  "title": "Resource Not Found",
-  "status": 404,
-  "detail": "No building found with id 12345."
-}
-
-
-
-
How to test
-
- Verify all responses with status code 4xx or 5xx have Content-Type set to application/problem+json or application/problem+xml and contain the fields status, title, and detail. Optional fields are type and instance. Verify no additional fields are present. -
-
-
- ## Versioning Changes in APIs are inevitable. APIs should therefore always be versioned, facilitating the transition between changes. From d39ef923d0388db139f168346ac5f28c95113636 Mon Sep 17 00:00:00 2001 From: Tim van der Lippe Date: Thu, 30 Oct 2025 14:57:07 +0100 Subject: [PATCH 16/22] Update designRules.md --- sections/designRules.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/sections/designRules.md b/sections/designRules.md index 13fdfb2..85ab0ef 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -496,7 +496,7 @@ https://api.example.org/v1/comments/456
Rationale

Providing problem details in a machine-readable format aids automation and debugging. By using a common error format, APIs do not need to define their own or misuse existing HTTP status codes.

-
+
+
How to test
- Verify all responses with status code 4xx or 5xx have Content-Type set to application/problem+json or application/problem+xml and contain the fields status, title, and detail. Optional fields are type and instance. Verify no additional fields are present. + Verify all responses with status code 4xx or 5xx have Content-Type set to application/problem+json or application/problem+xml and the body contains fields status, title, and detail.
@@ -541,6 +541,7 @@ Content-Type: application/problem+json{

Each error object MUST contain in and detail members, and MAY optionally contain location and code members.

  • in - where the error occurs: body or query.
    • +
  • detail - a human-readable message describing the violation.
  • location (optional) - a structured locator for the offending value:
    • For JSON request bodies: an object with a pointer property containing a JSON Pointer [[rfc6901]] expression pointing to the value.
      • @@ -550,9 +551,8 @@ Content-Type: application/problem+json      { For body errors, the location member may be omitted, in case the error refers to the body as a whole (e.g. syntax errors).
    • code (optional) - a short, stable machine-readable code as a rule identifier (e.g. date.format). If a type URI is provided on the message-level, dereferencing this URI SHOULD result in a page describing all possible code values including a description for each value.
      • -
    • detail - a human-readable message describing the violation.
    -
    +
    -
    + +
    +
    Rationale
    From ba26c05caac59edf2505afafd1d1376beb11a147 Mon Sep 17 00:00:00 2001 From: Tim van der Lippe Date: Thu, 30 Oct 2025 15:51:28 +0100 Subject: [PATCH 17/22] Voeg Spectral linter regels toe --- linter/spectral.yml | 38 ++- .../expected-output.txt | 6 + .../error-type-bad-request/openapi.json | 295 ++++++++++++++++++ .../error-type-extra-field/openapi.json | 2 +- .../expected-output.txt | 2 +- .../error-type-missing-required/openapi.json | 2 +- .../testcases/error-type/expected-output.txt | 2 +- linter/testcases/error-type/openapi.json | 2 +- sections/designRules.md | 19 +- 9 files changed, 352 insertions(+), 16 deletions(-) create mode 100644 linter/testcases/error-type-bad-request/expected-output.txt create mode 100644 linter/testcases/error-type-bad-request/openapi.json diff --git a/linter/spectral.yml b/linter/spectral.yml index ede765f..f7fe610 100644 --- a/linter/spectral.yml +++ b/linter/spectral.yml @@ -170,7 +170,7 @@ rules: functionOptions: match: ^https://.* - #/core/problem-details + #/core/error-handling/problem-details nlgov:use-problem-schema: severity: error message: "The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457." @@ -200,6 +200,42 @@ rules: - title - detail + #/core/error-handling/bad-request + nlgov:problem-schema-members-bad-request: + severity: error + given: $..[responses][?(@property && @property.match(400))].content[?(@property=="application/problem+json" || @property=="application/problem+xml")]..schema.properties + then: + function: schema + functionOptions: + schema: + type: object + properties: + errors: + type: object + properties: + properties: + type: object + properties: + in: {} + detail: {} + location: + type: object + properties: + properties: + type: object + properties: + pointer: {} + name: {} + index: {} + additionalProperties: false + code: {} + required: + - in + - detail + additionalProperties: false + required: + - errors + nlgov:property-casing: severity: warn given: diff --git a/linter/testcases/error-type-bad-request/expected-output.txt b/linter/testcases/error-type-bad-request/expected-output.txt new file mode 100644 index 0000000..7dbc0bd --- /dev/null +++ b/linter/testcases/error-type-bad-request/expected-output.txt @@ -0,0 +1,6 @@ + +/testcases/error-type-bad-request/openapi.json + 133:58 error nlgov:problem-schema-members-bad-request Property "extra" is not expected to be here paths./properties-correct.get.responses[400].content.application/problem+json.schema.properties.errors.properties + 254:66 error nlgov:problem-schema-members-bad-request Property "pointer2" is not expected to be here paths./properties-location-verkeerd-format.get.responses[400].content.application/problem+json.schema.properties.errors.properties.location.properties + +✖ 2 problems (2 errors, 0 warnings, 0 infos, 0 hints) diff --git a/linter/testcases/error-type-bad-request/openapi.json b/linter/testcases/error-type-bad-request/openapi.json new file mode 100644 index 0000000..1ee4a30 --- /dev/null +++ b/linter/testcases/error-type-bad-request/openapi.json @@ -0,0 +1,295 @@ +{ + "openapi": "3.0.3", + "info": { + "title": "Baseline", + "description": "Deze OpenAPI specification bevat het minimale om aan alle regels te voldoen.", + "contact": { + "name": "Beheerder", + "url": "https://www.example.com", + "email": "mail@example.com" + }, + "version": "1.0.0" + }, + "servers": [ + { + "url": "https://example.com/api/v1" + } + ], + "security": [ + { + "default": [] + } + ], + "tags": [ + { + "name": "openapi" + }, + { + "name": "resource" + } + ], + "paths": { + "/openapi.json": { + "get": { + "tags": [ + "openapi" + ], + "description": "OpenAPI document", + "operationId": "getOpenapiJSON", + "parameters": [], + "responses": { + "200": { + "description": "OK", + "headers": { + "API-Version": { + "description": "De huidige versie van de applicatie", + "style": "simple", + "schema": { + "type": "string" + } + }, + "access-control-allow-origin": { + "description": "Alle origins mogen bij deze resource", + "schema": { + "type": "string" + } + } + } + }, + "400": { + "description": "OK", + "content": { + "application/problem+json": { + "schema": { + "type": "object", + "properties": { + "status": { "type": "integer" }, + "title": { "type": "string" }, + "detail": { "type": "string" }, + "errors": { + "type": "object", + "properties": { + "in": { "type": "string" }, + "location": { + "type": "object", + "properties": { + "pointer": { "type": "string" }, + "name": { "type": "string" }, + "index": { "type": "integer" } + } + }, + "code": { "type": "string" }, + "detail": { "type": "string" } + } + } + }, + "required": ["status", "title", "detail", "errors"], + "additionalProperties": false + } + } + } + } + }, + "security": [ + { + "default": [] + } + ] + } + }, + "/properties-correct": { + "get": { + "tags": [ + "resource" + ], + "description": "Resource", + "operationId": "getPropertiesCorrect", + "parameters": [], + "responses": { + "200": { + "description": "OK", + "headers": { + "API-Version": { + "description": "De huidige versie van de applicatie", + "style": "simple", + "schema": { + "type": "string" + } + } + } + }, + "400": { + "description": "OK", + "content": { + "application/problem+json": { + "schema": { + "type": "object", + "properties": { + "status": { "type": "integer" }, + "title": { "type": "string" }, + "detail": { "type": "string" }, + "errors": { + "type": "object", + "properties": { + "in": { "type": "string" }, + "location": { + "type": "object", + "properties": { + "pointer": { "type": "string" }, + "name": { "type": "string" }, + "index": { "type": "integer" } + } + }, + "code": { "type": "string" }, + "detail": { "type": "string" }, + "extra": { "type": "string" } + } + } + }, + "required": ["status", "title", "detail", "errors"], + "additionalProperties": false + } + } + } + } + }, + "security": [ + { + "default": [] + } + ] + } + }, + "/properties-zonder-location": { + "get": { + "tags": [ + "resource" + ], + "description": "Resource", + "operationId": "getResource", + "parameters": [], + "responses": { + "200": { + "description": "OK", + "headers": { + "API-Version": { + "description": "De huidige versie van de applicatie", + "style": "simple", + "schema": { + "type": "string" + } + } + } + }, + "400": { + "description": "OK", + "content": { + "application/problem+json": { + "schema": { + "type": "object", + "properties": { + "status": { "type": "integer" }, + "title": { "type": "string" }, + "detail": { "type": "string" }, + "errors": { + "type": "object", + "properties": { + "in": { "type": "string" }, + "code": { "type": "string" }, + "detail": { "type": "string" } + } + } + }, + "required": ["status", "title", "detail", "errors"], + "additionalProperties": false + } + } + } + } + }, + "security": [ + { + "default": [] + } + ] + } + }, + "/properties-location-verkeerd-format": { + "get": { + "tags": [ + "resource" + ], + "description": "Resource", + "operationId": "getPropertiesVerkeerdFormat", + "parameters": [], + "responses": { + "200": { + "description": "OK", + "headers": { + "API-Version": { + "description": "De huidige versie van de applicatie", + "style": "simple", + "schema": { + "type": "string" + } + } + } + }, + "400": { + "description": "OK", + "content": { + "application/problem+json": { + "schema": { + "type": "object", + "properties": { + "status": { "type": "integer" }, + "title": { "type": "string" }, + "detail": { "type": "string" }, + "errors": { + "type": "object", + "properties": { + "in": { "type": "string" }, + "location": { + "type": "object", + "properties": { + "pointer2": { "type": "string" }, + "name": { "type": "string" }, + "index": { "type": "integer" } + } + }, + "code": { "type": "string" }, + "detail": { "type": "string" } + } + } + }, + "required": ["status", "title", "detail", "errors"], + "additionalProperties": false + } + } + } + } + }, + "security": [ + { + "default": [] + } + ] + } + } + }, + "components": { + "schemas": { + }, + "securitySchemes": { + "default": { + "type": "oauth2", + "flows": { + "implicit": { + "authorizationUrl": "https://test.com", + "scopes": {} + } + } + } + } + } +} \ No newline at end of file diff --git a/linter/testcases/error-type-extra-field/openapi.json b/linter/testcases/error-type-extra-field/openapi.json index c7ce4f5..d1917c2 100644 --- a/linter/testcases/error-type-extra-field/openapi.json +++ b/linter/testcases/error-type-extra-field/openapi.json @@ -53,7 +53,7 @@ } } }, - "400": { + "404": { "description": "NOK", "content": { "application/problem+json": { diff --git a/linter/testcases/error-type-missing-required/expected-output.txt b/linter/testcases/error-type-missing-required/expected-output.txt index f23f2f9..60a8b61 100644 --- a/linter/testcases/error-type-missing-required/expected-output.txt +++ b/linter/testcases/error-type-missing-required/expected-output.txt @@ -1,5 +1,5 @@ /testcases/error-type-missing-required/openapi.json - 62:50 error nlgov:problem-schema-members "properties" property must have required property "detail". These fields are required: status, title and detail. paths./openapi.json.get.responses[400].content.application/problem+json.schema.properties + 62:50 error nlgov:problem-schema-members "properties" property must have required property "detail". These fields are required: status, title and detail. paths./openapi.json.get.responses[404].content.application/problem+json.schema.properties ✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints) diff --git a/linter/testcases/error-type-missing-required/openapi.json b/linter/testcases/error-type-missing-required/openapi.json index 3cb69cc..bf748b3 100644 --- a/linter/testcases/error-type-missing-required/openapi.json +++ b/linter/testcases/error-type-missing-required/openapi.json @@ -53,7 +53,7 @@ } } }, - "400": { + "404": { "description": "NOK", "content": { "application/problem+json": { diff --git a/linter/testcases/error-type/expected-output.txt b/linter/testcases/error-type/expected-output.txt index 11b48f6..c79b49a 100644 --- a/linter/testcases/error-type/expected-output.txt +++ b/linter/testcases/error-type/expected-output.txt @@ -1,5 +1,5 @@ /testcases/error-type/openapi.json - 58:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./openapi.json.get.responses[400].content + 58:35 error nlgov:use-problem-schema The content type of an error response should be application/problem+json or application/problem+xml to match RFC 9457. paths./openapi.json.get.responses[404].content ✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints) diff --git a/linter/testcases/error-type/openapi.json b/linter/testcases/error-type/openapi.json index 4f28a88..176c0d3 100644 --- a/linter/testcases/error-type/openapi.json +++ b/linter/testcases/error-type/openapi.json @@ -53,7 +53,7 @@ } } }, - "400": { + "404": { "description": "OK", "content": { "application/hal+json": { diff --git a/sections/designRules.md b/sections/designRules.md index 85ab0ef..88e31fd 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -485,7 +485,7 @@ https://api.example.org/v1/comments/456 ## Error handling -
    +

    Use problem details for error responses

    Statement
    @@ -514,12 +514,13 @@ Content-Type: application/problem+json    {
    -
    +

    Use status code 400 for invalid input

    Statement
    -

    API requests containing invalid input MUST result in HTTP status code 400 Bad Request. Invalid input includes syntax errors, missing or invalid query parameters and schema violations for the request payload.

    +

    API requests containing invalid input MUST result in HTTP status code 400 (Bad Request). + Invalid input includes syntax errors, missing or invalid query parameters and schema violations for the request payload.

    Rationale
    @@ -532,13 +533,13 @@ Content-Type: application/problem+json{
    -
    -

    Add specific errors for bad-request responses

    +
    +

    Add specific errors for Bad Request responses

    Statement
    -

    Problem details with status code 400 (Bad Request) MUST include an additional member errors containing an ordered list of validation error objects, as specified below.

    -

    Each error object MUST contain in and detail members, and MAY optionally contain location and code members.

    +

    Problem details with status code 400 (Bad Request) MUST include an additional member errors containing an ordered list of validation error objects, as specified below. +

    Each error object MUST contain in and detail members, and MAY optionally contain location and code members.

    • in - where the error occurs: body or query.
    • detail - a human-readable message describing the violation.
      • @@ -553,7 +554,6 @@ Content-Type: application/problem+json      {
    • code (optional) - a short, stable machine-readable code as a rule identifier (e.g. date.format). If a type URI is provided on the message-level, dereferencing this URI SHOULD result in a page describing all possible code values including a description for each value.
    How to test
    - Verify that content schemas for media types application/problem+json or application/problem+xml as part of responses with status 400, contain a required errors member conforming to the requirements above. + Verify all responses with status code 400 contain a required errors member conforming to the requirements above.
    From 77ed6da8e89b6eebb5f561fad6f6a95225d21733 Mon Sep 17 00:00:00 2001 From: Tim van der Lippe Date: Thu, 30 Oct 2025 16:51:41 +0100 Subject: [PATCH 18/22] Voeg linter regel toe voor Bad Request vereiste --- linter/spectral.yml | 17 ++ .../expected-output.txt | 7 + .../error-type-bad-invalid-input/openapi.json | 232 ++++++++++++++++++ .../paths-kebab-variables/expected-output.txt | 8 +- .../expected-output.txt | 5 +- 5 files changed, 266 insertions(+), 3 deletions(-) create mode 100644 linter/testcases/error-type-bad-invalid-input/expected-output.txt create mode 100644 linter/testcases/error-type-bad-invalid-input/openapi.json diff --git a/linter/spectral.yml b/linter/spectral.yml index f7fe610..430c748 100644 --- a/linter/spectral.yml +++ b/linter/spectral.yml @@ -200,6 +200,23 @@ rules: - title - detail + #/core/error-handling/invalid-input + nlgov:problem-invalid-input: + severity: error + message: "Endpoints that have parameters must be able to return a 400 response" + given: + - $.paths..[?( @property.match(/(get)|(put)|(post)/) && @.parameters && @.parameters.length > 0 )] + then: + function: schema + functionOptions: + schema: + type: object + properties: + responses: + type: object + required: + - "400" + #/core/error-handling/bad-request nlgov:problem-schema-members-bad-request: severity: error diff --git a/linter/testcases/error-type-bad-invalid-input/expected-output.txt b/linter/testcases/error-type-bad-invalid-input/expected-output.txt new file mode 100644 index 0000000..eacc692 --- /dev/null +++ b/linter/testcases/error-type-bad-invalid-input/expected-output.txt @@ -0,0 +1,7 @@ + +/testcases/error-type-bad-invalid-input/openapi.json + 119:29 error nlgov:problem-invalid-input Endpoints that have parameters must be able to return a 400 response paths./invalid-response-vereist.get.responses + 157:29 error nlgov:problem-invalid-input Endpoints that have parameters must be able to return a 400 response paths./invalid-response-vereist.put.responses + 195:29 error nlgov:problem-invalid-input Endpoints that have parameters must be able to return a 400 response paths./invalid-response-vereist.post.responses + +✖ 3 problems (3 errors, 0 warnings, 0 infos, 0 hints) diff --git a/linter/testcases/error-type-bad-invalid-input/openapi.json b/linter/testcases/error-type-bad-invalid-input/openapi.json new file mode 100644 index 0000000..c8ab926 --- /dev/null +++ b/linter/testcases/error-type-bad-invalid-input/openapi.json @@ -0,0 +1,232 @@ +{ + "openapi": "3.0.3", + "info": { + "title": "Baseline", + "description": "Deze OpenAPI specification bevat het minimale om aan alle regels te voldoen.", + "contact": { + "name": "Beheerder", + "url": "https://www.example.com", + "email": "mail@example.com" + }, + "version": "1.0.0" + }, + "servers": [ + { + "url": "https://example.com/api/v1" + } + ], + "security": [ + { + "default": [] + } + ], + "tags": [ + { + "name": "openapi" + }, + { + "name": "resource" + } + ], + "paths": { + "/openapi.json": { + "get": { + "tags": [ + "openapi" + ], + "description": "OpenAPI document", + "operationId": "getOpenapiJSON", + "parameters": [], + "responses": { + "200": { + "description": "OK", + "headers": { + "API-Version": { + "description": "De huidige versie van de applicatie", + "style": "simple", + "schema": { + "type": "string" + } + }, + "access-control-allow-origin": { + "description": "Alle origins mogen bij deze resource", + "schema": { + "type": "string" + } + } + } + }, + "400": { + "description": "OK", + "content": { + "application/problem+json": { + "schema": { + "type": "object", + "properties": { + "status": { "type": "integer" }, + "title": { "type": "string" }, + "detail": { "type": "string" }, + "errors": { + "type": "object", + "properties": { + "in": { "type": "string" }, + "location": { + "type": "object", + "properties": { + "pointer": { "type": "string" }, + "name": { "type": "string" }, + "index": { "type": "integer" } + } + }, + "code": { "type": "string" }, + "detail": { "type": "string" } + } + } + }, + "required": ["status", "title", "detail", "errors"], + "additionalProperties": false + } + } + } + } + }, + "security": [ + { + "default": [] + } + ] + } + }, + "/invalid-response-vereist": { + "get": { + "tags": [ + "resource" + ], + "description": "Resource", + "operationId": "getPropertiesCorrect", + "parameters": [ + { + "name": "expand", + "in": "query", + "description": "Schakelaar om details van gekoppelde organisaties (subOIN of OINhouder) op te vragen (default false = geen details)", + "schema": { + "type": "boolean" + }, + "style": "form", + "explode": true + } + ], + "responses": { + "200": { + "description": "OK", + "headers": { + "API-Version": { + "description": "De huidige versie van de applicatie", + "style": "simple", + "schema": { + "type": "string" + } + } + } + } + }, + "security": [ + { + "default": [] + } + ] + }, + "put": { + "tags": [ + "resource" + ], + "description": "Resource", + "operationId": "putPropertiesCorrect", + "parameters": [ + { + "name": "expand", + "in": "query", + "description": "Schakelaar om details van gekoppelde organisaties (subOIN of OINhouder) op te vragen (default false = geen details)", + "schema": { + "type": "boolean" + }, + "style": "form", + "explode": true + } + ], + "responses": { + "200": { + "description": "OK", + "headers": { + "API-Version": { + "description": "De huidige versie van de applicatie", + "style": "simple", + "schema": { + "type": "string" + } + } + } + } + }, + "security": [ + { + "default": [] + } + ] + }, + "post": { + "tags": [ + "resource" + ], + "description": "Resource", + "operationId": "postPropertiesCorrect", + "parameters": [ + { + "name": "expand", + "in": "query", + "description": "Schakelaar om details van gekoppelde organisaties (subOIN of OINhouder) op te vragen (default false = geen details)", + "schema": { + "type": "boolean" + }, + "style": "form", + "explode": true + } + ], + "responses": { + "200": { + "description": "OK", + "headers": { + "API-Version": { + "description": "De huidige versie van de applicatie", + "style": "simple", + "schema": { + "type": "string" + } + } + } + } + }, + "security": [ + { + "default": [] + } + ] + } + } + }, + "components": { + "schemas": { + }, + "securitySchemes": { + "default": { + "type": "oauth2", + "flows": { + "implicit": { + "authorizationUrl": "https://test.com", + "scopes": {} + } + } + } + } + } +} \ No newline at end of file diff --git a/linter/testcases/paths-kebab-variables/expected-output.txt b/linter/testcases/paths-kebab-variables/expected-output.txt index 95cc954..c0bfcd8 100644 --- a/linter/testcases/paths-kebab-variables/expected-output.txt +++ b/linter/testcases/paths-kebab-variables/expected-output.txt @@ -1 +1,7 @@ -No results with a severity of 'error' found! + +/testcases/paths-kebab-variables/openapi.json + 87:29 error nlgov:problem-invalid-input Endpoints that have parameters must be able to return a 400 response paths./organisaties/{id}.get.responses + 128:29 error nlgov:problem-invalid-input Endpoints that have parameters must be able to return a 400 response paths./organisaties/{id}/nested.get.responses + 169:29 error nlgov:problem-invalid-input Endpoints that have parameters must be able to return a 400 response paths./organisaties/{organisationId}/pad.get.responses + +✖ 3 problems (3 errors, 0 warnings, 0 infos, 0 hints) diff --git a/linter/testcases/paths-kebab-zoek-uitzondering/expected-output.txt b/linter/testcases/paths-kebab-zoek-uitzondering/expected-output.txt index d1c1822..f641557 100644 --- a/linter/testcases/paths-kebab-zoek-uitzondering/expected-output.txt +++ b/linter/testcases/paths-kebab-zoek-uitzondering/expected-output.txt @@ -1,5 +1,6 @@ /testcases/paths-kebab-zoek-uitzondering/openapi.json - 125:19 warning path-keys-no-trailing-slash Path must not end with slash. paths./_zoek/ + 125:19 warning path-keys-no-trailing-slash Path must not end with slash. paths./_zoek/ + 174:29 error nlgov:problem-invalid-input Endpoints that have parameters must be able to return a 400 response paths./organisaties/{id}/nested/_zoek.get.responses -✖ 1 problem (0 errors, 1 warning, 0 infos, 0 hints) +✖ 2 problems (1 error, 1 warning, 0 infos, 0 hints) From f4c6d2169c50c70679f7b08c53712ca1141b8749 Mon Sep 17 00:00:00 2001 From: Tim van der Lippe Date: Wed, 19 Nov 2025 14:36:36 +0100 Subject: [PATCH 19/22] Verwerken feedback van werksessie --- sections/designRules.md | 55 ++++++++++++++++++++++++++++++----------- 1 file changed, 41 insertions(+), 14 deletions(-) diff --git a/sections/designRules.md b/sections/designRules.md index 88e31fd..37b9269 100644 --- a/sections/designRules.md +++ b/sections/designRules.md @@ -539,18 +539,19 @@ Content-Type: application/problem+json{
    Statement

    Problem details with status code 400 (Bad Request) MUST include an additional member errors containing an ordered list of validation error objects, as specified below. -

    Each error object MUST contain in and detail members, and MAY optionally contain location and code members. +

    Each error object MUST contain in and detail members, and MAY optionally contain location, index and code members.

    • in - where the error occurs: body or query.
    • detail - a human-readable message describing the violation.
      • -
    • location (optional) - a structured locator for the offending value: +
    • location (optional) - a locator for the offending value:
        -
      • For JSON request bodies: an object with a pointer property containing a JSON Pointer [[rfc6901]] expression pointing to the value.
        • -
      • For XML request bodies: an element with a path attribute containing an absolute XPath v3.1 [[xpath-31]] expression pointing to the value.
        • -
      • For query parameters: an object with a name property (the parameter name). When the same name appears multiple times, include an index property, containing a zero-based index position.
        • +
      • For JSON request bodies: a JSON Pointer [[rfc6901]] expression pointing to the value.
        • +
      • For XML request bodies: an absolute XPath v3.1 [[xpath-31]] expression pointing to the value.
        • +
      • For query parameters: the parameter name.
      For body errors, the location member may be omitted, in case the error refers to the body as a whole (e.g. syntax errors).
      • +
    • index (optional) - a zero-based index position when multiple query parameters have the same name.
    • code (optional) - a short, stable machine-readable code as a rule identifier (e.g. date.format). If a type URI is provided on the message-level, dereferencing this URI SHOULD result in a page describing all possible code values including a description for each value.