Skip to content

Comments

docs: update documentation for CS architecture and REST API Gateway#191

Open
cuioss-oliver wants to merge 2 commits intomainfrom
feature/188-documentation-updates
Open

docs: update documentation for CS architecture and REST API Gateway#191
cuioss-oliver wants to merge 2 commits intomainfrom
feature/188-documentation-updates

Conversation

@cuioss-oliver
Copy link
Collaborator

Summary

  • Update all specification documents, PlantUML diagrams, READMEs, and guides to reflect the architectural evolution introduced in Add self-contained REST API gateway processor with built-in JWT auth and JSON Schema validation #137
  • Add REST API Gateway requirements (NIFI-GATEWAY-1 through NIFI-GATEWAY-8) covering embedded Jetty server, multi-route configuration, RFC 9457 errors, CORS, and management endpoints
  • Fix 15+ broken cross-references, create module READMEs for nifi-cuioss-api, nifi-cuioss-common, and nifi-cuioss-rest-processors, and correct misplaced nifi-cuioss-processors/README.md content

Changes by category

  • PlantUML diagrams (5 updated + PNGs regenerated): architecture, component interaction, API gateway pattern, configuration overview, test environment
  • Core spec docs (4 rewritten): technical-components, configuration, error-handling, observability
  • Remaining spec docs (10 updated): configuration-ui, integration-patterns, jwt-rest-api, end-to-end-testing, testing, security, token-validation, internationalization, configuration-static, LogMessages
  • Top-level docs (3 updated): README.adoc, Specification.adoc, Requirements.adoc
  • Guides (1 updated): QuickStart.adoc — added RestApiGateway quick start
  • Module READMEs (3 created): nifi-cuioss-api, nifi-cuioss-common, nifi-cuioss-rest-processors
  • Fixed (1): nifi-cuioss-processors/README.md — was incorrectly describing JS development
  • Reference (1): agents.md — added missing module sections

Closes #188

Test plan

  • ./mvnw -Ppre-commit clean install -DskipTests passes
  • ./mvnw clean install passes (all tests green)
  • PlantUML PNGs regenerated from updated .puml sources
  • Review rendered .adoc files for formatting correctness
  • Verify cross-references resolve correctly on GitHub

🤖 Generated with Claude Code

… API Gateway (#188)

Update all specification documents, PlantUML diagrams, READMEs, and guides
to reflect the architectural evolution: Controller Service pattern,
RestApiGatewayProcessor with embedded Jetty, 4-module structure, and
unified Custom UI. Add gateway requirements (NIFI-GATEWAY-1 through
NIFI-GATEWAY-8), fix 15+ broken cross-references, create module READMEs,
and correct misplaced nifi-cuioss-processors/README.md content.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @cuioss-oliver, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the NiFi Extensions project by introducing a powerful new REST API Gateway processor, alongside a thorough overhaul of the project's documentation and internal architecture. The changes aim to provide a more robust and streamlined solution for handling JWT-authenticated REST APIs within Apache NiFi, improving both developer experience and operational visibility. The architectural refactoring ensures better modularity and adherence to NiFi 2.x standards, while comprehensive documentation updates ensure clarity and ease of use for all new and existing features.

Highlights

  • Introduction of REST API Gateway Processor: A new RestApiGatewayProcessor has been introduced, providing a self-contained REST API gateway with an embedded Jetty HTTP server, multi-route configuration, per-route JWT authorization, and RFC 9457 error responses. This significantly simplifies building JWT-authenticated REST endpoints in NiFi.
  • Comprehensive Documentation Updates: All specification documents, PlantUML diagrams, READMEs, and guides have been updated to reflect the new architectural evolution, including the RestApiGatewayProcessor and the separation of API interfaces. This includes new requirements, configuration details, integration patterns, and UI descriptions.
  • Module Architecture Refinement: The project's module architecture has been refined, separating Controller Service API interfaces into a new nifi-cuioss-api module and its NAR (nifi-cuioss-api-nar) to comply with NiFi 2.x classloader requirements. A new nifi-cuioss-rest-processors module now houses the REST API Gateway.
  • Enhanced Error Handling and Observability: Error handling for the REST API Gateway now adheres to RFC 9457 for problem details, and new security event counters have been added. Observability features have been expanded to include a dedicated /metrics endpoint for the gateway, supporting Prometheus and JSON formats.
  • Improved UI and Testing: The Custom UI has been unified to serve both processors, dynamically adapting its tabs based on the processor type. End-to-end (E2E) tests have been extended to cover the new gateway's UI and functionality.
Changelog
  • README.adoc
    • Updated the project title to 'NiFi Extensions -- JWT Authentication and REST API Gateway'.
    • Revised the 'What is it?', 'The Problem', and 'The Solution' sections to describe both the MultiIssuerJWTTokenAuthenticator and the new RestApiGatewayProcessor.
    • Added a new 'Working Example: RestApiGateway' section with instructions for testing the gateway.
    • Updated the 'Components' table to include the new nifi-cuioss-rest-processors module and revised the description for nifi-cuioss-ui.
    • Added a new link to the 'REST API Error Reference (RFC 9457)' under the 'Development' section.
  • agents.md
    • Introduced new module architecture sections for nifi-cuioss-api and nifi-cuioss-api-nar.
    • Updated the description for nifi-cuioss-common to reflect its role as the Controller Service implementation.
    • Added a new module architecture section for nifi-cuioss-rest-processors.
    • Expanded the nifi-cuioss-ui module description to include new servlets and JavaScript modules.
    • Updated 'Core processor logic', 'UI configuration', and 'Tests' sections to include references to the new REST API Gateway components and related files.
  • doc/LogMessages.adoc
    • Renamed the document title from 'Log Messages for MultiIssuerJWTTokenAuthenticator' to 'NiFi Extensions Log Messages'.
  • doc/Requirements.adoc
    • Updated the document title to 'NiFi Extensions Requirements'.
    • Revised the 'Overview' to describe both the MultiIssuerJWTTokenAuthenticator and the RestApiGatewayProcessor.
    • Added a comprehensive 'REST API Gateway Requirements' section, detailing 8 new requirements (NIFI-GATEWAY-1 through NIFI-GATEWAY-8) covering embedded HTTP server, multi-route configuration, JWT authorization, RFC 9457 errors, CORS, management endpoints, request size limits, and back-pressure integration.
  • doc/Specification.adoc
    • Updated the document title to 'NiFi Extensions Specification'.
    • Revised the 'Overview' to encompass both processors and their shared Controller Service.
    • Updated the 'Document Structure' to reflect the inclusion of the REST API Gateway and new documentation.
    • Expanded 'Libraries and Dependencies' to include Eclipse Jetty 12, cui-http, and Jakarta JSON Processing.
    • Added a new 'Controller Service Architecture' section.
    • Updated various sections ('Multiple Issuer Support', 'Token Validation', 'Static Configuration Support', 'Caching and Performance', 'Security Features', 'Error Handling', 'Testing Framework', 'Architecture Overview', 'NiFi Compatibility', 'OAuth-Sheriff Integration') to reflect the project-wide scope and the new REST API Gateway features.
    • Added a new 'REST API Gateway' section summarizing its key features.
  • doc/guides/QuickStart.adoc
    • Updated the guide's title and overview to cover both the MultiIssuerJWTTokenAuthenticator and the RestApiGatewayProcessor.
    • Added a new 'RestApiGateway Quick Start' section with detailed steps for adding the processor, configuring the Controller Service, setting gateway properties, adding routes via dynamic properties, connecting relationships, and testing the gateway.
    • Added a new link to the 'REST API error reference' in the 'Next Steps' section.
  • doc/plantuml/api-gateway-pattern.puml
    • Completely rewrote the PlantUML diagram to illustrate the detailed request flow within the RestApiGatewayProcessor, including the embedded Jetty, request pipeline stages (input sanitization, route matching, JWT authentication, authorization, body validation, enqueue), management endpoint, and interaction with the JwtIssuerConfigService.
  • doc/plantuml/architecture.puml
    • Completely rewrote the PlantUML diagram to reflect the new modular architecture, including nifi-cuioss-api, nifi-cuioss-common, nifi-cuioss-processors, nifi-cuioss-rest-processors, and nifi-cuioss-ui.
    • Integrated the RestApiGatewayProcessor and its internal components into the architecture diagram.
    • Updated relationships to show how both processors interact with the shared Controller Service.
  • doc/plantuml/component-interaction.puml
    • Completely rewrote the PlantUML diagram to illustrate the component interactions within the updated architecture, focusing on the APIClient interacting with the RestApiGatewayProcessor, and both processors utilizing the shared JwtIssuerConfigService.
  • doc/plantuml/configuration-overview.puml
    • Updated the diagram title to 'NiFi Extensions Configuration Overview'.
    • Restructured configuration categories to clearly separate Controller Service configuration, MultiIssuerJWTTokenAuthenticator properties, and RestApiGatewayProcessor properties.
    • Added detailed configuration properties for the RestApiGatewayProcessor, including listening port, SSL, request limits, CORS, management API key, and dynamic route configurations.
  • doc/plantuml/test-environment-architecture.puml
    • Added the RestApiGateway component to the Docker node in the test environment.
    • Updated relationships to show the NiFi instance interacting with the Gateway processor and Playwright performing E2E tests on the Gateway.
  • doc/specification/configuration-static.adoc
    • Renamed the document title to 'NiFi Extensions Static Configuration'.
  • doc/specification/configuration-ui.adoc
    • Updated the document title to 'NiFi Extensions UI Configuration'.
    • Revised the 'UI Configuration Overview' to cover both processors.
    • Added a new 'Unified Custom UI Architecture' section explaining how the UI dynamically adapts to the processor type.
    • Detailed new 'JWT Processor Tabs' and 'Gateway Processor Tabs' sections.
    • Introduced 'Gateway UI Sub-Endpoints' for proxying requests to the running gateway.
  • doc/specification/configuration.adoc
    • Updated the document title to 'NiFi Extensions Configuration Reference'.
    • Revised the 'Overview' to describe configuration for both processors and the shared Controller Service.
    • Updated 'Implementation Classes' to include RestApiGatewayConstants and RestApiGatewayProcessor.
    • Modified 'Configuration Architecture' to detail how properties are split between the Controller Service and both processors.
    • Added a new 'RestApiGateway Processor Properties' section, detailing static properties and dynamic route configuration.
    • Updated 'Controller Service Properties' to clarify that both processors reference the same CS instance.
    • Added a new property jwt.validation.jwks.allow.private.network.addresses.
    • Updated 'UI Configuration' and 'Static Configuration Files' sections to reflect changes for both processors/CS.
    • Revised 'Common Configuration Scenarios' to include 'RestApiGateway with Multiple Routes' and renamed 'Advanced Configuration with Multiple Issuers' to 'Multiple Issuers (Shared CS)'.
  • doc/specification/end-to-end-testing.adoc
    • Updated the document title to 'NiFi Extensions End-to-End Testing'.
    • Adjusted implementation links for existing E2E tests (JWKS validation, token verification, metrics, help tabs).
    • Added a new 'Gateway Tabs' section for E2E testing the RestApiGatewayProcessor's custom UI.
  • doc/specification/error-handling.adoc
    • Updated the document title to 'NiFi Extensions Error Handling'.
    • Revised the 'Error Handling Overview' to describe error handling for both processors, including RFC 9457 problem details for the Gateway.
    • Updated 'Implementation Classes' to include ProblemDetail and GatewaySecurityEvents.
    • Renamed 'Error Handling Implementation' to 'JWT Processor Error Handling'.
    • Added a comprehensive 'REST API Gateway Error Handling' section, detailing RFC 9457 format, various error types (400-503), security event tracking, CORS, and WWW-Authenticate headers.
    • Updated 'Integration with OAuth-Sheriff' and 'Security Event Monitoring' to apply to both processors.
    • Renamed 'Error Response Generation' to 'Error Response Design Principles' and updated it to include RFC 9457 and RFC 6750.
    • Removed the 'Enhanced Error Messages' section.
    • Added links to ../rest-errors/index.adoc and observability.adoc in the 'See Also' section.
  • doc/specification/integration-patterns.adoc
    • Updated the document title to 'NiFi Extensions Integration Patterns'.
    • Updated 'Implementation Classes' to include RestApiGatewayProcessor.
    • Added a new 'RestApiGateway Pattern (Primary)' section, detailing its advantages over a manual gateway, providing a configuration example, and clarifying when to use it versus the MultiIssuerJWTTokenAuthenticator.
  • doc/specification/internationalization.adoc
    • Updated the document title to 'NiFi Extensions Internationalization'.
    • Revised the 'Internationalization Overview' to state that both processors support i18n.
    • Updated the link for 'Key Existence Testing'.
  • doc/specification/jwt-rest-api.adoc
    • Updated the 'Overview' to cover REST API endpoints from both the Custom UI backend and the RestApiGatewayProcessor.
    • Added a new 'Gateway Management Endpoints' section for the /metrics endpoint.
    • Introduced a 'Gateway Proxy Endpoints (Custom UI)' section for /api/gateway/config, /api/gateway/metrics, and /api/gateway/test.
    • Added a 'Component Info Endpoint' for /api/component-info to aid UI detection.
    • Removed BackendEndpointsIntegrationTest from verification and updated the E2E test link.
  • doc/specification/observability.adoc
    • Updated the document title to 'NiFi Extensions Observability'.
    • Revised the 'Observability Overview' to describe how both processors expose metrics.
    • Updated 'Implementation Classes' to include GatewaySecurityEvents.
    • Renamed 'Security Event Metrics' to 'JWT Processor Metrics' and 'Available Metrics' to 'SecurityEventCounter Metrics'.
    • Removed several outdated sections related to NiFi UI integration and Prometheus integration for the JWT processor.
    • Added a comprehensive 'REST API Gateway Metrics' section, detailing metric sources, gateway security event types, the /metrics endpoint (authentication, Prometheus, and JSON formats), and Custom UI integration.
    • Updated 'NiFi UI Integration' and 'External Monitoring Integration' to reflect the new gateway metrics.
    • Updated 'Security Considerations' to include the gateway's API key and UI metrics.
    • Added a link to error-handling.adoc in the 'Related Implementation' section.
  • doc/specification/security.adoc
    • Updated the document title to 'NiFi Extensions Security Specification'.
    • Updated 'Implementation Classes' and 'Verification' sections to reference ComponentConfigReader instead of ProcessorConfigReader.
  • doc/specification/technical-components.adoc
    • Updated the document title to 'NiFi Extensions Technical Components'.
    • Added a new 'Module Architecture' section detailing the purpose of each new and existing module.
    • Introduced a new 'Controller Service Architecture' section.
    • Restructured 'Implementation Status' into specific sections for MultiIssuerJWTTokenAuthenticator, RestApiGateway, Controller Service, Shared Infrastructure, and UI classes.
    • Revised the 'Overview' to cover both processors.
    • Added a detailed 'RestApiGateway Processor' section, outlining its lifecycle, relationships, FlowFile output attributes, and a 6-stage request pipeline.
    • Updated 'Dependencies' to include Jetty 12, cui-http, and Jakarta JSON Processing.
    • Updated 'Multi-Issuer Support' and 'TokenValidator Lifecycle in NiFi' to reflect the shared Controller Service.
    • Updated 'Error Handling' to mention RFC 9457 for the Gateway.
    • Added a link to ../rest-errors/index.adoc in the 'Implementation' section.
  • doc/specification/testing.adoc
    • Updated the document title to 'NiFi Extensions Testing'.
    • Expanded 'Test Directories' to include nifi-cuioss-rest-processors and nifi-cuioss-common test paths, and clarified Playwright E2E and WCAG accessibility tests.
  • doc/specification/token-validation.adoc
    • Renamed the document title to 'NiFi Extensions Token Validation'.
  • nifi-cuioss-api/README.adoc
    • Added a new README file detailing the purpose, key classes (JwtIssuerConfigService, JwtAuthenticationConfig), and module relationships of the nifi-cuioss-api module, emphasizing its role in NiFi 2.x classloader separation.
  • nifi-cuioss-common/README.adoc
    • Added a new README file detailing the purpose, key classes (StandardJwtIssuerConfigService, AuthorizationValidator, TokenClaimMapper, ErrorContext, IssuerConfigurationParser, JWTAttributes), and module relationships of the nifi-cuioss-common module for shared JWT infrastructure.
  • nifi-cuioss-processors/README.md
    • Rewrote the README to focus exclusively on the MultiIssuerJWTTokenAuthenticator processor, removing all previous JavaScript development details.
    • Added dedicated sections for 'Purpose', 'Key Classes', 'Module Relationships', and 'Building' specific to the processor.
  • nifi-cuioss-rest-processors/README.adoc
    • Added a new README file detailing the purpose, key classes (RestApiGatewayProcessor, RestApiGatewayConstants, GatewayRequestHandler, ManagementEndpointHandler, ProblemDetail, GatewaySecurityEvents, RouteConfiguration, HttpRequestContainer), and route configuration for the nifi-cuioss-rest-processors module, which provides the REST API Gateway.
Activity
  • The pull request was generated using Claude Code.
  • The pull request addresses and closes issue Phase 5: Documentation Updates for RestApiGateway and Architecture Refactoring #188.
  • The author has verified that ./mvnw -Ppre-commit clean install -DskipTests passes.
  • The author has verified that ./mvnw clean install passes with all tests green.
  • The author has regenerated PlantUML PNGs from updated .puml sources.
  • The author plans to review rendered .adoc files for formatting correctness.
  • The author plans to verify cross-references resolve correctly on GitHub.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This is an excellent and thorough documentation update that aligns all specifications, diagrams, and guides with the new architecture including the RestApiGatewayProcessor. The new module READMEs and restructured integration patterns significantly improve clarity for developers and users. I've found a couple of minor issues in the documentation that could be addressed to make it even better, detailed in the comments below.

Remove reference to non-existent 02-verify-custom-ui-tabs.spec.js — this
test file does not exist on disk. The actual test numbering starts at 01
(configuration), 02 (JWKS validation), 03 (token verification), etc.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Phase 5: Documentation Updates for RestApiGateway and Architecture Refactoring

1 participant