diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md deleted file mode 100644 index a3d0e5d..0000000 --- a/.github/copilot-instructions.md +++ /dev/null @@ -1,12 +0,0 @@ -# Copilot Instructions - -All project conventions, code style guidelines, architecture documentation, and contribution requirements are maintained in a single source of truth: - -**📄 See [AGENTS.md](../AGENTS.md) for all instructions.** - -This includes: -- Code style and C# conventions -- Commit message format (Conventional Commits) -- Testing patterns and frameworks -- Build and run commands -- Architecture and project structure diff --git a/AGENTS.md b/AGENTS.md index 136b89e..2c6d552 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,497 +2,87 @@ ## Overview -Azure Event Grid Simulator is a local development simulator that provides HTTPS endpoints mimicking Azure Event Grid topics and subscribers. It is compatible with the Microsoft.Azure.EventGrid client library and supports both EventGrid and CloudEvents v1.0 schemas. +Azure Event Grid Simulator - local HTTPS simulator for Azure Event Grid topics/subscribers. Compatible with Microsoft.Azure.EventGrid client library, supports EventGrid and CloudEvents v1.0 schemas. -**Technology Stack:** - -- .NET 10.0 (ASP.NET Core Web API) -- C# with latest language features -- Serilog for structured logging -- xUnit for testing with Shouldly assertions and NSubstitute for mocking - -**Key Features:** - -- Multi-topic support with individual HTTPS endpoints -- HTTP webhook, Azure Service Bus, Storage Queue, and Event Hub subscriber delivery -- Azure Event Grid-compatible retry with exponential backoff -- Dead-letter support with local JSON file output -- Event filtering (subject-based and advanced) -- Schema transformation between EventGrid and CloudEvents -- Authentication via aeg-sas-key or aeg-sas-token headers - -## Development Environment - -### Prerequisites - -- .NET 10.0 SDK (specified in `global.json`) -- Docker (optional, for containerized development) -- HTTPS development certificate (required for local testing) - -### Initial Setup - -```bash -# Clone and navigate to the repository -cd /src/AzureEventGridSimulator - -# Restore .NET tools (includes CSharpier formatter) -dotnet tool restore - -# Restore dependencies -dotnet restore /src/AzureEventGridSimulator/src/AzureEventGridSimulator.sln - -# Trust the development certificate (required for HTTPS) -dotnet dev-certs https --trust -``` - -### Configuration - -The simulator uses `appsettings.json` for topic and subscriber configuration. Key settings: - -- **Topics**: Each topic requires `name`, `port`, and optional `key` for authentication -- **Subscribers**: Supports HTTP webhooks, Azure Service Bus (queues/topics), Storage Queues, and Event Hubs -- **Filtering**: Event type, subject-based, and advanced filtering per subscriber -- **Retry Policy**: Configurable per subscriber (maxDeliveryAttempts, eventTimeToLiveInMinutes, enabled) -- **Dead-Letter**: Configurable per subscriber (enabled, folderPath for JSON output) - -Example configuration: `/src/AzureEventGridSimulator/src/AzureEventGridSimulator/appsettings.json` - -Docker configuration: `/src/AzureEventGridSimulator/docker/appsettings.docker.json` +**Stack:** .NET 10.0, C#, Serilog, xUnit/Shouldly/NSubstitute ## Commands -### Build and Run - ```bash -# Build the solution -dotnet build /src/AzureEventGridSimulator/src/AzureEventGridSimulator.sln --configuration Release +# Build +dotnet build src/AzureEventGridSimulator.sln --configuration Release -# Run the simulator -dotnet run --project /src/AzureEventGridSimulator/src/AzureEventGridSimulator/AzureEventGridSimulator.csproj +# Test +dotnet test src/AzureEventGridSimulator.sln --configuration Release +dotnet test src/AzureEventGridSimulator.sln --filter "Category=unit" +dotnet test src/AzureEventGridSimulator.sln --filter "Category=integration" -# Run with custom config file -dotnet run --project /src/AzureEventGridSimulator/src/AzureEventGridSimulator/AzureEventGridSimulator.csproj -- --ConfigFile=/path/to/config.json -``` - -### Testing - -```bash -# Run all tests -dotnet test /src/AzureEventGridSimulator/src/AzureEventGridSimulator.sln --configuration Release - -# Run only unit tests -dotnet test /src/AzureEventGridSimulator/src/AzureEventGridSimulator.sln --filter "Category=unit" +# Run +dotnet run --project src/AzureEventGridSimulator/AzureEventGridSimulator.csproj -# Run only integration tests -dotnet test /src/AzureEventGridSimulator/src/AzureEventGridSimulator.sln --filter "Category=integration" - -# Run tests with coverage -dotnet test /src/AzureEventGridSimulator/src/AzureEventGridSimulator.sln --collect:"XPlat Code Coverage" +# Format (runs automatically on build) +dotnet csharpier format src ``` -### Code Formatting - -CSharpier runs automatically on build (via MSBuild target in `Directory.Build.props`). To format manually: - -```bash -# Format all code -dotnet csharpier format /src/AzureEventGridSimulator/src +## Project Structure -# Check formatting without applying changes -dotnet csharpier check /src/AzureEventGridSimulator/src ``` - -### Docker - -```bash -# Build Docker image -docker build -t azureeventgridsimulator:dev -f /src/AzureEventGridSimulator/Dockerfile /src/AzureEventGridSimulator - -# Run with docker-compose (includes Service Bus emulator, SQL Server, Seq, and Azurite) -docker-compose -f /src/AzureEventGridSimulator/docker-compose.yml up --build --detach - -# Stop docker-compose services -docker-compose -f /src/AzureEventGridSimulator/docker-compose.yml down +src/ +├── AzureEventGridSimulator/ # Main application +│ ├── Controllers/ # API endpoints +│ ├── Domain/ # Business logic +│ │ ├── Commands/ # Command handlers +│ │ ├── Entities/ # Domain models +│ │ └── Services/ # Domain services (Delivery/, Retry/) +│ ├── Infrastructure/ # Cross-cutting concerns +│ │ ├── Mediator/ # Custom mediator (no MediatR) +│ │ ├── Middleware/ # HTTP middleware +│ │ └── Settings/ # Configuration models +│ └── Program.cs # Entry point +├── AzureEventGridSimulator.Tests/ # Tests (UnitTests/, IntegrationTests/) +├── Directory.Build.props # Shared MSBuild properties +└── Directory.Packages.props # Central Package Management ``` -## Architecture - -### Project Structure - -``` -/src -├── AzureEventGridSimulator/ # Main application -│ ├── Controllers/ # API endpoints -│ ├── Domain/ # Business logic layer -│ │ ├── Commands/ # Command handlers -│ │ ├── Entities/ # Domain models (including PendingDelivery, DeadLetterEvent) -│ │ └── Services/ # Domain services -│ │ ├── Delivery/ # Event delivery services (HTTP, ServiceBus, StorageQueue, EventHub) -│ │ └── Retry/ # Retry infrastructure (queue, scheduler, dead-letter) -│ ├── Infrastructure/ # Cross-cutting concerns -│ │ ├── Extensions/ # Extension methods -│ │ ├── Mediator/ # Custom mediator implementation -│ │ ├── Middleware/ # HTTP middleware -│ │ └── Settings/ # Configuration models -│ │ └── Subscribers/ # Subscriber settings (including RetryPolicy, DeadLetter) -│ └── Program.cs # Application entry point -├── AzureEventGridSimulator.Tests/ # Test project -│ ├── UnitTests/ # Unit tests -│ │ └── Retry/ # Retry-specific tests -│ ├── IntegrationTests/ # Integration tests -│ └── ActualSimulatorTests/ # End-to-end tests -├── Directory.Build.props # Shared MSBuild properties -└── Directory.Packages.props # Central Package Management (CPM) -``` - -### Key Design Patterns - -- **Mediator Pattern**: Custom in-process mediator for command handling (no MediatR dependency) -- **Middleware Pipeline**: Request validation, authentication, and logging -- **Domain-Driven Design**: Clear separation between domain logic, infrastructure, and API -- **File-Scoped Namespaces**: Required by .editorconfig (C# 10+) -- **Dependency Injection**: ASP.NET Core built-in DI container - -### Central Package Management - -This project uses Central Package Management (CPM). Package versions are defined in `/src/AzureEventGridSimulator/src/Directory.Packages.props` and referenced without versions in `.csproj` files. - ## Code Style -### Language and Formatting - -- **Target Framework**: .NET 10.0 (`net10.0`) -- **Language Version**: Latest C# features enabled -- **Nullable**: Disabled project-wide -- **Formatter**: CSharpier (100 character line width) -- **Line Endings**: CRLF (Windows-style) -- **Encoding**: UTF-8 with BOM -- **Indentation**: 4 spaces for C#, 2 spaces for JSON/XML - -### C# Conventions (from .editorconfig) - -- **Namespaces**: File-scoped namespaces required (`warning` level) -- **Usings**: Outside namespace, system directives first -- **var**: Preferred for all variable declarations -- **Expression bodies**: Preferred for properties/indexers/lambdas, block bodies for methods -- **Pattern matching**: Preferred over `is` with cast and `as` with null check -- **Primary constructors**: Preferred for simple classes and records -- **Collection expressions**: Preferred (`[]` syntax) - -### Code Organization - -- All domain logic in `/Domain` namespace -- Infrastructure concerns in `/Infrastructure` namespace -- API controllers use API versioning (`Asp.Versioning.Mvc`) -- Use `InternalsVisibleTo` for test access (see `Program.cs`) +- **Framework:** .NET 10.0, latest C# features +- **Nullable:** Enabled +- **Formatter:** CSharpier (100 char width, runs on build) +- **Namespaces:** File-scoped required +- **Preferences:** `var` for variables, pattern matching, primary constructors, collection expressions (`[]`) ## Testing -### Test Framework - -- **Framework**: xUnit 2.9.3 -- **Assertions**: Shouldly (fluent assertions) -- **Mocking**: NSubstitute -- **Integration Tests**: `Microsoft.AspNetCore.Mvc.Testing` - -### Test Categories - -Tests are organized using xUnit traits: - -```csharp -[Trait("Category", "unit")] // Unit tests -[Trait("Category", "integration")] // Integration tests -``` - -Run specific categories using `--filter "Category=unit"`. - -### Test Structure - -- **UnitTests/**: Fast, isolated tests for individual components -- **IntegrationTests/**: Tests involving multiple components or external dependencies -- **ActualSimulatorTests/**: End-to-end tests running the full simulator - -### Writing Tests - ```csharp -using Shouldly; -using Xunit; - -namespace AzureEventGridSimulator.Tests.UnitTests.YourFeature; - -[Trait("Category", "unit")] -public class YourFeatureTests +[Trait("Category", "unit")] // or "integration" +public class MyTests { [Fact] - public void Should_DoSomething_When_Condition() - { - // Arrange - var sut = new YourClass(); - - // Act - var result = sut.DoSomething(); - - // Assert - result.ShouldNotBeNull(); - result.Value.ShouldBe(expected); - } -} -``` - -## Common Tasks - -### Adding a New Topic Configuration - -Edit `appsettings.json` or `docker/appsettings.docker.json`: - -```json -{ - "topics": [ + public void Should_DoX_When_Y() { - "name": "MyNewTopic", - "port": 60102, - "key": "YourSecretKey=", - "subscribers": { - "http": [ - { - "name": "MyWebhook", - "endpoint": "https://example.com/webhook", - "disableValidation": false - } - ], - "eventHub": [ - { - "name": "MyEventHubSubscriber", - "connectionString": "Endpoint=sb://my-namespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=abc123", - "eventHubName": "my-event-hub", - "deliverySchema": "CloudEventV1_0", - "properties": { - "Label": { "type": "dynamic", "value": "Subject" }, - "Region": { "type": "static", "value": "west-us" } - } - } - ] - } + // Arrange/Act/Assert with Shouldly + result.ShouldBe(expected); } - ] } ``` -Event Hub subscribers support: -- **connectionString**: Full connection string (or use namespace/sharedAccessKeyName/sharedAccessKey separately) -- **eventHubName**: Target Event Hub name (required) -- **deliverySchema**: Output schema (EventGridSchema or CloudEventV1_0) -- **properties**: Custom properties added to EventData (static or dynamic values from event fields) -- **filter**: Subject-based and advanced event filtering -- **retryPolicy**: Retry configuration (maxDeliveryAttempts, eventTimeToLiveInMinutes) -- **deadLetter**: Dead-letter configuration for failed deliveries - -### Testing Event Publishing - -```bash -# Using cURL (EventGrid schema) -curl -k \ - -H "Content-Type: application/json" \ - -H "aeg-sas-key: TheLocal+DevelopmentKey=" \ - -X POST "https://localhost:60101/api/events?api-version=2018-01-01" \ - -d '[{"id":"123","subject":"/test","data":{},"eventType":"Test.Event","eventTime":"2025-01-01T00:00:00Z","dataVersion":"1"}]' - -# Using cURL (CloudEvents structured mode) -curl -k \ - -H "Content-Type: application/cloudevents+json" \ - -H "aeg-sas-key: TheLocal+DevelopmentKey=" \ - -X POST "https://localhost:60101/api/events?api-version=2018-01-01" \ - -d '{"specversion":"1.0","type":"com.example.test","source":"/test","id":"123","data":{}}' -``` - -### Debugging with Logs - -The simulator uses Serilog with console and file sinks. Logs are written to `log_YYYYMMDD.txt` in the application directory. - -Configure log levels in `appsettings.json`: - -```json -{ - "Serilog": { - "MinimumLevel": { - "Default": "Information", - "Override": { - "Microsoft": "Warning", - "System": "Warning" - } - } - } -} -``` - -For Docker environments, use Seq (included in docker-compose) at `http://localhost:8081`. - -### Generating SSL Certificates - -For local development: - -```bash -dotnet dev-certs https --trust -``` - -For Docker: - -```bash -dotnet dev-certs https \ - --export-path /src/AzureEventGridSimulator/docker/azureEventGridSimulator.pfx \ - --password Y0urSup3rCrypt1cPa55w0rd! -``` - -### Adding a New Subscriber Type - -1. Create delivery service in `/Domain/Services/Delivery/` -2. Implement `ISubscriberEventDeliveryHandler` interface -3. Register service in `Program.cs` DI configuration -4. Add configuration model in `/Infrastructure/Settings/` -5. Update configuration parsing in settings classes - -## Important Considerations - -### HTTPS Only - -Azure Event Grid only accepts HTTPS connections. The simulator enforces HTTPS for all topic endpoints. The `Microsoft.Azure.EventGrid` client library will always use HTTPS regardless of the URL scheme provided. - -### Topic Endpoint Pattern - -- Azure Event Grid pattern: `https://topic-name.location.eventgrid.azure.net/api/events` -- Simulator pattern: `https://localhost:{port}/api/events?api-version=2018-01-01` -- Each topic uses a unique port to distinguish between topics -- The EventGrid client reduces URLs to `https://host:port` and drops query strings - -### Authentication Validation - -- If a topic has a `key` configured, requests must include `aeg-sas-key` or `aeg-sas-token` header -- Set `key` to `null` to disable authentication for a topic -- SAS tokens must be generated according to Azure Event Grid specifications - -### Message Size Limits - -- Overall message body: ≤ 1,048,576 bytes (1 MB) -- Individual event: ≤ 1,048,576 bytes (1 MB) -- Enforced in validation middleware - -### Event Schemas - -- **EventGrid schema**: Azure's proprietary format (default) -- **CloudEvents v1.0**: CNCF standard, supports structured and binary content modes -- Schema can be auto-detected, explicitly set per topic, or transformed on output -- See `Domain/Services/EventSchemaDetector.cs` for detection logic - -### Subscriber Validation - -The simulator mimics Azure Event Grid's subscription validation handshake: - -1. On startup, sends validation event to each subscriber -2. Expects `validationCode` echo response -3. Disables subscriber if validation fails (unless `disableValidation: true`) - -### Event Filtering - -Advanced filters support up to 25 conditions per subscriber. String comparisons are case-insensitive. "Not" operators return `true` when the key doesn't exist. Dot notation supports nested data property access. - -### CI/CD - -GitHub Actions workflows in `.github/workflows/`: - -- **ci.yml**: Multi-platform build and test (Windows, Linux, macOS) -- **release.yml**: GitHub releases with binaries -- **docker-release.yml**: Docker Hub image publishing - -Builds require warnings-as-errors to pass (`TreatWarningsAsErrors=true`). - -### Build-Time Code Formatting - -CSharpier runs automatically before every build of the main project. To skip formatting (e.g., in Docker builds), set `DesignTimeBuild=true`: - -```bash -dotnet build -p:DesignTimeBuild=true -``` - -### No Nullable Reference Types - -This project has nullable reference types disabled (`disable`). Do not add nullable annotations (`?`, `!`) to code. - -### Custom Mediator Implementation - -The project previously used MediatR but now uses a custom in-process mediator (`Infrastructure/Mediator/`). This is due to trimming and single-file publishing constraints. Command handlers implement `ICommandHandler`. - -### Testing Against Real Azure Service Bus - -Integration tests can use real Azure Service Bus if connection strings are provided via environment variables. Otherwise, use the Service Bus emulator included in docker-compose. - -### Branch Strategy - -- Main branch: `master` -- Active development may occur on feature branches -- CI runs on `master`, `main`, and pull requests - -### Commit Message Convention - -This project uses [Conventional Commits](https://www.conventionalcommits.org/) for automated versioning via Release Please. Commit messages must follow this format: - -``` -[optional scope]: - -[optional body] - -[optional footer(s)] -``` - -**Types that trigger version bumps:** - -| Type | Description | Version Bump | -|------|-------------|--------------| -| `feat` | New feature | Minor (4.1.0 → 4.2.0) | -| `fix` | Bug fix | Patch (4.1.0 → 4.1.1) | -| `feat!` or `fix!` | Breaking change (note the `!`) | Major (4.1.0 → 5.0.0) | - -**Types that do NOT trigger version bumps:** - -- `docs` - Documentation changes -- `style` - Formatting, whitespace -- `refactor` - Code restructuring without behavior change -- `perf` - Performance improvements -- `test` - Adding or updating tests -- `build` - Build system or dependencies -- `ci` - CI/CD configuration -- `chore` - Other maintenance tasks - -**Examples:** - -```bash -feat: add Azure Storage Queue subscriber support -fix: correct SAS token validation for special characters -feat!: change configuration schema for multi-topic setup -docs: update README with Docker instructions -chore: update NuGet dependencies -``` - -### Release Process +## Key Constraints -Releases are automated via Release Please: +- **HTTPS only** - all topic endpoints require HTTPS +- **Authentication** - `aeg-sas-key` or `aeg-sas-token` headers when topic has `key` configured +- **Message limits** - 1 MB max per event and overall body +- **Schemas** - EventGrid (default) or CloudEvents v1.0, auto-detected or configured -1. Push commits to `master` using conventional commit messages -2. Release Please creates/updates a "Release PR" with changelog -3. Merge the Release PR to trigger: - - Git tag creation (e.g., `4.2.0`) - - GitHub Release with generated changelog - - NuGet package publish - - Docker image build and push - - Platform binaries upload +## Commits & Releases -## Documentation Maintenance +Uses [Conventional Commits](https://www.conventionalcommits.org/) with Release Please: -When making changes to the codebase, keep documentation in sync: +- `feat:` = minor bump, `fix:` = patch bump, `feat!:`/`fix!:` = major bump +- `docs`, `style`, `refactor`, `test`, `chore` = no version bump -- **AGENTS.md**: Single source of truth for all AI/LLM coding assistants. Update when changing architecture, conventions, build processes, or development workflows -- **README.md**: Update when changing user-facing features, configuration options, or usage instructions -- **DOCKER.md**: Update when changing Docker configuration, environment variables, or container behavior +Merging Release Please PRs triggers GitHub releases, NuGet publish, and Docker builds. -Note: `.github/copilot-instructions.md` redirects to this file (AGENTS.md) to maintain a single source of truth. +## Configuration -Documentation should be updated in the same commit or PR as the related code changes. +Topics configured in `appsettings.json`. Each topic needs `name`, `port`, optional `key`. Subscribers support HTTP webhooks, Service Bus, Storage Queue, and Event Hub with filtering, retry, and dead-letter options.