Skip to content

Production Deployment#38

Merged
edinstance merged 42 commits intomainfrom
dev
Sep 1, 2025
Merged

Production Deployment#38
edinstance merged 42 commits intomainfrom
dev

Conversation

@edinstance
Copy link
Owner

@edinstance edinstance commented Sep 1, 2025

Summary by CodeRabbit

  • New Features
    • Monthly account statements: automatic generation as PDF, stored in a new secure bucket and delivered via email (attachment or link), with API access to request a statement by period.
    • New endpoint to fetch account transactions with flexible date range filtering.
    • Background orchestration using a new state machine and continuation processing for reliability.
    • Faster transaction lookups via a new DynamoDB index.
  • Documentation
    • Added setup guidance for SES no‑reply email configuration.
  • Chores
    • Dependency updates and additions for email/PDF generation and observability.

…ated process transaction function to use this
…count

Signed-off-by: edinstance <edinstance@gmail.com>
…e monthly reports state machine

Signed-off-by: edinstance <edinstance@gmail.com>
edinstance and others added 12 commits September 2, 2025 00:30
Signed-off-by: edinstance <edinstance@gmail.com>
Signed-off-by: edinstance <edinstance@gmail.com>
Signed-off-by: edinstance <edinstance@gmail.com>
Signed-off-by: edinstance <edinstance@gmail.com>
Signed-off-by: edinstance <edinstance@gmail.com>
Signed-off-by: edinstance <edinstance@gmail.com>
Docstrings generation was requested by @edinstance.

* #36 (comment)

The following files were modified:

* `functions/accounts/get_account_transactions/get_account_transactions/app.py`
* `functions/accounts/get_account_transactions/get_account_transactions/date_helpers.py`
* `functions/accounts/get_account_transactions/get_account_transactions/transaction_helpers.py`
* `functions/monthly_reports/accounts/create_report/create_report/app.py`
* `functions/monthly_reports/accounts/create_report/create_report/generate_pdf.py`
* `functions/monthly_reports/accounts/notify_client/notify_client/app.py`
* `functions/monthly_reports/accounts/notify_client/notify_client/date_helpers.py`
* `functions/monthly_reports/accounts/notify_client/notify_client/processing.py`
* `functions/monthly_reports/accounts/notify_client/notify_client/send_report.py`
* `functions/monthly_reports/accounts/process_pending_reports/process_pending_reports/app.py`
* `functions/monthly_reports/accounts/trigger/trigger/app.py`
* `functions/transactions/process_transactions/process_transactions/app.py`
* `functions/transactions/process_transactions/process_transactions/sqs.py`
* `layers/python/authentication/authentication/user_details.py`
* `layers/python/helpers/dynamodb.py`
* `layers/python/helpers/s3.py`
* `layers/python/helpers/ses.py`
* `layers/python/helpers/sfn.py`
* `layers/python/helpers/sqs.py`
* `layers/python/monthly_reports/monthly_reports/helpers.py`
* `layers/python/monthly_reports/monthly_reports/metrics.py`
* `layers/python/monthly_reports/monthly_reports/processing.py`
* `layers/python/monthly_reports/monthly_reports/responses.py`
* `layers/python/monthly_reports/monthly_reports/sfn.py`
* `layers/python/monthly_reports/monthly_reports/sqs.py`
* `tests/conftest.py`
* `tests/functions/accounts/get_account_transactions/conftest.py`
* `tests/functions/accounts/get_account_transactions/test_app.py`
* `tests/functions/accounts/get_account_transactions/test_date_helpers.py`
* `tests/functions/monthly_reports/accounts/create_report/conftest.py`
* `tests/functions/monthly_reports/accounts/create_report/test_app.py`
* `tests/functions/monthly_reports/accounts/create_report/test_exceptions.py`
* `tests/functions/monthly_reports/accounts/create_report/test_generate_pdf.py`
* `tests/functions/monthly_reports/accounts/notify_client/conftest.py`
* `tests/functions/monthly_reports/accounts/notify_client/test_app.py`
* `tests/functions/monthly_reports/accounts/trigger_tests/conftest.py`
Signed-off-by: edinstance <edinstance@gmail.com>
Signed-off-by: edinstance <edinstance@gmail.com>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 1, 2025

Caution

Review failed

The pull request is closed.

Walkthrough

Adds a monthly account reports pipeline (State Machine, Lambdas, SQS queues, S3 bucket, layer utilities), a new GetAccountTransactions Lambda with HTTP and direct invocation paths, refactors transaction DLQ handling to generic SQS messaging, updates helper libraries (SES/S3/SFN/SQS/DynamoDB), bumps dependencies, and introduces extensive tests. Template gains SES_NO_REPLY_EMAIL and a DynamoDB GSI.

Changes

Cohort / File(s) Summary
Documentation & Infra config
README.md, pytest.ini, template.yml
Adds SES no-reply SSM guidance; extends pytest pythonpath; introduces State Machine and multiple Lambdas/queues/bucket/log groups; adds SES_NO_REPLY_EMAIL; updates TransactionsTable with AccountDateIndex GSI.
Dependencies
dev-requirements.txt, functions/.../requirements.txt, layers/python/*/requirements.txt, layers/python/monthly_reports/requirements.txt, tests/requirements.txt
Bumps aws_lambda_powertools to 3.17.0; adds xhtml2pdf and Jinja2; defines per-function/layer dependency manifests.
GetAccountTransactions feature
functions/accounts/get_account_transactions/get_account_transactions/app.py, .../date_helpers.py, .../exceptions.py, .../transaction_helpers.py, functions/accounts/get_account_transactions/requirements.txt
New Lambda with API Gateway and non-HTTP paths to query transactions via DynamoDB GSI using validated date ranges; structured errors and logging; per-function requirements.
Monthly reports — Lambdas
functions/monthly_reports/accounts/create_report/.../{app.py,exceptions.py,generate_pdf.py,template.html}, functions/monthly_reports/accounts/notify_client/.../{app.py,date_helpers.py,processing.py,send_report.py}, functions/monthly_reports/accounts/process_pending_reports/process_pending_reports/app.py, functions/monthly_reports/accounts/trigger/trigger/app.py, functions/monthly_reports/accounts/create_report/requirements.txt
New report creation (HTML→PDF→S3→presigned URL), notification (attachment or link), EventBridge trigger and SQS continuation processor; input validation, error mapping, and logging; env-driven configuration.
Monthly reports — shared layer
layers/python/monthly_reports/monthly_reports/{helpers.py,metrics.py,processing.py,responses.py,sfn.py,sqs.py}, layers/python/monthly_reports/requirements.txt
Adds helpers for statement period, metrics, batched SFN execution with retries, continuation/DLQ SQS messaging, response shaping; declares layer dependency.
Helpers layers updates
layers/python/helpers/{dynamodb.py,s3.py,ses.py,sfn.py,sqs.py}
Adds DynamoDB pagination, S3/SFN client factories, refactors SQS to generic send_message_to_sqs, enhances SES: return response dicts, stricter validation, and new attachment-sending API.
Authentication layer
layers/python/authentication/authentication/user_details.py, layers/python/authentication/requirements.txt
Adds get_user_attributes using Cognito AdminGetUser; updates powertools version.
Transactions processing refactor
functions/transactions/process_transactions/process_transactions/app.py, .../sqs.py
Replaces DLQ record API with generic SQS messaging and message attributes; expands error routing; updates return schema and preconditions.
Tests — GetAccountTransactions
tests/functions/accounts/get_account_transactions/*
Adds fixtures and tests covering HTTP/Step Functions paths, date range validation, and DynamoDB query behaviour.
Tests — Monthly reports
tests/functions/monthly_reports/accounts/create_report/*, tests/functions/monthly_reports/accounts/notify_client/*, tests/functions/monthly_reports/accounts/process_pending_reports/*, tests/functions/monthly_reports/accounts/trigger_tests/*, tests/layers/monthly_reports/*
Extensive unit tests for report generation, notification modes, trigger/continuation flows, metrics, responses, SFN retry logic, and SQS helpers.
Tests — Helpers & layers
tests/layers/helpers/{test_dynamodb.py,test_s3.py,test_ses.py,test_sfn.py,test_sqs.py}, tests/layers/authentication/test_user_details.py, tests/functions/transactions/process_transactions/{test_app.py,test_sqs.py}, tests/conftest.py
Adds/updates tests for helper clients/APIs, SES behaviours (including attachments), SQS message API, Cognito user attributes, and transaction processing DLQ refactor; new fixtures for mocked AWS clients.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Cron as EventBridge (cron)
  participant Trigger as Lambda: Trigger
  participant DDB as DynamoDB: Accounts
  participant SQS as SQS: Continuation
  participant SFN as Step Functions: StateMachine
  participant GAT as Lambda: GetAccountTransactions
  participant CR as Lambda: CreateReport
  participant NC as Lambda: NotifyClient
  participant S3 as S3: MonthlyReportsBucket
  participant SES as SES
  participant Cognito as Cognito IdP

  Cron->>Trigger: Invoke (monthly)
  Trigger->>DDB: Scan (paged)
  alt Has pages and time left
    Trigger->>SFN: StartExecution per account
  else Low remaining time
    Trigger-->>SQS: Send continuation message
    Trigger-->>Cron: Return TIMEOUT_CONTINUATION
  end

  par StateMachine execution
    SFN->>GAT: Invoke (accountId, period)
    GAT->>DDB: Query AccountDateIndex
    SFN->>CR: Invoke (transactions, balance, period)
    CR->>S3: PutObject (PDF)
    CR->>S3: Get presigned URL
    SFN->>NC: Invoke (accountId, userId, period)
    NC->>Cognito: AdminGetUser (email/name)
    NC->>S3: HeadObject (size)
    alt Size <= threshold
      NC->>S3: GetObject (PDF bytes)
      NC->>SES: SendRawEmail (attachment)
    else Large object
      NC->>S3: Get presigned URL
      NC->>SES: SendEmail (link)
    end
  and On error
    SFN-->>SQS: Send to Creation DLQ
  end
Loading
sequenceDiagram
  autonumber
  actor User as API Client
  participant API as API Gateway
  participant GAT as Lambda: GetAccountTransactions
  participant DDB as DynamoDB: Transactions (GSI)

  User->>API: GET /accounts/{id}/transactions[?period|start|end]
  API->>GAT: Proxy event
  GAT->>DDB: Query AccountDateIndex (id, date range)
  DDB-->>GAT: Items
  GAT-->>API: 200 { transactions, statementPeriod }

  note over GAT: Non-HTTP invocation<br/>expects { accountId, ... } and returns merged { transactions }
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

Poem

A rabbit taps keys with a gentle thrum,
GSI hums, reports now come.
PDFs nest in S3’s boughs,
SES wings them to clients’ brows.
Queues pass notes when time is thin—
Step Functions nod, “Begin… begin!”
Hop! Another month tucked in. 🐇📄✨

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch dev

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@edinstance edinstance merged commit d003039 into main Sep 1, 2025
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant