aws-lambda-python

An implementation of the Open MPIC API using AWS-Lambda serverless functions written in Python as well as AWS API Gateway.

Deployment

As this API runs serverless in the cloud, it is not installed but rather deployed. The below instructions deploy the API in a user's AWS account and create a unique API endpoint for that user.

Requirements

All requirements for running the API are packaged and uploaded to AWS as a lambda layer. However, the machine deploying the API must have the following requirements

• The AWS CLI (https://aws.amazon.com/cli/) installed with default profile login credentials. The script currently uses the "default" profile to deploy the API to. If you have multiple AWS profiles, ensure the one you want to use for the API in listed as default in ~/.aws/credentials. You can alternatively change the AWS module parameters in main.tf to use an alternate profile.
• Open Tofu (https://opentofu.org/) is installed. This is an open-source fork of Terraform and the configs in this project are largely interoperable between the two.
• Python 3.11.9 which can be run with the command python3.11 and python3. pyenv is one option for getting this specific version of python and not modifying any other python installs on the system.
• Bash. Several deployment scripts are written for bash.
• Hatch (https://hatch.pypa.io/) for building and running the project. This is a Python project manager that can be installed via pip install hatch.

Deployment Steps

1. Create config.yaml in the root directory of the repo to contain the proper values needed for the deployment. A default config.yaml for a 6-perspective deployment with the controller in us-east-2 is included in this repo as config.example.yaml. This config can be made the active config by running cp config.example.yaml config.yaml in the root directory.
2. Create a virtual Python environment in the layer directory and install the project dependencies via pip. This can be executed by running hatch run lambda:layer-install.
3. Package two AWS layers by executing package-layer.sh. This will make two files: python3_layer_content.zip and mpic_coordinator_layer_content.zip which will later be referenced by Open Tofu. This can be done by running ./package-layer.sh or hatch run lambda:layer-package.
4. Run configure.py from the root directory of the repo to generate Open Tofu files from templates. This can be separately executed by running hatch run ./configure.py or hatch run lambda:configure-tf.
5. Zip all Lambda functions. AWS Lambda functions are usually deployed from zip files. This can be separately executed by running ./zip-all.sh or hatch run lambda:zip-all.
6. Deploy the entire package with Open Tofu. cd to the open-tofu directory where .tf files are located. Then run tofu init. Then run tofu apply and type yes at the confirmation prompt. This provides a standard install with DNSSEC enabled which causes the system to incur expenses even when it is not in use (due to the AWS VPC NAT Gateways needed). To reduce the AWS bill, DNSSEC can also be disabled by appending -var="dnssec_enabled=false" to tofu apply (i.e., tofu apply -var="dnssec_enabled=false").
7. Get the URL of the deployed API endpoint by running hatch run ./get_api_url.py in the root directory.
8. Get the API Key generated by AWS by running hatch run ./get_api_key.py in the root directory. The deployment is configured to reject any API call that does not have this key passed via the x-api-key HTTP header.

For convenience:

• hatch run lambda:prepare will run steps 2-5 in a single command.
• hatch run lambda:deploy-dnssec or hatch run lambda:deploy-no-dnssec will clean the environment and then run steps 2-6 with DNSSEC validation enabled or disabled respectively.

Note: the above commands do not run tofu init. During first time environment setup this will need to be run in the open-tofu dir for these commands to work.

Testing

The following is an example of a test API call that uses bash command substitution to fill in the proper values for the API URL and the API key.

curl -H 'Content-Type: application/json' -H "x-api-key: $(hatch run ./get_api_key.py)" \
      -d '{
  "check_type": "caa",
  "domain_or_ip_target": "example.com"
}' \
      -X POST \
      "$(hatch run ./get_api_url.py)/mpic"

The above sample must be run from the root directory of a deployed Open MPIC aws-lambda-python implementation for the bash command substitution to work. You can also run hatch run ./get_api_key.py and hatch run ./get_api_url.py, store these values and then substitute them into the above command. Once deployed, the API is globally accessible and authenticates requests via the x-api-key header, so the curl command with both of these values substituted can be run from any Internet-connected machine to trigger the API.

The API is compliant with the Open MPIC Specification.

There is documentation based on the API specification used in this version.

Development

Code changes can easily be deployed by editing the .py files and then rezipping the project via ./zip-all.sh and ./2-package.sh in the layer directory. Then, running tofu apply run from the open-tofu directory will update only on the required resources and leave the others unchanged. If any .tf.template files are changed or config.yaml is edited, hatch run ./configure.py must be rerun followed by tofu apply in the open-tofu directory.

.generated.tf files should not be edited directly and are not checked into git. Edit .tf.template files and regenerate the files via ./configure.py.

Unit Testing

Pytest is used for unit testing. To run the tests, use any of the following commands from the root directory of the project:

python -m pytest tests/unit

hatch test

hatch run test:unit

To generate an HTML coverage report (located in htmlcov/index.html), run:

hatch run test:coverage

Tests are configured in pyproject.toml

If you encounter issues running unit tests with any of these commands, contact the project maintainers.

Integration Testing

Pytest also runs integration tests in this project. These tests are currently intended to run against a deployed, live API, so they are effectively end-to-end tests. To run the tests, use either of the following commands:

python -m pytest tests/integration

hatch test tests/integration

hatch run test:integration

Note: integration tests cannot currently be invoked by running pytest tests/integration because PYTHONPATH is not set correctly in that case.

The integration tests test a live, deployed API and point the API at real web resources that are provisioned for testing purposes. The integration test content is currently hosted on caatestsuite.com (maintained by SSLMate) and integration-testing.open-mpic.org (maintained by the maintainers of the Open MPIC project). The zone file for caatestsuite.com is available in SSLMate/caatestsuite along with more info about the tests at caatestsuite.com, and the zone file for integration-testing.open-mpic.org is available in open-mpic/open-mpic-integration-zone. integration-testing.open-mpic.org also runs an HTTP(S) server for DCV integration testing. The webroot and configs for the webserver is available in open-mpic/open-mpic-integration-webroot. Both of these services are maintained on a best-effort basis. If there is an issue with integration tests, check the availability of these two services and the responses from the sub-domains used in the integration tests. THESE SERVICES ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND. They are not intended for use outside of development integration testing (i.e., no excessive query volume, no use in production systems).

If you encounter issues running integration tests with the above commands, contact the project maintainers.

Tear-down

If you would like to take the API down, run tofu destroy in the open-tofu directory and type yes at the prompt. This will remove all AWS resources created by Open Tofu for the API. There is currently a situation where AWS takes a very long time to release internal IP prefix resources from a VPC upon teardown. This can lead to tofu destroy hanging for a long time. One mitigation is that tofu destroy can be run (which deprovisions almost all resources within minutes) and then aborted several minutes later with ctrl+c. Then, after several hours, tofu destroy can be rerun which will deprovision the remaining IP prefix resources after sufficient time has passed for AWS to register the IP prefix objects are no longer associated with the VPC.

After tofu destroy, ./clean.sh in the root directory also clears generated/zip files.

hatch run lambda:destroy-tf can be run as an alternative to tofu destroy

EventBridge Warmer

The AWS Lambda deployment takes approximately 8 seconds to perform the fist Open MPIC call because of Lambda cold starts. Subsequent calls take approximately 1 second. To avoid a cold start and get a ~1 second response time even on the first API call, the EventBridge Warmer can optionally be used to keep the Lambda functions "hot." To deploy with this feature, add -var="eventbridge_warmer_enabled=true" to the tofu apply command (e.g., tofu apply -var="dnssec_enabled=false" -var="eventbridge_warmer_enabled=true"). The use of the EventBridge Warmer will increase costs and incur costs even when no API calls are being made.

Additional Implementation Tasks

There are several features that may be of interest to the community, but we don't yet have a specific completion timeline. These may be given higher priority based on feedback and community interest.

• Support for CAA extensions. CAA issue tags can potentially have extensions to specify things like account ID or validation method per RFC 8657. The API could potentially take validation method or account id as an optional parameter and perform the processing on these CAA extensions to have them correctly impact the API response.