osl-utils

Repo of shared resources (gh actions, dockerfiles, etc) for automating OSL deployments

Using HyP3's actions as an example to work from.

Reusable Workflows

A collection of reusable workflows that are shared across this organization.

reusable-setup-env.yaml

Sets up variables/tags for each repo. Use like:

name: Setup Environment

on:
  push:
    branches:
      - main

jobs:
  call-setup-env-workflow:
    # Docs at: https://github.com/ASFOpenSARlab/osl-utils?tab=readme-ov-file#reusable-setup-envyaml
    uses: ASFOpenSARlab/osl-utils/.github/workflows/reusable-setup-env.yaml@v#.#.#
    with:
      tagname: ${{ github.ref_name }} # Optional; default shown
      aws-region: us-west-2           # Optional; default shown

This is used to figure out what environment to use, figure out when you're running on a tag, etc.

Other Workflows

docker-build.yaml

This is to build a docker container, that has all our CloudFormation/etc tools, that we can use inside other actions or locally to have a similar environment. It is made to be used in other repos like so:

jobs:
  # This job makes sure the container uri is all lowercase:
  docker-tag:
    # Docs at: https://github.com/ASFOpenSARlab/osl-utils?tab=readme-ov-file#docker-buildyaml
    env:
      IMAGE_REPO: ghcr.io/ASFOpenSARlab/osl-utils
      IMAGE_REPO_TAG: v0.0.2
    runs-on: ubuntu-latest
    outputs:
      container-uri: ${{ steps.container.outputs.uri }}
    steps:
      - name: Save Container Info
        id: container
        run: echo "uri=${{ env.IMAGE_REPO }}:${{ env.IMAGE_REPO_TAG }}" | tr '[:upper:]' '[:lower:]' >> $GITHUB_OUTPUT

  # Run the actions steps inside of the container:
  your-job-here:
    runs-on: ubuntu-latest
    container:
      image: ${{ needs.docker-tag.outputs.container-uri }}
      credentials:
        username: ASFOpenSARlab-bot
        password: ${{ secrets.PAT_PACKAGES_READ_ONLY }}
    steps:
      - uses: actions/checkout@v4
      - run: echo "Inside the Custom Image!"
      # ...

When the image is built, it's tagged with the info from reusable-setup-env.yaml. You can use this to test changes here in other actions (Change their tag from v0.0.2 to dev for example), before merging the changes up the maturities here.

Linting Workflows

To use these locally, add the following job to the Makefile inside your repo:

.PHONY := docker-shell
docker-shell:
  echo "Starting Docker Shell..."
  echo ""
  docker run \
    -v "$$(pwd):/code" \
    -it \
    --rm \
    --pull always \
  ghcr.io/asfopensarlab/osl-utils:main \
    bash

Then you can do inside bash:

make docker-shell
make help # See all the linting options. They're also what are called below inside any `reusable-code-quality-*.yaml` file.
cd /code  # If you need to access the repo files inside the docker container

reusable-code-quality-dockerfile.yaml

Uses hadolint to lint and format Dockerfiles.

name: Lint Dockerfiles
on:
    pull_request: 

jobs:
  shell:
    uses: ASFOpenSARlab/osl-utils/.github/workflows/reusable-code-quality-dockerfile.yaml@v#.#.#
    with:
      # The osl-utils docker tag. Can also be `dev`, initials, etc. Should match the tag above in prod.
      osl-utils-tag: v#.#.#

reusable-code-quality-jinja2.yaml

Uses djlint to lint and format Jinja2 templates. Per-repo configurations are available with a configuration file, pyproject.toml/djlint.toml/.djlintrc (see the djlint docs).

name: Lint Jinja Templates
on:
    pull_request: 

jobs:
  shell:
    uses: ASFOpenSARlab/osl-utils/.github/workflows/reusable-code-quality-jinja2.yaml@v#.#.#
    with:
      # The osl-utils docker tag. Can also be `dev`, initials, etc. Should match the tag above in prod.
      osl-utils-tag: v#.#.#

reusable-code-quality-python.yaml

Uses Ruff to lint and format Python source files.

Configure via a pyproject.toml, ruff.toml, or .ruff.toml file (see the Ruff docs).

name: Lint Python
on:
    pull_request: 

jobs:
  shell:
    uses: ASFOpenSARlab/osl-utils/.github/workflows/reusable-code-quality-python.yaml@v#.#.#
    with:
      # The osl-utils docker tag. Can also be `dev`, initials, etc. Should match the tag above in prod.
      osl-utils-tag: v#.#.#

reusable-code-quality-shell.yaml

Uses shellcheck and shfmt to lint and format (respectively) shell code.

name: Lint Shell
on:
    pull_request: 

jobs:
  shell:
    uses: ASFOpenSARlab/osl-utils/.github/workflows/reusable-code-quality-shell.yaml@v#.#.#
    with:
      # The osl-utils docker tag. Can also be `dev`, initials, etc. Should match the tag above in prod.
      osl-utils-tag: v#.#.#

reusable-code-quality-yaml.yaml

Uses yamllint and yamlfmt to lint and format (respectively) YAML.

name: Lint yaml
on:
    pull_request: 

jobs:
  shell:
    uses: ASFOpenSARlab/osl-utils/.github/workflows/reusable-code-quality-yaml.yaml@v#.#.#
    with:
      # The osl-utils docker tag. Can also be `dev`, initials, etc. Should match the tag above in prod.
      osl-utils-tag: v#.#.#

reusable-code-quality-markdown.yaml

Uses markdownlint-cli2 to lint the markdown files themselves, and markdownlint-rule-relative-links to verify relative links.

name: Lint Markdown
on:
    pull_request:

jobs:
  shell:
    uses: ASFOpenSARlab/osl-utils/.github/workflows/reusable-code-quality-markdown.yaml@v#.#.#
    with:
      # The osl-utils docker tag. Can also be `dev`, initials, etc. Should match the tag above in prod.
      osl-utils-tag: v#.#.#