-
Notifications
You must be signed in to change notification settings - Fork 0
chore: add strict codecov and zero-issue quality gates #16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
dd6ad2f
7906704
cb94c6e
4409906
912ddfa
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| name: Codacy Zero | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main, master] | ||
| pull_request: | ||
| branches: [main, master] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| codacy-zero: | ||
| name: Codacy Zero | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - name: Assert Codacy zero-open gate | ||
| env: | ||
| CODACY_API_TOKEN: ${{ secrets.CODACY_API_TOKEN }} | ||
| run: | | ||
| python3 scripts/quality/check_codacy_zero.py \ | ||
| --owner "${GITHUB_REPOSITORY_OWNER}" \ | ||
| --repo "${GITHUB_REPOSITORY#*/}" \ | ||
| --out-json "codacy-zero/codacy.json" \ | ||
| --out-md "codacy-zero/codacy.md" | ||
| - name: Upload Codacy artifacts | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: codacy-zero | ||
| path: codacy-zero |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| name: Codecov Analytics | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main, master] | ||
| pull_request: | ||
| branches: [main, master] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| codecov-analytics: | ||
| name: Codecov Analytics | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - uses: actions/setup-python@v6 | ||
| with: | ||
| python-version: '3.12' | ||
| - uses: actions/setup-node@v6 | ||
| with: | ||
| node-version: '20' | ||
|
|
||
| - name: Backend coverage | ||
| run: | | ||
| mkdir -p coverage | ||
| python -m pip install --upgrade pip | ||
| python -m pip install -r backend/requirements.txt pytest pytest-cov | ||
| python -m pytest backend --cov=backend --cov-report=xml:backend/coverage.xml | ||
| - name: Frontend coverage | ||
| run: | | ||
| npm --prefix frontend/webcoder_ui ci | ||
| npm --prefix frontend/webcoder_ui test -- --coverage --watch=false | ||
| - name: Upload coverage to Codecov | ||
| uses: codecov/codecov-action@v5 | ||
|
Check warning on line 39 in .github/workflows/codecov-analytics.yml
|
||
Check warningCode scanning / CodeQL Unpinned tag for a non-immutable Action in workflow Medium
Unpinned 3rd party Action 'Codecov Analytics' step
Uses Step Error loading related location Loading |
||
| with: | ||
| files: backend/coverage.xml,frontend/webcoder_ui/coverage/lcov.info | ||
| flags: backend,frontend | ||
| fail_ci_if_error: true | ||
| verbose: true | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| name: Coverage 100 | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main, master] | ||
| pull_request: | ||
| branches: [main, master] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| coverage-100: | ||
| name: Coverage 100 Gate | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - uses: actions/setup-python@v6 | ||
| with: | ||
| python-version: '3.12' | ||
| - uses: actions/setup-node@v6 | ||
| with: | ||
| node-version: '20' | ||
|
|
||
| - name: Backend coverage | ||
| run: | | ||
| mkdir -p coverage | ||
| python -m pip install --upgrade pip | ||
| python -m pip install -r backend/requirements.txt pytest pytest-cov | ||
| python -m pytest backend --cov=backend --cov-report=xml:backend/coverage.xml | ||
| - name: Frontend coverage | ||
| run: | | ||
| npm --prefix frontend/webcoder_ui ci | ||
| npm --prefix frontend/webcoder_ui test -- --coverage --watch=false | ||
|
|
||
| - name: Enforce 100% coverage | ||
| run: | | ||
| python3 scripts/quality/assert_coverage_100.py \ | ||
| --xml "backend=backend/coverage.xml" \ | ||
| --lcov "frontend=frontend/webcoder_ui/coverage/lcov.info" \ | ||
| --out-json "coverage-100/coverage.json" \ | ||
| --out-md "coverage-100/coverage.md" | ||
| - name: Upload coverage artifacts | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: coverage-100 | ||
| path: | | ||
| coverage | ||
| **/coverage | ||
| **/TestResults |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| name: DeepScan Zero | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main, master] | ||
| pull_request: | ||
| branches: [main, master] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
| checks: read | ||
|
|
||
| jobs: | ||
| deepscan-zero: | ||
| name: DeepScan Zero | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - name: Assert DeepScan vendor check is green | ||
| run: | | ||
| python3 scripts/quality/check_required_checks.py \ | ||
| --repo "${GITHUB_REPOSITORY}" \ | ||
| --sha "${GITHUB_SHA}" \ | ||
| --required-context "DeepScan" \ | ||
| --timeout-seconds 1200 \ | ||
| --poll-seconds 20 \ | ||
| --out-json "deepscan-zero/deepscan.json" \ | ||
| --out-md "deepscan-zero/deepscan.md" | ||
| - name: Upload DeepScan artifacts | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: deepscan-zero | ||
| path: deepscan-zero |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| name: Quality Zero Gate | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main, master] | ||
| pull_request: | ||
| branches: [main, master] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
Check noticeCode scanning / SonarCloud Read permissions should be defined at the job level Low
Move this read permission from workflow level to job level. See more on SonarQube Cloud
|
||
|
|
||
|
Comment on lines
+10
to
+12
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 2. Github api perms missing Quality Zero Gate restricts workflow token permissions to contents: read, but then calls the GitHub Checks/Statuses APIs to query check results; this can cause 403 errors and a permanently failing gate. Agent Prompt
|
||
| jobs: | ||
| secrets-preflight: | ||
| name: Quality Secrets Preflight | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }} | ||
| CODACY_API_TOKEN: ${{ secrets.CODACY_API_TOKEN }} | ||
| CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} | ||
| SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }} | ||
| SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }} | ||
| SENTRY_ORG: ${{ vars.SENTRY_ORG }} | ||
| SENTRY_PROJECT: ${{ vars.SENTRY_PROJECT }} | ||
| DEEPSCAN_POLICY_MODE: ${{ vars.DEEPSCAN_POLICY_MODE }} | ||
| DEEPSCAN_OPEN_ISSUES_URL: ${{ vars.DEEPSCAN_OPEN_ISSUES_URL }} | ||
| DEEPSCAN_API_TOKEN: ${{ secrets.DEEPSCAN_API_TOKEN }} | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - name: Run quality secrets preflight | ||
| run: | | ||
| python3 scripts/quality/check_quality_secrets.py \ | ||
| --out-json quality-secrets/secrets.json \ | ||
| --out-md quality-secrets/secrets.md | ||
| - name: Upload secrets preflight artifact | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: quality-secrets | ||
| path: quality-secrets | ||
|
|
||
| quality-zero-gate: | ||
| name: Quality Zero Gate | ||
| if: always() | ||
| runs-on: ubuntu-latest | ||
| needs: | ||
| - secrets-preflight | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - name: Assert secrets preflight succeeded | ||
| run: | | ||
| if [ "${{ needs.secrets-preflight.result }}" != "success" ]; then | ||
| echo "Quality Secrets Preflight failed or was not successful." >&2 | ||
| exit 1 | ||
| fi | ||
| - name: Assert required quality contexts are green | ||
| run: | | ||
| python3 scripts/quality/check_required_checks.py \ | ||
| --repo "${GITHUB_REPOSITORY}" \ | ||
| --sha "${GITHUB_SHA}" \ | ||
| --required-context "Coverage 100 Gate" \ | ||
| --required-context "Codecov Analytics" \ | ||
| --required-context "Sonar Zero" \ | ||
| --required-context "Codacy Zero" \ | ||
| --required-context "Snyk Zero" \ | ||
| --required-context "Sentry Zero" \ | ||
| --required-context "DeepScan Zero" \ | ||
| --required-context "SonarCloud Code Analysis" \ | ||
| --required-context "Codacy Static Code Analysis" \ | ||
| --required-context "DeepScan" \ | ||
| --timeout-seconds 1500 \ | ||
| --poll-seconds 20 \ | ||
| --out-json quality-zero-gate/required-checks.json \ | ||
| --out-md quality-zero-gate/required-checks.md | ||
| - name: Upload aggregate artifact | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: quality-zero-gate | ||
| path: quality-zero-gate | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| name: Sentry Zero | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main, master] | ||
| pull_request: | ||
| branches: [main, master] | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| sentry-zero: | ||
| name: Sentry Zero | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }} | ||
| SENTRY_ORG: ${{ vars.SENTRY_ORG }} | ||
| SENTRY_PROJECT: ${{ vars.SENTRY_PROJECT }} | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - name: Assert Sentry unresolved issues are zero | ||
| run: | | ||
| python3 scripts/quality/check_sentry_zero.py \ | ||
| --project "${SENTRY_PROJECT}" \ | ||
| --out-json "sentry-zero/sentry.json" \ | ||
| --out-md "sentry-zero/sentry.md" | ||
| - name: Upload Sentry artifacts | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: sentry-zero | ||
| path: sentry-zero |
Uh oh!
There was an error while loading. Please reload this page.