oblachno-oss/devx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

DEVX-126: ci: consolidate CI and post-merge workflows

Browse Source
This commit is contained in:
emil User
2026-07-12 01:52:40 +00:00
parent ed0dfce98b
commit cb84dae050
7 changed files with 421 additions and 558 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tech/architecture.md
+48 -43
View File
@@ -337,7 +337,7 @@ Configures repository branch protection and labels via the Gitea REST API.
Sets up master branch protection (required status checks, block on rejected
reviews, block on outdated branch) and creates standard labels. Status check
contexts are read from `DEVX_STATUS_CHECKS` or default to
`CI / quality (pull_request)`.
`CI / validate (pull_request)`.
### `generate_badges.py`
@@ -456,13 +456,14 @@ Developer pushes and creates PR (title: "DEVX-N: <vikunja task title>")
▼
CI workflow (ci.yml) triggers:
│
├── quality (lint, tests, coverage, test speed, doc coverage,
│ translation check, dependency scan, workflow dry-run)
│
├── detect-changes (classify_changes.py → user-facing or workflow-only)
│ └── if user-facing → release-dry-run (release.py --dry-run)
│
├── pr-review (pr_review.py → posts COMMENT or REQUEST_CHANGES)
├── validate (single job: quality + detect-changes +
│ release-dry-run + pr-review + pre-merge validation)
│ ├── quality steps (lint, tests, coverage, test speed, doc coverage,
│ │ translation check, dependency scan, workflow dry-run)
│ ├── detect-changes (classify_changes.py → user-facing or workflow-only)
│ │ └── if user-facing → release-dry-run (release.py --dry-run)
│ ├── pre-merge validation (check_auto_merge_ready.py)
│ └── pr-review (pr_review.py → posts COMMENT or REQUEST_CHANGES)
│
└── auto-merge (auto_merge.py)
├── validate PR title format
@@ -483,50 +484,54 @@ Push to master (squash-merge commit: "DEVX-N <conventional commit>")
▼
Post-merge workflow (post-merge.yml) triggers:
│
├── detect-type (detect_release_commit.py)
│ └── is-release? → skip all jobs except badges
├── detect-and-configure (single job)
│ ├── configure-repo (configure_repo.py)
│ ├── detect-type (detect_release_commit.py)
│ │ └── is-release? → skip all steps except badges
│ └── validate-commit-msg (validate_commit_msg.py --branch master)
│
├── validate-commit-msg (validate_commit_msg.py --branch master)
│
├── release (release.py)
│ ├── classify_changes.py → skip if workflow-only
│ ├── git-cliff → calculate next version
│ ├── update __version__ in __init__.py
│ ├── update CHANGELOG.md
│ ├── run make lint-ruff && make pytest-cov
│ ├── commit "release: vX.Y.Z [skip ci]"
│ ├── create annotated tag vX.Y.Z
│ └── push commit + tag to master
│ │
│ ▼
│ Tag push triggers publish workflow (see below)
│
├── sync-wiki (sync_wiki.py --strict)
│ └── sync docs/ to Gitea wiki with integrity check
│
├── badges (push_badges.py) [ALWAYS runs, even on release commits]
│ ├── fetch latest master
│ ├── generate_badges.py → SVG files
│ ├── push to orphan badges branch
│ └── update README.md + docs/index.md with cache-busting URLs
│
├── vikunja (post_merge.py)
│ ├── extract task ID from commit message
│ ├── mark Vikunja task as done
│ └── post comment with merge SHA
│
└── configure-repo (configure_repo.py)
└── ensure branch protection and labels
└── release-and-maintain (needs detect-and-configure)
├── release (release.py) [skip if release commit or workflow-only]
│ ├── classify_changes.py → skip if workflow-only
│ ├── git-cliff → calculate next version
│ ├── update __version__ in __init__.py
│ ├── update CHANGELOG.md
│ ├── run make lint-ruff && make pytest-cov
│ ├── commit "release: vX.Y.Z [skip ci]"
│ ├── create annotated tag vX.Y.Z
│ └── push commit + tag to master
│ │
│ ▼
│ publish (publish.py) [if release created a tag]
│ ├── build package (python -m build)
│ ├── publish to Gitea PyPI registry (twine upload)
│ │ OR publish to standard PyPI (if PYPI_TOKEN set)
│ │ OR skip publish (if --skip-build)
│ └── create Gitea release with git-cliff notes
│
├── sync-wiki (sync_wiki.py --strict) [skip if automated]
│ └── sync docs/ to Gitea wiki with integrity check
│
├── vikunja (post_merge.py) [skip if automated]
│ ├── extract task ID from commit message
│ ├── mark Vikunja task as done
│ └── post comment with merge SHA
│
└── badges (push_badges.py) [ALWAYS runs, even on release commits]
├── fetch latest master
├── generate_badges.py → SVG files
├── push to orphan badges branch
└── update README.md + docs/index.md with cache-busting URLs
```
### Publish flow
```text
Tag push (vX.Y.Z) triggers publish workflow (publish.yml):
Within release-and-maintain job (after release step creates a tag):
│
▼
├── install build, twine, git-cliff, tea
├── configure tea login
├── checkout release tag
│
└── publish (publish.py)
├── build package (python -m build)
docs/tech/ci-cd-workflow.md
+146 -108
View File
@@ -1,32 +1,29 @@
# CI/CD Workflow
devx uses Gitea Actions for CI/CD automation. Three workflows implement a
complete pipeline: pull request validation, post-merge release automation, and
tag-triggered publishing.
devx uses Gitea Actions for CI/CD automation. Two workflows implement a
complete pipeline: pull request validation and post-merge release
automation (including publishing).
## Workflow overview
```text
PR opened/synchronized ──► CI (ci.yml)
│ ├── quality
│ ├── detect-changes
│ ├── release-dry-run (if user-facing)
│ ├── pr-review
│ ├── validate (quality + detect-changes +
│ │ release-dry-run + pr-review +
│ │ pre-merge validation)
│ └── auto-merge ──► squash-merge to master
│ │
▼ ▼
Push to master ──► Post-merge (post-merge.yml)
├── detect-type
├── validate-commit-msg
├── release ──► tag vX.Y.Z
├── sync-wiki │
├── badges │
├── vikunja │
└── configure-repo │
│
▼
Tag push (v*) ──► Publish (publish.yml)
└── publish ──► Gitea PyPI registry + Gitea release
├── detect-and-configure (detect-type +
│ validate-commit-msg +
│ configure-repo)
└── release-and-maintain
├── release ──► tag vX.Y.Z
├── publish ──► Gitea PyPI registry + Gitea release
├── sync-wiki
├── vikunja
└── badges (always runs)
```
## CI workflow (`ci.yml`)
@@ -35,9 +32,15 @@ Runs on pull requests (opened and synchronize) and manual dispatch.
### Jobs
#### `quality`
#### `validate`
The main quality gate. Runs on every PR:
The single validation job. Consolidates the former `quality`,
`detect-changes`, `release-dry-run`, `pr-review`, and `pre-merge-check`
jobs into one job to save checkout+setup overhead. Runs on every PR.
**Quality steps**
The main quality gate:
1. **Lint all** — ruff check, ruff format check, pyright, bandit, actionlint
(via `make lint-all`)
@@ -52,21 +55,21 @@ The main quality gate. Runs on every PR:
7. **Workflow dry-run validation** — `make workflow-dryrun` via act_runner
(best-effort, skipped if act_runner is not installed)
#### `detect-changes`
**`detect-changes` step**
Classifies changes between `origin/master` and the PR head as user-facing or
workflow-only using `python -m devx.ci.classify_changes --github-output`.
Writes `user-facing-changed=true|false` to the job output for use by
downstream jobs.
downstream steps.
#### `release-dry-run`
**`release-dry-run` step**
Depends on `quality` and `detect-changes`. Only runs if user-facing changes
are detected. Runs `python -m devx.ci.release --dry-run` to validate that
the release script can calculate the next version and generate the changelog
without making changes. Non-blocking (uses `|| true`).
Only runs if the detect-changes step detected user-facing changes. Runs
`python -m devx.ci.release --dry-run` to validate that the release script
can calculate the next version and generate the changelog without making
changes. Non-blocking (uses `|| true`).
#### `pr-review`
**`pr-review` step**
Runs on every pull request. Executes `python -m devx.ci.pr_review` with the
PR number and repository. Fetches the PR diff via the Gitea API and runs
@@ -87,11 +90,24 @@ Checks performed:
7. Test coverage — source changes must include test updates
8. Commit conventions — conventional commit format on PR commits
**Pre-merge validation step**
Runs on every pull request. Executes
`python -m devx.ci.check_auto_merge_ready` with the branch name, PR title,
repository, and PR number. Validates auto-merge preconditions before the
`auto-merge` job runs:
1. **Branch name** — must contain a valid task ID (for example,
`DEVX-12-fix-foo` → `DEVX-12`)
2. **PR title format** — must be `{PREFIX}-N: <vikunja task title>`
3. **Vikunja task** — must exist and the title must match the PR title
4. **Branch state** — must not be behind master
#### `auto-merge`
Depends on `quality`, `detect-changes`, and `pr-review`. The final job in the
CI workflow. Runs `python -m devx.ci.auto_merge` with the branch name, PR
title, repository, and PR number:
Depends on `validate`. The final job in the CI workflow. Runs
`python -m devx.ci.auto_merge` with the branch name, PR title, repository,
and PR number:
1. **Read task ID** from branch name (for example, `DEVX-12-fix-foo` → `DEVX-12`)
2. **Validate PR title format** — must be `{PREFIX}-N: <vikunja task title>`
@@ -107,8 +123,9 @@ The merge commit push to master triggers the post-merge workflow.
### Smart CI: user-facing vs workflow-only changes
Not all changes require a new release. The `detect-changes` job classifies
changes using `python -m devx.ci.classify_changes`:
Not all changes require a new release. The `detect-changes` step in the
`validate` job classifies changes using
`python -m devx.ci.classify_changes`:
**Workflow-only paths** (infrastructure — no release needed):
- `.gitea/**` — Gitea Actions workflows
@@ -137,55 +154,90 @@ Rule priority (first match wins):
## Post-merge workflow (`post-merge.yml`)
Runs on every push to master. A single workflow with conditional jobs
replaces separate workflows for release, wiki sync, badges, and Vikunja task
updates.
Runs on every push to master. Consolidated into 2 jobs (from 7) to reduce
runner overhead: `detect-and-configure` (detect-type + validate-commit-msg +
configure-repo) and `release-and-maintain` (release + publish + sync-wiki +
badges + vikunja). Individual steps within `release-and-maintain` are
conditional on the `detect-and-configure` job's outputs.
### Job dependency graph
```text
detect-type ──┬── validate-commit-msg (skip if release commit)
├── release (skip if release commit)
│ │
│ ├── sync-wiki (needs release)
│ ├── badges (needs release, ALWAYS runs)
│ └── vikunja (needs release)
└── configure-repo (independent, skip if release commit)
detect-and-configure
├── configure-repo (independent, skip if release commit)
├── detect-type → is-release? is-automated?
└── validate-commit-msg (skip if release commit)
│
▼
release-and-maintain (needs detect-and-configure)
├── release (skip if release commit or workflow-only)
│ └── publish (if release created a tag)
├── sync-wiki (skip if automated)
├── vikunja (skip if automated)
└── badges (always runs)
```
`sync-wiki` and `vikunja` depend on `release` succeeding so that the wiki and
task tracker are only updated when the code is actually released. If release
fails, they are skipped to avoid leaving the wiki or Vikunja in an
inconsistent state.
`sync-wiki` and `vikunja` run only on non-automated commits (that is, real PR
merges) so that the wiki and task tracker are only updated when a human
change lands. They skip on release commits and automated commits.
The `badges` job uses `if: always()` with no is-release condition so it runs
on every push to master, including release commits. This ensures badges
(tests, coverage, version, etc.) are always current.
The `badges` step always runs (even on release commits) so badges (tests,
coverage, version, etc.) are always current. It runs last so it picks up
any version bump the release step created.
When `release` creates a `release: vX.Y.Z` commit, the release commit's
post-merge run still updates badges (the version badge picks up the new
version). Other jobs skip. The tag push triggers `publish.yml`.
version). Other steps skip. The `publish` step builds and publishes the
package to the Gitea PyPI registry within the same `release-and-maintain`
job (it checks out the release tag).
### Post-merge jobs
#### `detect-type`
#### `detect-and-configure`
The first post-merge job. Consolidates the former `detect-type`,
`validate-commit-msg`, and `configure-repo` jobs. Outputs `is-release`,
`is-automated`, and `user-facing-changed` for the `release-and-maintain`
job.
**`detect-type` step**
Checks if the latest commit is a release commit (`release: vX.Y.Z [skip ci]`)
using `python -m devx.ci.detect_release_commit`. Writes `is-release=true` or
`is-release=false` to the job output. All subsequent jobs use this to
conditionally skip for release commits.
`is-release=false` (and `is-automated`) to the job output. The
`release-and-maintain` job uses these to conditionally skip steps for
release commits.
#### `validate-commit-msg`
**`validate-commit-msg` step**
Depends on `detect-type`. Skips for release commits. Validates the latest
commit message using `python -m devx.ci.validate_commit_msg --branch master`.
On master, commits must follow `{PREFIX}-N: <conventional commit>` format
(added by auto-merge).
Skips for release/automated commits. Validates the latest commit message
using `python -m devx.ci.validate_commit_msg --branch master`. On master,
commits must follow `{PREFIX}-N: <conventional commit>` format (added by
auto-merge).
#### `release`
**`configure-repo` step**
Depends on `detect-type`. Skips for release commits. The core release
automation job. Runs `python -m devx.ci.release`:
Ensures branch protection and labels are configured using
`python -m devx.tools.configure_repo --repo <name> --owner <owner>`:
- Sets up master branch protection (required status checks, block on rejected
reviews, block on outdated branch)
- Creates standard labels
- Status check contexts read from `DEVX_STATUS_CHECKS` or default to
`CI / validate (pull_request)`
On failure, the `notify_failure` step creates a Gitea issue.
#### `release-and-maintain`
Depends on `detect-and-configure`. The second post-merge job. Consolidates
the former `release`, `publish`, `sync-wiki`, `badges`, and `vikunja` jobs.
Individual steps are conditional on the `detect-and-configure` job's outputs.
**`release` step**
Skips for release commits and workflow-only changes. The core release
automation step. Runs `python -m devx.ci.release`:
1. **Classify changes** — calls `classify_changes.py` to check for user-facing
changes. If only infrastructure files changed, exits without releasing.
@@ -225,11 +277,10 @@ tag/version/commit alignment.
On failure, the `notify_failure` step creates a Gitea issue via
`python -m devx.ci.notify_failure`.
#### `sync-wiki`
**`sync-wiki` step**
Depends on `detect-type` and `release`. Skips for release commits. Syncs
documentation from `docs/` to the Gitea wiki using
`python -m devx.ci.sync_wiki --repo <owner/repo> --strict`:
Skips for automated commits. Syncs documentation from `docs/` to the Gitea
wiki using `python -m devx.ci.sync_wiki --repo <owner/repo> --strict`:
1. Reads `docs/mapping.json` to map file paths to wiki page titles
2. Lists existing wiki pages via the Gitea API
@@ -243,15 +294,14 @@ deleted).
On failure, the `notify_failure` step creates a Gitea issue.
#### `badges`
**`badges` step**
Depends on `detect-type` and `release`. Uses `if: always()` so it runs on
every push to master, including release commits. Generates and pushes quality
badges using `python -m devx.ci.push_badges`:
Always runs (even on release commits). Generates and pushes quality badges
using `python -m devx.ci.push_badges`:
1. **Fetch latest master** — `git fetch origin master && git reset --hard
origin/master` (ensures the version badge reflects the current state,
even if the release job recently pushed a new version)
even if the release step recently pushed a new version)
2. **Generate badges** — calls `devx.tools.generate_badges` which runs
pytest-cov, doc-coverage, lint checks, and version extraction, then writes
SVG files: `coverage.svg`, `tests.svg`, `docs.svg`, `quality.svg`,
@@ -268,11 +318,10 @@ and waits 10s between attempts).
On failure, the `notify_failure` step creates a Gitea issue.
#### `vikunja`
**`vikunja` step**
Depends on `detect-type` and `release`. Skips for release commits. Updates
the Vikunja task after a merge using `python -m devx.ci.post_merge --git-sha
<sha>`:
Skips for automated commits. Updates the Vikunja task after a merge using
`python -m devx.ci.post_merge --git-sha <sha>`:
1. Extracts the task ID from the first line of the commit message
2. Marks the corresponding Vikunja task as done
@@ -280,26 +329,11 @@ the Vikunja task after a merge using `python -m devx.ci.post_merge --git-sha
On failure, the `notify_failure` step creates a Gitea issue.
#### `configure-repo`
**`publish` step**
Depends on `detect-type`. Skips for release commits. Ensures branch
protection and labels are configured using
`python -m devx.tools.configure_repo --repo <name> --owner <owner>`:
- Sets up master branch protection (required status checks, block on rejected
reviews, block on outdated branch)
- Creates standard labels
- Status check contexts read from `DEVX_STATUS_CHECKS` or default to
`CI / quality (pull_request)`
On failure, the `notify_failure` step creates a Gitea issue.
## Publish workflow (`publish.yml`)
Runs on tag pushes matching `v*`. Triggered by the `release` job in the
post-merge workflow when it creates and pushes a new version tag.
### Job: `publish`
Only runs if the `release` step created a tag. Builds and publishes the
package within the same `release-and-maintain` job (checks out the release
tag). Runs `python -m devx.ci.publish <tag> <owner/repo>`:
1. **Install dependencies** — build, twine, requests, python-dotenv, click,
and the project itself
@@ -518,25 +552,29 @@ The complete release process from PR to published package:
1. **PR merged** — `auto-merge` squash-merges the PR to master with
`{PREFIX}-N <conventional commit>` title
2. **Post-merge triggers** — the merge push triggers `post-merge.yml`
3. **detect-type** — confirms the commit is not a release commit
4. **release** — `release.py` calculates the next version, updates files,
runs tests, commits `release: vX.Y.Z [skip ci]`, creates tag `vX.Y.Z`,
and pushes to master
5. **Tag push triggers publish** — the tag push triggers `publish.yml`
6. **publish** — `publish.py` builds the package, publishes to the Gitea PyPI
registry, and creates a Gitea release with git-cliff notes
7. **sync-wiki** — documentation is synced to the Gitea wiki
8. **badges** — quality badges are regenerated and pushed to the `badges`
branch; README and docs/index.md are updated with cache-busting URLs
9. **vikunja** — the corresponding Vikunja task is marked as done
10. **configure-repo** — branch protection and labels are ensured
3. **detect-and-configure** — detects release commit, validates commit
message, and ensures branch protection/labels
4. **release** (step in `release-and-maintain`) — `release.py` calculates
the next version, updates files, runs tests, commits
`release: vX.Y.Z [skip ci]`, creates tag `vX.Y.Z`, and pushes to master
5. **publish** (step in `release-and-maintain`) — `publish.py` builds the
package, publishes to the Gitea PyPI registry, and creates a Gitea
release with git-cliff notes (checks out the release tag within the
same job)
6. **sync-wiki** (step in `release-and-maintain`) — documentation is synced
to the Gitea wiki
7. **vikunja** (step in `release-and-maintain`) — the corresponding Vikunja
task is marked as done
8. **badges** (step in `release-and-maintain`) — quality badges are
regenerated and pushed to the `badges` branch; README and docs/index.md
are updated with cache-busting URLs
The release commit's post-merge run skips all jobs except `badges` (which
The release commit's post-merge run skips all steps except `badges` (which
picks up the new version number). This prevents infinite loops.
## Failure handling
Every job in the post-merge and publish workflows has a `notify_failure` step
Every job in the CI and post-merge workflows has a `notify_failure` step
that runs `if: failure()`. This creates a Gitea issue with the workflow name,
run ID, and commit SHA, ensuring failures that would otherwise go unnoticed
in the Actions tab are surfaced as issues. The issue is created via the tea