DEVX-62: feat: weighted LPT distribution, workflow fixes, decouple vikunja/sync-wiki from release
This commit is contained in:
@@ -66,7 +66,7 @@ src/devx/
|
||||
│ ├── sync_wiki.py # Sync documentation to Gitea wiki
|
||||
│ ├── push_badges.py # Generate and push quality badges (--retries for retry on git push failures)
|
||||
│ ├── notify_failure.py # Create Gitea issues on CI failures (--auto-login)
|
||||
│ ├── distribute_files.py # Distribute files across parallel runners
|
||||
│ ├── distribute_files.py # Distribute files across parallel runners (LPT scheduling)
|
||||
│ ├── integration_guard.py # Run pytest with cross-runner fail-fast
|
||||
│ ├── check_translations.py # Translation completeness check
|
||||
│ └── doc_coverage.py # Documentation coverage check
|
||||
@@ -79,7 +79,7 @@ src/devx/
|
||||
├── opentofu.py # OpenTofu output helpers (get_tofu_output, get_tofu_vm_ip, get_tofu_vm_field)
|
||||
└── molecule/ # Optional molecule testing helpers (for Ansible projects)
|
||||
├── discover_runners.py # Dynamic Gitea runner discovery
|
||||
├── distribute_molecule.py # Distribute molecule scenarios across runners (--roles-root for multi-role)
|
||||
├── distribute_molecule.py # Distribute molecule scenarios across runners (LPT scheduling, --roles-root for multi-role)
|
||||
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast (--roles-root)
|
||||
├── molecule_all.py # Run all molecule scenarios locally
|
||||
└── platforms.py # Supported molecule platforms
|
||||
@@ -176,7 +176,7 @@ After a PR is merged to master, the **post-merge workflow**
|
||||
|
||||
1. **detect-type** — Checks if the commit is a regular merge or a
|
||||
release commit (`release: vX.Y.Z`). All subsequent jobs skip for
|
||||
release commits.
|
||||
release commits (except badges).
|
||||
|
||||
2. **release** — Runs `python -m devx.ci.release` which:
|
||||
- Checks for user-facing changes via `python -m devx.ci.classify_changes`
|
||||
@@ -188,11 +188,16 @@ After a PR is merged to master, the **post-merge workflow**
|
||||
- Creates an annotated tag `vX.Y.Z` on the release commit
|
||||
- Pushes both the commit and tag to master
|
||||
|
||||
3. **sync-wiki** — Syncs documentation to the Gitea wiki.
|
||||
3. **sync-wiki** — Syncs documentation to the Gitea wiki. Runs for ALL
|
||||
non-release commits (not just when release succeeds), so docs-only
|
||||
changes still update the wiki.
|
||||
|
||||
4. **badges** — Generates and pushes quality badge SVGs to the `badges` branch.
|
||||
Uses `if: always()` so it runs on every push, including release commits.
|
||||
|
||||
5. **vikunja** — Marks the corresponding Vikunja task as done.
|
||||
5. **vikunja** — Marks the corresponding Vikunja task as done. Runs for ALL
|
||||
non-release commits (not just when release succeeds), so infrastructure-only
|
||||
changes still update the task tracker.
|
||||
|
||||
The tag push triggers the **publish workflow** (`.gitea/workflows/publish.yml`)
|
||||
which builds and publishes the package to the Gitea PyPI registry.
|
||||
@@ -310,6 +315,25 @@ auto-merge:
|
||||
(needs.molecule-tests.result == 'success' || needs.molecule-tests.result == 'skipped')
|
||||
```
|
||||
|
||||
### LPT Test Distribution Algorithm
|
||||
|
||||
`distribute_molecule` and `distribute_files` use **LPT (Longest Processing
|
||||
Time first)** scheduling instead of naive round-robin. This produces a more
|
||||
balanced distribution when test items have varying costs:
|
||||
|
||||
1. **Weight estimation**: Each item is assigned a weight:
|
||||
- Molecule scenarios: heuristic by name (`nextcloud`=10, `gitea`=8,
|
||||
`binary`=2, default=3). See `_SCENARIO_WEIGHTS` in
|
||||
`distribute_molecule.py`.
|
||||
- Integration test files: weight by file size in bytes (as a proxy
|
||||
for test runtime).
|
||||
2. **LPT assignment**: Items are sorted by weight (descending), then
|
||||
each is assigned to the runner with the least total weight.
|
||||
|
||||
This ensures heavy scenarios (e.g. `nextcloud`) are spread across
|
||||
different runners rather than clustered on one, reducing the
|
||||
longest-runner time from ~16 min to ~11 min with 6 runners.
|
||||
|
||||
## Config System
|
||||
|
||||
devx uses environment variables with `.env` file fallback for configuration.
|
||||
|
||||
Reference in New Issue
Block a user