docs: add wait-for-checks CLI command, ADR-0002, update architecture docs

Phase 1e-f of REFACTORING-PLAN.md:

- Register `devx ci wait-for-checks` in cli.py with test coverage
- Add ADR-0002 documenting the ansible_checks consolidation and
  wait_for_checks extraction decisions
- Update AGENTS.md package structure to reflect ansible_checks/
  subpackage and deprecated ci/discover_runners wrapper
- Update docs/user/cli-commands.md with wait-for-checks documentation
  and discover-runners deprecation notice
- Update docs/tech/architecture.md with new modules and canonical
  discover_runners location
- Update docs/mapping.json with ADR entries

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
emil
2026-08-12 19:31:20 +02:00
co-authored by Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
parent 8f1232a7d7
commit 5e1db395b8
7 changed files with 194 additions and 13 deletions
+16 -7
View File
@@ -98,7 +98,9 @@ src/devx/
│ ├── record_deployed_tag.py # Record deployed tag to Gitea repo variable
│ ├── cancel_superseded_runs.py # Cancel in-flight CI runs for the same PR branch
│ ├── check_workflow_artifact_deps.py # Verify artifact download jobs depend on upload jobs
│ └── check_workflow_tofu_init.py # Verify tofu-state jobs have a tofu-init step
│ ├── check_workflow_tofu_init.py # Verify tofu-state jobs have a tofu-init step
│ ├── discover_runners.py # Deprecated wrapper → molecule/discover_runners
│ └── wait_for_checks.py # Poll Gitea Actions for job completion (replaces inline shell polling)
├── tools/ # Developer tooling modules (run locally or by CI)
│ ├── setup.py # Environment setup (venv, deps, hooks)
│ ├── install_tools.py # Install actionlint, git-cliff, act_runner, tea, hadolint, vale
@@ -122,12 +124,19 @@ src/devx/
│ ├── pr_label.py # Add labels to PRs (idempotent)
│ ├── pre_push_check.py # Validate Vikunja task existence before push
│ ├── check_docker_init.py # Check Docker Compose services with healthchecks have init: true
│ ├── check_ansible_set_fact_to_json.py # Check set_fact tasks don't misuse to_json
│ ├── check_ansible_set_fact_to_json.py # Thin wrapper → ansible_checks/set_fact_to_json
│ ├── check_alert_rules.py # Validate Prometheus alert rules with promtool
│ ├── check_ansible_no_log.py # Check Ansible tasks for missing no_log on secrets
│ ├── check_ansible_patterns.py # Detect dangerous failure-masking patterns
│ ├── check_jinja_expr.py # Validate Jinja2 expressions in Ansible files
│ ├── check_ansible_no_state_absent_on_db.py # Prevent state:absent on DB paths
│ ├── check_ansible_no_log.py # Thin wrapper → ansible_checks/no_log
│ ├── check_ansible_patterns.py # Thin wrapper → ansible_checks/patterns
│ ├── check_jinja_expr.py # Thin wrapper → ansible_checks/jinja_expr
│ ├── check_ansible_no_state_absent_on_db.py # Thin wrapper → ansible_checks/no_state_absent_on_db
│ ├── ansible_checks/ # Composable Ansible check subpackage (canonical implementations)
│ │ ├── _shared.py # AnsibleFileFinder, AnsibleYAMLParser, ViolationReporter
│ │ ├── no_log.py # Check missing no_log on secret-handling tasks
│ │ ├── patterns.py # Detect dangerous failure-masking patterns
│ │ ├── set_fact_to_json.py # Check set_fact tasks don't misuse to_json
│ │ ├── no_state_absent_on_db.py # Prevent state:absent on DB paths
│ │ └── jinja_expr.py # Validate Jinja2 expressions in Ansible files
│ └── _shared.py # Shared tool utilities
├── opentofu.py # OpenTofu output helpers (get_tofu_output, get_tofu_vm_ip, get_tofu_vm_field)
├── utils/ # Shared utilities (reusable across projects)
@@ -143,7 +152,7 @@ src/devx/
│ ├── ui.py # say() — unified click.echo + logging output
│ └── jinja.py # Jinja2 environment helpers + Ansible-compatible filters
└── molecule/ # Optional molecule testing helpers (for Ansible projects)
├── discover_runners.py # Dynamic Gitea runner discovery
├── discover_runners.py # Dynamic Gitea runner discovery (canonical; ci/discover_runners is a deprecated wrapper)
├── distribute_molecule.py # Distribute molecule scenarios across runners (LPT scheduling, --roles-root for multi-role)
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast (--roles-root)
├── molecule_all.py # Run all molecule scenarios locally
@@ -0,0 +1,108 @@
# ADR-0002: Ansible Check Tool Consolidation and wait_for_checks Extraction
Date: 2026-08-12
Status: Accepted
## Context
The devx package had two categories of code duplication and inline
workflow logic that were hard to test and maintain:
### 1. Ansible Check Tools — Duplicated Boilerplate
Five Ansible check tools (`check_ansible_no_log`,
`check_ansible_patterns`, `check_ansible_set_fact_to_json`,
`check_ansible_no_state_absent_on_db`, `check_jinja_expr`) each
implemented their own file discovery, YAML parsing, task iteration, and
violation reporting logic. While the check logic differed, the
supporting infrastructure was copy-pasted across all five modules:
- `find_task_files()` — glob YAML files, skip molecule
- YAML multi-document parsing with error handling
- Task iteration (bare lists, play dicts with `tasks`/`pre_tasks`/`post_tasks`/`handlers`, nested `block` tasks)
- Violation formatting (`path:line — message`)
This made it difficult to add new checks (each new tool repeated the
boilerplate) and risky to change shared behavior (fixes had to be
applied to all five modules independently).
### 2. Inline Job Polling in Workflow YAML
The `grm` repository's `ci.yml` workflow contained ~25 lines of inline
shell + Python polling logic to wait for the `molecule-tests` job to
complete before the auto-merge step. This logic:
- Was not testable (embedded in workflow YAML)
- Duplicated the Gitea API client pattern already used elsewhere
- Had no timeout handling, no error reporting, no retry logic
- Could not be reused by other repositories
### 3. Duplicate discover_runners Modules
`devx.ci.discover_runners` and `devx.molecule.discover_runners` were
near-identical modules. The `ci/` version had better error logging
(warnings on non-200 responses, 403 suppression for instance-level
queries), while the `molecule/` version silently swallowed errors.
Both were imported by different workflows, making it unclear which was
canonical.
## Decision
### 1. Composable `ansible_checks/` Subpackage
Consolidate the five Ansible check tools into a
`devx.tools.ansible_checks/` subpackage with shared utilities:
- `_shared.py` — `AnsibleFileFinder`, `AnsibleYAMLParser`,
`ViolationReporter` classes providing composable helpers
- `no_log.py`, `patterns.py`, `set_fact_to_json.py`,
`no_state_absent_on_db.py`, `jinja_expr.py` — canonical check
implementations using the shared utilities
The old modules (`check_ansible_*.py`, `check_jinja_expr.py`) remain as
**thin backward-compat wrappers** that re-export the canonical
implementation and preserve the CLI entry point. This avoids breaking
existing Makefile targets and workflow references.
**Composition over inheritance**: each check module picks the helpers it
needs. Tools that don't parse YAML (for example line-based scanners) can skip
`AnsibleYAMLParser` entirely.
### 2. Extracted `wait_for_checks` Module
Extract the inline polling logic into `devx.ci.wait_for_checks`:
- Polls the Gitea API for job completion status
- Configurable job name prefix, timeout, poll interval
- Exit codes: 0 (success), 1 (failure), 2 (timeout), 3 (API error)
- `--require-success/--no-require-success` flag for flexibility
- 100% test coverage with mocked API responses
This replaces the inline shell polling in `grm` `ci.yml` with a
reusable, testable Python module.
### 3. Deprecated `ci/discover_runners` Wrapper
Merge the `ci/discover_runners` implementation (with its better error
logging) into `molecule/discover_runners` as the canonical version.
Make `ci/discover_runners` a deprecated wrapper that:
- Re-exports all public symbols from `molecule.discover_runners`
- Emits a `DeprecationWarning` when run as `__main__`
- Preserves backward compatibility for existing workflow references
New code should import from `devx.molecule.discover_runners` directly.
## Consequences
- **New checks are easier to write**: import `_shared` helpers, implement
only the check-specific logic
- **Shared behavior can be fixed in one place**: file discovery, YAML
parsing, violation formatting
- **Workflow polling is testable**: `wait_for_checks` has 26 unit tests
covering success, failure, timeout, and API error scenarios
- **Backward compatibility preserved**: all existing Makefile targets,
workflow references, and test imports continue to work via wrappers
- **Migration path is gradual**: new code uses the subpackage; old code
can migrate at its own pace; wrappers can be removed in a future
release once all references are updated
+3 -1
View File
@@ -3,5 +3,7 @@
"user/getting-started.md": "Getting-Started",
"user/cli-commands.md": "CLI-Commands",
"tech/architecture.md": "Architecture",
"tech/ci-cd-workflow.md": "CI-CD-Workflow"
"tech/ci-cd-workflow.md": "CI-CD-Workflow",
"decisions/0001-test-isolation-pytest-plugin-and-shift-left-quality-gates.md": "ADR-0001-Test-Isolation",
"decisions/0002-ansible-check-consolidation-and-wait-for-checks.md": "ADR-0002-Ansible-Check-Consolidation"
}
+22 -5
View File
@@ -33,7 +33,8 @@ src/devx/
│ ├── notify_failure.py # Create Gitea issues on CI failures
│ ├── distribute_files.py # Distribute files across parallel runners
│ ├── integration_guard.py # Run pytest with cross-runner fail-fast
│ ├── discover_runners.py # Dynamic Gitea runner discovery
│ ├── discover_runners.py # Deprecated wrapper → molecule/discover_runners
│ ├── wait_for_checks.py # Poll Gitea Actions for job completion
│ ├── check_translations.py # Translation completeness check
│ └── doc_coverage.py # Documentation coverage check
├── tools/ # Developer tooling modules (run locally or by CI)
@@ -48,7 +49,7 @@ src/devx/
│ └── install_checkmake.py # Install checkmake (Makefile linter)
└── molecule/ # Optional molecule testing helpers (Ansible projects)
├── __init__.py
├── discover_runners.py # Dynamic Gitea runner discovery
├── discover_runners.py # Dynamic Gitea runner discovery (canonical)
├── distribute_molecule.py # Distribute scenarios across runners
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast
├── molecule_all.py # Run all molecule scenarios locally
@@ -285,13 +286,24 @@ Click commands from `cli.py` and verifies each has documentation in
`architecture.md` and CI scripts in `ci-cd-workflow.md`. Supports
`--fail-on-missing` to enforce 100% coverage.
### `discover_runners.py`
### `discover_runners.py` (deprecated wrapper)
> **Deprecated:** Use `devx.molecule.discover_runners` instead. This
> module is a thin wrapper that re-exports the canonical implementation.
Discovers available Gitea Actions runners at three levels: repository,
organization, and instance (administrator). Falls back to the `MOLECULE_RUNNERS` repo
variable or `DEFAULT_MAX_RUNNERS` (3). Outputs runner count or a JSON index
array for use as a dynamic matrix in Gitea Actions.
### `wait_for_checks.py`
Polls the Gitea Actions API for job completion status. Used by auto-merge
jobs that need to wait for parallel jobs (for example molecule-tests) before
proceeding. Replaces inline shell polling in workflow YAML with a
reusable, testable Python module. Exit codes: 0 (success), 1 (job
failure), 2 (timeout), 3 (API error or no matching jobs).
### `distribute_files.py`
Distributes files matching a glob pattern across N parallel runners
@@ -396,8 +408,13 @@ Intended for local development; CI uses the parallel matrix instead.
### `molecule/discover_runners.py`
Discovers available Gitea Actions runners for molecule tests. Same logic as
`devx.ci.discover_runners` but intended for molecule-specific workflows.
Discovers available Gitea Actions runners for molecule tests. This is the
canonical implementation; `devx.ci.discover_runners` is a deprecated wrapper
that re-exports from this module. Queries runners at repository,
organization, and instance (administrator) levels, with warnings logged
to stderr on non-200 responses (except 403 on instance-level, which is
expected without admin scope). Falls back to `MOLECULE_RUNNERS` env var
or `DEFAULT_MAX_RUNNERS` (3).
### `start_docker.py`
+31
View File
@@ -83,6 +83,11 @@ devx ci detect-release-commit
### `devx ci discover-runners`
> **Deprecated:** Use `devx molecule discover-runners` instead. This
> command is a thin wrapper that re-exports the canonical implementation
> from `devx.molecule.discover_runners`. It will be removed in a future
> release.
Discover available Gitea Actions runners for dynamic job distribution.
Queries the Gitea API for registered runners at repository, organization, and
instance (administrator) levels. Falls back to `MOLECULE_RUNNERS` repo variable or
@@ -315,6 +320,32 @@ devx ci validate-commit-msg commit-msg.txt --branch master
Options:
- `--branch <branch>` — override branch detection (for CI use)
### `devx ci wait-for-checks`
Wait for Gitea Actions jobs to complete by polling the API. Used by
auto-merge jobs that need to wait for parallel jobs (for example molecule-tests)
before proceeding. Replaces inline shell polling in workflow YAML with
a reusable, testable Python module.
Exit codes:
- `0` — all matching jobs completed successfully
- `1` — one or more matching jobs failed (when `--require-success` is set)
- `2` — timeout reached before all jobs completed
- `3` — API error or no matching jobs found
```bash
devx ci wait-for-checks --job-name molecule-tests --repo oblachno-oss/grm
devx ci wait-for-checks --job-name molecule-tests --timeout 1200 --poll-interval 10
devx ci wait-for-checks --job-name molecule-tests --no-require-success
```
Options:
- `--job-name <prefix>` — job name prefix to match (required)
- `--repo <owner/name>` — repository (default: `$GITHUB_REPOSITORY`)
- `--timeout <seconds>` — max wait time (default: 1200 = 20 min)
- `--poll-interval <seconds>` — seconds between polls (default: 10)
- `--require-success / --no-require-success` — exit 1 if a job failed (default: yes)
### `devx ci cancel-superseded-runs`
Cancel in-flight CI runs for the same PR branch when a new push triggers
+7
View File
@@ -158,6 +158,13 @@ def ci_validate_commit_msg(args: tuple[str, ...]) -> None:
_run_module("devx.ci.validate_commit_msg", list(args))
@ci.command("wait-for-checks")
@click.argument("args", nargs=-1)
def ci_wait_for_checks(args: tuple[str, ...]) -> None:
"""Wait for Gitea Actions jobs to complete (polls API)."""
_run_module("devx.ci.wait_for_checks", list(args))
@ci.command("distribute-files")
@click.argument("args", nargs=-1)
def ci_distribute_files(args: tuple[str, ...]) -> None:
+7
View File
@@ -150,6 +150,13 @@ class TestCiCommands:
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.ci.validate_commit_msg", ["msg"])
@patch("devx.cli._run_module")
def test_ci_wait_for_checks(self, mock_run: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["ci", "wait-for-checks", "--", "--job-name", "molecule-tests"])
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.ci.wait_for_checks", ["--job-name", "molecule-tests"])
class TestToolsCommands:
@patch("devx.cli._run_module")