Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
18632bd543 | ||
|
|
1fc7a03c1a | ||
|
|
808e7a2e42 | ||
|
|
dd8e6c69e9 | ||
|
|
ccb7023965 | ||
|
|
7dd15f1461 | ||
|
|
d4e4621fa1 | ||
|
|
a6f814c446 | ||
|
|
04aa5acb1f | ||
|
|
6973f9d851 | ||
|
|
2669a0ea73 | ||
|
|
03f057b55a | ||
|
|
706d6dafe0 | ||
|
|
03ddce427c | ||
|
|
9642d6884c | ||
|
|
b2074d6635 | ||
|
|
a6dddf25e7 | ||
|
|
01130a7385 | ||
|
|
07580c9280 | ||
|
|
6601d90bee | ||
|
|
0df79fed53 | ||
|
|
cf8287e683 | ||
|
|
9f1bdc4cf1 |
@@ -1,135 +0,0 @@
|
||||
# dependency-graph
|
||||
|
||||
Map of the oblachno ecosystem. Knows which repo produces what, which
|
||||
repos depend on which, and the correct order for cross-repo changes.
|
||||
|
||||
## When to Invoke
|
||||
|
||||
Invoke this skill when:
|
||||
- Changes span multiple repos
|
||||
- A change in one repo requires version bumps in downstream repos
|
||||
- Deploying infrastructure that depends on published packages/images
|
||||
- Verifying the ecosystem is in a consistent state before deployment
|
||||
- Determining which repos to update and in what order
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- All repos cloned under `/home/emo/dev/ideas/oblachno/`
|
||||
- `.env` with `DEVELOPER_GITEA_API_TOKEN` in each repo
|
||||
|
||||
## Ecosystem Map
|
||||
|
||||
```
|
||||
devx (PyPI package)
|
||||
/ | \
|
||||
/ | \
|
||||
grm sso-bridge infra
|
||||
(PyPI) (PyPI+Docker) (deploys all)
|
||||
| | |
|
||||
v v v
|
||||
infra bump infra bump staging
|
||||
(auto PR) (auto PR) production
|
||||
|
|
||||
mattermost-oidc (Docker image)
|
||||
(infra pulls :latest at deploy)
|
||||
```
|
||||
|
||||
## Repositories
|
||||
|
||||
| Repo | Produces | Consumers | Release Trigger |
|
||||
|------|----------|-----------|-----------------|
|
||||
| `devx` | PyPI package `devx` | grm, sso-bridge, infra | User-facing changes to `src/devx/**` |
|
||||
| `grm` | PyPI package `grm` | infra | User-facing changes to `src/grm/**` or `ansible/**` |
|
||||
| `sso-bridge` | PyPI package `sso_bridge` + Docker image | infra | User-facing changes to `src/sso_bridge/**` or `ansible/**` |
|
||||
| `infra` | Staging/production deployment | (end users) | User-facing changes + nightly gate |
|
||||
| `mattermost-oidc` | Docker image `mattermost-oidc` | infra (pulls at deploy) | `Dockerfile` or `build.yml` changes |
|
||||
|
||||
## Dependency Chain
|
||||
|
||||
### devx → all repos
|
||||
|
||||
devx publishes to the Gitea PyPI registry. grm, sso-bridge, and infra
|
||||
pin devx in `pyproject.toml`:
|
||||
```toml
|
||||
"devx @ git+https://git.oblachno.oblachno.fyi/oblachno-oss/devx.git@vX.Y.Z"
|
||||
```
|
||||
|
||||
When devx publishes a new version:
|
||||
1. grm, sso-bridge, and infra must bump their pinned devx version
|
||||
2. This is currently manual — no auto-dependency-PR from devx
|
||||
3. Each repo must `make setup` to pick up the new version
|
||||
|
||||
### grm → infra
|
||||
|
||||
grm publishes to PyPI. Its post-merge workflow auto-creates an infra
|
||||
dependency PR via `devx.ci.create_dependency_pr --repo oblachno/infra
|
||||
--package grm`. The PR bumps the pinned grm version in infra's
|
||||
`pyproject.toml`.
|
||||
|
||||
### sso-bridge → infra
|
||||
|
||||
sso-bridge publishes to PyPI AND builds a Docker image. Its post-merge
|
||||
workflow auto-creates an infra dependency PR via
|
||||
`devx.ci.create_dependency_pr --repo oblachno/infra --package
|
||||
sso_bridge`. The PR bumps the pinned sso_bridge version.
|
||||
|
||||
The Docker image is pulled by infra at deploy time (`sso-bridge:latest`).
|
||||
|
||||
### mattermost-oidc → infra
|
||||
|
||||
mattermost-oidc builds a Docker image tagged `:latest` and `:MM_VERSION`.
|
||||
infra pulls `mattermost-oidc:latest` at deploy time. There is no
|
||||
auto-dependency-PR — infra simply pulls the latest image.
|
||||
|
||||
### infra → staging/production
|
||||
|
||||
infra deploys to staging and production. The deployment:
|
||||
1. Provisions VMs from golden images
|
||||
2. Runs Ansible roles (including grm and sso-bridge roles)
|
||||
3. Pulls Docker images (sso-bridge, mattermost-oidc)
|
||||
4. Configures services
|
||||
|
||||
## Correct Order for Cross-Repo Changes
|
||||
|
||||
When a change spans multiple repos, follow this order:
|
||||
|
||||
1. **devx first** — if the change starts in devx, merge and publish devx
|
||||
first. Wait for the PyPI publish job to complete.
|
||||
2. **Bump devx in consumers** — in grm/sso-bridge/infra, bump the pinned
|
||||
devx version, run `make setup`, verify tests pass, merge.
|
||||
3. **grm/sso-bridge second** — merge and publish grm/sso-bridge. Wait
|
||||
for the PyPI publish + Docker image build to complete.
|
||||
4. **Auto-dependency-PRs** — grm/sso-bridge post-merge auto-creates infra
|
||||
PRs to bump pinned versions. Wait for these PRs to appear.
|
||||
5. **Merge infra dependency PRs** — review and merge the auto-created
|
||||
infra PRs.
|
||||
6. **infra last** — deploy to staging, validate, promote to production.
|
||||
|
||||
## State Verification Before Deployment
|
||||
|
||||
Before deploying infra, verify:
|
||||
|
||||
1. **devx version consistent** — all repos pin the same devx version
|
||||
2. **grm published** — latest grm tag exists in PyPI
|
||||
3. **sso-bridge published** — latest sso_bridge tag exists in PyPI
|
||||
4. **sso-bridge image built** — latest sso-bridge Docker image exists
|
||||
5. **mattermost-oidc image built** — latest mattermost-oidc image exists
|
||||
6. **infra pins match published versions** — no stale pins
|
||||
7. **Nightly gate green** — `NIGHTLY_STATUS` is not `failed`
|
||||
|
||||
## Quick Check Commands
|
||||
|
||||
```bash
|
||||
# Check latest devx version
|
||||
curl -sS https://git.oblachno.oblachno.fyi/api/v1/repos/oblachno-oss/devx/releases/latest | python3 -c "import json,sys; print(json.load(sys.stdin).get('tag_name','?'))"
|
||||
|
||||
# Check pinned devx version in each repo
|
||||
for repo in grm sso-bridge infra; do
|
||||
echo -n "$repo: "; grep 'devx @' /home/emo/dev/ideas/oblachno/$repo/pyproject.toml | grep -oP 'v[\d.]+'
|
||||
done
|
||||
|
||||
# Check latest sso-bridge image build
|
||||
curl -sS -H "Authorization: token $DEVELOPER_GITEA_API_TOKEN" \
|
||||
"https://git.oblachno.oblachno.fyi/api/v1/repos/oblachno/sso-bridge/actions/runs?per_page=5" \
|
||||
| python3 -c "import json,sys; [print(r['id'],r['status'],r['conclusion']) for r in json.load(sys.stdin).get('workflow_runs',[]) if r.get('event')=='push']"
|
||||
```
|
||||
@@ -1,90 +0,0 @@
|
||||
# deployment-coordination
|
||||
|
||||
How devx releases propagate to downstream repos. devx is the base
|
||||
package — all other repos pin it. A devx release must complete before
|
||||
consumers can bump.
|
||||
|
||||
## When to Invoke
|
||||
|
||||
Invoke this skill when:
|
||||
- Changes to devx affect downstream repos (grm, sso-bridge, infra)
|
||||
- Preparing a devx release that other repos depend on
|
||||
- Verifying downstream repos have bumped to the latest devx version
|
||||
- Coordinating a multi-repo change that starts in devx
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- devx repo at `/home/emo/dev/ideas/oblachno/devx`
|
||||
- `.env` with `DEVELOPER_GITEA_API_TOKEN`
|
||||
- See `dependency-graph` skill for the full ecosystem map
|
||||
|
||||
## What devx Produces
|
||||
|
||||
devx publishes a Python package to the Gitea PyPI registry. Downstream
|
||||
repos pin it:
|
||||
```toml
|
||||
"devx @ git+https://git.oblachno.oblachno.fyi/oblachno-oss/devx.git@vX.Y.Z"
|
||||
```
|
||||
|
||||
## Release Flow
|
||||
|
||||
1. PR merged to master
|
||||
2. Post-merge workflow runs `devx.ci.release` — classifies changes
|
||||
3. If user-facing changes: git-cliff bumps version, creates tag, pushes
|
||||
4. `devx.ci.publish` builds and publishes to Gitea PyPI registry
|
||||
5. Downstream repos must bump their pinned devx version
|
||||
|
||||
## Downstream Consumers
|
||||
|
||||
| Repo | Pin location | Auto-bump? |
|
||||
|------|-------------|------------|
|
||||
| grm | `pyproject.toml` | No — manual |
|
||||
| sso-bridge | `pyproject.toml` | No — manual |
|
||||
| infra | `pyproject.toml` | No — manual |
|
||||
|
||||
devx does NOT auto-create dependency PRs in downstream repos. Bumping
|
||||
is manual: create a PR in each downstream repo to update the pinned
|
||||
version.
|
||||
|
||||
## Coordinating a devx Change
|
||||
|
||||
When a change to devx affects downstream repos:
|
||||
|
||||
1. **Merge devx PR** — wait for post-merge publish to complete
|
||||
2. **Verify publish** — check the new tag exists:
|
||||
```bash
|
||||
curl -sS -H "Authorization: token $DEVELOPER_GITEA_API_TOKEN" \
|
||||
https://git.oblachno.oblachno.fyi/api/v1/repos/oblachno-oss/devx/releases/latest \
|
||||
| python3 -c "import json,sys; print(json.load(sys.stdin).get('tag_name','?'))"
|
||||
```
|
||||
3. **Bump downstream repos** — for each repo (grm, sso-bridge, infra):
|
||||
- Update `devx @ ...@vX.Y.Z` in `pyproject.toml`
|
||||
- Run `make setup` to install the new version
|
||||
- Run `make pytest-cov` to verify compatibility
|
||||
- Create and merge a PR
|
||||
4. **Verify infra deploys** — after infra bumps, verify staging deploy
|
||||
picks up the new devx version
|
||||
|
||||
## State Verification
|
||||
|
||||
Before starting a devx change, verify current state:
|
||||
```bash
|
||||
# Current devx version
|
||||
grep '__version__' /home/emo/dev/ideas/oblachno/devx/src/devx/__init__.py
|
||||
|
||||
# What each repo pins
|
||||
for repo in grm sso-bridge infra; do
|
||||
echo -n "$repo pins: "
|
||||
grep 'devx @' /home/emo/dev/ideas/oblachno/$repo/pyproject.toml | grep -oP 'v[\d.]+'
|
||||
done
|
||||
```
|
||||
|
||||
If pins are inconsistent across repos, bump them to the latest published
|
||||
version before starting new work.
|
||||
|
||||
## Common Mistakes
|
||||
|
||||
- Merging a devx PR and immediately merging downstream PRs without
|
||||
waiting for the publish job to complete
|
||||
- Forgetting to bump infra (it has the most complex deploy pipeline)
|
||||
- Bumping only one downstream repo when the change affects all three
|
||||
@@ -2,26 +2,17 @@
|
||||
|
||||
Quick reference for devx tools when working on the devx repo itself.
|
||||
|
||||
## When to Invoke
|
||||
|
||||
Invoke this skill when creating PRs, checking CI status, adding
|
||||
labels, rebasing branches, or performing any PR lifecycle operation.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `.venv` exists (run `make setup` if not)
|
||||
- `.env` with `DEVELOPER_GITEA_API_TOKEN`, `VIKUNJA_TOKEN`
|
||||
|
||||
## PR Workflow (use these, not raw git/tea/MCP)
|
||||
|
||||
| Task | Command |
|
||||
|------|---------|
|
||||
| Create Vikunja task | `.venv/bin/python -m devx.tools.create_task --title "..." --description "..."` (make target doesn't forward args) |
|
||||
| Create Vikunja task | `make create-task -- --title "..." --description "..."` |
|
||||
| Create PR | `make create-pr` |
|
||||
| Push + create PR | `make push-with-pr` |
|
||||
| Check CI status | `make devx-pr-status` or `make devx-pr-status PR=42 WAIT=1` |
|
||||
| Fetch CI failure logs | `make devx-pr-logs` or `make devx-pr-logs PR=42 JOB=quality TAIL=50` |
|
||||
| Add ready-to-merge label | `make devx-pr-label` or `make devx-pr-label PR=42` |
|
||||
| Post PR review | `make devx-pr-review PR=42 EVENT=APPROVE BODY="..." CHECKLIST=1,2,3,4,5,6,7,8,9,10,11,12,13` |
|
||||
| Rebase current branch | `make rebase` |
|
||||
| Rebase PR via API | `make pr-rebase` or `make pr-rebase PR=42` |
|
||||
|
||||
@@ -33,15 +24,6 @@ When the `ready-to-merge` label is added and all CI checks pass:
|
||||
3. The rebase triggers a new CI run; the next auto-merge attempt merges
|
||||
4. No manual rebase needed unless the API rebase fails
|
||||
|
||||
## Spec-Driven CI Gates (Pre-merge)
|
||||
|
||||
Every PR must pass these gates before merge:
|
||||
|
||||
| Gate | Module | What it checks |
|
||||
|------|--------|----------------|
|
||||
| Spec validation | `devx.ci.validate_spec` | Spec file exists at `docs/specs/<TASK-ID>.md`, has REQ-IDs, all ACs checked |
|
||||
| PR size | `devx.ci.check_pr_size` | Max 500 lines / 10 files (excludes CHANGELOG, badges, locks) |
|
||||
|
||||
## Key Rules
|
||||
|
||||
- Never manually merge via API — always use auto-merge with `ready-to-merge` label
|
||||
|
||||
@@ -1,272 +0,0 @@
|
||||
# pr-review
|
||||
|
||||
Deep, critical PR review with auto-fix. This skill guides the agent
|
||||
through a thorough review of a pull request, posting inline comments
|
||||
for each issue found, auto-fixing them, resolving the discussion threads,
|
||||
and marking the PR as ready-to-merge when no blocking issues remain.
|
||||
|
||||
## When to Invoke
|
||||
|
||||
Invoke this skill when asked to review a PR, or when a PR is open and
|
||||
needs review before merge. Do NOT invoke automatically on every PR —
|
||||
this is an on-demand deep review, not a CI gate.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- The PR must be open in a Gitea repo
|
||||
- The agent needs Gitea MCP access (gitea server)
|
||||
- The agent needs git push access to the PR's head branch
|
||||
- The PR should have passed CI (validate job) before deep review
|
||||
|
||||
## Review Categories
|
||||
|
||||
Review every PR against these 8 categories. For each issue found, post
|
||||
an inline comment on the specific line, then auto-fix it.
|
||||
|
||||
### 1. Functional Correctness
|
||||
|
||||
- Does the code actually do what the spec/PR title claims?
|
||||
- Are edge cases handled? (empty input, null, boundary values, concurrent access)
|
||||
- Are error paths tested? Not just happy path.
|
||||
- Does the code handle all return values? (ignored errors, unchecked None)
|
||||
- Are there off-by-one errors, wrong comparisons, inverted conditions?
|
||||
- Do loops terminate correctly? (no infinite loops, correct break/continue)
|
||||
- Are regex patterns correct? (anchored, escaped, non-greedy where needed)
|
||||
- Are API responses validated before use? (status codes, response shape)
|
||||
|
||||
### 2. Completeness
|
||||
|
||||
- Are all requirements from the spec implemented? (check each REQ-ID)
|
||||
- Are all acceptance criteria in the spec checked off?
|
||||
- Are tests written for all new code paths?
|
||||
- Are error messages user-facing (wrapped in `_()`)?
|
||||
- Are new CLI commands documented in `docs/user/cli-commands.md`?
|
||||
- Are new modules added to architecture docs?
|
||||
- Are CHANGELOG entries added for user-facing changes?
|
||||
- Are translations added for new user-facing strings?
|
||||
|
||||
### 3. Architecture
|
||||
|
||||
- Does the code follow the repo's layer separation? (no business logic in CLI, no direct subprocess in CLI)
|
||||
- Are new dependencies justified? (no unnecessary new packages)
|
||||
- Is configuration via env vars / config.py, not hardcoded?
|
||||
- Are new modules placed in the correct directory? (ci/ vs tools/ vs molecule/)
|
||||
- Does the code reuse existing utilities? (no reimplemented helpers)
|
||||
- Are imports circular? (check import chains)
|
||||
- Is the code testable? (injectable dependencies, no hidden global state)
|
||||
- Does the code follow existing patterns in the codebase?
|
||||
|
||||
### 4. Reliability
|
||||
|
||||
- Are external API calls retried with backoff?
|
||||
- Are timeouts set on all network operations?
|
||||
- Are file operations atomic? (write to temp, rename)
|
||||
- Are database operations transactional where needed?
|
||||
- Are there race conditions? (check shared mutable state)
|
||||
- Are resources cleaned up in all paths? (finally blocks, context managers)
|
||||
- Can the code handle partial failures? (one service down, others up)
|
||||
- Are idempotency guarantees maintained? (safe to retry)
|
||||
|
||||
### 5. Robustness
|
||||
|
||||
- Does the code fail gracefully? (meaningful error messages, not stack traces)
|
||||
- Are unexpected inputs handled? (type checking, validation)
|
||||
- Are there any crash-on-bad-input paths?
|
||||
- Does the code degrade under load? (backpressure, queue limits)
|
||||
- Are there resource leaks? (file handles, connections, memory)
|
||||
- Does the code survive network partitions? (retry, circuit breaker)
|
||||
- Are there any unhandled exceptions that could crash the process?
|
||||
- Is logging sufficient to diagnose production issues?
|
||||
|
||||
### 6. Security
|
||||
|
||||
- Are there hardcoded secrets, tokens, or passwords?
|
||||
- Is `shell=True` used with user input? (command injection)
|
||||
- Is `eval()` or `exec()` used? (code injection)
|
||||
- Are SQL queries parameterized? (no string concatenation)
|
||||
- Are file paths validated? (no path traversal)
|
||||
- Are user inputs sanitized before display? (XSS in web contexts)
|
||||
- Are SSL/TLS verifications disabled without justification?
|
||||
- Are secrets logged in error messages or debug output?
|
||||
- Are permissions checked before privileged operations?
|
||||
- Is sensitive data in memory longer than necessary?
|
||||
|
||||
### 7. Technical Excellence
|
||||
|
||||
- Are functions under 50 lines? (refactor if longer)
|
||||
- Is cyclomatic complexity reasonable? (no deeply nested if/else chains)
|
||||
- Are names meaningful? (no single-letter vars, no misleading names)
|
||||
- Is dead code removed? (no commented-out blocks, no unused imports)
|
||||
- Are comments explaining WHY, not WHAT?
|
||||
- Is the code DRY? (no copy-pasted blocks that should be shared)
|
||||
- Is the code SOLID? (single responsibility, open/closed)
|
||||
- Are magic numbers extracted to named constants?
|
||||
- Is the code formatted per the repo's linter config?
|
||||
- Are type hints present on all function signatures?
|
||||
|
||||
### 8. Test Quality
|
||||
|
||||
- Do tests actually test the behavior? (not just that code runs)
|
||||
- Are tests independent? (no shared mutable state, no order dependency)
|
||||
- Are tests fast? (no real sleeps, no real network calls, mocked)
|
||||
- Are edge cases tested? (empty, None, boundary, error paths)
|
||||
- Are test names descriptive? (test_what_condition_expected_result)
|
||||
- Are mocks set up correctly? (mocking the right object, not too broad)
|
||||
- Is coverage 100% for new code? (every branch, every line)
|
||||
- Are integration tests added for cross-module changes?
|
||||
- Do tests clean up after themselves? (tmp_path, fixtures)
|
||||
|
||||
## Review Procedure
|
||||
|
||||
### Step 1: Gather Context
|
||||
|
||||
```
|
||||
1. Read the PR spec (if exists): docs/specs/<TASK-ID>.md
|
||||
2. Fetch PR details via Gitea MCP: pull_request_read (get_pr, list_pr_files)
|
||||
3. Read the full diff: git diff origin/master...HEAD
|
||||
4. Read the PR description and any existing review comments
|
||||
5. Identify the repo's task prefix (OBL-INFRA, GRM, SSO, DEVX)
|
||||
```
|
||||
|
||||
### Step 2: Review Each File
|
||||
|
||||
For each changed file in the PR:
|
||||
|
||||
1. Read the full file (not just the diff) to understand context
|
||||
2. Go through all 8 review categories
|
||||
3. For each issue found, note: file path, line number, category, severity, description, suggested fix
|
||||
|
||||
### Step 3: Post Inline Comments
|
||||
|
||||
For each issue found, post an inline review comment using the Gitea MCP:
|
||||
|
||||
```
|
||||
mcp_call_tool: gitea / pull_request_review_write
|
||||
method: create
|
||||
owner: <owner>
|
||||
repo: <repo>
|
||||
pull_number: <PR number>
|
||||
state: PENDING (accumulate comments before submitting)
|
||||
body: "" (empty for now, summary added on submit)
|
||||
comments: [
|
||||
{
|
||||
path: "<file path>",
|
||||
new_line_num: <line number>,
|
||||
body: "**[<category>] [<severity>]** <description>\n\n**Suggested fix:**\n```<lang>\n<fixed code>\n```"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Comment format:
|
||||
```
|
||||
**[Security] [error]** `shell=True` used with user input — command injection risk.
|
||||
|
||||
**Suggested fix:**
|
||||
```python
|
||||
subprocess.run(["git", "log", commit], check=True)
|
||||
```
|
||||
```
|
||||
|
||||
Severity levels:
|
||||
- `error` — must fix before merge (security, correctness, crash)
|
||||
- `warning` — should fix before merge (reliability, best practice)
|
||||
- `info` — consider fixing (style, minor improvement)
|
||||
|
||||
### Step 4: Auto-Fix Issues
|
||||
|
||||
For each issue that can be safely auto-fixed:
|
||||
|
||||
1. Edit the file using the `edit` tool
|
||||
2. Commit with message: `fix: address review comment — <short description>`
|
||||
3. Push to the PR's head branch: `git push origin HEAD`
|
||||
4. Wait for CI to re-run on the push
|
||||
|
||||
Auto-fix ALL issues unless:
|
||||
- The fix requires an architectural decision (ask the user)
|
||||
- The fix changes public API behavior (ask the user)
|
||||
- The fix is ambiguous (multiple valid approaches, ask the user)
|
||||
|
||||
### Step 5: Resolve Discussion Threads
|
||||
|
||||
After auto-fixing an issue and CI passes:
|
||||
|
||||
1. Find the review comment thread for that issue
|
||||
2. Post a reply: `Fixed in <commit-sha>. Closing this thread.`
|
||||
3. Resolve the discussion (if Gitea supports it via API)
|
||||
4. If resolving via API is not available, the reply comment serves as resolution
|
||||
|
||||
### Step 6: Submit Final Review
|
||||
|
||||
After all issues are addressed (fixed or discussed):
|
||||
|
||||
```
|
||||
mcp_call_tool: gitea / pull_request_review_write
|
||||
method: submit
|
||||
owner: <owner>
|
||||
repo: <repo>
|
||||
pull_number: <PR number>
|
||||
review_id: <from step 3 create>
|
||||
state: COMMENT (or APPROVED if no blocking issues remain)
|
||||
body: <summary — see below>
|
||||
```
|
||||
|
||||
### Step 7: Post Summary
|
||||
|
||||
Post a brief summary as a PR comment (via `issue_write / add_comment`):
|
||||
|
||||
```
|
||||
## Deep Review Summary
|
||||
|
||||
- **Files reviewed:** N
|
||||
- **Issues found:** N (N auto-fixed, N require attention)
|
||||
- **Categories:** security (N), correctness (N), architecture (N), ...
|
||||
|
||||
**Outcome:** ✅ Ready to merge — all issues addressed.
|
||||
**OR**
|
||||
**Outcome:** ⚠️ N blocking issue(s) remain — see inline comments.
|
||||
```
|
||||
|
||||
Keep the summary to 5-10 bullet points. Do not paste the full review.
|
||||
|
||||
### Step 8: Mark PR Ready
|
||||
|
||||
If all issues are addressed and no blocking issues remain:
|
||||
|
||||
```
|
||||
mcp_call_tool: gitea / issue_write
|
||||
method: add_labels
|
||||
owner: <owner>
|
||||
repo: <repo>
|
||||
issue_number: <PR number>
|
||||
labels: [<label_id for "ready-to-merge">]
|
||||
```
|
||||
|
||||
If blocking issues remain, do NOT add the label. Post a comment
|
||||
explaining what needs to be resolved before the PR can merge.
|
||||
|
||||
## Gitea MCP Tools Reference
|
||||
|
||||
| Action | MCP tool | Method |
|
||||
|--------|----------|--------|
|
||||
| Get PR details | `pull_request_read` | `get_pr` |
|
||||
| List PR files | `pull_request_read` | `list_pr_files` |
|
||||
| Get PR diff | `pull_request_read` | `get_pr_diff` |
|
||||
| Create review (pending) | `pull_request_review_write` | `create` (state: PENDING) |
|
||||
| Submit review | `pull_request_review_write` | `submit` (state: APPROVED/COMMENT/REQUEST_CHANGES) |
|
||||
| Post PR comment | `issue_write` | `add_comment` |
|
||||
| Add label | `issue_write` | `add_labels` |
|
||||
| List labels | `label_read` | `list_repo_labels` |
|
||||
| Merge PR | `pull_request_write` | `merge` (do NOT use — auto-merge handles this) |
|
||||
|
||||
## Important Rules
|
||||
|
||||
- **Never merge the PR yourself.** Add the `ready-to-merge` label and let
|
||||
the auto-merge workflow handle it. This ensures CI passes and the
|
||||
commit message follows the `<PREFIX>-N: <conventional>` format.
|
||||
- **Never approve your own PR.** If the agent created the PR, post
|
||||
COMMENT state, not APPROVED.
|
||||
- **Always push fixes to the PR branch**, not directly to master.
|
||||
- **Wait for CI after each push** before resolving the discussion thread.
|
||||
- **Post one review with all comments**, not multiple reviews.
|
||||
- **The summary must be brief** — 5-10 bullet points max.
|
||||
- **Severity matters**: only `error` severity blocks the `ready-to-merge` label.
|
||||
@@ -1,142 +0,0 @@
|
||||
# skill-creation
|
||||
|
||||
How to create, validate, and maintain Devin skills. Skills must be
|
||||
clear, succinct, and actionable — no AI slop.
|
||||
|
||||
## When to Invoke
|
||||
|
||||
Invoke this skill when:
|
||||
- Creating a new skill
|
||||
- Amending an existing skill
|
||||
- Evaluating whether a skill is needed
|
||||
- Reviewing a PR that adds or modifies skills
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Skill directory: `.devin/skills/<skill-name>/SKILL.md`
|
||||
- Validator: `tests/test_skills.py` (infra) or reference to it
|
||||
- Tests: `tests/unit/test_skills_validation.py` (infra)
|
||||
|
||||
## When to Create a Skill
|
||||
|
||||
Create a skill when:
|
||||
- An agent struggles with a task repeatedly (branch hygiene, PR order)
|
||||
- A workflow has non-obvious ordering constraints (deployment coordination)
|
||||
- A task requires specific tool usage over raw commands (CI monitoring)
|
||||
- Multiple agents need shared context (dependency graph)
|
||||
|
||||
Do NOT create a skill for:
|
||||
- One-off tasks (use a spec instead)
|
||||
- Tasks already covered by AGENTS.md
|
||||
- Tasks that are obvious from the Makefile or README
|
||||
- Tasks that change frequently (skills should be stable)
|
||||
|
||||
## Skill Structure
|
||||
|
||||
Every skill MUST have:
|
||||
|
||||
```markdown
|
||||
# <skill-name>
|
||||
|
||||
One-line description of what the skill does.
|
||||
|
||||
## When to Invoke
|
||||
|
||||
2-4 bullet points describing when to use this skill.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
What must exist before using the skill (venv, .env, tools).
|
||||
|
||||
## <Core Content>
|
||||
|
||||
The actual guidance. Keep it actionable.
|
||||
|
||||
## Verification (if applicable)
|
||||
|
||||
How to verify the skill's guidance works.
|
||||
|
||||
## Common Mistakes (if applicable)
|
||||
|
||||
What agents get wrong without this skill.
|
||||
```
|
||||
|
||||
## Quality Standards
|
||||
|
||||
### Do
|
||||
- **Be specific.** Reference exact make targets, file paths, commands.
|
||||
- **Be concise.** Each section should be scannable in under 30 seconds.
|
||||
- **Be actionable.** Every paragraph should tell the agent what to DO.
|
||||
- **Use tables** for command reference, mappings, and comparisons.
|
||||
- **Use code blocks** for commands the agent should run.
|
||||
- **Link to other skills** when related (e.g., "See `dependency-graph` skill").
|
||||
|
||||
### Don't
|
||||
- **No preamble.** Don't start with "This skill helps agents..." — just state what it does.
|
||||
- **No filler.** Don't repeat information from AGENTS.md or other skills.
|
||||
- **No vague advice.** "Be careful with branches" is useless. "Run `git branch --show-current` before every commit" is useful.
|
||||
- **No AI slop.** Don't write "In this comprehensive guide, we will explore..." — just give the guidance.
|
||||
- **No redundant sections.** If "Common Mistakes" would repeat "When to Invoke", skip it.
|
||||
- **No marketing.** Don't describe the skill as "powerful" or "comprehensive".
|
||||
|
||||
## Scope Rules
|
||||
|
||||
- **One skill per concern.** Don't mix branch hygiene with CI monitoring.
|
||||
- **Project-specific, not generic.** Skills reference this repo's make targets, file paths, and conventions — not abstract advice.
|
||||
- **Shared skills must be identical across repos.** Use `SHARED_SKILLS` in the validator to enforce this.
|
||||
- **Per-repo skills must reflect that repo's reality.** Don't copy infra-specific targets to sso-bridge.
|
||||
|
||||
## Automated Validation
|
||||
|
||||
Every skill must pass the validator (`tests/test_skills.py`). The validator checks:
|
||||
|
||||
1. **Structure** — H1 title, "When to Invoke" section, "Prerequisites" section
|
||||
2. **Commands** — referenced `make <target>` commands exist in Makefile or devx.mak
|
||||
3. **Paths** — referenced file paths exist in the repo
|
||||
4. **Shared skills** — identical content across repos (SHA-256 comparison)
|
||||
5. **No drift** — no references to nonexistent commands or files
|
||||
|
||||
Run the validator:
|
||||
```bash
|
||||
python3 tests/test_skills.py --repo infra --repo sso-bridge
|
||||
```
|
||||
|
||||
## Effectiveness Evaluation
|
||||
|
||||
### Static Checks (automated, CI)
|
||||
|
||||
The validator runs in CI as part of `make pytest-cov`. A failing skill
|
||||
test blocks the PR. This catches:
|
||||
- Missing sections
|
||||
- Invalid commands
|
||||
- Broken file references
|
||||
- Cross-repo drift
|
||||
|
||||
### Runtime Metrics (manual, periodic)
|
||||
|
||||
Track these signals to evaluate skill effectiveness:
|
||||
- **Skill invocation frequency** — how often agents invoke the skill
|
||||
- **Success rate when invoked** — did the skill prevent the mistake it targets?
|
||||
- **Feedback issues** — agents create Gitea issues with `feedback` label when a skill is unclear or wrong
|
||||
- **Mistake recurrence** — if agents still make the mistake the skill targets, the skill needs improvement
|
||||
|
||||
### Retrospective Review
|
||||
|
||||
Periodically (monthly or after major incidents) review skills:
|
||||
1. List all skills and their last-modified dates
|
||||
2. Check for feedback issues tagged `skill-improvement`
|
||||
3. Verify referenced commands still exist (run validator)
|
||||
4. Remove skills that are no longer relevant
|
||||
5. Update skills where mistakes still recur
|
||||
6. Document lessons in this skill's "Common Mistakes" section
|
||||
|
||||
## Creating a New Skill — Checklist
|
||||
|
||||
- [ ] Identify the repeated struggle or non-obvious workflow
|
||||
- [ ] Check no existing skill covers it
|
||||
- [ ] Write the skill following the structure above
|
||||
- [ ] Run `python3 tests/test_skills.py` — must pass
|
||||
- [ ] Run `make pytest-cov` — must pass with 100% coverage
|
||||
- [ ] If shared across repos, copy identical content to each repo
|
||||
- [ ] Add the skill to `SHARED_SKILLS` in the validator if shared
|
||||
- [ ] Create PR, verify CI passes, merge
|
||||
@@ -1,139 +0,0 @@
|
||||
# Spec-Driven Development
|
||||
|
||||
## When to Invoke
|
||||
|
||||
Invoke this skill when starting any change — every PR requires a spec
|
||||
at `docs/specs/<TASK-ID>.md` that CI validates before merge.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- A Vikunja task ID (`DEVX-N`) — see `vikunja-tasks` skill
|
||||
|
||||
## Overview
|
||||
|
||||
Every change starts with a spec. No spec, no code. No code, no PR.
|
||||
|
||||
The spec is a markdown file at `docs/specs/<TASK-ID>.md` in the repo.
|
||||
It contains structured requirements (REQ-IDs) and acceptance criteria
|
||||
(AC checklist) that CI validates before merge.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Create Vikunja task** — `make create-task -- --title "Title" --description "..."`
|
||||
2. **Write spec** — Create `docs/specs/<TASK-ID>.md` (see template below)
|
||||
3. **Create branch** — `git checkout -b <PREFIX>-N-short-description`
|
||||
4. **Implement** — Write code with `# Implements: REQ-N` comments
|
||||
5. **Check ACs** — Tick all acceptance criteria checkboxes in the spec
|
||||
6. **Push and create PR** — `make push-with-pr`
|
||||
7. **CI validates** — Spec validation, PR size check, fast molecule, lint, tests
|
||||
8. **Auto-merge** — Add `ready-to-merge` label after review
|
||||
9. **Auto-deploy** — Post-merge deploys to staging (if nightly gate is green)
|
||||
|
||||
## Spec Template
|
||||
|
||||
```markdown
|
||||
# <TASK-ID>: <Title>
|
||||
|
||||
## Problem
|
||||
<What is broken or missing? Why does this change exist?>
|
||||
|
||||
## Approach
|
||||
<How will you solve it? What are the key design decisions?>
|
||||
|
||||
REQ-1: <First requirement description>
|
||||
REQ-2: <Second requirement description>
|
||||
REQ-3: <Third requirement description>
|
||||
|
||||
## Test Plan
|
||||
- <How will you verify each REQ is implemented correctly?>
|
||||
- <Include unit tests, molecule scenarios, integration tests>
|
||||
|
||||
## Deploy Plan
|
||||
- <How will this change be deployed?>
|
||||
- <What order do components need to deploy in?>
|
||||
- <Are there migrations or one-time operations?>
|
||||
|
||||
## Rollback Plan
|
||||
- <How do you revert if something goes wrong?>
|
||||
- <What data/state changes are irreversible?>
|
||||
|
||||
## Acceptance Criteria
|
||||
- [ ] REQ-1: <criterion that proves REQ-1 is done>
|
||||
- [ ] REQ-2: <criterion that proves REQ-2 is done>
|
||||
- [ ] REQ-3: <criterion that proves REQ-3 is done>
|
||||
```
|
||||
|
||||
## CI Validation
|
||||
|
||||
The `devx.ci.validate_spec` module checks:
|
||||
|
||||
1. **Spec file exists** at `docs/specs/<TASK-ID>.md` (TASK-ID from branch name)
|
||||
2. **Required sections present**: Problem, Approach, Test Plan, Deploy Plan, Rollback Plan, Acceptance Criteria
|
||||
3. **At least one REQ-ID** line (format: `REQ-N: <description>`)
|
||||
4. **All AC checkboxes checked** (`- [x]`, not `- [ ]`)
|
||||
|
||||
If any check fails, CI blocks the PR before expensive jobs run.
|
||||
|
||||
## PR Size Limits
|
||||
|
||||
CI enforces max 500 lines / 10 files changed (excluding CHANGELOG.md,
|
||||
README.md, badges, lock files). Oversized PRs are rejected. Split your
|
||||
work into smaller PRs.
|
||||
|
||||
## Code-to-Spec Linking
|
||||
|
||||
Each function, task, or template that implements a requirement should
|
||||
have a comment:
|
||||
|
||||
```python
|
||||
# Implements: REQ-1
|
||||
def install_sso_bridge():
|
||||
...
|
||||
```
|
||||
|
||||
```yaml
|
||||
# Implements: REQ-2
|
||||
- name: Clone infra repo
|
||||
git:
|
||||
...
|
||||
```
|
||||
|
||||
## Fast Molecule (Pre-merge)
|
||||
|
||||
CI runs molecule only for **changed roles** (detected via git diff),
|
||||
with converge + verify only, single platform. This gives quick feedback
|
||||
(~5-10 min) without the full molecule suite.
|
||||
|
||||
## Full Molecule (Nightly)
|
||||
|
||||
The complete molecule suite (all scenarios, all platforms) runs nightly
|
||||
at 02:00 CET on master. If it fails:
|
||||
- A Gitea issue is created with the `feedback` label
|
||||
- The `NIGHTLY_STATUS` repo variable is set to `failed:<run_id>`
|
||||
- All staging deploys are blocked until nightly passes again
|
||||
|
||||
## Auto-Deploy on Merge
|
||||
|
||||
Every merged PR auto-deploys to staging (if nightly gate is green).
|
||||
No manual trigger needed. The deploy runs the full pipeline:
|
||||
provision → deploy-observability → deploy-customer → configure-oidc.
|
||||
|
||||
For grm/sso-bridge: post-merge publishes the package, then auto-creates
|
||||
an infra PR to bump the pinned version. That infra PR auto-deploys when
|
||||
merged.
|
||||
|
||||
## Key Commands
|
||||
|
||||
```bash
|
||||
# Validate spec locally (before pushing)
|
||||
python -m devx.ci.validate_spec --branch <PREFIX>-N-description
|
||||
|
||||
# Check PR size locally
|
||||
python -m devx.ci.check_pr_size --base origin/master --head HEAD
|
||||
|
||||
# See which roles need fast molecule
|
||||
python -m devx.ci.fast_molecule --base origin/master --head HEAD
|
||||
|
||||
# Check nightly gate status
|
||||
python -m devx.ci.nightly_gate --repo oblachno/infra --action check
|
||||
```
|
||||
@@ -3,17 +3,6 @@
|
||||
Make targets for testing, debugging, and CI investigation. **Use these
|
||||
instead of raw `pytest`, `ruff`, or `actionlint` commands.**
|
||||
|
||||
## When to Invoke
|
||||
|
||||
Invoke this skill when running tests, investigating CI failures, or
|
||||
linting before push. Also invoke when asked to "run tests", "check
|
||||
coverage", or "debug a failure".
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `.venv` exists (run `make setup` if not)
|
||||
- Tools installed (run `make install-tools` for actionlint/act_runner)
|
||||
|
||||
## Why Make Targets
|
||||
|
||||
Make targets encapsulate the correct venv activation, PYTHONPATH, env
|
||||
@@ -56,13 +45,6 @@ This runs `lint-all` + `pytest-cov`. The pre-push git hook only
|
||||
validates the Vikunja task exists — it does NOT run tests. You must
|
||||
run `make pre-push` manually.
|
||||
|
||||
### Spec-Driven Workflow
|
||||
|
||||
Every PR requires a spec file at `docs/specs/<TASK-ID>.md`. See the
|
||||
`spec-driven-development` skill for the full workflow and template.
|
||||
CI validates the spec (via `devx.ci.validate_spec`) and checks PR size
|
||||
(via `devx.ci.check_pr_size`) before running expensive jobs.
|
||||
|
||||
## CI Failure Investigation
|
||||
|
||||
When investigating a CI failure:
|
||||
|
||||
@@ -1,74 +0,0 @@
|
||||
# vikunja-tasks
|
||||
|
||||
Vikunja task lifecycle beyond `create`: querying status, closing, and
|
||||
recovering when the tracker is unreachable.
|
||||
|
||||
## When to Invoke
|
||||
|
||||
- Creating, closing, or checking a Vikunja task
|
||||
- A spec workflow step needs the task ID or done state
|
||||
- `vikunja.oblachno.oblachno.fyi` fails to resolve / times out
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `.env` with `VIKUNJA_TOKEN`
|
||||
- Project ID comes from `[tool.devx]` in `pyproject.toml`
|
||||
(`DEVX_VIKUNJA_PROJECT_ID`)
|
||||
|
||||
## Create
|
||||
|
||||
`make create-task` does **not** forward arguments — call the module:
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m devx.tools.create_task \
|
||||
--title "Task title (no DEVX-N prefix)" \
|
||||
--description "<h2>Context</h2><p>...</p>"
|
||||
```
|
||||
|
||||
Prints `DEVX-N` + next steps. Title must not include the task-ID
|
||||
prefix (auto-merge prepends it; a manual prefix double-prefixes the
|
||||
PR title and fails validation).
|
||||
|
||||
## Query / Close
|
||||
|
||||
```bash
|
||||
# Task details (ID = numeric part of DEVX-N)
|
||||
curl -sf -H "Authorization: Bearer $VIKUNJA_TOKEN" \
|
||||
"https://vikunja.oblachno.oblachno.fyi/api/v1/tasks/<N>"
|
||||
|
||||
# Close: mark done
|
||||
curl -sf -X POST -H "Authorization: Bearer $VIKUNJA_TOKEN" \
|
||||
-H "Content-Type: application/json" -d '{"done":true}' \
|
||||
"https://vikunja.oblachno.oblachno.fyi/api/v1/tasks/<N>"
|
||||
```
|
||||
|
||||
Post-merge automation marks the task done when the PR squash-merges —
|
||||
manual close is only needed for abandoned/superseded tasks.
|
||||
|
||||
## Task-ID / Spec Collisions
|
||||
|
||||
Vikunja IDs can collide with historical spec files (an old task reused
|
||||
the number). Convention: preserve the old file as
|
||||
`docs/specs/<ID>-<topic>-historical.md`, then write the new spec at
|
||||
`docs/specs/<ID>.md`. Check `git log` on the existing spec before
|
||||
moving it.
|
||||
|
||||
## Tracker Unreachable
|
||||
|
||||
If the Vikunja host fails DNS/TLS:
|
||||
|
||||
1. Don't block the whole workflow — record the intended task title in
|
||||
the spec draft and retry `create_task` before branching.
|
||||
2. Never invent an ID — branch/PR titles must match a real task or
|
||||
`pre_push_check` / auto-merge validation fails.
|
||||
3. DNS failures observed so far were transient; retry after a few
|
||||
minutes before escalating.
|
||||
|
||||
## Common Mistakes
|
||||
|
||||
- `make create-task -- --title ...` — args are dropped; use the module
|
||||
call above (forwarding fix is S11 scope).
|
||||
- Including `DEVX-N:` in the task title — double prefix breaks
|
||||
auto-merge.
|
||||
- Closing a task whose PR is still open — auto-merge's post-merge
|
||||
step handles the close; manual close confuses the audit trail.
|
||||
@@ -77,9 +77,6 @@ jobs:
|
||||
CI_GITEA_API_TOKEN: ${{ secrets.CI_GITEA_API_TOKEN }}
|
||||
CI_GITEA_USERNAME: ${{ vars.CI_GITEA_USERNAME }}
|
||||
PYTHONPATH: src
|
||||
# Serialize blob uploads to avoid Gitea registry race condition
|
||||
# (BlobUploader.Append offset mismatch — see DEVX-162).
|
||||
DOCKER_MAX_CONCURRENT_UPLOADS: "1"
|
||||
run: |
|
||||
. .venv/bin/activate
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
|
||||
+26
-67
@@ -33,40 +33,21 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
# Implements: REQ-1 (DEVX-176) — docs-only changes skip the heavy
|
||||
# quality steps. Detection needs only git, so it runs before setup.
|
||||
- name: Detect docs-only change
|
||||
id: docs-only
|
||||
if: github.event_name == 'pull_request'
|
||||
run: |
|
||||
HEAD="${{ github.event.pull_request.head.sha || github.sha }}"
|
||||
DOCS_ONLY=true
|
||||
while IFS= read -r f; do
|
||||
case "$f" in
|
||||
docs/*|*.md|.devin/*) ;;
|
||||
*) DOCS_ONLY=false; break;;
|
||||
esac
|
||||
done < <(git diff --name-only "origin/master...$HEAD")
|
||||
echo "docs-only=$DOCS_ONLY" >> "$GITHUB_OUTPUT"
|
||||
echo "docs-only=$DOCS_ONLY"
|
||||
- name: Set up environment
|
||||
env:
|
||||
CI_GITEA_API_TOKEN: ${{ secrets.CI_GITEA_API_TOKEN }}
|
||||
run: make setup-image
|
||||
# --- quality steps ---
|
||||
- name: Lint all
|
||||
if: steps.docs-only.outputs.docs-only != 'true'
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
make lint-all
|
||||
- name: Unit tests with 100% coverage
|
||||
if: steps.docs-only.outputs.docs-only != 'true'
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
make pytest-cov
|
||||
- name: Check unit test speed
|
||||
if: steps.docs-only.outputs.docs-only != 'true'
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
python3 -m devx.tools.check_test_speed --max-seconds 15 --max-single-seconds 0.5
|
||||
@@ -79,12 +60,10 @@ jobs:
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
make devx-docs-check
|
||||
- name: Translation completeness check
|
||||
if: steps.docs-only.outputs.docs-only != 'true'
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
python3 -m devx.ci.check_translations
|
||||
- name: Dependency security scan
|
||||
if: steps.docs-only.outputs.docs-only != 'true'
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
# Install pip in venv if missing (needed by pip-audit)
|
||||
@@ -92,7 +71,6 @@ jobs:
|
||||
PIPAPI_PYTHON_LOCATION=$PWD/.venv/bin/python \
|
||||
pip-audit --desc --skip-editable 2>&1 || true
|
||||
- name: Workflow dry-run validation
|
||||
if: steps.docs-only.outputs.docs-only != 'true'
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
@@ -128,30 +106,14 @@ jobs:
|
||||
--pr-title "$PR_TITLE" \
|
||||
--repo "$REPOSITORY" \
|
||||
--pr-number "$PR_NUMBER"
|
||||
- name: Validate spec file
|
||||
- name: Run automated PR review
|
||||
if: github.event_name == 'pull_request'
|
||||
env:
|
||||
DEVX_TASK_PREFIX: DEVX
|
||||
PYTHONPATH: ${{ env.PYTHONPATH }}
|
||||
HEAD_REF: ${{ github.head_ref }}
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
python3 -m devx.ci.validate_spec \
|
||||
--branch "$HEAD_REF" \
|
||||
--github-output
|
||||
- name: Check PR size
|
||||
if: github.event_name == 'pull_request'
|
||||
env:
|
||||
PYTHONPATH: ${{ env.PYTHONPATH }}
|
||||
CI_GITEA_API_TOKEN: ${{ secrets.CI_GITEA_API_TOKEN }}
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
python3 -m devx.ci.check_pr_size \
|
||||
--base "origin/master" \
|
||||
--head "${{ github.event.pull_request.head.sha || github.sha }}" \
|
||||
--repo "${{ github.repository }}" \
|
||||
--pr-number "${{ github.event.number }}" \
|
||||
--github-output
|
||||
set -euo pipefail
|
||||
python3 -m devx.ci.pr_review \
|
||||
"${{ github.event.number }}" \
|
||||
"${{ github.repository }}"
|
||||
# --- release-dry-run step (conditional) ---
|
||||
- name: Release dry-run validation
|
||||
if: steps.detect.outputs.user-facing-changed == 'true'
|
||||
@@ -159,9 +121,19 @@ jobs:
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
python3 -m devx.ci.release --dry-run
|
||||
# Implements: REQ-2 (DEVX-176) — no failure-issue step in PR CI;
|
||||
# auto-created issues are for deploy-pipeline failures only
|
||||
# (post-merge keeps its notification).
|
||||
- name: Notify on failure
|
||||
if: failure()
|
||||
env:
|
||||
CI_GITEA_API_TOKEN: ${{ secrets.CI_GITEA_API_TOKEN }}
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
python3 -m devx.ci.notify_failure \
|
||||
--repo "${{ github.repository }}" \
|
||||
--run-id "${{ github.run_id }}" \
|
||||
--workflow "ci/validate" \
|
||||
--commit "${{ github.sha }}" \
|
||||
--auto-login
|
||||
|
||||
auto-merge:
|
||||
# Auto-merge runs after validate passes. It reads the task ID
|
||||
@@ -195,29 +167,16 @@ jobs:
|
||||
REVIEWER_GITEA_API_TOKEN: ${{ secrets.REVIEWER_GITEA_API_TOKEN }}
|
||||
CI_GITEA_API_TOKEN: ${{ secrets.CI_GITEA_API_TOKEN }}
|
||||
PR_NUMBER: ${{ github.event.number }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
REPOSITORY: ${{ github.repository }}
|
||||
run: |
|
||||
. .venv/bin/activate 2>/dev/null || true
|
||||
# Post APPROVE review via Gitea API to satisfy branch protection.
|
||||
# Try REVIEWER_GITEA_API_TOKEN first; fall back to CI_GITEA_API_TOKEN
|
||||
# (CI bot account) if the reviewer token is the same user as the PR
|
||||
# creator (Gitea rejects self-approvals).
|
||||
for TOKEN in "${REVIEWER_GITEA_API_TOKEN}" "${CI_GITEA_API_TOKEN}"; do
|
||||
[ -z "$TOKEN" ] && continue
|
||||
RESPONSE=$(curl -s -w "\n%{http_code}" -X POST \
|
||||
"${GITHUB_SERVER_URL}/api/v1/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews" \
|
||||
-H "Authorization: token ${TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"event":"APPROVED","body":"Auto-approved: all CI checks passed (validate job)."}')
|
||||
HTTP_CODE=$(echo "$RESPONSE" | tail -1)
|
||||
BODY=$(echo "$RESPONSE" | head -n -1)
|
||||
if [ "$HTTP_CODE" = "200" ] || [ "$HTTP_CODE" = "201" ]; then
|
||||
echo "Approval posted successfully (HTTP $HTTP_CODE)."
|
||||
break
|
||||
fi
|
||||
echo "::warning::Approval with token failed (HTTP $HTTP_CODE): ${BODY}"
|
||||
done
|
||||
python3 -m devx.ci.pr_review \
|
||||
"$PR_NUMBER" \
|
||||
"$REPOSITORY" \
|
||||
--event APPROVE \
|
||||
--checklist-confirmed \
|
||||
--checklist-categories 1,2,3,4,5,6,7,8,9,10,11,12,13 \
|
||||
--body "Auto-approved: all CI checks passed (validate job)."
|
||||
- name: Squash merge with task ID
|
||||
env:
|
||||
CI_GITEA_API_TOKEN: ${{ secrets.CI_GITEA_API_TOKEN }}
|
||||
|
||||
@@ -21,15 +21,10 @@ name: Post-merge
|
||||
on:
|
||||
push:
|
||||
branches: [master]
|
||||
# Implements: REQ-1 — manual recovery when a [skip ci] squash title
|
||||
# suppresses the push-triggered post-merge run.
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: post-merge-${{ github.ref }}
|
||||
# Implements: REQ-3 (DEVX-176) — queue instead of killing an in-flight
|
||||
# release/publish; a cancelled release can leave tag-without-publish.
|
||||
cancel-in-progress: false
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
PIP_BREAK_SYSTEM_PACKAGES: "1"
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Joseph Kato
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -1,38 +0,0 @@
|
||||
# The words behind a SMOG or Gunning fog score: each one of three or more
|
||||
# syllables. The scores say a paragraph reads hard; this says which words
|
||||
# made it so. Syllables are counted by vowel groups, with the silent e and
|
||||
# the -ed and -es endings taken off, which agrees with the scores' count on
|
||||
# nearly every word and errs on the long side for the rest.
|
||||
#
|
||||
# A suggestion, since a paragraph can carry a few. Turn it on while bringing
|
||||
# a score down, and off again after. See issue 8.
|
||||
extends: script
|
||||
message: "'%s' has three or more syllables. It counts toward the SMOG and Gunning fog scores."
|
||||
link: https://en.wikipedia.org/wiki/SMOG
|
||||
level: suggestion
|
||||
scope: paragraph
|
||||
script: |
|
||||
text := import("text")
|
||||
|
||||
matches := []
|
||||
for m in text.re_find(`[A-Za-z]+(?:'[A-Za-z]+)?`, scope, -1) {
|
||||
w := text.to_lower(m[0].text)
|
||||
if len(w) < 5 {
|
||||
continue
|
||||
}
|
||||
groups := text.re_find(`[aeiouy]+`, w, -1)
|
||||
n := groups == undefined ? 0 : len(groups)
|
||||
if text.has_suffix(w, "e") && !text.has_suffix(w, "le") && !text.has_suffix(w, "ee") {
|
||||
n--
|
||||
}
|
||||
if text.has_suffix(w, "ed") && !text.re_match(`[td]ed$`, w) {
|
||||
n--
|
||||
}
|
||||
if text.has_suffix(w, "es") && !text.re_match(`[sxz]es$|[cs]hes$`, w) {
|
||||
n--
|
||||
}
|
||||
if n >= 3 {
|
||||
matches = append(matches, {begin: m[0].begin, end: m[0].end})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,4 @@
|
||||
{
|
||||
"feed": "https://github.com/vale-cli/readability/releases.atom",
|
||||
"vale_version": ">=2.13.0",
|
||||
"license_files": [
|
||||
"LICENSE"
|
||||
]
|
||||
}
|
||||
"feed": "https://github.com/errata-ai/Readability/releases.atom",
|
||||
"vale_version": ">=2.13.0"
|
||||
}
|
||||
@@ -83,6 +83,7 @@ src/devx/
|
||||
│ ├── classify_changes.py # User-facing vs infrastructure change detection
|
||||
│ ├── detect_release_commit.py # Detect release commits on master
|
||||
│ ├── validate_commit_msg.py # Conventional commit validation
|
||||
│ ├── pr_review.py # Automated PR review + manual reviews (--event, --body, --checklist-confirmed)
|
||||
│ ├── post_merge.py # Vikunja task updates after merge
|
||||
│ ├── sync_wiki.py # Sync documentation to Gitea wiki
|
||||
│ ├── push_badges.py # Generate and push quality badges (--retries for retry on git push failures)
|
||||
@@ -144,6 +145,7 @@ src/devx/
|
||||
└── molecule/ # Optional molecule testing helpers (for Ansible projects)
|
||||
├── discover_runners.py # Dynamic Gitea runner discovery
|
||||
├── distribute_molecule.py # Distribute molecule scenarios across runners (LPT scheduling, --roles-root for multi-role)
|
||||
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast (--roles-root)
|
||||
├── molecule_all.py # Run all molecule scenarios locally
|
||||
├── molecule_changed.py # Detect which Ansible roles changed and output molecule scenarios
|
||||
├── start_docker.py # Ensure Docker daemon is running for molecule tests
|
||||
@@ -157,37 +159,8 @@ src/devx/
|
||||
- **PYTHONPATH: src** — Workflows set `PYTHONPATH: src` (NOT `.:src` since there are no scripts at repo root)
|
||||
- **Config via env vars** — `DEVX_*` environment variables with `.env` file fallback
|
||||
|
||||
|
||||
## Spec-Driven Development
|
||||
|
||||
Every change starts with a spec. No spec, no code.
|
||||
|
||||
**Workflow:**
|
||||
1. Create Vikunja task → get `<PREFIX>-N` task ID
|
||||
2. Write spec at `docs/specs/<TASK-ID>.md` (see template in `.devin/skills/spec-driven-development/SKILL.md`)
|
||||
3. Create branch, implement with `# Implements: REQ-N` comments
|
||||
4. Tick all acceptance criteria checkboxes in spec
|
||||
5. Push and create PR — CI validates spec before expensive jobs
|
||||
|
||||
**CI gates (pre-merge):**
|
||||
- `devx.ci.validate_spec` — checks spec exists, has required sections, REQ-IDs, all ACs checked
|
||||
- `devx.ci.check_pr_size` — max 500 lines / 10 files (excludes CHANGELOG, badges, locks)
|
||||
- `devx.ci.fast_molecule` — full `molecule test` for changed roles only (scoped, single platform)
|
||||
|
||||
**Nightly (infra only):**
|
||||
- Full molecule suite (all scenarios, all platforms) + staging deploy + integration tests
|
||||
- On failure: sets `NIGHTLY_STATUS=failed`, blocks staging deploys
|
||||
- Post-merge auto-deploy to staging checks this gate before deploying
|
||||
|
||||
**Post-merge:**
|
||||
- Infra: auto-deploys to staging (if nightly gate is green)
|
||||
- GRM/sso-bridge: auto-publishes package, auto-creates infra dependency PR to bump pinned version
|
||||
|
||||
**Skill:** `.devin/skills/spec-driven-development/SKILL.md` — full template and workflow details.
|
||||
|
||||
## PR Workflow (Mandatory)
|
||||
|
||||
|
||||
Every change to master goes through this workflow. No exceptions.
|
||||
|
||||
### Branch Protection (Required Gitea Settings)
|
||||
@@ -238,7 +211,7 @@ docs: update README
|
||||
### 6. Review the PR
|
||||
|
||||
**Automated review (CI `validate` job):** Every PR triggers an automated
|
||||
review via the `pr-review` skill (agent-invoked, not a CI step).
|
||||
review via `python -m devx.ci.pr_review` as a step in the `validate` job.
|
||||
This posts a review with
|
||||
`COMMENT` (no issues) or `REQUEST_CHANGES` (issues found):
|
||||
|
||||
|
||||
+68
-216
@@ -2,262 +2,114 @@
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
## [0.56.0] - 2026-09-30
|
||||
|
||||
### Features
|
||||
|
||||
- *(molecule)* Log lease owner and UTC expiry on emission
|
||||
|
||||
## [0.55.5] - 2026-09-28
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Strip [skip ci] tokens and prefer newest commit in squash titles
|
||||
|
||||
## [0.55.4] - 2026-09-23
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Fail-open molecule selection and honest fast-path contract
|
||||
|
||||
## [0.55.3] - 2026-09-21
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Honor repo bandit config in badge quality check
|
||||
|
||||
## [0.55.2] - 2026-09-19
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(ci)* Supply git identity for dep-PR manifest commits
|
||||
|
||||
## [0.55.1] - 2026-09-19
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(ci)* Resolve dep-PR container digest via registry v2 API
|
||||
|
||||
## [0.55.0] - 2026-09-19
|
||||
|
||||
### Features
|
||||
|
||||
- *(ci)* Retry dep-PR container verification until publish lands
|
||||
|
||||
## [0.54.0] - 2026-09-19
|
||||
|
||||
### Features
|
||||
|
||||
- *(ci)* Dep PRs carry target-repo task ID and generated spec
|
||||
|
||||
## [0.53.3] - 2026-09-19
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(ci)* Check out API-created dep branch via remote-tracking ref
|
||||
|
||||
## [0.53.2] - 2026-09-19
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(ci)* Create dep-PR branches via POST /branches
|
||||
|
||||
## [0.53.1] - 2026-09-19
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(ci)* Handle list-shaped ref response in create_dependency_pr
|
||||
|
||||
## [0.53.0] - 2026-09-19
|
||||
|
||||
### Features
|
||||
|
||||
- *(ci)* Manifest-aware dependency PRs and cleanup protection
|
||||
|
||||
## [0.52.0] - 2026-09-19
|
||||
|
||||
### Features
|
||||
|
||||
- *(ci)* Verify producer container artifact before opening dependency PR
|
||||
|
||||
## [0.51.12] - 2026-09-19
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Create_dependency_pr clones target repo instead of editing producer checkout
|
||||
|
||||
## [0.51.11] - 2026-09-19
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(ci)* Retry Vikunja lookups and surface self-approval merge failures
|
||||
|
||||
## [0.51.10] - 2026-09-17
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Read Gitea repo-variable data field; fail closed on unknown nightly status
|
||||
|
||||
## [0.51.9] - 2026-08-26
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Exclude docs/plans/* from PR size check
|
||||
|
||||
## [0.51.8] - 2026-08-26
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Accept deps: as valid conventional commit type
|
||||
|
||||
## [0.51.7] - 2026-08-26
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Increase HTTP 500 retry count to 5 with longer backoff and visible logging
|
||||
|
||||
## [0.51.6] - 2026-08-26
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Use stderr=STDOUT to capture all docker push output in one stream
|
||||
|
||||
## [0.51.5] - 2026-08-26
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Check stdout for HTTP 500 in _run_push (docker sends to stdout)
|
||||
|
||||
## [0.51.4] - 2026-08-26
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Serialize registry uploads and retry on HTTP 500
|
||||
|
||||
### Refactor
|
||||
|
||||
- Remove cross-repo contract tests from devx
|
||||
|
||||
## [0.51.3] - 2026-08-26
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Fall back to CI bot token for auto-merge approval
|
||||
|
||||
## [0.51.2] - 2026-08-26
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Push-first strategy in build_image to avoid losing latest tag
|
||||
|
||||
## [0.51.1] - 2026-08-25
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Delete existing manifest before push (Gitea #31964 workaround)
|
||||
|
||||
## [0.51.0] - 2026-08-25
|
||||
|
||||
### Features
|
||||
|
||||
- Add role defaults path to create_dependency_pr search
|
||||
|
||||
## [0.50.2] - 2026-08-24
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Update check_pr_size usage example with --repo and --pr-number args
|
||||
|
||||
## [0.50.1] - 2026-08-15
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Prefer rootless Docker socket over low-space inner DinD daemon
|
||||
- Check /run/host-docker.sock for host Docker daemon
|
||||
- Use /dev/shm/docker as data-root for inner dockerd
|
||||
- Kill existing dockerd before starting /dev/shm/docker daemon
|
||||
- Use /dev/shm/docker.sock socket for local dockerd
|
||||
- Add --iptables=false to local dockerd
|
||||
- Disable bridge and ip6tables for local dockerd
|
||||
- Use host Docker when root dir is inaccessible (free=0)
|
||||
- Prefer /run/host-docker.sock over inner dockerd
|
||||
- Start local dockerd instead of using low-space inner dockerd
|
||||
- Use container overlay for local dockerd data root
|
||||
- Kill inner dockerd with SIGKILL, use alt socket if alive
|
||||
- Kill dockerd by PID when pkill fails, use /dev/shm for alive daemon
|
||||
- Trust /var/run/docker.sock with free=0 when no inner dockerd exists
|
||||
|
||||
## [0.50.0] - 2026-08-09
|
||||
|
||||
### Features
|
||||
|
||||
- Add 5 standalone lint scripts from infra
|
||||
|
||||
## [0.49.0] - 2026-08-09
|
||||
## [0.50.0] - 2026-08-12
|
||||
|
||||
### Features
|
||||
|
||||
- Sync missing features from v0.49.x line to master
|
||||
## [0.49.5] - 2026-08-07
|
||||
|
||||
## [0.48.2] - 2026-08-09
|
||||
### Performance
|
||||
|
||||
- Skip dep resolution in setup-image with --no-deps
|
||||
## [0.49.4] - 2026-08-07
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Remove dead translation keys and add missing one
|
||||
|
||||
## [0.48.1] - 2026-08-08
|
||||
- Add container.credentials for private registry auth
|
||||
## [0.49.3] - 2026-08-07
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- *(setup)* Extract version from filename for mirror installs
|
||||
- Retry ansible-galaxy collection install on transient timeouts
|
||||
## [0.49.2] - 2026-08-07
|
||||
|
||||
## [0.48.0] - 2026-08-08
|
||||
### Bug Fixes
|
||||
|
||||
- Add fallback URL for tea download
|
||||
## [0.49.1] - 2026-08-07
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Add container images to build-images workflow
|
||||
## [0.49.0] - 2026-08-07
|
||||
|
||||
### Features
|
||||
|
||||
- *(setup)* Mirror Ansible collections from Gitea registry with auth
|
||||
- Add --include-roles and --exclude-roles to distribute_molecule
|
||||
## [0.48.0] - 2026-07-22
|
||||
|
||||
## [0.47.10] - 2026-08-05
|
||||
### Features
|
||||
|
||||
- Extract reusable components from infra and grm into devx
|
||||
|
||||
## [0.49.5] - 2026-08-07
|
||||
|
||||
### Performance
|
||||
|
||||
- Skip dep resolution in setup-image with --no-deps
|
||||
|
||||
## [0.49.4] - 2026-08-07
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Unique molecule container names per CI runner
|
||||
- Add container.credentials for private registry auth
|
||||
|
||||
## [0.47.9] - 2026-08-03
|
||||
## [0.49.3] - 2026-08-07
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Unique molecule container names per CI runner
|
||||
- Retry ansible-galaxy collection install on transient timeouts
|
||||
|
||||
## [0.47.8] - 2026-08-03
|
||||
## [0.49.2] - 2026-08-07
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Increase CI_SCALE_FACTOR default from 4 to 6
|
||||
- Add fallback URL for tea download
|
||||
|
||||
## [0.47.7] - 2026-08-03
|
||||
## [0.49.1] - 2026-08-07
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Scale check_test_speed limits on CI runners
|
||||
- Add container images to build-images workflow
|
||||
|
||||
## [0.47.6] - 2026-08-03
|
||||
## [0.49.0] - 2026-08-07
|
||||
|
||||
### Bug Fixes
|
||||
### Features
|
||||
|
||||
- Configure git auth in setup_image for git+https deps
|
||||
- Add --include-roles and --exclude-roles to distribute_molecule
|
||||
## [0.48.0] - 2026-07-22
|
||||
|
||||
## [0.47.5] - 2026-08-03
|
||||
### Features
|
||||
|
||||
### Bug Fixes
|
||||
- Extract reusable components from infra and grm into devx
|
||||
|
||||
- Push wiki to main branch instead of master
|
||||
## [0.48.0] - 2026-07-22
|
||||
|
||||
## [0.47.4] - 2026-08-03
|
||||
### Features
|
||||
|
||||
### Bug Fixes
|
||||
- Extract reusable components from infra and grm into devx
|
||||
|
||||
- Add User-Agent header to _download in install_tools
|
||||
## [Unreleased]
|
||||
|
||||
### Features
|
||||
|
||||
- Extract reusable components from infra and grm into devx:
|
||||
- `devx.utils.ui.say()` — unified click.echo + logging output
|
||||
- `devx.utils.api.APIClient` — base HTTP API client class with retry logic
|
||||
- `devx.utils.jinja` — Jinja2 environment helpers with Ansible-compatible filters
|
||||
- `devx.i18n.configure_i18n()` — configurable `lang_env_var` and `translations_path_env_var`
|
||||
- `devx.ci.cancel_superseded_runs` — cancel in-flight CI runs for the same PR branch
|
||||
- `devx.ci.check_workflow_artifact_deps` — verify artifact download jobs depend on upload jobs
|
||||
- `devx.ci.check_workflow_tofu_init` — verify tofu-state jobs have a tofu-init step
|
||||
- `devx.tools.check_docker_init` — check Docker Compose services with healthchecks have init: true
|
||||
- `devx.tools.check_ansible_set_fact_to_json` — check set_fact tasks don't misuse to_json
|
||||
- `devx.tools.check_alert_rules` — validate Prometheus alert rules with promtool
|
||||
- Add `jinja2` and `pyyaml` as core dependencies (previously in `deploy` extras only)
|
||||
- Register new CLI commands: `devx ci cancel-superseded-runs`, `devx ci check-workflow-artifact-deps`,
|
||||
`devx ci check-workflow-tofu-init`, `devx tools check-docker-init`,
|
||||
`devx tools check-ansible-set-fact-to-json`, `devx tools check-alert-rules`
|
||||
- Add Makefile targets for all new check tools
|
||||
|
||||
## [0.47.3] - 2026-07-17
|
||||
|
||||
|
||||
@@ -16,12 +16,12 @@ quality badges.
|
||||
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/LICENSE)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
|
||||
[](https://www.python.org/downloads/)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
|
||||
[](https://www.python.org/downloads/)
|
||||
|
||||
## Why devx?
|
||||
|
||||
@@ -87,7 +87,7 @@ extra index and list devx in your dependencies:
|
||||
```toml
|
||||
[project]
|
||||
dependencies = [
|
||||
"devx>=0.56.0",
|
||||
"devx>=0.50.0",
|
||||
]
|
||||
|
||||
[tool.pip]
|
||||
@@ -101,8 +101,8 @@ pip install -e .
|
||||
```
|
||||
|
||||
> **Note:** If your project requires a specific devx version, pin it in
|
||||
> `dependencies` (for example, `"devx==0.56.0"`) or use a version constraint
|
||||
> (for example, `"devx>=0.56.0,<0.57"`).
|
||||
> `dependencies` (for example, `"devx==0.50.0"`) or use a version constraint
|
||||
> (for example, `"devx>=0.50.0,<0.51"`).
|
||||
|
||||
### Optional extras
|
||||
|
||||
|
||||
+8
-8
@@ -12,12 +12,12 @@ project to be reusable across all oblachno-oss repositories.
|
||||
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/LICENSE)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
|
||||
[](https://www.python.org/downloads/)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
|
||||
[](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
|
||||
[](https://www.python.org/downloads/)
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -74,14 +74,14 @@ Add devx to your `pyproject.toml` dependencies and configure the registry:
|
||||
```toml
|
||||
[project]
|
||||
dependencies = [
|
||||
"devx>=0.56.0",
|
||||
"devx>=0.50.0",
|
||||
]
|
||||
|
||||
[tool.pip]
|
||||
extra-index-url = "https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple"
|
||||
```
|
||||
|
||||
Pin a specific version if needed: `"devx==0.56.0"` or `"devx>=0.56.0,<0.57"`.
|
||||
Pin a specific version if needed: `"devx==0.50.0"` or `"devx>=0.50.0,<0.51"`.
|
||||
|
||||
### Optional extras
|
||||
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
# DEVX-155: Replace pr_review with spec-driven CI gates and pr-review skill
|
||||
|
||||
## Problem
|
||||
The `devx.ci.pr_review` module was a monolithic automated PR review tool that
|
||||
ran in CI and posted COMMENT/REQUEST_CHANGES reviews. It duplicated logic now
|
||||
better handled by an agent-invoked skill, and it blocked the introduction of
|
||||
spec-driven development gates (validate_spec, check_pr_size) that should run
|
||||
before expensive CI jobs.
|
||||
|
||||
## Approach
|
||||
Remove `pr_review` and replace it with lightweight, focused CI gates plus a
|
||||
new `pr-review` skill for deep agent-invoked reviews.
|
||||
|
||||
REQ-1: Add `devx.ci.validate_spec` — validates spec file exists, has required sections, REQ-IDs, all ACs checked
|
||||
REQ-2: Add `devx.ci.check_pr_size` — enforces max 500 lines / 10 files (excludes CHANGELOG, badges, locks)
|
||||
REQ-3: Add `devx.ci.fast_molecule` — detects changed roles, outputs fast molecule commands (converge+verify, single platform)
|
||||
REQ-4: Add `devx.ci.nightly_gate` — checks/sets NIGHTLY_STATUS repo variable to block staging deploys on nightly failure
|
||||
REQ-5: Add `devx.ci.create_dependency_pr` — auto-creates infra PR to bump pinned package version after grm/sso-bridge release
|
||||
REQ-6: Remove `devx.ci.pr_review` module and `tests/unit/test_pr_review.py`
|
||||
REQ-7: Update CI workflows to replace pr_review steps with validate_spec + check_pr_size + curl-based APPROVE
|
||||
REQ-8: Add `spec-driven-development` and `pr-review` skills under `.devin/skills/`
|
||||
REQ-9: Update AGENTS.md and skill docs to document the new spec-driven workflow
|
||||
|
||||
## Test Plan
|
||||
- Unit tests for each new module (test_validate_spec, test_check_pr_size, test_fast_molecule, test_nightly_gate, test_create_dependency_pr, test_spec_driven_workflows)
|
||||
- Remove test_pr_review.py and pr_review references from test_cli.py (pr_review.py deleted from source)
|
||||
- Verify CI workflow YAML passes actionlint
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master via auto-merge workflow
|
||||
- devx post-merge publishes new version; downstream repos (grm, infra, sso-bridge) bump their devx pin
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit; downstream repos keep their current devx pin
|
||||
- pr_review.py can be restored from git history if needed
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: `devx.ci.validate_spec` module exists with `--branch` and `--github-output` options
|
||||
- [x] REQ-2: `devx.ci.check_pr_size` module exists with `--base`, `--head`, `--github-output` options
|
||||
- [x] REQ-3: `devx.ci.fast_molecule` module exists and outputs changed roles + commands
|
||||
- [x] REQ-4: `devx.ci.nightly_gate` module exists with `--action check/set-passed/set-failed`
|
||||
- [x] REQ-5: `devx.ci.create_dependency_pr` module exists with `--repo`, `--package`, `--new-version` options
|
||||
- [x] REQ-6: The pr_review CI module and its test file are deleted from source tree
|
||||
- [x] REQ-7: CI workflow uses validate_spec + check_pr_size + curl APPROVE instead of pr_review
|
||||
- [x] REQ-8: `.devin/skills/spec-driven-development/SKILL.md` and `.devin/skills/pr-review/SKILL.md` exist
|
||||
- [x] REQ-9: AGENTS.md documents spec-driven development workflow and pr-review skill
|
||||
@@ -1,34 +0,0 @@
|
||||
# DEVX-156: Fix commit message format and release new CI modules
|
||||
|
||||
## Problem
|
||||
The DEVX-155 merge commit on master has an invalid format
|
||||
('DEVX-155: Replace...' missing conventional commit type). This blocks
|
||||
the post-merge release workflow's `validate_commit_msg` step, preventing
|
||||
`validate_spec`, `check_pr_size`, `nightly_gate`, and `create_dependency_pr`
|
||||
from being published to the Gitea PyPI registry. All downstream repos
|
||||
(grm, infra, sso-bridge) are blocked — their CI fails with
|
||||
`No module named devx.ci.validate_spec`.
|
||||
|
||||
## Approach
|
||||
Add a trivial user-facing change (version doc comment) with a proper
|
||||
conventional commit format to trigger the post-merge release workflow.
|
||||
The release will publish the new CI modules that DEVX-155 introduced.
|
||||
|
||||
REQ-1: Add a user-facing change to src/devx/ to trigger release
|
||||
REQ-2: Ensure the commit message follows conventional format (type: description)
|
||||
|
||||
## Test Plan
|
||||
- Verify post-merge workflow runs successfully after merge
|
||||
- Verify a new release tag is created (v0.51.0 or similar)
|
||||
- Verify devx.ci.validate_spec is importable from the published package
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master via auto-merge workflow
|
||||
- Post-merge workflow auto-releases and publishes
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit if release fails
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: A user-facing change is added to src/devx/
|
||||
- [x] REQ-2: Commit message follows conventional format
|
||||
@@ -1,24 +0,0 @@
|
||||
# DEVX-157: Add role defaults path to create_dependency_pr search
|
||||
|
||||
## Problem
|
||||
`create_dependency_pr` only searches `pyproject.toml` and the infra images vars file for pinned versions. The sso-bridge role pins its version in its role defaults file via `sso_bridge_version`, which is not searched.
|
||||
|
||||
## Approach
|
||||
Add the sso-bridge role defaults path to the search paths.
|
||||
|
||||
REQ-1: Add ROLE_DEFAULTS_PATH constant pointing to the sso-bridge role defaults file
|
||||
REQ-2: Include ROLE_DEFAULTS_PATH in the search loop
|
||||
|
||||
## Test Plan
|
||||
- Verify existing tests pass
|
||||
- Verify find_pinned_version finds sso_bridge_version in the defaults file
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master, auto-release new devx version
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: ROLE_DEFAULTS_PATH constant added
|
||||
- [x] REQ-2: search loop includes ROLE_DEFAULTS_PATH
|
||||
@@ -1,38 +0,0 @@
|
||||
# DEVX-158: Fix build-images workflow: delete existing manifest before push
|
||||
|
||||
## Problem
|
||||
Gitea 1.27 has a known bug (#31964) where pushing a Docker image tag that
|
||||
already exists in the container registry fails with HTTP 500 "package
|
||||
version already exists." The build-images workflow has been failing for weeks because
|
||||
every push to `ci-base:latest`, `ci-quality:latest`, and `ci-full:latest`
|
||||
hits this error.
|
||||
|
||||
## Approach
|
||||
Add a `delete_remote_manifest` function that deletes the existing manifest
|
||||
via the Docker registry v2 API before pushing. This works around the Gitea
|
||||
bug by ensuring the tag doesn't exist when the push starts.
|
||||
|
||||
REQ-1: Add `delete_remote_manifest` function using Docker registry v2 API
|
||||
REQ-2: Call `delete_remote_manifest` before each `docker push` in `push_image`
|
||||
REQ-3: Pass registry credentials from `main` to `push_image`
|
||||
REQ-4: Handle errors gracefully — never block the push if delete fails
|
||||
REQ-5: 100% test coverage for new code
|
||||
|
||||
## Test Plan
|
||||
- Unit tests for `delete_remote_manifest` (success, 404, 500, network error)
|
||||
- Unit tests for `push_image` with and without credentials
|
||||
- Verify existing tests still pass
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master, auto-release new devx version
|
||||
- The build-images workflow will use the new code on the next run
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: `delete_remote_manifest` function added
|
||||
- [x] REQ-2: Called before each push in `push_image`
|
||||
- [x] REQ-3: Credentials passed from `main` to `push_image`
|
||||
- [x] REQ-4: Errors don't block the push (returns True on failure)
|
||||
- [x] REQ-5: 100% test coverage
|
||||
@@ -1,41 +0,0 @@
|
||||
# DEVX-159: Fix build_image push-first strategy to avoid losing latest tag
|
||||
|
||||
## Problem
|
||||
The `push_image` function in `build_image.py` deletes the existing
|
||||
manifest *before* pushing (Gitea #31964 workaround). When the push
|
||||
fails for other reasons (HTTP 500), the old tag is lost, breaking all
|
||||
CI jobs that use that image.
|
||||
|
||||
This caused `ci-base:latest` to disappear from the registry when
|
||||
build-images run #4104 failed with HTTP 500 on push, after already
|
||||
deleting the old `latest` manifest.
|
||||
|
||||
## Approach
|
||||
Switch to a push-first strategy:
|
||||
1. Try pushing directly
|
||||
2. Only if push fails with "already exists" (Gitea #31964), delete
|
||||
the old manifest and retry
|
||||
3. If push fails for any other reason, the old manifest is preserved
|
||||
|
||||
REQ-1: Push first, no pre-emptive delete
|
||||
REQ-2: Delete + retry only on "already exists" error
|
||||
REQ-3: Old manifest preserved on non-already-exists failures
|
||||
REQ-4: 100% test coverage of new logic
|
||||
|
||||
## Test Plan
|
||||
- Unit tests for all push paths (success, already-exists retry,
|
||||
non-already-exists failure, retry-also-fails)
|
||||
- Verify existing tests still pass
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master, build-images workflow uses new push logic on next
|
||||
image rebuild
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: Push first, no pre-emptive delete
|
||||
- [x] REQ-2: Delete + retry only on "already exists" error
|
||||
- [x] REQ-3: Old manifest preserved on non-already-exists failures
|
||||
- [x] REQ-4: 100% test coverage of new logic
|
||||
@@ -1,31 +0,0 @@
|
||||
# DEVX-160: Remove cross-repo contract tests from devx
|
||||
|
||||
## Problem
|
||||
devx unit tests (`test_spec_driven_workflows.py`) were validating workflow
|
||||
YAML and skill files in infra, grm, sso-bridge, and Mattermost OIDC repos.
|
||||
This is an architecture violation — devx must not be aware of other repos.
|
||||
Those repos consume devx; devx does not test them.
|
||||
|
||||
## Approach
|
||||
Rewrite `test_spec_driven_workflows.py` to only test devx's own workflows
|
||||
and skills. Remove all references to `_OBLACHNO_ROOT`, `_INFRA`, `_GRM`,
|
||||
`_SSO_BRIDGE`, and parametrized repo lists.
|
||||
|
||||
REQ-1: No references to other repos in devx tests
|
||||
REQ-2: All devx workflow/skill tests still pass
|
||||
REQ-3: 100% coverage maintained
|
||||
|
||||
## Test Plan
|
||||
- Run `pytest tests/unit/test_spec_driven_workflows.py` — all pass
|
||||
- Run full test suite with coverage — 100%
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: No references to other repos in devx tests
|
||||
- [x] REQ-2: All devx workflow/skill tests still pass
|
||||
- [x] REQ-3: 100% coverage maintained
|
||||
@@ -1,27 +0,0 @@
|
||||
# DEVX-161: Fix auto-merge self-approval: use CI bot token fallback
|
||||
|
||||
## Problem
|
||||
The auto-merge workflow posts an APPROVE review using
|
||||
`REVIEWER_GITEA_API_TOKEN`. When this token belongs to the same user
|
||||
who created the PR, Gitea rejects the self-approval, causing the merge
|
||||
to fail with HTTP 405 "Does not have enough approvals."
|
||||
|
||||
## Approach
|
||||
Try `REVIEWER_GITEA_API_TOKEN` first; if it fails (self-approval
|
||||
rejection), fall back to `CI_GITEA_API_TOKEN` (kireto — CI bot account).
|
||||
|
||||
REQ-1: Auto-merge posts approval with fallback to CI bot token
|
||||
REQ-2: Approval step reports which token succeeded
|
||||
|
||||
## Test Plan
|
||||
- Create a PR and observe auto-merge succeeds
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: Auto-merge posts approval with fallback to CI bot token
|
||||
- [x] REQ-2: Approval step reports which token succeeded
|
||||
@@ -1,41 +0,0 @@
|
||||
# DEVX-162: Fix registry push race condition: serialize uploads + retry on HTTP 500
|
||||
|
||||
## Problem
|
||||
The Gitea container registry (v1.27.2) has a known race condition in
|
||||
`BlobUploader.Append()` where concurrent blob uploads cause the file
|
||||
offset and DB model to get out of sync, producing HTTP 500 "offset
|
||||
mismatch between file and model" errors. This causes the build-images
|
||||
workflow to fail intermittently when pushing runner images.
|
||||
|
||||
The `package_blob_upload` table accumulates stale entries from failed
|
||||
uploads that worsen the problem over time.
|
||||
|
||||
## Approach
|
||||
Two fixes in devx (a third fix — scheduled cleanup — is tracked
|
||||
separately as OBL-INFRA-537):
|
||||
|
||||
1. Set `DOCKER_MAX_CONCURRENT_UPLOADS=1` in the build-images workflow
|
||||
to serialize blob uploads and avoid the race condition.
|
||||
|
||||
2. Add HTTP 500 retry logic to `push_image` in `build_image.py`.
|
||||
When a push fails with HTTP 500 (not "already exists"), retry up
|
||||
to 3 times with exponential backoff (5s, 10s, 20s).
|
||||
|
||||
REQ-1: Build-images workflow sets DOCKER_MAX_CONCURRENT_UPLOADS=1
|
||||
REQ-2: push_image retries on HTTP 500 with exponential backoff
|
||||
REQ-3: All existing tests pass with 100% coverage
|
||||
|
||||
## Test Plan
|
||||
- Unit tests for retry logic (mock subprocess)
|
||||
- Manual: trigger build-images workflow and verify push succeeds
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: Build-images workflow sets DOCKER_MAX_CONCURRENT_UPLOADS=1
|
||||
- [x] REQ-2: push_image retries on HTTP 500 with exponential backoff
|
||||
- [x] REQ-3: All existing tests pass with 100% coverage
|
||||
@@ -1,58 +0,0 @@
|
||||
# DEVX-162: Fix Gitea repo-variable read contract and fail closed on unknown nightly status
|
||||
|
||||
## Problem
|
||||
|
||||
`GiteaClient.get_repo_variable()` reads `body["value"]`, but the deployed
|
||||
Gitea returns the variable payload in the `data` field:
|
||||
|
||||
```json
|
||||
{"owner_id":0,"repo_id":1,"name":"NIGHTLY_STATUS","data":"passed:5842","description":""}
|
||||
```
|
||||
|
||||
Every read therefore returns `None`. `devx.ci.nightly_gate --action check`
|
||||
interprets `None` as "bootstrap — allow deploy," so a real `failed:<run>` status
|
||||
is invisible and the gate is permanently fail-open. Infra nightly run 5800 set
|
||||
`NIGHTLY_STATUS=passed:5800` while platform/customer integration tests were
|
||||
still failing — and even a correct `failed` value would have been ignored.
|
||||
|
||||
Additionally, an unrecognized non-empty status currently allows deploys
|
||||
(fail-open instead of fail-closed).
|
||||
|
||||
Verified against the live API: `POST`/`PUT` accept `{"value": ...}` and work;
|
||||
only the GET response uses `data`. The earlier `DEVX-162` spec (registry push
|
||||
race) is preserved as `DEVX-162-registry-push-race-historical.md`.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: `src/devx/api_clients.py` — `get_repo_variable` reads `data` first
|
||||
and falls back to `value` for older server/fixture compatibility. Write
|
||||
path unchanged (PUT/POST `{"value": ...}` verified live: 201/204).
|
||||
|
||||
REQ-2: `src/devx/ci/nightly_gate.py` — unknown non-empty status blocks the
|
||||
deploy (fail closed) instead of allowing it. Unset (bootstrap) still
|
||||
allows.
|
||||
|
||||
REQ-3: Tests cover the `data` field, the `value` fallback, and fail-closed
|
||||
unknown status.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- `pytest tests/unit/test_api_clients.py tests/unit/test_nightly_gate.py`
|
||||
- `make lint-all` (ruff, pyright, bandit, translations)
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
Merge via auto-merge; post-merge workflow publishes a new devx package to the
|
||||
Gitea PyPI registry and opens the infra dependency-bump PR automatically.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the commit; infra's pinned devx version keeps the previous behavior
|
||||
until the dependency PR lands.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] `get_repo_variable` returns the `data` field and falls back to `value`.
|
||||
- [x] Unknown nightly status exits non-zero and writes `nightly-gate-passed=false`.
|
||||
- [x] Unit tests cover `data`, `value` fallback and fail-closed unknown status.
|
||||
- [x] `make lint-all` and unit tests pass.
|
||||
@@ -1,33 +0,0 @@
|
||||
# DEVX-163: Fix _run_push to check stdout for HTTP 500
|
||||
|
||||
## Problem
|
||||
`_run_push` only checked `result.stderr` for HTTP 500, but docker push
|
||||
sends the "received unexpected HTTP status: 500 Internal Server Error"
|
||||
message to **stdout**, not stderr. This means the tenacity retry logic
|
||||
added in DEVX-162 never triggered — the push failed immediately without
|
||||
retrying.
|
||||
|
||||
## Approach
|
||||
Check both `result.stdout` and `result.stderr` for the "500" status code.
|
||||
Also update the "already exists" check in `push_image` to check both
|
||||
streams, since docker may send that message to stdout as well.
|
||||
|
||||
REQ-1: _run_push checks both stdout and stderr for HTTP 500
|
||||
REQ-2: push_image "already exists" check uses combined stdout+stderr
|
||||
REQ-3: All existing tests pass with 100% coverage
|
||||
|
||||
## Test Plan
|
||||
- Unit tests for stdout 500 detection
|
||||
- Unit tests for stderr 500 detection
|
||||
- Manual: trigger build-images workflow and verify retry works
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: _run_push checks both stdout and stderr for HTTP 500
|
||||
- [x] REQ-2: push_image "already exists" check uses combined stdout+stderr
|
||||
- [x] REQ-3: All existing tests pass with 100% coverage
|
||||
@@ -1,46 +0,0 @@
|
||||
# DEVX-163: Add vikunja-tasks skill and skill validation tests, fix create-task docs
|
||||
|
||||
## Problem
|
||||
|
||||
The OBL-INFRA-548 programme audit found devx lacks a Vikunja
|
||||
task-lifecycle skill and has no skill validation tests (infra and
|
||||
sso-bridge have them; grm gained them under GRM-171).
|
||||
`devx-workflow` documents `make create-task -- --title`, which fails
|
||||
because `devx-create-task` forwards no arguments.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Add `vikunja-tasks` skill: create via module call, query,
|
||||
close, spec-collision convention, unreachable-tracker handling.
|
||||
REQ-2: Add `tests/unit/test_skills_validation.py` covering
|
||||
structure, make-target, file-ref checks + existence tests for all
|
||||
skills.
|
||||
REQ-3: Fix broken `make create-task -- --title` documentation in
|
||||
`devx-workflow` skill; add missing When to Invoke / Prerequisites
|
||||
sections to older-format skills.
|
||||
|
||||
Preserve the colliding spec as
|
||||
[DEVX-163-run-push-stdout-historical](DEVX-163-run-push-stdout-historical.md).
|
||||
|
||||
## Test Plan
|
||||
|
||||
- `pytest tests/unit/test_skills_validation.py` passes (10 tests).
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
Documentation/skills only — auto-merge to master; no runtime deploy.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the squash-merge commit; skills are inert documentation.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: `vikunja-tasks` skill exists.
|
||||
- [x] REQ-2: `tests/unit/test_skills_validation.py` exists and passes.
|
||||
- [x] REQ-3: create-task docs corrected.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Fixing `devx-create-task` argument forwarding (S11 backlog: the
|
||||
devx.mak target takes no args; needs env-var or arg forwarding).
|
||||
@@ -1,34 +0,0 @@
|
||||
# DEVX-164: Increase HTTP 500 retry count and backoff for docker push
|
||||
|
||||
## Problem
|
||||
The HTTP 500 retry logic (DEVX-162, DEVX-163) works correctly — 3 retry
|
||||
attempts are made. But all 3 attempts fail because the Gitea registry's
|
||||
"offset mismatch" race condition needs more than ~15s to recover. The
|
||||
current backoff is 5s-20s with 3 attempts (total ~15s of waiting).
|
||||
|
||||
## Approach
|
||||
Increase retry count from 3 to 5 and backoff from 5-20s to 10-60s,
|
||||
giving the registry up to ~2 minutes to recover. Add visible logging
|
||||
between retry attempts so the CI logs show the retry happening.
|
||||
|
||||
REQ-1: Increase retry count from 3 to 5
|
||||
REQ-2: Increase backoff from 5-20s to 10-60s exponential
|
||||
REQ-3: Add visible logging between retry attempts (click.echo)
|
||||
REQ-4: All tests pass with 100% coverage
|
||||
|
||||
## Test Plan
|
||||
- Unit tests verify retry count and backoff parameters
|
||||
- Unit tests verify logging output on retry
|
||||
- Manual: trigger build-images workflow and verify retries visible in logs
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: Increase retry count from 3 to 5
|
||||
- [x] REQ-2: Increase backoff from 5-20s to 10-60s exponential
|
||||
- [x] REQ-3: Add visible logging between retry attempts (click.echo)
|
||||
- [x] REQ-4: All tests pass with 100% coverage
|
||||
@@ -1,57 +0,0 @@
|
||||
# DEVX-164: auto-merge resilience — self-approval and Vikunja outage handling
|
||||
|
||||
## Problem
|
||||
|
||||
Two defects hit auto-merge during S02 work:
|
||||
|
||||
1. `get_vikunja_task_title` crashes on transient Vikunja errors. The
|
||||
Vikunja API returned 404/502 during a restart window (run 6093);
|
||||
`list_project_tasks` treats 4xx as non-retryable `APIError`, so the
|
||||
job failed immediately instead of riding out a short outage.
|
||||
2. When a PR author and the workflow's reviewer token map to the same
|
||||
Gitea user, the auto-approve step is rejected ("approve your own
|
||||
pull is not allowed") and the merge fails `405: not enough
|
||||
approvals`. The generic merge error gives no hint that an external
|
||||
approval is the fix (hit on sso-bridge #18, #19, and infra #1647's
|
||||
approvals-only failure mode).
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Wrap the task-list pagination in `get_vikunja_task_title` with a
|
||||
bounded retry (tenacity, ~4 attempts, exponential backoff) covering
|
||||
`APIError` and `requests.RequestException`. A genuinely missing task
|
||||
still ends in the same "Could not find" ClickException.
|
||||
|
||||
REQ-2: On merge `HTTP 405`, fetch PR reviews; when zero `APPROVED`
|
||||
reviews exist, extend the error with the self-approval explanation and
|
||||
the remediation (approve via a non-author account).
|
||||
|
||||
REQ-3: Regression tests for both behaviors.
|
||||
|
||||
## Files Affected
|
||||
|
||||
- `src/devx/ci/auto_merge.py`
|
||||
- `tests/unit/test_auto_merge.py`
|
||||
|
||||
## Test Plan
|
||||
|
||||
- New tests: retry-then-success on transient APIError; retry-exhaustion
|
||||
still raises; missing task still raises; 405 error includes
|
||||
approvals diagnostic.
|
||||
- `make pytest-cov`, `make lint-all`.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
- Merge → next release publishes the package; consuming repos pick it
|
||||
up on their next CI run (devx is pinned per-repo, bump via the usual
|
||||
dependency PR flow).
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
- Revert; previous behavior returns.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: Vikunja task-list retries transient API failures
|
||||
- [x] REQ-2: 405 merge error reports approval state + self-approval hint
|
||||
- [x] REQ-3: Regression tests added and passing
|
||||
@@ -1,31 +0,0 @@
|
||||
# DEVX-165: Accept deps: as valid conventional commit type
|
||||
|
||||
## Problem
|
||||
The commit validator rejects `deps:` as a conventional commit type, causing
|
||||
post-merge CI failures on grm and sso-bridge repos where automated dependency
|
||||
bump PRs use `deps: bump devx...` as the commit message.
|
||||
|
||||
## Approach
|
||||
REQ-1: Add `deps` to `CONVENTIONAL_RE` in `src/devx/config.py`
|
||||
REQ-2: Update the allowed types list in the error message in
|
||||
`src/devx/ci/validate_commit_msg.py`
|
||||
REQ-3: Add test coverage for `deps:` type in `tests/unit/test_config.py`
|
||||
and `tests/unit/test_validate_commit_msg.py`
|
||||
|
||||
## Test Plan
|
||||
- `make pytest-cov` passes with 100% coverage
|
||||
- `make lint-all` passes
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master → post-merge auto-publishes new devx version
|
||||
- grm and sso-bridge bump devx version to pick up the fix
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: Add `deps` to `CONVENTIONAL_RE` in `src/devx/config.py`
|
||||
- [x] REQ-2: Update the allowed types list in the error message in
|
||||
`src/devx/ci/validate_commit_msg.py`
|
||||
- [x] REQ-3: Add test coverage for `deps:` type in `tests/unit/test_config.py`
|
||||
and `tests/unit/test_validate_commit_msg.py`
|
||||
@@ -1,63 +0,0 @@
|
||||
# DEVX-165: Manifest-aware dependency PRs and registry cleanup protection
|
||||
|
||||
## Problem
|
||||
|
||||
S03 (OBL-INFRA-548 REQ-3) requires an immutable delivery contract: infra
|
||||
pins the sso-bridge release as {version, git ref, image tag, OCI digest}
|
||||
in a JSON manifest. Two gaps in devx block that:
|
||||
|
||||
1. `create_dependency_pr` only regex-bumps a version string in
|
||||
pyproject/ansible vars — it cannot update a structured manifest with
|
||||
the resolved image digest, and it does not verify the producer
|
||||
artifact exists before opening the PR.
|
||||
2. `clean_images` deletes all but the newest N tags — a tag/digest that
|
||||
infra still pins gets deleted once newer releases land, breaking
|
||||
deploys.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: `create_dependency_pr` gains `--manifest <path>` +
|
||||
`--verify-container <owner/name>` + `--container-tag` +
|
||||
`--source-ref`: before opening the PR it resolves the container tag's
|
||||
OCI digest via the packages API (`manifest.json` blob sha256), then
|
||||
updates manifest fields `{version, git_ref, image_tag, image_digest,
|
||||
source_run_id, updated_at}` in the PR branch instead of a regex bump.
|
||||
`--container-tag` decouples the image tag from the release version
|
||||
(sso-bridge images tag `__init__.py.__version__`, not the git tag).
|
||||
|
||||
REQ-1b: `create_dependency_pr` clones the *target* repo into a tempdir
|
||||
and performs all file lookups and git operations inside it. Previously
|
||||
it operated on CWD — the producer repo's own checkout — so file updates
|
||||
silently targeted the wrong repo and the whole dep-PR path no-oped
|
||||
behind `|| echo warning`.
|
||||
|
||||
REQ-2: `clean_images` gains `--protect` (repeatable): named versions are
|
||||
never deleted regardless of `--keep` trimming.
|
||||
|
||||
REQ-3: Regression tests for manifest update, digest resolution,
|
||||
verify-then-PR ordering, and protect filtering.
|
||||
|
||||
## Files Affected
|
||||
|
||||
- `src/devx/ci/create_dependency_pr.py`
|
||||
- `src/devx/tools/clean_images.py`
|
||||
- `tests/unit/test_create_dependency_pr.py`
|
||||
- `tests/unit/test_clean_images.py`
|
||||
|
||||
## Test Plan
|
||||
|
||||
- New unit tests per REQ; `make pytest-cov`, `make lint-all`.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
- Merge → devx release → consumer repos pick up via dependency PRs.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
- Revert; regex version bump and unprotected cleanup return.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: Manifest update + pre-PR OCI digest verification
|
||||
- [x] REQ-2: `--protect` exempts versions from cleanup
|
||||
- [x] REQ-3: Regression tests added and passing
|
||||
@@ -1,47 +0,0 @@
|
||||
# DEVX-166: create_dependency_pr must clone the target repo
|
||||
|
||||
## Problem
|
||||
|
||||
`create_dependency_pr` resolves the pinned-version file and runs all
|
||||
git operations in the current working directory. Producer post-merge
|
||||
workflows (grm, sso-bridge) invoke it from the *producer* checkout, so
|
||||
it searches/modifies the wrong repository: `find_pinned_version` reads
|
||||
files that do not exist there, and the git fetch/checkout/commit/push
|
||||
sequence runs in the producer clone. The failure is silent — producer
|
||||
workflows append `|| echo warning`, so a no-op looks like success.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Clone the target repo (`--repo`) into a temporary directory with
|
||||
an authenticated `http.extraHeader`, then run every file lookup and git
|
||||
operation (fetch, checkout, add, commit, push) inside that clone. The
|
||||
push uses the same auth header config.
|
||||
|
||||
REQ-2: Tests mock `subprocess.run` so no real clone happens in the unit
|
||||
suite (test-isolation gate).
|
||||
|
||||
## Files Affected
|
||||
|
||||
- `src/devx/ci/create_dependency_pr.py`
|
||||
- `tests/unit/test_create_dependency_pr.py`
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Existing CLI tests keep passing with the subprocess mock in place.
|
||||
- Verify the clone command targets the `--repo` URL and that git ops
|
||||
run with `cwd=<clone>` (asserted via the mock's call list).
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
Merge via auto-merge after green CI. The fix takes effect the next time
|
||||
a producer post-merge workflow invokes `create_dependency_pr`.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the squash-merge commit on master; the previous (broken) CWD
|
||||
behavior returns, which is strictly worse — no state is created.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: target repo cloned to tempdir; all file/git ops run in the clone
|
||||
- [x] REQ-2: unit tests never spawn a real git subprocess
|
||||
@@ -1,64 +0,0 @@
|
||||
# DEVX-167: Add dependency-graph, deployment-coordination, and skill-creation skills
|
||||
|
||||
## Problem
|
||||
Agents working across the oblachno ecosystem lack shared, persistent
|
||||
context for three recurring pain points:
|
||||
|
||||
1. **Cross-repo dependency ordering** — agents frequently merge
|
||||
downstream PRs before the upstream publish job completes, or forget
|
||||
to bump infra. There is no single reference for which repo produces
|
||||
what and in what order changes must propagate.
|
||||
2. **devx release coordination** — devx is the base package pinned by
|
||||
grm, sso-bridge, and infra. Agents repeatedly merge devx PRs and
|
||||
immediately merge downstream bumps without waiting for the PyPI
|
||||
publish job, or bump only one consumer when a change affects all
|
||||
three.
|
||||
3. **Skill quality drift** — skills are created ad hoc with inconsistent
|
||||
structure, vague advice, and no automated validation reference. New
|
||||
skills miss required sections, reference nonexistent make targets,
|
||||
and drift across repos.
|
||||
|
||||
## Approach
|
||||
Add three SKILL.md files under `.devin/skills/`:
|
||||
|
||||
REQ-1: `dependency-graph` — shared skill mapping the oblachno ecosystem
|
||||
(repos, what each produces, consumers, release triggers, correct
|
||||
cross-repo change order, state verification checklist)
|
||||
|
||||
REQ-2: `deployment-coordination` — devx-specific skill covering the
|
||||
devx release flow, downstream consumers, manual bump procedure, and
|
||||
common mistakes when coordinating a devx change
|
||||
|
||||
REQ-3: `skill-creation` — shared skill defining skill structure,
|
||||
quality standards, scope rules, automated validation reference, and a
|
||||
creation checklist
|
||||
|
||||
## Files Affected
|
||||
- `.devin/skills/dependency-graph/SKILL.md` (new)
|
||||
- `.devin/skills/deployment-coordination/SKILL.md` (new)
|
||||
- `.devin/skills/skill-creation/SKILL.md` (new)
|
||||
- `docs/specs/DEVX-167.md` (new)
|
||||
|
||||
## Test Plan
|
||||
- Verify all three SKILL.md files follow the required structure (H1
|
||||
title, When to Invoke, Prerequisites sections)
|
||||
- Verify referenced make targets and file paths exist
|
||||
- Run `make pytest-cov` — skill validation tests must pass
|
||||
|
||||
## Deploy Plan
|
||||
- Merge to master; skills are consumed by agents immediately on next
|
||||
invocation — no build or deploy step required
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge commit; remove the three skill directories
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: dependency-graph skill exists with ecosystem map, repo
|
||||
table, dependency chain, cross-repo change order, and state
|
||||
verification checklist
|
||||
- [x] REQ-2: deployment-coordination skill exists with devx release
|
||||
flow, downstream consumer table, coordination steps, and common
|
||||
mistakes
|
||||
- [x] REQ-3: skill-creation skill exists with structure template,
|
||||
quality standards, scope rules, validation reference, and
|
||||
creation checklist
|
||||
@@ -1,57 +0,0 @@
|
||||
# DEVX-167: S03 artifact integrity — verify producer artifact + cleanup protection
|
||||
|
||||
## Problem
|
||||
|
||||
S03 (OBL-INFRA-548 REQ-3) requires that dependency PRs only open after
|
||||
the producer artifact exists and is content-addressable, and that
|
||||
registry cleanup never deletes a version pinned by a release manifest.
|
||||
Two gaps:
|
||||
|
||||
1. `create_dependency_pr` opens a bump PR unconditionally — if the
|
||||
producer's publish job lagged or failed, the consumer pins a
|
||||
nonexistent artifact.
|
||||
2. The sso-bridge image tag is derived from `__init__.py.__version__`,
|
||||
which does not always equal the release git tag, so the tag to
|
||||
verify must be decoupled from `--new-version`.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: `create_dependency_pr` gains `--verify-container <owner/name>`
|
||||
and `--container-tag <tag>`: before any branch/PR work it resolves the
|
||||
OCI digest of the image tag via the Gitea packages API (`manifest.json`
|
||||
blob sha256) and refuses the PR when the artifact is missing or
|
||||
unreadable. `--container-tag` decouples the image tag from the release
|
||||
version (sso-bridge tags images from `__init__.py.__version__`, not the
|
||||
git tag).
|
||||
|
||||
REQ-2: Regression tests cover digest resolution, verification-failure
|
||||
aborts, invalid container format, and the `--container-tag` override.
|
||||
|
||||
## Files Affected
|
||||
|
||||
- `src/devx/ci/create_dependency_pr.py`
|
||||
- `src/devx/translations.json`
|
||||
- `tests/unit/test_create_dependency_pr.py`
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Unit tests for `resolve_container_digest` (digest from manifest blob,
|
||||
missing tag, missing blob, connection error).
|
||||
- CLI tests: verify runs before version lookup, digest resolution,
|
||||
invalid format rejection, container-tag override.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
Merge via auto-merge after green CI. Producer post-merge workflows adopt
|
||||
the new flags in their own PRs (sso-bridge SSO-22 already passes them).
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the squash-merge commit; the new options disappear and callers
|
||||
without them behave exactly as before.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: pre-PR OCI digest verification with --verify-container/--container-tag
|
||||
- [x] REQ-2: regression tests for all new behavior
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
# DEVX-168: Fix create_dependency_pr master-ref parsing
|
||||
|
||||
## Problem
|
||||
|
||||
`devx.ci.create_dependency_pr` crashes with
|
||||
`AttributeError: 'list' object has no attribute 'get'` when creating the
|
||||
dependency branch. Gitea's `GET /repos/{o}/{r}/git/refs/heads/master`
|
||||
returns a JSON **array** of matching refs, not a single object. Observed
|
||||
in production when sso-bridge v0.4.1's post-merge dep-PR step ran.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Normalize the ref response — if it is a list, select the entry
|
||||
whose `ref` field equals `refs/heads/master` or fall back to the first
|
||||
entry; proceed to extract `object.sha` as before. An empty or absent
|
||||
SHA still fails closed.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Unit test: list-shaped response resolves SHA and creates the PR.
|
||||
- Unit test: empty list fails with the master-SHA error message.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
devx releases as a version tag; sso-bridge/infra pin bumps pick it up via
|
||||
their normal dep-PR flow.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the commit; dep-PR creation stays broken on Gitea (status quo).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: list-shaped ref response is handled; SHA extraction works;
|
||||
empty list still errors. Covered by unit tests at 100% coverage.
|
||||
@@ -1,33 +0,0 @@
|
||||
# DEVX-169: create_dependency_pr uses branches API
|
||||
|
||||
## Problem
|
||||
|
||||
After DEVX-168, dep-PR creation fails at `POST /git/refs` with
|
||||
`HTTP 405 Method Not Allowed` — this Gitea version does not implement the
|
||||
createRef endpoint. Observed creating the sso_bridge 0.4.1 infra
|
||||
dependency PR.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Create the dependency branch via `POST /branches` with
|
||||
`new_branch_name`/`old_branch_name` (from master). An already-exists
|
||||
error is tolerated; other API errors fail closed.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Branch creation calls `POST /branches` with the expected payload.
|
||||
- 422 already-exists is tolerated and the PR is still created.
|
||||
- Other API errors abort with a branch-creation failure.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
devx release tag; producers pick it up via pin bumps.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert; dep-PR creation stays broken (status quo).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: branch creation uses the branches API; already-exists
|
||||
tolerated; failures surface. Covered by unit tests at 100% coverage.
|
||||
@@ -1,33 +0,0 @@
|
||||
# DEVX-170: create_dependency_pr checks out the API-created branch
|
||||
|
||||
## Problem
|
||||
|
||||
After DEVX-169 the dependency branch is created via `POST /branches`, but
|
||||
`git fetch origin <branch>` only populates `FETCH_HEAD`. The follow-up
|
||||
`git checkout <branch>` fails silently (`check=False`), commits land on
|
||||
the wrong ref, and `git push origin <branch>` fails with
|
||||
`src refspec does not match any`. Observed creating the sso_bridge 0.4.1
|
||||
infra dependency PR.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Fetch the API-created branch into `refs/remotes/origin/<branch>`
|
||||
and force-create the local branch with `git checkout -B <branch>
|
||||
origin/<branch>`, both with `check=True` so failures surface.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Unit test asserts the fetch refspec and `checkout -B` invocations.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
devx release tag; producers pick it up via pin bumps.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert; dep-PR creation stays broken (status quo).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: the clone checks out the API-created branch so commit and
|
||||
push target `deps/<pkg>-<version>`; covered by unit tests.
|
||||
@@ -1,36 +0,0 @@
|
||||
# DEVX-171: create_dependency_pr uses target-project task ID + generated spec
|
||||
|
||||
## Problem
|
||||
|
||||
Dep PRs fail target-repo validation: infra requires `OBL-INFRA-N` in the
|
||||
branch name and `docs/specs/<TASK>.md` in the diff. The tool created the
|
||||
Vikunja task in the *producer's* project and named the branch
|
||||
`deps/<pkg>-<ver>` with no task ID, and wrote no spec.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Create the tracking task before branch creation, in the target
|
||||
repo's Vikunja project via `--task-project-id` (default:
|
||||
DEVX_VIKUNJA_PROJECT_ID). Embed the returned identifier in the branch
|
||||
name (`deps/<TASK>-<pkg>-<ver>`) and PR title, and commit a generated
|
||||
`docs/specs/<TASK>.md` with the required sections alongside the pin bump.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Branch name and PR title carry the target task ID; spec file added.
|
||||
- `create_vikunja_task(project_id=N)` passes N to VikunjaClient.
|
||||
- Existing no-task behavior preserved when Vikunja is unreachable.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
devx release tag; producer workflows pass `--task-project-id` for infra
|
||||
(project 3).
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert; dep PRs keep failing target validation (status quo).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: `--task-project-id` supported; branch/spec/PR title embed
|
||||
the target task ID; covered by unit tests at 100% coverage.
|
||||
@@ -1,32 +0,0 @@
|
||||
# DEVX-172: dep-PR retries container verification until publish lands
|
||||
|
||||
## Problem
|
||||
|
||||
Post-merge dep-PR runs concurrently with the producer's image-build
|
||||
workflow. `--verify-container` hits HTTP 404 before the push lands and
|
||||
skips PR creation — observed for sso-bridge v0.4.1 and v0.4.3.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: `resolve_container_digest(..., timeout_s)` retries a 404 lookup
|
||||
every 15s until the deadline.
|
||||
REQ-2: `--verify-timeout` CLI option (default 600s, 0 disables) wires the
|
||||
retry into the dep-PR step.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Unit test: 404-then-200 resolves the digest without raising.
|
||||
- Existing no-retry path (timeout_s=0) still fails immediately.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
devx release tag; producers inherit the 600s default on next pin bump.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert; dep-PR verification fails fast on 404 again (status quo).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: 404 responses retry until `timeout_s` deadline.
|
||||
- [x] REQ-2: `--verify-timeout` option exposed, default 600.
|
||||
@@ -1,49 +0,0 @@
|
||||
# DEVX-173: Resolve container digest via registry v2 API
|
||||
|
||||
## Problem
|
||||
|
||||
`resolve_container_digest` returns the sha256 of the `manifest.json`
|
||||
blob listed by the Gitea packages API (`/packages/<owner>/container/
|
||||
<name>/<tag>/files`). That blob digest is NOT the OCI manifest digest —
|
||||
`docker pull repo@sha256:<blob>` fails with "not found." Production
|
||||
deploy run 6251 died pulling `sso-bridge@sha256:5ca9...` which the dep
|
||||
PR had recorded in `deploy/sso-bridge-release.json`. The registry serves
|
||||
0.4.1 as `sha256:c0212ed1...` — different digest entirely.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Keep the packages-API call as the existence check (it has the
|
||||
404-retry semantics needed for the publish race) but resolve the
|
||||
pullable digest via the registry v2 API: `GET /v2/token` with
|
||||
`scope=repository:<owner>/<name>:pull` (basic-auth with the Gitea
|
||||
token), then `GET /v2/<owner>/<name>/manifests/<tag>` with OCI/Docker
|
||||
manifest Accept headers and read `Docker-Content-Digest`. Registry base
|
||||
URL is derived from `GITEA_API_URL` (strip `/api/v1`).
|
||||
|
||||
REQ-2: If the v2 digest lookup fails (non-2xx, missing header), fail
|
||||
closed with a ClickException — never record a blob sha256 as a pullable
|
||||
digest.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Mocked v2 token + manifest endpoints return digest; recorded value is
|
||||
the `Docker-Content-Digest` header.
|
||||
- 404 retry semantics on the packages-API existence check unchanged.
|
||||
- Missing digest header / non-2xx manifest response → ClickException.
|
||||
- Unit tests cover token request scope and Accept headers.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
devx release tag; the sso-bridge dep-PR pin bump follows in its own PR.
|
||||
The wrong digest already recorded in infra's manifest is corrected by
|
||||
re-running the dep-PR with the fixed version.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert; dep-PR records the (unpullable) blob digest again — deploys must
|
||||
then use tag pulls until a corrected manifest lands.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: Digest resolved from `Docker-Content-Digest` via v2 API.
|
||||
- [x] REQ-2: Lookup failures fail closed; no blob-sha256 fallback.
|
||||
@@ -1,31 +0,0 @@
|
||||
# DEVX-174: Set git identity in dep-PR target clone
|
||||
|
||||
## Problem
|
||||
|
||||
`create_dependency_pr` commits the manifest bump in a fresh clone of the
|
||||
target repo. The clone has no `user.name`/`user.email`, so `git commit`
|
||||
exits 128 ("Please tell me who you are") — observed in sso-bridge
|
||||
post-merge run 6284 after the digest and manifest bump both succeeded.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Pass a CI bot identity inline (`git -c user.name=... -c
|
||||
user.email=... commit`) so the commit works in any environment without
|
||||
mutating global git config.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Unit test asserting the commit invocation carries `-c user.name` /
|
||||
`-c user.email` arguments.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
devx release; sso-bridge picks it up via its devx pin on next bump.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert; dep-PR commit fails again in containers without git identity.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: dep-PR commit supplies explicit identity flags.
|
||||
@@ -1,40 +0,0 @@
|
||||
# DEVX-175: Badge generation must honor repo bandit configuration
|
||||
|
||||
## Problem
|
||||
|
||||
`collect_quality()` in `generate_badges.py` runs `bandit -r src/` with no
|
||||
config file. Repos that declare a `[tool.bandit]` section in
|
||||
`pyproject.toml` (skip lists aligned with infrastructure patterns — for
|
||||
example sso-bridge skips B404/B603/B501) lint clean via `make lint-all`, but the
|
||||
quality badge still reports `bandit: FAIL` — a false-FAIL badge that
|
||||
contradicts the actual lint gate.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: When `pyproject.toml` exists in the target repo and contains a
|
||||
`[tool.bandit]` section, pass `-c pyproject.toml` to the bandit invocation
|
||||
in `collect_quality()`.
|
||||
|
||||
REQ-2: Repos without `[tool.bandit]` keep the current invocation unchanged.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Unit test: repo fixture with `[tool.bandit]` → command includes
|
||||
`-c pyproject.toml`; without the section → plain invocation.
|
||||
- Manual: `python -m devx.tools.generate_badges` in sso-bridge reports
|
||||
`bandit: pass`.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
devx release → repos pick it up on next devx pin bump; no infra changes.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the commit; badges revert to unconfigured bandit runs.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1 implemented with `# Implements: REQ-1` comment
|
||||
- [x] REQ-2 preserved (no config → unchanged command)
|
||||
- [x] Unit tests cover both branches
|
||||
- [x] `make lint-all` + `make pytest-cov` pass in devx
|
||||
@@ -1,43 +0,0 @@
|
||||
# DEVX-176: CI hygiene — docs fast-path, failure-notify scoping, post-merge cancel
|
||||
|
||||
## Problem
|
||||
|
||||
devx CI has the same inefficiencies fixed in infra (OBL-INFRA-613/615/616):
|
||||
docs-only PRs run the full quality suite (~10 min), CI failures auto-create
|
||||
issues (noise — the user decided issues are for deploy failures only), and
|
||||
post-merge `cancel-in-progress: true` can kill a release mid-publish.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Docs-only PRs skip heavy validate steps (lint-all, unit tests,
|
||||
test-speed, translation check, security scan, workflow dry-run). Docs gate,
|
||||
spec validation, PR size, and auto-merge preconditions still run.
|
||||
Restricted to pull_request events.
|
||||
|
||||
REQ-2: Remove the failure-issue step from `ci.yml` validate job.
|
||||
Post-merge keeps failure notification (release/publish failures are
|
||||
deploy-pipeline events).
|
||||
|
||||
REQ-3: post-merge `cancel-in-progress: false` — a new push queues instead
|
||||
of killing an in-flight release/publish.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- `make workflow-lint` passes.
|
||||
- Docs-only PR: quality steps skipped, docs gate + spec + size + merge run.
|
||||
- Non-docs PR: unchanged behavior.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
Workflow-only change; takes effect on merge. No release needed.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the commit.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1 implemented — early docs-only step + step-level `if` gates
|
||||
- [x] REQ-2 implemented — notify step removed from ci.yml only
|
||||
- [x] REQ-3 implemented — post-merge concurrency flipped
|
||||
- [x] `make workflow-lint` passes
|
||||
@@ -1,76 +0,0 @@
|
||||
# DEVX-177: Honest molecule selection — fail-open coverage and real command contract
|
||||
|
||||
## Problem
|
||||
|
||||
`devx.molecule.molecule_changed` and `devx.ci.fast_molecule` violate the
|
||||
S09/REQ-9 honesty contract in three ways:
|
||||
|
||||
1. **Silent skips.** `detect_changed_roles` only knows four playbooks and
|
||||
`ansible/roles/`. Changes to `restore.yml`, `deploy-sso-bridge.yml`,
|
||||
`upgrade-postgres.yml`, `rolling-update-gitea.yml`,
|
||||
`update-alertmanager.yml`, `build-image.yml`, `playbooks/_tasks/`,
|
||||
`playbooks/tasks/`, or `ansible/group_vars/` trigger zero molecule
|
||||
coverage — the fast path passes by skipping work.
|
||||
2. **Phantom targets.** `ROLE_TARGET_MAP` includes `sso_config` (role
|
||||
moved to the sso-bridge repo) and emits `molecule-crowdsec` /
|
||||
`molecule-disk-cleanup` targets that do not exist in infra's Makefile.
|
||||
Playbook mappings also inject absent roles, so `make molecule-changed`
|
||||
can select nonexistent targets.
|
||||
3. **Dishonest contract.** `fast_molecule` documents "converge + verify
|
||||
only, no idempotence" and builds `molecule test -s X --destroy=never
|
||||
--platform-name=...` commands that nothing executes — CI actually runs
|
||||
full `molecule test -s X` via `run_molecule_scenario.py` (which does
|
||||
include idempotence where the scenario defines it).
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Fail-open coverage in `detect_changed_roles` — add the missing
|
||||
playbook→role mappings; `ansible/group_vars/**` and any other
|
||||
`ansible/playbooks/**` file (including `_tasks/`/`tasks/`) not explicitly
|
||||
mapped select all molecule-covered roles. `ansible/environments/` stays
|
||||
unmapped (env data is covered by unit/deploy tests, not molecule) and is
|
||||
documented as such.
|
||||
|
||||
REQ-2: Selection only emits roles that exist — `detect_changed_roles`
|
||||
gains a `roles_dir` parameter; role names from file paths and playbook
|
||||
maps are kept only when `<roles_dir>/<role>` exists. Shared-path and
|
||||
fail-open "all roles" resolution returns only directories under
|
||||
`roles_dir` containing a `molecule/` dir (untestable roles select
|
||||
nothing rather than phantom targets).
|
||||
|
||||
REQ-3: Honest fast-molecule contract — `build_molecule_commands` emits
|
||||
exactly what the CI runner executes (`molecule test -s <scenario>`);
|
||||
docstrings state the real sequence (full `molecule test` per scenario on
|
||||
changed roles, single platform as configured by the scenario) instead of
|
||||
the old "converge + verify only" claim.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Update `test_molecule_changed.py`: unmapped playbook → all present
|
||||
roles; `group_vars` change → all; mapped playbook → mapped roles only;
|
||||
role absent from `roles_dir` → filtered out; shared path → only roles
|
||||
with `molecule/` dirs.
|
||||
- Update `test_fast_molecule.py`: emitted commands are `molecule test -s
|
||||
<scenario>` with no `--destroy`/`--platform-name` flags.
|
||||
- `make pytest-cov` (100% gate), `make lint-all`.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
Merge → devx release publishes automatically → infra dep-PR bumps the
|
||||
pin; the honest selection takes effect on the next infra CI run.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the squash commit; previous (under-covering) selection returns —
|
||||
acceptable short-term because coverage only widens with this change.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: All `ansible/playbooks/**` and `ansible/group_vars/**`
|
||||
changes select molecule coverage; unmapped files fail open to all
|
||||
testable roles.
|
||||
- [x] REQ-2: No nonexistent roles or make targets are emitted; `sso_config`
|
||||
no longer appears when absent from `roles_dir`.
|
||||
- [x] REQ-3: `build_molecule_commands` output matches the executed CI
|
||||
command shape; module docstrings describe the real sequence.
|
||||
- [x] Unit tests cover every new behavior; 100% coverage maintained.
|
||||
@@ -1,38 +0,0 @@
|
||||
# DEVX-178: PLAYBOOK_ROLE_MAP entry for deploy-controller playbook
|
||||
|
||||
## Problem
|
||||
|
||||
OBL-INFRA-655 adds the `deploy-controller.yml` playbook in the infra repo to the infra
|
||||
repo (drives the new `deploy_controller` role). The infra molecule
|
||||
coverage guard (`test_every_playbook_is_mapped`) requires every playbook
|
||||
under `ansible/playbooks/` to appear in `PLAYBOOK_ROLE_MAP`. Without the
|
||||
entry, the guard fails and unmapped playbooks fail open to all testable
|
||||
roles — correct but wasteful.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Add a `deploy-controller.yml` → `['deploy_controller']` entry
|
||||
to `PLAYBOOK_ROLE_MAP` in `src/devx/molecule/molecule_changed.py` so
|
||||
changes to the playbook select the `deploy_controller` molecule scenario.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Existing `test_molecule_changed.py` mapping tests cover new entries
|
||||
generically; add an explicit assertion that the deploy-controller
|
||||
playbook maps to `deploy_controller`.
|
||||
- `make pytest-cov` (100% gate), `make lint-all`.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
Merge → devx release publishes automatically → infra dep-PR bumps the
|
||||
pin; the mapping takes effect on the next infra CI run.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the squash commit; infra playbook falls back to fail-open all-role
|
||||
selection (safe, slower).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: the deploy-controller playbook key maps to `["deploy_controller"]` in `PLAYBOOK_ROLE_MAP`.
|
||||
- [x] Unit test pins the mapping; 100% coverage maintained.
|
||||
@@ -1,49 +0,0 @@
|
||||
# DEVX-179: auto-merge must never select `[skip ci]` squash titles
|
||||
|
||||
## Problem
|
||||
|
||||
PR #351 squash-merged as `DEVX-178: chore: update badge URLs to commit
|
||||
4422c70b [skip ci]`. `extract_conventional_msg` picked a commit subject
|
||||
carrying `[skip ci]` — a master badge commit surfaced in the PR's commit
|
||||
list after force-push/amend rewrote branch history. The resulting merge
|
||||
commit suppressed the post-merge push run: no release, no publish, no
|
||||
wiki sync, no Vikunja close.
|
||||
|
||||
PR #352 shows the secondary defect: two `ci:` commits on the branch and
|
||||
the older one won the tie, so the squash title described a throwaway
|
||||
retrigger commit instead of the real change.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Subjects containing `[skip ci]`, `[ci skip]`, or `[skip actions]`
|
||||
(case-insensitive) are ineligible merge titles. They are excluded before
|
||||
priority scoring, so a badge/chore/`[skip ci]` commit can never suppress
|
||||
the post-merge pipeline again.
|
||||
|
||||
REQ-2: Equal-priority ties resolve to the newest commit in the returned
|
||||
list (Gitea returns PR commits newest-first; iterate in returned order
|
||||
and keep the first best candidate).
|
||||
|
||||
## Test Plan
|
||||
|
||||
- `test_auto_merge.py`: commits `[ci retrigger, badge-chore-skipci]` →
|
||||
the `ci` subject wins; commits `[older-ci, newer-ci]` → newest wins;
|
||||
all commits `[skip ci]` → falls back to newest non-skipped subject.
|
||||
- `make pytest-cov` (100% gate), `make lint-all`.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
Merge as `fix:` → post-merge cuts a release that also ships DEVX-178's
|
||||
PLAYBOOK_ROLE_MAP entry.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the squash commit; extract behavior returns to the previous
|
||||
(unhardened) selection.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: `[skip ci]`/`[ci skip]`/`[skip actions]` subjects are never
|
||||
selected.
|
||||
- [x] REQ-2: Equal-priority ties pick the newest commit.
|
||||
- [x] Unit tests cover both; 100% coverage maintained.
|
||||
@@ -1,35 +0,0 @@
|
||||
# DEVX-180: Add workflow_dispatch trigger to post-merge workflow
|
||||
|
||||
## Problem
|
||||
|
||||
When auto-merge produces a `[skip ci]` squash title (DEVX-179), the
|
||||
post-merge push run is suppressed — release, publish, wiki sync, Vikunja
|
||||
close, and badges are all skipped with no manual recovery path, because
|
||||
`post-merge.yml` only triggers on `push`.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: Add `workflow_dispatch:` to the `on:` block of
|
||||
`.gitea/workflows/post-merge.yml` so a missed or skipped post-merge run
|
||||
can be dispatched manually on master.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- `make workflow-check` (actionlint + act_runner dryrun) validates the
|
||||
trigger syntax.
|
||||
- Post-merge: dispatch the workflow manually once and confirm it runs
|
||||
on master.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
Merge → post-merge push run fires normally → then dispatch
|
||||
`post-merge.yml` on master to recover the suppressed DEVX-178 release.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Revert the one-line trigger addition.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: `workflow_dispatch` present in `post-merge.yml` `on:` block;
|
||||
actionlint and act_runner dryrun pass.
|
||||
@@ -1,48 +0,0 @@
|
||||
# DEVX-181: Emit runner resource leases for CI molecule jobs (S08 producer side)
|
||||
|
||||
## Problem
|
||||
GRM-173 (S08) shipped consumer-side runner leases: `runner-cleanup.sh`
|
||||
honors `org.oblachno.lease-until` / `org.oblachno.owner` labels on
|
||||
containers, volumes, and networks. But no CI producer emits them:
|
||||
molecule containers/networks created on the runner's host rootless Docker
|
||||
socket carry only `owner=molecule`, so a pressure/critical cleanup can
|
||||
still wipe a molecule object that is stopped for a moment mid-run.
|
||||
Docker labels cannot be added post-creation — producers must emit them
|
||||
at create time.
|
||||
|
||||
## Approach
|
||||
REQ-1: New `devx.molecule.lease` module computing `MOLECULE_LEASE_UNTIL`
|
||||
(epoch = now + TTL, default 4h) and `MOLECULE_OWNER` (`run-<id>-<job>`
|
||||
from GITEA_*/GITHUB_* env, else `local-<hostname>`), with
|
||||
`emit_github_env()` appending them to `$GITHUB_ENV` and a Click CLI
|
||||
printing `KEY=value` lines.
|
||||
REQ-2: `start_docker.main()` emits the lease env after a successful
|
||||
Docker start — every molecule CI job already calls it, so no workflow
|
||||
edits are needed.
|
||||
REQ-3: Consumers (infra/grm `molecule.yml`) add platform labels
|
||||
`org.oblachno.lease-until: "${MOLECULE_LEASE_UNTIL:-0}"` and
|
||||
`org.oblachno.owner: "${MOLECULE_OWNER:-molecule-local}"` — `0` is
|
||||
already expired, so local/unset runs keep today's unleased behavior.
|
||||
Downstream changes land in the infra and grm repos.
|
||||
|
||||
## Test Plan
|
||||
- `tests/unit/test_molecule_lease.py` covers `lease_owner` env precedence
|
||||
and fallbacks, `lease_env` arithmetic, `emit_github_env` write/no-op,
|
||||
and the CLI — 100% coverage enforced by `make pytest-cov`.
|
||||
- Existing `test_start_docker.py` still green (emission is additive).
|
||||
|
||||
## Deploy Plan
|
||||
- Merge via auto-merge → post-merge releases and publishes devx; infra
|
||||
and grm pin bumps consume it via their molecule.yml label additions.
|
||||
|
||||
## Rollback Plan
|
||||
- Revert the merge; `start_docker` stops exporting the vars and molecule
|
||||
interpolation falls back to expired leases (`:-0`).
|
||||
|
||||
## Acceptance Criteria
|
||||
- [x] REQ-1: `devx.molecule.lease` computes both env vars and writes them
|
||||
to `$GITHUB_ENV` when set.
|
||||
- [x] REQ-2: `start_docker` exports lease env on success without changing
|
||||
its existing behavior contract.
|
||||
- [x] REQ-3: the label contract (`org.oblachno.lease-until`,
|
||||
`org.oblachno.owner`) is exposed as module constants for consumers.
|
||||
@@ -1,49 +0,0 @@
|
||||
# DEVX-182: feat(molecule): release lease emission (ships DEVX-181)
|
||||
|
||||
- Task: DEVX-182
|
||||
- Status: draft
|
||||
|
||||
## Problem
|
||||
|
||||
DEVX-181 landed on master but was never released: the squash subject
|
||||
came out `DEVX-181: ci: retrigger validation` because the branch's last
|
||||
commit was an empty `ci:` retrigger. git-cliff found no bump-worthy
|
||||
commit, so `v0.55.5` is still latest and the S08 lease producer is
|
||||
unshipped. A `feat:` merge is needed to cut the release — and while
|
||||
shipping, the emission log line is upgraded from a bare count to the
|
||||
actual lease owner and UTC expiry, which is what an operator needs when
|
||||
debugging a cleanup race.
|
||||
|
||||
## Approach
|
||||
|
||||
REQ-1: `start_docker.main()` computes `lease_env()` once and logs
|
||||
`owner=` + ISO-8601 `until=` alongside the export confirmation. The
|
||||
single call also removes a latent second-boundary inconsistency where
|
||||
`emit_github_env` and the count message each re-derived `now`.
|
||||
|
||||
REQ-2: Merge squash commit carries `feat(molecule):` so git-cliff cuts
|
||||
the next minor release containing DEVX-181's emitter and this change.
|
||||
|
||||
## Test Plan
|
||||
|
||||
- Update `test_emits_lease_on_success` + `test_prints_and_exports` for
|
||||
the new log format; keep 100 % coverage.
|
||||
- `make pre-push` — full devx suite green.
|
||||
- Post-merge: verify a `v*` tag > v0.55.5 exists and
|
||||
`pip index versions devx` (Gitea PyPI) lists it.
|
||||
|
||||
## Deploy Plan
|
||||
|
||||
- Auto-merge; post-merge release job tags + publishes.
|
||||
- Infra (OBL-INFRA-658) and GRM (GRM-174) consume the new pin.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
- Revert merge; the unreleased module remains dormant regardless.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] REQ-1: emission log includes owner and ISO-8601 expiry; lease env
|
||||
computed once — covered by updated unit tests.
|
||||
- [x] REQ-2: merge commit uses `feat(molecule):` so the release job
|
||||
cuts a version bump.
|
||||
@@ -50,6 +50,7 @@ src/devx/
|
||||
├── __init__.py
|
||||
├── discover_runners.py # Dynamic Gitea runner discovery
|
||||
├── distribute_molecule.py # Distribute scenarios across runners
|
||||
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast
|
||||
├── molecule_all.py # Run all molecule scenarios locally
|
||||
├── start_docker.py # Ensure Docker is available for molecule
|
||||
└── platforms.py # Supported molecule platforms
|
||||
@@ -299,9 +300,9 @@ Distributes files matching a glob pattern across N parallel runners
|
||||
|
||||
### `integration_guard.py`
|
||||
|
||||
Runs pytest with cross-runner failure detection. A background thread polls
|
||||
the Gitea API. If any other integration-tests matrix runner reports failure,
|
||||
the current pytest subprocess is killed and this runner exits early.
|
||||
Runs pytest with the same cross-runner failure detection mechanism used by
|
||||
`molecule_ci_guard`. If any other integration-tests matrix runner reports
|
||||
failure, the current pytest subprocess is killed and this runner exits early.
|
||||
|
||||
## Developer tools (`devx.tools`)
|
||||
|
||||
@@ -381,6 +382,13 @@ the supported OS platform matrix. Supports `--roles-root` for multi-role
|
||||
repositories, `--list` to list scenarios, and `--list-platforms` to list
|
||||
platforms.
|
||||
|
||||
### `molecule_ci_guard.py`
|
||||
|
||||
Runs molecule tests sequentially while polling the Gitea API for other runner
|
||||
failures. If any other molecule matrix runner reports failure, the current
|
||||
molecule subprocess is killed and this runner exits early. Supports both
|
||||
single-role (4-part) and multi-role (5-part) pair encoding.
|
||||
|
||||
### `molecule_all.py`
|
||||
|
||||
Runs all molecule scenarios on all supported OS platforms sequentially.
|
||||
|
||||
@@ -478,6 +478,15 @@ python -m devx.molecule.distribute_molecule --list
|
||||
python -m devx.molecule.distribute_molecule --list-platforms
|
||||
```
|
||||
|
||||
### `molecule_ci_guard.py`
|
||||
|
||||
Runs molecule tests sequentially while polling the Gitea API for other runner
|
||||
failures. Aborts early if another runner fails the same job.
|
||||
|
||||
```bash
|
||||
python -m devx.molecule.molecule_ci_guard [--roles-root <dir>] pair1 pair2 ...
|
||||
```
|
||||
|
||||
### `validate_commit_msg.py`
|
||||
|
||||
Validates commit messages. On feature branches: conventional commits only
|
||||
|
||||
@@ -631,3 +631,29 @@ Options:
|
||||
- `--list-platforms` — list all platforms, one per line
|
||||
- `--roles-root <dir>` — roles root directory for multi-role repos (default:
|
||||
`ansible/roles`)
|
||||
|
||||
### `devx molecule guard`
|
||||
|
||||
Run molecule tests sequentially with CI failure polling. A background thread
|
||||
polls the Gitea API. If any other molecule matrix runner reports failure, the
|
||||
current molecule subprocess is killed and this runner exits early with code 1.
|
||||
|
||||
```bash
|
||||
devx molecule guard pair1 pair2 pair3
|
||||
devx molecule guard --roles-root ansible/roles pair1 pair2
|
||||
```
|
||||
|
||||
Each pair is encoded as:
|
||||
- **Single-role (4-part):** `scenario|platform_name|platform_image|platform_command`
|
||||
- **Multi-role (5-part):** `role|scenario|platform_name|platform_image|platform_command`
|
||||
|
||||
Options:
|
||||
- `--roles-root <dir>` — roles root directory for multi-role repos
|
||||
|
||||
Environment variables:
|
||||
- `GITEA_URL` — base URL of the Gitea instance
|
||||
- `CI_GITEA_TOKEN` — API token with repo access
|
||||
- `RUN_ID` — workflow run ID (`GITHUB_RUN_ID`)
|
||||
- `JOB_NAME` — base job name (`GITHUB_JOB`)
|
||||
- `MATRIX_INDEX` — current matrix index (runner-index)
|
||||
- `GITEA_REPOSITORY` — repository in `owner/repo` format
|
||||
|
||||
@@ -48,12 +48,12 @@ Add devx to your `pyproject.toml`:
|
||||
```toml
|
||||
[project]
|
||||
dependencies = [
|
||||
"devx>=0.56.0",
|
||||
"devx>=0.50.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"devx>=0.56.0",
|
||||
"devx>=0.50.0",
|
||||
]
|
||||
```
|
||||
|
||||
|
||||
@@ -1,9 +1,3 @@
|
||||
"""devx — reusable development and CI/CD tools for oblachno-oss projects.
|
||||
"""devx — reusable development and CI/CD tools for oblachno-oss projects."""
|
||||
|
||||
Provides CI/CD automation (validate_spec, check_pr_size, nightly_gate,
|
||||
create_dependency_pr, auto_merge, release, publish), developer tooling
|
||||
(setup, install_tools, configure_repo, create_task, create_pr), and
|
||||
molecule testing helpers for Ansible projects.
|
||||
"""
|
||||
|
||||
__version__ = "0.56.0"
|
||||
__version__ = "0.50.0"
|
||||
|
||||
@@ -392,10 +392,7 @@ class GiteaClient:
|
||||
"""
|
||||
try:
|
||||
r = self._request("GET", f"/actions/variables/{name}")
|
||||
body = r.json()
|
||||
if "data" in body:
|
||||
return body["data"]
|
||||
return body.get("value")
|
||||
return r.json().get("value")
|
||||
except APIError as e:
|
||||
if e.status == 404:
|
||||
return None
|
||||
|
||||
+17
-63
@@ -21,12 +21,10 @@ Usage:
|
||||
"""
|
||||
|
||||
import re
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import click
|
||||
import requests
|
||||
from dotenv import load_dotenv # pyright: ignore[reportMissingImports,reportUnknownVariableType]
|
||||
|
||||
from devx.api_clients import GiteaClient, VikunjaClient
|
||||
@@ -112,27 +110,6 @@ def validate_pr_title(pr_title: str, task_id: str) -> None:
|
||||
)
|
||||
|
||||
|
||||
_VIKUNJA_LOOKUP_ATTEMPTS = 4
|
||||
_VIKUNJA_LOOKUP_BACKOFF = 5.0
|
||||
|
||||
|
||||
def _list_project_tasks(client: VikunjaClient, page: int) -> list[dict[str, Any]]:
|
||||
"""List Vikunja tasks with bounded retries for transient outages.
|
||||
|
||||
Implements REQ-1: during a Vikunja restart the tasks endpoint can briefly
|
||||
return 404/502; retry a few times so title validation rides out the window
|
||||
instead of stranding an otherwise-valid PR.
|
||||
"""
|
||||
for attempt in range(1, _VIKUNJA_LOOKUP_ATTEMPTS + 1):
|
||||
try:
|
||||
return list(client.list_project_tasks(VIKUNJA_PROJECT_ID, page=page, per_page=DEFAULT_PER_PAGE))
|
||||
except (APIError, requests.RequestException):
|
||||
if attempt == _VIKUNJA_LOOKUP_ATTEMPTS:
|
||||
raise
|
||||
time.sleep(_VIKUNJA_LOOKUP_BACKOFF * attempt)
|
||||
raise AssertionError("unreachable") # pragma: no cover
|
||||
|
||||
|
||||
def get_vikunja_task_title(task_id: str) -> str:
|
||||
"""Fetch the Vikunja task title for the given DEVX-N identifier.
|
||||
|
||||
@@ -147,7 +124,7 @@ def get_vikunja_task_title(task_id: str) -> str:
|
||||
client = VikunjaClient(VIKUNJA_API_URL, token)
|
||||
page = 1
|
||||
while True:
|
||||
tasks = _list_project_tasks(client, page)
|
||||
tasks = client.list_project_tasks(VIKUNJA_PROJECT_ID, page=page, per_page=DEFAULT_PER_PAGE)
|
||||
if not tasks:
|
||||
break
|
||||
matches = [t for t in tasks if t.get("identifier") == task_id]
|
||||
@@ -184,31 +161,22 @@ def validate_pr_title_matches_vikunja(pr_title: str, task_id: str) -> None:
|
||||
)
|
||||
|
||||
|
||||
# Subjects carrying these tokens suppress post-merge CI entirely — strip them
|
||||
# before a commit message can become the squash title (see DEVX-179).
|
||||
_SKIP_CI_RE = re.compile(r"\s*\[(?:ci skip|skip ci|skip actions|actions skip|skip)\]", re.IGNORECASE)
|
||||
|
||||
|
||||
def extract_conventional_msg(commits: list[dict[str, Any]]) -> str:
|
||||
"""Extract the conventional commit message from PR commits.
|
||||
|
||||
Picks the highest-priority conventional commit message from the PR.
|
||||
Priority: feat > fix > refactor > docs > chore > other. The Gitea
|
||||
``/pulls/{n}/commits`` endpoint returns commits newest-first, so the
|
||||
first best-scoring subject wins equal-priority ties.
|
||||
``[skip ci]``-style tokens are stripped from every candidate so the
|
||||
merge title can never suppress the post-merge release pipeline.
|
||||
Priority: feat > fix > refactor > docs > chore > other.
|
||||
Falls back to the newest commit message if none match.
|
||||
"""
|
||||
priority = {"feat": 5, "fix": 4, "refactor": 3, "docs": 2, "chore": 1, "ci": 1, "style": 1, "test": 1}
|
||||
best_msg = ""
|
||||
best_score = 0
|
||||
for commit in commits:
|
||||
for commit in reversed(commits):
|
||||
commit_info = commit.get("commit", {})
|
||||
message = str(commit_info.get("message", "") if isinstance(commit_info, dict) else "").split("\n")[0]
|
||||
# Strip any leading task ID prefix (e.g. "OBL-INFRA-364: fix: ...") so
|
||||
# conventional commit matching works on the remainder; drop CI-skip
|
||||
# tokens so they never reach the merge title.
|
||||
stripped = _SKIP_CI_RE.sub("", _TASK_ID_PREFIX_RE.sub("", message)).strip()
|
||||
# conventional commit matching works on the remainder.
|
||||
stripped = _TASK_ID_PREFIX_RE.sub("", message)
|
||||
m = CONVENTIONAL_RE.match(stripped)
|
||||
if m:
|
||||
prefix = m.group(1).split("(")[0].strip() # e.g. "feat" from "feat(scope)"
|
||||
@@ -218,11 +186,11 @@ def extract_conventional_msg(commits: list[dict[str, Any]]) -> str:
|
||||
best_msg = stripped
|
||||
if best_msg:
|
||||
return best_msg
|
||||
# Fallback: the newest commit's first line (newest-first API order).
|
||||
# Fallback: use the newest commit's first line (strip task ID prefix if present)
|
||||
if commits:
|
||||
commit_info = commits[0].get("commit", {})
|
||||
commit_info = commits[-1].get("commit", {})
|
||||
raw = str(commit_info.get("message", "") if isinstance(commit_info, dict) else "").split("\n")[0]
|
||||
return _SKIP_CI_RE.sub("", _TASK_ID_PREFIX_RE.sub("", raw)).strip()
|
||||
return _TASK_ID_PREFIX_RE.sub("", raw)
|
||||
return ""
|
||||
|
||||
|
||||
@@ -304,28 +272,14 @@ def main(branch: str, pr_title: str, repo: str, pr_number: str) -> None:
|
||||
# Exit cleanly — the rebase triggers a new CI run that will retry.
|
||||
return
|
||||
else:
|
||||
hint = ""
|
||||
if e.status == 405:
|
||||
# Implements: REQ-2 — surface approval state. When the PR author
|
||||
# and the CI reviewer token map to the same Gitea user,
|
||||
# self-approval is rejected and the merge fails 405.
|
||||
try:
|
||||
reviews = client.get_pr_reviews(pr_num)
|
||||
if not any(r.get("state") == "APPROVED" for r in reviews):
|
||||
hint = _(
|
||||
"\nNo APPROVED review found on the PR. If the PR author and the"
|
||||
" CI reviewer token map to the same Gitea user, self-approval is"
|
||||
" rejected — approve the PR via a non-author account, then"
|
||||
" re-run the auto-merge job."
|
||||
)
|
||||
except APIError:
|
||||
pass
|
||||
msg = _(
|
||||
"Merge failed with HTTP {status}: {message}\nPlease check the PR is ready and you have merge rights.",
|
||||
status=e.status,
|
||||
message=e.message,
|
||||
)
|
||||
raise click.ClickException(msg + hint) from None
|
||||
raise click.ClickException(
|
||||
_(
|
||||
"Merge failed with HTTP {status}: {message}\n"
|
||||
"Please check the PR is ready and you have merge rights.",
|
||||
status=e.status,
|
||||
message=e.message,
|
||||
)
|
||||
) from None
|
||||
|
||||
click.echo(
|
||||
_(
|
||||
|
||||
@@ -1,213 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# Implements: REQ-2
|
||||
"""Check PR size and reject oversized PRs.
|
||||
|
||||
Enforces max lines changed and max files changed to keep PRs small
|
||||
and deployable. Generated/excluded files are not counted.
|
||||
|
||||
PRs with the ``refactoring`` label bypass the size check — large but
|
||||
legitimate refactoring PRs that touch many files in a coordinated way.
|
||||
|
||||
Usage:
|
||||
python -m devx.ci.check_pr_size --base origin/master --head HEAD \\
|
||||
--repo oblachno-oss/grm --pr-number 123
|
||||
|
||||
In CI, pass ``--github-output`` to set ``pr-size-ok`` and ``pr-size-detail``
|
||||
for downstream steps.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess # nosec B404
|
||||
|
||||
import click
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from devx.api_clients import GiteaClient
|
||||
from devx.ci._shared import write_github_output
|
||||
from devx.config import GITEA_API_URL
|
||||
from devx.i18n import _
|
||||
from devx.tokens import get_ci_token
|
||||
|
||||
load_dotenv()
|
||||
|
||||
# Files/patterns excluded from size counting (generated, badges, locks, etc.)
|
||||
DEFAULT_EXCLUDED_PATTERNS = [
|
||||
"CHANGELOG.md",
|
||||
"README.md",
|
||||
"docs/index.md",
|
||||
"docs/plans/*",
|
||||
"*.svg",
|
||||
"uv.lock",
|
||||
"poetry.lock",
|
||||
"Pipfile.lock",
|
||||
"package-lock.json",
|
||||
"yarn.lock",
|
||||
"go.sum",
|
||||
]
|
||||
|
||||
DEFAULT_MAX_LINES = 500
|
||||
DEFAULT_MAX_FILES = 10
|
||||
REFACTORING_LABEL = "refactoring"
|
||||
|
||||
|
||||
def has_refactoring_label(repo: str, pr_number: int) -> bool:
|
||||
"""Check if a PR has the 'refactoring' label (bypasses size check)."""
|
||||
try:
|
||||
token = get_ci_token()
|
||||
owner, repo_name = repo.split("/", 1)
|
||||
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
|
||||
pr = client.get_pr(pr_number)
|
||||
labels = pr.get("labels", [])
|
||||
return any(label.get("name") == REFACTORING_LABEL for label in labels)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def get_diff_stats(base: str, head: str) -> list[tuple[str, int, int]]:
|
||||
"""Get per-file diff stats (additions, deletions) between base and head.
|
||||
|
||||
Returns a list of (filename, additions, deletions) tuples.
|
||||
"""
|
||||
result = subprocess.run( # nosec B603 B607
|
||||
["git", "diff", "--numstat", base, head],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise click.ClickException(_("git diff --numstat failed: {stderr}", stderr=result.stderr.strip()))
|
||||
stats: list[tuple[str, int, int]] = []
|
||||
for line in result.stdout.strip().split("\n"):
|
||||
if not line:
|
||||
continue
|
||||
parts = line.split("\t")
|
||||
if len(parts) != 3:
|
||||
continue
|
||||
additions_s, deletions_s, filename = parts
|
||||
# Binary files show "-" for additions/deletions
|
||||
additions = int(additions_s) if additions_s.isdigit() else 0
|
||||
deletions = int(deletions_s) if deletions_s.isdigit() else 0
|
||||
stats.append((filename, additions, deletions))
|
||||
return stats
|
||||
|
||||
|
||||
def is_excluded(filename: str, excluded_patterns: list[str]) -> bool:
|
||||
"""Check if a filename matches any excluded pattern."""
|
||||
from fnmatch import fnmatch
|
||||
|
||||
return any(fnmatch(filename, pat) for pat in excluded_patterns)
|
||||
|
||||
|
||||
def check_size(
|
||||
stats: list[tuple[str, int, int]],
|
||||
max_lines: int,
|
||||
max_files: int,
|
||||
excluded_patterns: list[str],
|
||||
) -> tuple[bool, str]:
|
||||
"""Check diff stats against limits.
|
||||
|
||||
Returns (is_ok, detail_message).
|
||||
"""
|
||||
included = [(f, a, d) for f, a, d in stats if not is_excluded(f, excluded_patterns)]
|
||||
total_lines = sum(a + d for _, a, d in included)
|
||||
total_files = len(included)
|
||||
|
||||
if total_files == 0:
|
||||
return True, "No non-excluded files changed"
|
||||
|
||||
if total_files > max_files:
|
||||
return False, _(
|
||||
"PR has {file_count} files changed (max {max_files}). Excluded: {excluded_count} files.",
|
||||
file_count=total_files,
|
||||
max_files=max_files,
|
||||
excluded_count=len(stats) - total_files,
|
||||
)
|
||||
|
||||
if total_lines > max_lines:
|
||||
return False, _(
|
||||
"PR has {line_count} lines changed (max {max_lines}). Excluded: {excluded_count} files.",
|
||||
line_count=total_lines,
|
||||
max_lines=max_lines,
|
||||
excluded_count=len(stats) - total_files,
|
||||
)
|
||||
|
||||
return True, _(
|
||||
"PR size OK: {file_count} files, {line_count} lines (max {max_files} files, {max_lines} lines).",
|
||||
file_count=total_files,
|
||||
line_count=total_lines,
|
||||
max_files=max_files,
|
||||
max_lines=max_lines,
|
||||
)
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option("--base", default="origin/master", help=_("Base ref for diff"))
|
||||
@click.option("--head", default="HEAD", help=_("Head ref for diff"))
|
||||
@click.option(
|
||||
"--max-lines",
|
||||
type=int,
|
||||
default=DEFAULT_MAX_LINES,
|
||||
help=_("Max lines changed (excluded files not counted)"),
|
||||
)
|
||||
@click.option(
|
||||
"--max-files",
|
||||
type=int,
|
||||
default=DEFAULT_MAX_FILES,
|
||||
help=_("Max files changed (excluded files not counted)"),
|
||||
)
|
||||
@click.option(
|
||||
"--github-output",
|
||||
"github_output",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help=_("Write results to $GITHUB_OUTPUT"),
|
||||
)
|
||||
@click.option(
|
||||
"--excluded",
|
||||
"excluded",
|
||||
multiple=True,
|
||||
help=_("Additional excluded patterns (in addition to defaults)"),
|
||||
)
|
||||
@click.option("--repo", default=None, help=_("Repo (owner/name) for label check"))
|
||||
@click.option("--pr-number", type=int, default=None, help=_("PR number for label check"))
|
||||
def cli(
|
||||
base: str,
|
||||
head: str,
|
||||
max_lines: int,
|
||||
max_files: int,
|
||||
github_output: bool,
|
||||
excluded: tuple[str, ...],
|
||||
repo: str | None,
|
||||
pr_number: int | None,
|
||||
) -> None:
|
||||
"""Check PR size and reject oversized PRs."""
|
||||
# Check for refactoring label bypass
|
||||
if repo and pr_number and has_refactoring_label(repo, pr_number):
|
||||
detail = _("PR has 'refactoring' label — size check bypassed.")
|
||||
if github_output:
|
||||
write_github_output("pr-size-ok", "true")
|
||||
write_github_output("pr-size-detail", detail)
|
||||
click.echo(f"[pr-size] {detail}")
|
||||
return
|
||||
|
||||
excluded_patterns = list(DEFAULT_EXCLUDED_PATTERNS) + list(excluded)
|
||||
stats = get_diff_stats(base, head)
|
||||
is_ok, detail = check_size(stats, max_lines, max_files, excluded_patterns)
|
||||
|
||||
if github_output:
|
||||
write_github_output("pr-size-ok", "true" if is_ok else "false")
|
||||
write_github_output("pr-size-detail", detail)
|
||||
|
||||
if is_ok:
|
||||
click.echo(f"[pr-size] {detail}")
|
||||
else:
|
||||
click.echo(f"[pr-size] FAILED: {detail}", err=True)
|
||||
click.echo("", err=True)
|
||||
click.echo("Oversized PRs cannot be reliably reviewed or deployed independently.", err=True)
|
||||
click.echo("Split your work into smaller PRs, each addressing one concern.", err=True)
|
||||
raise click.ClickException(_("PR size check failed."))
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
cli()
|
||||
@@ -1,548 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# Implements: REQ-5
|
||||
"""Auto-create an infra PR to bump a pinned dependency version.
|
||||
|
||||
After grm or sso-bridge publishes a new package version, this module
|
||||
creates a PR in the infra repo to bump the pinned version in
|
||||
``pyproject.toml`` or ``ansible/group_vars/all/images.yml``.
|
||||
|
||||
Reuses ``devx.tools.create_pr`` for PR creation and Vikunja task linking.
|
||||
|
||||
Usage:
|
||||
python -m devx.ci.create_dependency_pr \
|
||||
--repo oblachno/infra \
|
||||
--package grm \
|
||||
--new-version 0.5.2 \
|
||||
--source-repo oblachno/grm \
|
||||
--source-run-id 12345
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess # nosec B404
|
||||
import tempfile
|
||||
import time
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
import requests
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from devx.api_clients import GiteaClient
|
||||
from devx.config import GITEA_API_URL, VIKUNJA_API_URL, VIKUNJA_PROJECT_ID
|
||||
from devx.exceptions import APIError
|
||||
from devx.i18n import _
|
||||
from devx.tokens import get_ci_token, get_vikunja_token
|
||||
from devx.tools.create_pr import find_existing_pr
|
||||
|
||||
load_dotenv()
|
||||
|
||||
# Where infra pins dependency versions
|
||||
PYPROJECT_PATH = "pyproject.toml"
|
||||
IMAGES_YML_PATH = "ansible/group_vars/all/images.yml"
|
||||
ROLE_DEFAULTS_PATH = "ansible/roles/sso_bridge/defaults/main.yml"
|
||||
|
||||
|
||||
def find_pinned_version(package: str, file_path: str) -> str | None:
|
||||
"""Find the currently pinned version of a package in a file.
|
||||
|
||||
Looks for patterns like:
|
||||
- ``"grm @ git+...@v0.5.1"``
|
||||
- ``grm = "0.5.1"``
|
||||
- ``grm_version: "0.5.1"``
|
||||
- ``grm_image_version: "0.5.1"``
|
||||
"""
|
||||
path = Path(file_path)
|
||||
if not path.exists():
|
||||
return None
|
||||
content = path.read_text(encoding="utf-8")
|
||||
# Match various pinning patterns
|
||||
patterns = [
|
||||
rf"{package}\s*@\s*git\+[^@]+@v?([\d.]+)", # pip: package @ git+url@vX.Y.Z
|
||||
rf'{package}\s*=\s*"([\d.]+)"', # pyproject: package = "X.Y.Z"
|
||||
rf'{package}_version:\s*"([\d.]+)"', # ansible vars: package_version: "X.Y.Z"
|
||||
rf'{package}_image_version:\s*"([\d.]+)"', # ansible vars: package_image_version: "X.Y.Z"
|
||||
]
|
||||
for pat in patterns:
|
||||
match = re.search(pat, content)
|
||||
if match:
|
||||
return match.group(1)
|
||||
return None
|
||||
|
||||
|
||||
def update_pinned_version(file_path: str, package: str, old_version: str, new_version: str) -> bool:
|
||||
"""Update the pinned version in a file. Returns True if changed."""
|
||||
path = Path(file_path)
|
||||
if not path.exists():
|
||||
return False
|
||||
content = path.read_text(encoding="utf-8")
|
||||
# Replace old version with new version in package-related lines
|
||||
patterns = [
|
||||
(rf"({package}\s*@\s*git\+[^@]+@v?){old_version}", rf"\g<1>{new_version}"),
|
||||
(rf'({package}\s*=\s*"){old_version}(")', rf"\g<1>{new_version}\g<2>"),
|
||||
(rf'({package}_version:\s*"){old_version}(")', rf"\g<1>{new_version}\g<2>"),
|
||||
(rf'({package}_image_version:\s*"){old_version}(")', rf"\g<1>{new_version}\g<2>"),
|
||||
]
|
||||
new_content = content
|
||||
changed = False
|
||||
for pat, replacement in patterns:
|
||||
new_content, n = re.subn(pat, replacement, new_content)
|
||||
if n > 0:
|
||||
changed = True
|
||||
if changed:
|
||||
path.write_text(new_content, encoding="utf-8")
|
||||
return changed
|
||||
|
||||
|
||||
_MANIFEST_ACCEPT = (
|
||||
"application/vnd.oci.image.index.v1+json, "
|
||||
"application/vnd.docker.distribution.manifest.list.v2+json, "
|
||||
"application/vnd.oci.image.manifest.v1+json, "
|
||||
"application/vnd.docker.distribution.manifest.v2+json"
|
||||
)
|
||||
|
||||
|
||||
# Implements: REQ-1 — existence check via packages API (keeps the 404-retry
|
||||
# semantics for the publish race); the pullable digest is then resolved via
|
||||
# the registry v2 API's Docker-Content-Digest header.
|
||||
def resolve_container_digest(api_url: str, owner: str, name: str, tag: str, token: str, timeout_s: int = 0) -> str:
|
||||
"""Verify the container tag exists, then resolve its pullable OCI digest.
|
||||
|
||||
The packages API proves the tag was published (and 404s while the
|
||||
producer's image-build workflow races us — ``timeout_s`` retries every
|
||||
15s). The packages-API ``manifest.json`` blob sha256 is NOT pullable
|
||||
via ``repo@sha256:...``, so the digest comes from the registry v2
|
||||
``Docker-Content-Digest`` header instead.
|
||||
"""
|
||||
url = f"{api_url}/packages/{owner}/container/{name}/{tag}/files"
|
||||
headers = {"Authorization": f"token {token}"}
|
||||
deadline = time.monotonic() + timeout_s
|
||||
while True:
|
||||
try:
|
||||
resp = requests.get(url, headers=headers, timeout=30) # nosec B310
|
||||
resp.raise_for_status()
|
||||
except requests.HTTPError as e:
|
||||
if e.response is not None and e.response.status_code == 404 and time.monotonic() < deadline:
|
||||
click.echo(
|
||||
_(
|
||||
"[dep-pr] {owner}/{name}:{tag} not published yet — retrying.",
|
||||
owner=owner,
|
||||
name=name,
|
||||
tag=tag,
|
||||
)
|
||||
)
|
||||
time.sleep(15)
|
||||
continue
|
||||
status = e.response.status_code if e.response is not None else "?"
|
||||
raise click.ClickException(
|
||||
_(
|
||||
"Container artifact {owner}/{name}:{tag} not found or unreadable (HTTP {status}). "
|
||||
"Refusing to open a dependency PR for an unpublished artifact.",
|
||||
owner=owner,
|
||||
name=name,
|
||||
tag=tag,
|
||||
status=status,
|
||||
)
|
||||
) from e
|
||||
except requests.RequestException as e:
|
||||
raise click.ClickException(
|
||||
_(
|
||||
"Registry lookup failed for {owner}/{name}:{tag}: {error}",
|
||||
owner=owner,
|
||||
name=name,
|
||||
tag=tag,
|
||||
error=e,
|
||||
)
|
||||
) from e
|
||||
break
|
||||
return _resolve_registry_digest(api_url, owner, name, tag, token)
|
||||
|
||||
|
||||
# Implements: REQ-2 — v2 digest resolution fails closed: non-2xx, missing
|
||||
# token, or absent Docker-Content-Digest all raise; a blob sha256 is never
|
||||
# recorded as a pullable digest.
|
||||
def _resolve_registry_digest(api_url: str, owner: str, name: str, tag: str, token: str) -> str:
|
||||
registry = api_url.removesuffix("/api/v1").removesuffix("/")
|
||||
repo = f"{owner}/{name}"
|
||||
try:
|
||||
tok_resp = requests.get( # nosec B310
|
||||
f"{registry}/v2/token",
|
||||
params={"service": "container_registry", "scope": f"repository:{repo}:pull"},
|
||||
auth=("ci", token),
|
||||
timeout=30,
|
||||
)
|
||||
tok_resp.raise_for_status()
|
||||
bearer = tok_resp.json().get("token", "")
|
||||
man_resp = requests.get( # nosec B310
|
||||
f"{registry}/v2/{repo}/manifests/{tag}",
|
||||
headers={"Authorization": f"Bearer {bearer}", "Accept": _MANIFEST_ACCEPT},
|
||||
timeout=30,
|
||||
)
|
||||
man_resp.raise_for_status()
|
||||
except requests.RequestException as e:
|
||||
status = getattr(getattr(e, "response", None), "status_code", "?")
|
||||
raise click.ClickException(
|
||||
_(
|
||||
"Registry v2 digest lookup failed for {repo}:{tag} (HTTP {status}): {error}",
|
||||
repo=repo,
|
||||
tag=tag,
|
||||
status=status,
|
||||
error=e,
|
||||
)
|
||||
) from e
|
||||
digest = man_resp.headers.get("Docker-Content-Digest", "")
|
||||
if not digest:
|
||||
raise click.ClickException(
|
||||
_("Registry returned no Docker-Content-Digest for {repo}:{tag}.", repo=repo, tag=tag)
|
||||
)
|
||||
return digest
|
||||
|
||||
|
||||
def update_manifest(file_path: str, section: str, fields: dict[str, str]) -> bool:
|
||||
"""Update a release-manifest JSON section in place. Returns True if changed.
|
||||
|
||||
Implements REQ-1: manifest fields record the immutable release contract
|
||||
(version, git ref, image tag, resolved OCI digest) in the target repo.
|
||||
"""
|
||||
import json as _json
|
||||
|
||||
path = Path(file_path)
|
||||
if not path.exists():
|
||||
raise click.ClickException(_("Manifest file not found: {file}", file=file_path))
|
||||
try:
|
||||
manifest = _json.loads(path.read_text(encoding="utf-8"))
|
||||
except _json.JSONDecodeError as e:
|
||||
raise click.ClickException(_("Manifest file {file} is not valid JSON: {error}", file=file_path, error=e)) from e
|
||||
existing = manifest.get(section)
|
||||
if not isinstance(existing, dict):
|
||||
raise click.ClickException(
|
||||
_("Manifest file {file} has no object section {section}", file=file_path, section=section)
|
||||
)
|
||||
changed = False
|
||||
for k, v in fields.items():
|
||||
if existing.get(k) != v:
|
||||
existing[k] = v
|
||||
changed = True
|
||||
if changed:
|
||||
path.write_text(_json.dumps(manifest, indent=2) + "\n", encoding="utf-8")
|
||||
return changed
|
||||
|
||||
|
||||
def read_manifest_version(file_path: str, section: str) -> str | None:
|
||||
"""Read the currently pinned version from a release manifest section."""
|
||||
import json as _json
|
||||
|
||||
path = Path(file_path)
|
||||
if not path.exists():
|
||||
return None
|
||||
try:
|
||||
manifest = _json.loads(path.read_text(encoding="utf-8"))
|
||||
except _json.JSONDecodeError:
|
||||
return None
|
||||
existing = manifest.get(section)
|
||||
if isinstance(existing, dict):
|
||||
version = existing.get("version")
|
||||
return str(version) if version is not None else None
|
||||
return None
|
||||
|
||||
|
||||
def create_vikunja_task(title: str, description: str, project_id: int = 0) -> str | None:
|
||||
"""Create a Vikunja task and return its identifier (e.g., OBL-INFRA-531)."""
|
||||
try:
|
||||
token = get_vikunja_token()
|
||||
except click.ClickException:
|
||||
return None
|
||||
|
||||
from devx.api_clients import VikunjaClient
|
||||
|
||||
client = VikunjaClient(VIKUNJA_API_URL, token)
|
||||
task = client.create_task(project_id or VIKUNJA_PROJECT_ID, title=title, description=description)
|
||||
return str(task.get("identifier", ""))
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option("--repo", default="oblachno/infra", help=_("Target repo (owner/name) to create PR in"))
|
||||
@click.option("--package", required=True, help=_("Package name to bump (e.g., grm, sso-bridge)"))
|
||||
@click.option("--new-version", required=True, help=_("New version to pin"))
|
||||
@click.option("--source-repo", required=True, help=_("Source repo that published (owner/name)"))
|
||||
@click.option("--source-run-id", default="", help=_("CI run ID that triggered the publish"))
|
||||
@click.option(
|
||||
"--manifest",
|
||||
"manifest_path",
|
||||
default="",
|
||||
help=_(
|
||||
"Path to a release-manifest JSON in the target repo. When set, the PR updates "
|
||||
"the manifest section named after --package instead of a regex version bump."
|
||||
),
|
||||
)
|
||||
@click.option(
|
||||
"--verify-container",
|
||||
default="",
|
||||
help=_(
|
||||
"Container to verify before opening the PR (owner/name). Resolves the OCI "
|
||||
"digest of the tag matching --new-version and records it in the manifest."
|
||||
),
|
||||
)
|
||||
@click.option(
|
||||
"--source-ref",
|
||||
default="",
|
||||
help=_("Git ref of the producer release (default: v<new-version>), recorded in the manifest."),
|
||||
)
|
||||
@click.option(
|
||||
"--container-tag",
|
||||
default="",
|
||||
help=_(
|
||||
"Container tag to verify with --verify-container (default: --new-version). "
|
||||
"Use when the image tag differs from the release version, e.g. a package "
|
||||
"__version__ tag vs a release git tag."
|
||||
),
|
||||
)
|
||||
@click.option(
|
||||
"--verify-timeout",
|
||||
type=int,
|
||||
default=600,
|
||||
help=_(
|
||||
"Seconds to keep retrying --verify-container while the artifact returns 404 "
|
||||
"(the image build races this step). 0 disables retries."
|
||||
),
|
||||
)
|
||||
@click.option(
|
||||
"--task-project-id",
|
||||
type=int,
|
||||
default=0,
|
||||
help=_(
|
||||
"Vikunja project for the tracking task (default: DEVX_VIKUNJA_PROJECT_ID). "
|
||||
"Use the target repo's project so the generated branch/spec satisfy its validation."
|
||||
),
|
||||
)
|
||||
@click.option("--dry-run", is_flag=True, default=False, help=_("Show what would be done without creating PR"))
|
||||
def cli(
|
||||
repo: str,
|
||||
package: str,
|
||||
new_version: str,
|
||||
source_repo: str,
|
||||
source_run_id: str,
|
||||
manifest_path: str,
|
||||
verify_container: str,
|
||||
source_ref: str,
|
||||
container_tag: str,
|
||||
verify_timeout: int,
|
||||
task_project_id: int,
|
||||
dry_run: bool,
|
||||
) -> None:
|
||||
"""Create an infra PR to bump a pinned dependency version."""
|
||||
token = get_ci_token()
|
||||
if "/" not in repo:
|
||||
raise click.ClickException(_("Invalid repo format: {repo}", repo=repo))
|
||||
owner, repo_name = repo.split("/", 1)
|
||||
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
|
||||
|
||||
# Implements: REQ-1 — verify the producer artifact exists and resolve its
|
||||
# digest before any branch/PR work begins.
|
||||
image_digest = ""
|
||||
if verify_container:
|
||||
if "/" not in verify_container:
|
||||
raise click.ClickException(
|
||||
_("Invalid container format: {container} (expected owner/name)", container=verify_container)
|
||||
)
|
||||
c_owner, c_name = verify_container.split("/", 1)
|
||||
image_tag = container_tag or new_version
|
||||
image_digest = resolve_container_digest(GITEA_API_URL, c_owner, c_name, image_tag, token, verify_timeout)
|
||||
click.echo(
|
||||
_(
|
||||
"[dep-pr] Verified {container}:{version} -> {digest}",
|
||||
container=verify_container,
|
||||
version=image_tag,
|
||||
digest=image_digest,
|
||||
)
|
||||
)
|
||||
|
||||
# Clone the target repo — this tool runs from the *producer* repo's CI,
|
||||
# so every file lookup and git operation must happen inside a clone of
|
||||
# the target repo, not the producer checkout in CWD.
|
||||
workdir = Path(tempfile.mkdtemp(prefix="dep-pr-"))
|
||||
clone_url = f"{GITEA_API_URL.removesuffix('/api/v1')}/{repo}.git"
|
||||
auth_cfg = f"http.extraHeader=Authorization: token {token}"
|
||||
subprocess.run( # nosec B603 B607
|
||||
["git", "-c", auth_cfg, "clone", "--depth", "50", clone_url, str(workdir)],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
|
||||
# Find current pinned version
|
||||
old_version = None
|
||||
changed_file = None
|
||||
if manifest_path:
|
||||
old_version = read_manifest_version(str(workdir / manifest_path), package)
|
||||
changed_file = manifest_path if old_version else None
|
||||
else:
|
||||
for f in [PYPROJECT_PATH, IMAGES_YML_PATH, ROLE_DEFAULTS_PATH]:
|
||||
old_version = find_pinned_version(package, str(workdir / f))
|
||||
if old_version:
|
||||
changed_file = f
|
||||
break
|
||||
|
||||
if not old_version:
|
||||
click.echo(_("[dep-pr] Could not find pinned version for {pkg} in infra repo.", pkg=package))
|
||||
if dry_run:
|
||||
return
|
||||
raise click.ClickException(_("Could not find pinned version for {pkg}", pkg=package))
|
||||
|
||||
if old_version == new_version:
|
||||
click.echo(_("[dep-pr] {pkg} already at {version} — no PR needed.", pkg=package, version=new_version))
|
||||
return
|
||||
|
||||
click.echo(
|
||||
_(
|
||||
"[dep-pr] Bumping {pkg} from {old} to {new} in {file}",
|
||||
pkg=package,
|
||||
old=old_version,
|
||||
new=new_version,
|
||||
file=changed_file,
|
||||
)
|
||||
)
|
||||
|
||||
if dry_run:
|
||||
click.echo(f"[dep-pr] DRY RUN: would update {changed_file} and create PR")
|
||||
return
|
||||
|
||||
# Implements: REQ-1 — create the tracking task in the *target* repo's
|
||||
# Vikunja project so its identifier satisfies the target's branch/PR-title
|
||||
# validation (e.g., OBL-INFRA-N for oblachno/infra).
|
||||
task_title = f"Bump {package} to {new_version}"
|
||||
task_desc = (
|
||||
f"<p>Auto-created dependency bump PR.</p>"
|
||||
f"<p>Package: {package}</p>"
|
||||
f"<p>Version: {old_version} → {new_version}</p>"
|
||||
f"<p>Source: {source_repo} (run #{source_run_id})</p>"
|
||||
)
|
||||
task_id = create_vikunja_task(task_title, task_desc, task_project_id)
|
||||
|
||||
# Create a branch — embed the task ID so target-repo validation accepts it.
|
||||
branch_name = f"deps/{task_id}-{package}-{new_version}" if task_id else f"deps/{package}-{new_version}"
|
||||
base_branch = "master"
|
||||
|
||||
# Check for existing PR (reuse from tools.create_pr)
|
||||
existing = find_existing_pr(client, branch_name)
|
||||
if existing:
|
||||
click.echo(_("[dep-pr] PR already exists: #{number}", number=existing.get("number", "?")))
|
||||
return
|
||||
|
||||
# Create branch via API
|
||||
try:
|
||||
# Implements: REQ-1 — Gitea lacks POST /git/refs; create the branch
|
||||
# from master via the branches API.
|
||||
client._request(
|
||||
"POST",
|
||||
"/branches",
|
||||
json={"new_branch_name": branch_name, "old_branch_name": base_branch},
|
||||
)
|
||||
except APIError as e:
|
||||
if "already exists" in str(e).lower():
|
||||
click.echo(f"[dep-pr] Branch {branch_name} already exists")
|
||||
else:
|
||||
raise click.ClickException(_("Failed to create branch: {error}", error=str(e))) from None
|
||||
|
||||
# Check out the API-created branch inside the target clone. A plain
|
||||
# fetch only populates FETCH_HEAD — fetch into the remote-tracking ref
|
||||
# and force-create the local branch from it.
|
||||
subprocess.run( # nosec B603 B607
|
||||
["git", "fetch", "origin", f"{branch_name}:refs/remotes/origin/{branch_name}"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
cwd=workdir,
|
||||
)
|
||||
subprocess.run( # nosec B603 B607
|
||||
["git", "checkout", "-B", branch_name, f"origin/{branch_name}"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
cwd=workdir,
|
||||
)
|
||||
|
||||
if manifest_path:
|
||||
fields = {
|
||||
"version": new_version,
|
||||
"git_ref": source_ref or f"v{new_version}",
|
||||
"image_tag": container_tag or new_version,
|
||||
"updated_at": datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
}
|
||||
if image_digest:
|
||||
fields["image_digest"] = image_digest
|
||||
if source_run_id:
|
||||
fields["source_run_id"] = source_run_id
|
||||
if not update_manifest(str(workdir / manifest_path), package, fields):
|
||||
raise click.ClickException(_("Failed to update {file}", file=manifest_path))
|
||||
elif not changed_file or not update_pinned_version(str(workdir / changed_file), package, old_version, new_version):
|
||||
raise click.ClickException(_("Failed to update {file}", file=changed_file))
|
||||
|
||||
assert changed_file is not None # nosec B101 — narrowed by the early exit above
|
||||
add_files = [changed_file]
|
||||
if task_id:
|
||||
# Implements: REQ-1 — spec-driven validation requires a spec file.
|
||||
spec_rel = f"docs/specs/{task_id}.md"
|
||||
spec_file = workdir / spec_rel
|
||||
spec_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
spec_file.write_text(
|
||||
f"# {task_id}: {task_title}\n\n"
|
||||
f"## Problem\n\n"
|
||||
f"{source_repo} released {package} {new_version}; this repo pins {old_version}.\n\n"
|
||||
f"## Approach\n\n"
|
||||
f"REQ-1: Update `{changed_file}` to pin {package} {new_version} "
|
||||
f"(auto-generated dependency PR).\n\n"
|
||||
f"## Test Plan\n\n"
|
||||
f"- Producer release CI verified the artifact "
|
||||
f"({source_repo} run #{source_run_id or 'n/a'}).\n\n"
|
||||
f"## Deploy Plan\n\n"
|
||||
f"Merge updates the pin; the next deploy applies it.\n\n"
|
||||
f"## Rollback Plan\n\n"
|
||||
f"Revert the pin bump.\n\n"
|
||||
f"## Acceptance Criteria\n\n"
|
||||
f"- [x] REQ-1: `{changed_file}` pins {package} {new_version}.\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
add_files.append(spec_rel)
|
||||
subprocess.run(["git", "add", *add_files], check=True, cwd=workdir) # nosec B603 B607
|
||||
commit_msg = f"deps: bump {package} from {old_version} to {new_version}"
|
||||
# Implements: REQ-1 — the fresh clone has no git identity in CI
|
||||
# containers; supply it inline (same bot identity as push_badges).
|
||||
subprocess.run( # nosec B603 B607
|
||||
[
|
||||
"git",
|
||||
"-c",
|
||||
"user.name=gitea-actions-bot",
|
||||
"-c",
|
||||
"user.email=actions@oblachno.fyi",
|
||||
"commit",
|
||||
"-m",
|
||||
commit_msg,
|
||||
],
|
||||
check=True,
|
||||
cwd=workdir,
|
||||
)
|
||||
subprocess.run( # nosec B603 B607
|
||||
["git", "-c", auth_cfg, "push", "origin", branch_name],
|
||||
check=True,
|
||||
cwd=workdir,
|
||||
)
|
||||
|
||||
# Create PR directly (dependency PRs have custom titles, not Vikunja-derived)
|
||||
pr_title = f"{task_id}: {task_title}" if task_id else task_title
|
||||
pr_body = (
|
||||
f"## Dependency Bump\n\n"
|
||||
f"Bumps **{package}** from `{old_version}` to `{new_version}`.\n\n"
|
||||
f"- **Source**: {source_repo}\n"
|
||||
f"- **Triggered by**: CI run #{source_run_id}\n"
|
||||
f"- **Changed file**: `{changed_file}`\n\n"
|
||||
f"This PR was auto-created by `devx.ci.create_dependency_pr`.\n"
|
||||
)
|
||||
if task_id:
|
||||
pr_body += f"\nCloses {task_id}"
|
||||
|
||||
pr = client.create_pr(title=pr_title, head=branch_name, base=base_branch, body=pr_body)
|
||||
click.echo(_("[dep-pr] Created PR #{number}: {title}", number=pr.get("number", "?"), title=pr_title))
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
cli()
|
||||
@@ -44,6 +44,7 @@ REQUIRED_SCRIPTS = [
|
||||
"auto_merge.py",
|
||||
"release.py",
|
||||
"publish.py",
|
||||
"pr_review.py",
|
||||
"notify_failure.py",
|
||||
"post_merge.py",
|
||||
"classify_changes.py",
|
||||
@@ -51,6 +52,7 @@ REQUIRED_SCRIPTS = [
|
||||
"detect_release_commit.py",
|
||||
"push_badges.py",
|
||||
"distribute_molecule.py",
|
||||
"molecule_ci_guard.py",
|
||||
"validate_commit_msg.py",
|
||||
]
|
||||
|
||||
|
||||
@@ -1,126 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# Implements: REQ-3
|
||||
"""Detect changed Ansible roles and output fast molecule test commands.
|
||||
|
||||
Reuses ``devx.molecule.molecule_changed`` for role detection (which handles
|
||||
playbook→role mapping and shared infrastructure paths).
|
||||
|
||||
Fast molecule = full ``molecule test`` for every scenario of each changed
|
||||
role, on the scenario's configured platform. "Fast" means *scoped* (only
|
||||
affected roles, single platform) — never skipped phases: create, converge,
|
||||
idempotence (when the scenario defines it), verify, and destroy all run,
|
||||
exactly as ``scripts/run_molecule_scenario.py`` executes them in CI.
|
||||
|
||||
Used in pre-merge CI to get quick feedback on Ansible changes without
|
||||
running the full all-roles/all-platforms suite (which runs nightly).
|
||||
|
||||
Usage:
|
||||
python -m devx.ci.fast_molecule --base origin/master --head HEAD
|
||||
|
||||
Outputs the list of changed roles and the molecule commands CI runs.
|
||||
In CI, pass ``--github-output`` to set ``fast-molecule-roles`` (space-
|
||||
separated) and ``fast-molecule-needed`` (true/false) for downstream steps.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from devx.ci._shared import write_github_output
|
||||
from devx.i18n import _
|
||||
from devx.molecule.molecule_changed import detect_changed_roles, get_changed_files
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
def get_molecule_scenarios(role_name: str, roles_dir: str = "ansible/roles") -> list[str]:
|
||||
"""Get list of molecule scenario names for a role."""
|
||||
mol_dir = Path(roles_dir) / role_name / "molecule"
|
||||
if not mol_dir.is_dir():
|
||||
return []
|
||||
scenarios = []
|
||||
for p in mol_dir.iterdir():
|
||||
if p.is_dir() and (p / "molecule.yml").exists():
|
||||
scenarios.append(p.name)
|
||||
return sorted(scenarios)
|
||||
|
||||
|
||||
def build_molecule_commands(
|
||||
roles: set[str],
|
||||
roles_dir: str = "ansible/roles",
|
||||
platform: str = "ubuntu-2604",
|
||||
) -> list[str]:
|
||||
"""Build the molecule commands CI executes for changed roles.
|
||||
|
||||
Emits exactly what ``scripts/run_molecule_scenario.py`` runs:
|
||||
``molecule test -s <scenario>`` — the full sequence (create, converge,
|
||||
idempotence, verify, destroy). The ``platform`` argument is accepted
|
||||
for interface stability but is informational: the scenario's
|
||||
``molecule.yml`` selects the platform, and CI distributes scenarios so
|
||||
each runs on a single platform.
|
||||
"""
|
||||
commands: list[str] = []
|
||||
for role in sorted(roles):
|
||||
for scenario in get_molecule_scenarios(role, roles_dir):
|
||||
commands.append(f"molecule test -s {scenario}")
|
||||
return commands
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option("--base", default="origin/master", help=_("Base ref for diff"))
|
||||
@click.option("--head", default="HEAD", help=_("Head ref for diff"))
|
||||
@click.option("--roles-dir", default="ansible/roles", help=_("Directory containing Ansible roles"))
|
||||
@click.option("--platform", default="ubuntu-2604", help=_("Single platform to test against"))
|
||||
@click.option(
|
||||
"--github-output",
|
||||
"github_output",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help=_("Write results to $GITHUB_OUTPUT"),
|
||||
)
|
||||
def cli(
|
||||
base: str,
|
||||
head: str,
|
||||
roles_dir: str,
|
||||
platform: str,
|
||||
github_output: bool,
|
||||
) -> None:
|
||||
"""Detect changed roles and output fast molecule test commands."""
|
||||
# Use molecule_changed for role detection (handles playbooks, shared infra)
|
||||
files = get_changed_files(base)
|
||||
if not files:
|
||||
click.echo("[fast-molecule] No files changed.")
|
||||
if github_output:
|
||||
write_github_output("fast-molecule-needed", "false")
|
||||
write_github_output("fast-molecule-roles", "")
|
||||
return
|
||||
|
||||
roles = detect_changed_roles(files, roles_dir)
|
||||
if not roles:
|
||||
click.echo("[fast-molecule] No Ansible roles changed.")
|
||||
if github_output:
|
||||
write_github_output("fast-molecule-needed", "false")
|
||||
write_github_output("fast-molecule-roles", "")
|
||||
return
|
||||
|
||||
commands = build_molecule_commands(roles, roles_dir, platform)
|
||||
|
||||
if github_output:
|
||||
write_github_output("fast-molecule-needed", "true" if commands else "false")
|
||||
write_github_output("fast-molecule-roles", " ".join(sorted(roles)))
|
||||
|
||||
click.echo(_("[fast-molecule] Changed roles: {roles}", roles=", ".join(sorted(roles))))
|
||||
if not commands:
|
||||
click.echo("[fast-molecule] No molecule scenarios found for changed roles.")
|
||||
return
|
||||
|
||||
click.echo(f"[fast-molecule] {len(commands)} scenario(s) to run:")
|
||||
for cmd in commands:
|
||||
click.echo(f" {cmd}")
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
cli()
|
||||
@@ -1,9 +1,10 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run integration tests with cross-runner failure detection.
|
||||
|
||||
Wraps ``pytest`` with Gitea API polling. If any other integration-tests
|
||||
matrix runner reports failure, the current pytest subprocess is killed
|
||||
and this runner exits early with code 1.
|
||||
Wraps ``pytest`` with the same Gitea API polling mechanism used by
|
||||
``molecule_ci_guard``. If any other integration-tests matrix runner
|
||||
reports failure, the current pytest subprocess is killed and this runner
|
||||
exits early with code 1.
|
||||
|
||||
Usage::
|
||||
|
||||
@@ -34,62 +35,17 @@ import threading
|
||||
import time
|
||||
|
||||
import click
|
||||
import requests
|
||||
|
||||
from devx.config import REPO_NAME, REPO_OWNER
|
||||
from devx.i18n import _
|
||||
from devx.molecule.molecule_ci_guard import (
|
||||
poll_for_other_failures,
|
||||
)
|
||||
from devx.tokens import get_ci_token
|
||||
|
||||
POLL_INTERVAL = 10
|
||||
|
||||
|
||||
def get_running_jobs(gitea_url: str, owner: str, repo: str, token: str, run_id: int) -> list[dict]:
|
||||
"""Return jobs for the given workflow run."""
|
||||
url = f"{gitea_url}/api/v1/repos/{owner}/{repo}/actions/runs/{run_id}/jobs"
|
||||
headers = {"Authorization": f"token {token}"}
|
||||
response = requests.get(url, headers=headers, timeout=30)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
return data.get("jobs", [])
|
||||
|
||||
|
||||
def any_other_runner_failed(jobs: list[dict], current_job_name: str, current_index: int) -> bool:
|
||||
"""Return True if any other matrix job has failed."""
|
||||
for job in jobs:
|
||||
name = job.get("name", "")
|
||||
if not name.startswith(current_job_name):
|
||||
continue
|
||||
if name == f"{current_job_name} ({current_index})" or name == current_job_name:
|
||||
continue
|
||||
if job.get("conclusion") == "failure":
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def poll_for_other_failures(
|
||||
gitea_url: str,
|
||||
owner: str,
|
||||
repo: str,
|
||||
token: str,
|
||||
run_id: int,
|
||||
job_name: str,
|
||||
current_index: int,
|
||||
stop_event: threading.Event,
|
||||
failed_event: threading.Event,
|
||||
) -> None:
|
||||
"""Background thread: poll API and signal if another runner fails."""
|
||||
while not stop_event.is_set():
|
||||
try:
|
||||
jobs = get_running_jobs(gitea_url, owner, repo, token, run_id)
|
||||
if any_other_runner_failed(jobs, job_name, current_index):
|
||||
click.echo(_("Another runner failed. Stopping this runner early."))
|
||||
failed_event.set()
|
||||
return
|
||||
except requests.RequestException as exc:
|
||||
click.echo(_("API poll warning: {exc}", exc=exc))
|
||||
stop_event.wait(POLL_INTERVAL)
|
||||
|
||||
|
||||
@click.command(context_settings={"ignore_unknown_options": True})
|
||||
@click.argument("pytest_args", nargs=-1, type=click.UNPROCESSED, required=True)
|
||||
def cli(pytest_args: tuple[str, ...]) -> None:
|
||||
|
||||
@@ -1,126 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# Implements: REQ-4
|
||||
"""Check if the nightly CI gate has passed; block staging deploys if it failed.
|
||||
|
||||
The nightly gate stores its status as a Gitea Actions repository variable
|
||||
named ``NIGHTLY_STATUS`` on the infra repo. Values:
|
||||
|
||||
- ``passed`` — nightly molecule + staging deploy + integration tests passed.
|
||||
- ``failed:<run_id>`` — nightly failed. Staging deploys are blocked until
|
||||
the nightly passes again.
|
||||
- (not set) — nightly hasn't run yet. First deploy is allowed (bootstrap).
|
||||
|
||||
Usage:
|
||||
python -m devx.ci.nightly_gate --repo oblachno/infra --action check
|
||||
python -m devx.ci.nightly_gate --repo oblachno/infra --action set-passed --run-id 12345
|
||||
python -m devx.ci.nightly_gate --repo oblachno/infra --action set-failed --run-id 12345
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import click
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from devx.api_clients import GiteaClient
|
||||
from devx.ci._shared import write_github_output
|
||||
from devx.config import GITEA_API_URL
|
||||
from devx.i18n import _
|
||||
from devx.tokens import get_ci_token
|
||||
|
||||
load_dotenv()
|
||||
|
||||
NIGHTLY_STATUS_VAR = "NIGHTLY_STATUS"
|
||||
|
||||
|
||||
def get_nightly_status(client: GiteaClient) -> str:
|
||||
"""Get the nightly status variable. Returns empty string if not set."""
|
||||
val = client.get_repo_variable(NIGHTLY_STATUS_VAR)
|
||||
return val or ""
|
||||
|
||||
|
||||
def set_nightly_status(client: GiteaClient, status: str) -> None:
|
||||
"""Set the nightly status variable."""
|
||||
client.set_repo_variable(NIGHTLY_STATUS_VAR, status)
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option("--repo", required=True, help=_("Repository in owner/name format"))
|
||||
@click.option(
|
||||
"--action",
|
||||
type=click.Choice(["check", "set-passed", "set-failed"]),
|
||||
required=True,
|
||||
help=_("Action to perform"),
|
||||
)
|
||||
@click.option("--run-id", default="", help=_("CI run ID (for set-failed/set-passed)"))
|
||||
@click.option(
|
||||
"--github-output",
|
||||
"github_output",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help=_("Write results to $GITHUB_OUTPUT"),
|
||||
)
|
||||
def cli(repo: str, action: str, run_id: str, github_output: bool) -> None:
|
||||
"""Check or set the nightly CI gate status."""
|
||||
token = get_ci_token()
|
||||
if "/" not in repo:
|
||||
raise click.ClickException(_("Invalid repo format: {repo}. Expected owner/name.", repo=repo))
|
||||
owner, repo_name = repo.split("/", 1)
|
||||
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
|
||||
|
||||
if action == "check":
|
||||
status = get_nightly_status(client)
|
||||
if not status:
|
||||
# Bootstrap: no nightly has run yet, allow deploy
|
||||
click.echo("[nightly-gate] No nightly status set — allowing deploy (bootstrap).")
|
||||
if github_output:
|
||||
write_github_output("nightly-gate-passed", "true")
|
||||
write_github_output("nightly-status", "")
|
||||
return
|
||||
|
||||
if status.startswith("passed"):
|
||||
click.echo("[nightly-gate] Nightly passed. Deploy allowed.")
|
||||
if github_output:
|
||||
write_github_output("nightly-gate-passed", "true")
|
||||
write_github_output("nightly-status", status)
|
||||
elif status.startswith("failed"):
|
||||
run_part = status.split(":", 1)[1] if ":" in status else ""
|
||||
run_link = f" (run #{run_part})" if run_part else ""
|
||||
click.echo(
|
||||
_(
|
||||
"[nightly-gate] Nightly FAILED{run}. Staging deploys are blocked until nightly passes.",
|
||||
run=run_link,
|
||||
),
|
||||
err=True,
|
||||
)
|
||||
if github_output:
|
||||
write_github_output("nightly-gate-passed", "false")
|
||||
write_github_output("nightly-status", status)
|
||||
raise click.ClickException(_("Nightly gate failed — staging deploy blocked."))
|
||||
else:
|
||||
click.echo(
|
||||
_(
|
||||
"[nightly-gate] Unknown nightly status: {status} — blocking deploy (fail closed).",
|
||||
status=status,
|
||||
),
|
||||
err=True,
|
||||
)
|
||||
if github_output:
|
||||
write_github_output("nightly-gate-passed", "false")
|
||||
write_github_output("nightly-status", status)
|
||||
raise click.ClickException(_("Unknown nightly status — staging deploy blocked."))
|
||||
|
||||
elif action == "set-passed":
|
||||
set_nightly_status(client, f"passed:{run_id}" if run_id else "passed")
|
||||
click.echo(_("[nightly-gate] Set NIGHTLY_STATUS=passed{run}", run=f":{run_id}" if run_id else ""))
|
||||
if github_output:
|
||||
write_github_output("nightly-status", f"passed:{run_id}" if run_id else "passed")
|
||||
|
||||
elif action == "set-failed":
|
||||
set_nightly_status(client, f"failed:{run_id}" if run_id else "failed")
|
||||
click.echo(_("[nightly-gate] Set NIGHTLY_STATUS=failed{run}", run=f":{run_id}" if run_id else ""))
|
||||
if github_output:
|
||||
write_github_output("nightly-status", f"failed:{run_id}" if run_id else "failed")
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
cli()
|
||||
@@ -0,0 +1,715 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Automated PR review: check architecture compliance, best practices, and quality.
|
||||
|
||||
Fetches the PR diff via the Gitea API, runs a series of automated checks,
|
||||
and posts a structured review using GiteaClient.create_review.
|
||||
|
||||
Checks performed:
|
||||
1. Architecture compliance — no business logic in CLI, no direct subprocess
|
||||
calls outside executor, no hardcoded config that should be in config.py
|
||||
2. Best practices — no bare except, no print() (use click.echo), no TODO/FIXME
|
||||
left in merged code, no functions > 50 lines
|
||||
3. Security — no secrets in code, no shell=True, no eval/exec
|
||||
4. i18n — no raw English strings in click.echo() without _() wrapper
|
||||
5. Resource management — no open() without with statement, no subprocess without cleanup
|
||||
6. Documentation — new CLI commands documented, new modules in architecture.md
|
||||
7. Test coverage — 100% enforced by pytest-cov (checked in quality job)
|
||||
8. Commit conventions — conventional commit format on branch commits
|
||||
|
||||
Usage:
|
||||
CI_GITEA_API_TOKEN=<token> [REVIEWER_GITEA_API_TOKEN=<token>] python3 -m devx.ci.pr_review <pr_number> <owner/repo>
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
import click
|
||||
from dotenv import load_dotenv # pyright: ignore[reportMissingImports,reportUnknownVariableType]
|
||||
|
||||
from devx.api_clients import GiteaClient
|
||||
from devx.config import GITEA_API_URL
|
||||
from devx.exceptions import APIError
|
||||
from devx.i18n import _
|
||||
from devx.tokens import get_ci_token, get_reviewer_token
|
||||
|
||||
load_dotenv()
|
||||
|
||||
# Files that are exempt from certain checks
|
||||
WORKFLOW_ONLY_SUFFIXES = (".yml", ".yaml", ".md", ".json", ".toml", ".cfg", ".ini", ".txt")
|
||||
PYTHON_SUFFIX = ".py"
|
||||
|
||||
# Architecture rules
|
||||
CLI_FILE = "src/devx/cli.py"
|
||||
EXECUTOR_FILE = "src/devx/executor.py"
|
||||
CONFIG_FILE = "src/devx/config.py"
|
||||
|
||||
# Patterns that indicate business logic in CLI (should be in runner_manager.py)
|
||||
BUSINESS_LOGIC_IN_CLI = [
|
||||
(r"subprocess\.(run|call|Popen|check_output|check_call)", "subprocess call in CLI — delegate to executor.py"),
|
||||
(r"\bos\.system\b", "os.system call in CLI — delegate to executor.py"),
|
||||
(r"\bansible-playbook\b", "ansible-playbook reference in CLI — delegate to executor.py"),
|
||||
]
|
||||
|
||||
# Patterns that indicate bad practices
|
||||
BAD_PRACTICES = [
|
||||
(r"\bprint\s*\(", "print() found — use click.echo() for user output"),
|
||||
(r"\beval\s*\(", "eval() found — security risk, avoid dynamic code execution"),
|
||||
(r"\bexec\s*\(", "exec() found — security risk, avoid dynamic code execution"),
|
||||
(r"shell\s*=\s*True", "shell=True found — security risk, use shell=False with list args"),
|
||||
(r"except\s*:", "bare except found — catch specific exceptions"),
|
||||
(r"except\s+Exception\s*:", "broad Exception catch — catch specific exceptions"),
|
||||
(r"#\s*(TODO|FIXME|HACK|XXX)", "TODO/FIXME found — resolve before merging"),
|
||||
]
|
||||
|
||||
# Patterns for hardcoded config values that should be in config.py
|
||||
HARDCODED_CONFIG = [
|
||||
(r"https?://[a-z]+\.[a-z]+\.[a-z]+", "hardcoded URL — move to config.py with env var override"),
|
||||
]
|
||||
|
||||
|
||||
@dataclass
|
||||
class ReviewResult:
|
||||
"""Result of automated review checks."""
|
||||
|
||||
issues: list[dict[str, Any]] = field(default_factory=list)
|
||||
summary: list[str] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def has_issues(self) -> bool:
|
||||
return bool(self.issues)
|
||||
|
||||
def add_issue(self, file_path: str, line: int, message: str, severity: str = "warning") -> None:
|
||||
self.issues.append(
|
||||
{
|
||||
"path": file_path,
|
||||
"body": f"[{severity}] {message}",
|
||||
"new_position": line,
|
||||
}
|
||||
)
|
||||
|
||||
def add_summary(self, text: str) -> None:
|
||||
self.summary.append(text)
|
||||
|
||||
|
||||
def is_python_file(path: str) -> bool:
|
||||
"""Check if a file is a Python source file."""
|
||||
return path.endswith(PYTHON_SUFFIX) and not path.startswith("tests/")
|
||||
|
||||
|
||||
def is_workflow_only(path: str) -> bool:
|
||||
"""Check if a file is workflow/config/docs only (not Python source)."""
|
||||
return path.endswith(WORKFLOW_ONLY_SUFFIXES) or path.startswith((".gitea/", "docs/", "ansible/"))
|
||||
|
||||
|
||||
def check_architecture_compliance(files: list[dict[str, Any]], result: ReviewResult) -> None:
|
||||
"""Check that changes follow the documented architecture."""
|
||||
for f in files:
|
||||
path = f.get("filename", "")
|
||||
if not is_python_file(path):
|
||||
continue
|
||||
|
||||
patch = f.get("patch", "")
|
||||
if not patch:
|
||||
continue
|
||||
|
||||
lines = patch.split("\n")
|
||||
current_line = 0
|
||||
for line in lines:
|
||||
if line.startswith("@@"):
|
||||
match = re.search(r"\+(\d+)", line)
|
||||
if match:
|
||||
current_line = int(match.group(1)) - 1
|
||||
continue
|
||||
if line.startswith("+") and not line.startswith("+++"):
|
||||
current_line += 1
|
||||
content = line[1:]
|
||||
|
||||
# Check for business logic in CLI
|
||||
if path == CLI_FILE:
|
||||
for pattern, msg in BUSINESS_LOGIC_IN_CLI:
|
||||
if re.search(pattern, content):
|
||||
result.add_issue(path, current_line, msg, "error")
|
||||
|
||||
if not result.issues:
|
||||
result.add_summary("- Architecture compliance: OK")
|
||||
|
||||
|
||||
def check_best_practices(files: list[dict[str, Any]], result: ReviewResult) -> None:
|
||||
"""Check for common code quality issues."""
|
||||
for f in files:
|
||||
path = f.get("filename", "")
|
||||
if not is_python_file(path):
|
||||
continue
|
||||
|
||||
patch = f.get("patch", "")
|
||||
if not patch:
|
||||
continue
|
||||
|
||||
lines = patch.split("\n")
|
||||
current_line = 0
|
||||
for line in lines:
|
||||
if line.startswith("@@"):
|
||||
match = re.search(r"\+(\d+)", line)
|
||||
if match:
|
||||
current_line = int(match.group(1)) - 1
|
||||
continue
|
||||
if line.startswith("+") and not line.startswith("+++"):
|
||||
current_line += 1
|
||||
content = line[1:]
|
||||
|
||||
for pattern, msg in BAD_PRACTICES:
|
||||
if re.search(pattern, content):
|
||||
result.add_issue(path, current_line, msg, "warning")
|
||||
|
||||
if not any(i["body"].startswith("[warning]") for i in result.issues):
|
||||
result.add_summary("- Best practices: OK")
|
||||
|
||||
|
||||
def check_security(files: list[dict[str, Any]], result: ReviewResult) -> None:
|
||||
"""Check for security issues in changed files."""
|
||||
for f in files:
|
||||
path = f.get("filename", "")
|
||||
if not is_python_file(path):
|
||||
continue
|
||||
|
||||
patch = f.get("patch", "")
|
||||
if not patch:
|
||||
continue
|
||||
|
||||
lines = patch.split("\n")
|
||||
current_line = 0
|
||||
for line in lines:
|
||||
if line.startswith("@@"):
|
||||
match = re.search(r"\+(\d+)", line)
|
||||
if match:
|
||||
current_line = int(match.group(1)) - 1
|
||||
continue
|
||||
if line.startswith("+") and not line.startswith("+++"):
|
||||
current_line += 1
|
||||
content = line[1:]
|
||||
|
||||
# Check for hardcoded secrets
|
||||
secret_re = r'(token|password|secret|key)\s*=\s*["\'][^"\']{8,}["\']' # nosec B105
|
||||
is_secret = re.search(secret_re, content, re.IGNORECASE)
|
||||
is_comment = content.strip().startswith("#")
|
||||
is_example = "your-" in content or "example" in content
|
||||
if is_secret and not is_comment and not is_example:
|
||||
result.add_issue(
|
||||
path,
|
||||
current_line,
|
||||
"potential hardcoded secret — use environment variable",
|
||||
"error",
|
||||
)
|
||||
|
||||
if not any(i["body"].startswith("[error]") and "secret" in i["body"] for i in result.issues):
|
||||
result.add_summary("- Security: OK")
|
||||
|
||||
|
||||
def check_i18n(files: list[dict[str, Any]], result: ReviewResult) -> None:
|
||||
"""Check that user-facing strings are wrapped in _().
|
||||
|
||||
Detects ``click.echo()`` calls with raw string literals that are not
|
||||
wrapped in ``_()``. Only checks ``src/`` files, not tests or scripts.
|
||||
"""
|
||||
# Pattern: click.echo("...") or click.echo(f"...") without _() wrapper
|
||||
raw_echo_re = re.compile(r'click\.echo\s*\(\s*["\']([^"\']+)["\']')
|
||||
raw_fstring_re = re.compile(r'click\.echo\s*\(\s*f["\']')
|
||||
# Also check click.ClickException and raise with string
|
||||
raw_exception_re = re.compile(r'click\.ClickException\s*\(\s*["\']([^"\']+)["\']')
|
||||
|
||||
for f in files:
|
||||
path = f.get("filename", "")
|
||||
if not is_python_file(path) or not path.startswith("src/"):
|
||||
continue
|
||||
|
||||
patch = f.get("patch", "")
|
||||
if not patch:
|
||||
continue
|
||||
|
||||
lines = patch.split("\n")
|
||||
current_line = 0
|
||||
for line in lines:
|
||||
if line.startswith("@@"):
|
||||
match = re.search(r"\+(\d+)", line)
|
||||
if match:
|
||||
current_line = int(match.group(1)) - 1
|
||||
continue
|
||||
if line.startswith("+") and not line.startswith("+++"):
|
||||
current_line += 1
|
||||
content = line[1:]
|
||||
|
||||
# Skip comments and docstrings
|
||||
stripped = content.strip()
|
||||
if stripped.startswith("#") or stripped.startswith('"""') or stripped.startswith("'''"):
|
||||
continue
|
||||
|
||||
# Check for raw strings in click.echo without _()
|
||||
for regex, msg in [
|
||||
(raw_echo_re, "click.echo() with raw string — wrap in _() for i18n"),
|
||||
(raw_fstring_re, "click.echo() with f-string — wrap in _() for i18n"),
|
||||
(raw_exception_re, "ClickException with raw string — wrap in _() for i18n"),
|
||||
]:
|
||||
if regex.search(content):
|
||||
result.add_issue(path, current_line, msg, "warning")
|
||||
|
||||
if not any("i18n" in i["body"] for i in result.issues):
|
||||
result.add_summary("- i18n: OK")
|
||||
|
||||
|
||||
def check_resource_management(files: list[dict[str, Any]], result: ReviewResult) -> None:
|
||||
"""Check for resource leaks: open() without with, subprocess without cleanup.
|
||||
|
||||
Detects:
|
||||
- ``open()`` calls not in a ``with`` statement
|
||||
- ``subprocess.Popen()`` without ``.wait()`` or ``.communicate()``
|
||||
"""
|
||||
# Pattern: open("...") not preceded by "with" on the same line
|
||||
open_re = re.compile(r"(?<!with\s)\bopen\s*\(")
|
||||
popen_re = re.compile(r"subprocess\.Popen\s*\(")
|
||||
|
||||
for f in files:
|
||||
path = f.get("filename", "")
|
||||
if not is_python_file(path):
|
||||
continue
|
||||
|
||||
patch = f.get("patch", "")
|
||||
if not patch:
|
||||
continue
|
||||
|
||||
lines = patch.split("\n")
|
||||
current_line = 0
|
||||
for line in lines:
|
||||
if line.startswith("@@"):
|
||||
match = re.search(r"\+(\d+)", line)
|
||||
if match:
|
||||
current_line = int(match.group(1)) - 1
|
||||
continue
|
||||
if line.startswith("+") and not line.startswith("+++"):
|
||||
current_line += 1
|
||||
content = line[1:]
|
||||
|
||||
# Skip comments
|
||||
if content.strip().startswith("#"):
|
||||
continue
|
||||
|
||||
# Check for open() without with
|
||||
if open_re.search(content) and "with " not in content:
|
||||
result.add_issue(
|
||||
path, current_line, "open() without with statement — potential resource leak", "warning"
|
||||
)
|
||||
|
||||
# Check for Popen without communicate/wait on same line
|
||||
if popen_re.search(content) and ".communicate" not in content and ".wait" not in content:
|
||||
result.add_issue(
|
||||
path,
|
||||
current_line,
|
||||
"subprocess.Popen() without immediate .communicate() or .wait() — ensure cleanup",
|
||||
"warning",
|
||||
)
|
||||
|
||||
if not any("resource" in i["body"].lower() for i in result.issues):
|
||||
result.add_summary("- Resource management: OK")
|
||||
|
||||
|
||||
def check_function_length(files: list[dict[str, Any]], result: ReviewResult) -> None:
|
||||
"""Check that no new function is excessively long (> 50 lines)."""
|
||||
for f in files:
|
||||
path = f.get("filename", "")
|
||||
if not is_python_file(path):
|
||||
continue
|
||||
|
||||
patch = f.get("patch", "")
|
||||
if not patch:
|
||||
continue
|
||||
|
||||
# Count consecutive added lines within a function
|
||||
lines = patch.split("\n")
|
||||
current_line = 0
|
||||
func_start = 0
|
||||
func_name = ""
|
||||
added_in_func = 0
|
||||
|
||||
for line in lines:
|
||||
if line.startswith("@@"):
|
||||
if func_name and added_in_func > 50:
|
||||
result.add_issue(
|
||||
path,
|
||||
func_start,
|
||||
f"function '{func_name}' adds {added_in_func} lines — consider splitting (> 50 lines)",
|
||||
"warning",
|
||||
)
|
||||
match = re.search(r"\+(\d+)", line)
|
||||
if match:
|
||||
current_line = int(match.group(1)) - 1
|
||||
func_name = ""
|
||||
added_in_func = 0
|
||||
continue
|
||||
if line.startswith("+") and not line.startswith("+++"):
|
||||
current_line += 1
|
||||
content = line[1:]
|
||||
func_match = re.match(r"\s*def\s+(\w+)\s*\(", content)
|
||||
if func_match:
|
||||
if func_name and added_in_func > 50:
|
||||
result.add_issue(
|
||||
path,
|
||||
func_start,
|
||||
f"function '{func_name}' adds {added_in_func} lines — consider splitting (> 50 lines)",
|
||||
"warning",
|
||||
)
|
||||
func_name = func_match.group(1)
|
||||
func_start = current_line
|
||||
added_in_func = 0
|
||||
else:
|
||||
added_in_func += 1
|
||||
elif line.startswith(" ") or line.startswith("-"):
|
||||
pass # context or removed line
|
||||
|
||||
# Check last function
|
||||
if func_name and added_in_func > 50:
|
||||
result.add_issue(
|
||||
path,
|
||||
func_start,
|
||||
f"function '{func_name}' adds {added_in_func} lines — consider splitting (> 50 lines)",
|
||||
"warning",
|
||||
)
|
||||
|
||||
|
||||
def check_documentation(files: list[dict[str, Any]], result: ReviewResult) -> None:
|
||||
"""Check that documentation is updated for relevant changes."""
|
||||
has_src_changes = any(
|
||||
is_python_file(f.get("filename", "")) and f.get("filename", "").startswith("src/") for f in files
|
||||
)
|
||||
has_doc_changes = any(
|
||||
f.get("filename", "").startswith("docs/") or f.get("filename", "") in ("README.md", "AGENTS.md", "CHANGELOG.md")
|
||||
for f in files
|
||||
)
|
||||
has_ansible_changes = any(f.get("filename", "").startswith("ansible/") for f in files)
|
||||
has_tofu_changes = any(f.get("filename", "").startswith("tofu/") for f in files)
|
||||
has_workflow_changes = any(f.get("filename", "").startswith(".gitea/") for f in files)
|
||||
|
||||
# Check for TODO/FIXME in changed docs
|
||||
todo_issues: list[str] = []
|
||||
for f in files:
|
||||
filename = f.get("filename", "")
|
||||
if filename.endswith(".md") and filename.startswith(("docs/", "README", "AGENTS")):
|
||||
# Can't check file content from PR API easily, but flag if patch adds TODO
|
||||
patch = f.get("patch", "")
|
||||
if patch and re.search(r"^\+.*\b(TODO|FIXME|HACK|XXX)\b", patch, re.IGNORECASE):
|
||||
todo_issues.append(f"{filename}: new TODO/FIXME added in documentation")
|
||||
|
||||
if has_src_changes and not has_doc_changes:
|
||||
result.add_summary("- Documentation: WARNING — source files changed but no docs updated")
|
||||
elif has_ansible_changes and not has_doc_changes:
|
||||
result.add_summary("- Documentation: WARNING — Ansible role changed but no docs updated")
|
||||
elif has_tofu_changes and not has_doc_changes:
|
||||
result.add_summary("- Documentation: WARNING — OpenTofu changes but no docs updated")
|
||||
elif has_workflow_changes and not has_doc_changes:
|
||||
result.add_summary("- Documentation: INFO — workflow changes (consider updating CI docs if behavior changed)")
|
||||
else:
|
||||
result.add_summary("- Documentation: OK")
|
||||
|
||||
if todo_issues:
|
||||
for issue in todo_issues:
|
||||
result.add_summary(f"- Documentation: WARNING — {issue}")
|
||||
|
||||
|
||||
def check_test_coverage(files: list[dict[str, Any]], result: ReviewResult) -> None:
|
||||
"""Check that tests are updated for source changes."""
|
||||
has_src_changes = any(
|
||||
is_python_file(f.get("filename", "")) and f.get("filename", "").startswith("src/") for f in files
|
||||
)
|
||||
has_test_changes = any(f.get("filename", "").startswith("tests/") for f in files)
|
||||
|
||||
if has_src_changes and not has_test_changes:
|
||||
result.add_summary("- Tests: WARNING — source files changed but no test files updated")
|
||||
else:
|
||||
result.add_summary("- Tests: OK")
|
||||
|
||||
|
||||
def check_commit_conventions(client: GiteaClient, pr_number: str, result: ReviewResult) -> None:
|
||||
"""Check that PR commits follow conventional commit format.
|
||||
|
||||
Verifies that at least one commit on the PR branch matches the
|
||||
conventional commit pattern (type: description). Merge commits
|
||||
and revert commits are exempt.
|
||||
"""
|
||||
try:
|
||||
commits = client.get_pr_commits(pr_number)
|
||||
except APIError as e:
|
||||
result.add_summary(f"- Commit conventions: ERROR — could not fetch commits: {e.message}")
|
||||
return
|
||||
|
||||
if not commits:
|
||||
result.add_summary("- Commit conventions: OK (no commits to check)")
|
||||
return
|
||||
|
||||
from devx.config import CONVENTIONAL_RE
|
||||
|
||||
has_conventional = False
|
||||
non_conventional: list[str] = []
|
||||
|
||||
for commit in commits:
|
||||
commit_info = commit.get("commit", {})
|
||||
message = str(commit_info.get("message", "") if isinstance(commit_info, dict) else "").split("\n")[0]
|
||||
# Skip merge commits and revert commits
|
||||
if message.startswith(("Merge", "Revert")):
|
||||
continue
|
||||
if CONVENTIONAL_RE.match(message):
|
||||
has_conventional = True
|
||||
else:
|
||||
non_conventional.append(message[:60])
|
||||
|
||||
if has_conventional:
|
||||
result.add_summary("- Commit conventions: OK")
|
||||
elif non_conventional:
|
||||
result.add_summary(
|
||||
f"- Commit conventions: WARNING — no conventional commit found. "
|
||||
f"Non-conventional commits: {', '.join(non_conventional[:3])}"
|
||||
)
|
||||
else:
|
||||
result.add_summary("- Commit conventions: OK (all commits are merges/reverts)")
|
||||
|
||||
|
||||
def run_review(client: GiteaClient, pr_number: str) -> ReviewResult:
|
||||
"""Run all review checks and return the result."""
|
||||
result = ReviewResult()
|
||||
|
||||
try:
|
||||
files = client.get_pr_files(pr_number)
|
||||
except APIError as e:
|
||||
result.add_summary(f"- ERROR: Could not fetch PR files: {e.message}")
|
||||
return result
|
||||
|
||||
if not files:
|
||||
result.add_summary("- No files changed in this PR")
|
||||
return result
|
||||
|
||||
# Run all checks
|
||||
check_architecture_compliance(files, result)
|
||||
check_best_practices(files, result)
|
||||
check_security(files, result)
|
||||
check_i18n(files, result)
|
||||
check_resource_management(files, result)
|
||||
check_function_length(files, result)
|
||||
check_documentation(files, result)
|
||||
check_test_coverage(files, result)
|
||||
check_commit_conventions(client, pr_number, result)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def build_review_body(result: ReviewResult) -> str:
|
||||
"""Build the review body text from the review result."""
|
||||
lines = ["## Automated PR Review", ""]
|
||||
|
||||
for item in result.summary:
|
||||
lines.append(item)
|
||||
|
||||
if result.issues:
|
||||
lines.append("")
|
||||
lines.append(f"**{len(result.issues)} issue(s) found:**")
|
||||
lines.append("")
|
||||
for issue in result.issues:
|
||||
lines.append(f"- `{issue['path']}:{issue['new_position']}` — {issue['body']}")
|
||||
else:
|
||||
lines.append("")
|
||||
lines.append("No issues found by automated checks.")
|
||||
|
||||
lines.append("")
|
||||
lines.append("---")
|
||||
lines.append("**Auto-merge:** If all CI checks pass, this PR will be merged automatically.")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def post_review(client: GiteaClient, pr_number: str, result: ReviewResult) -> dict[str, Any]:
|
||||
"""Post the review to the PR.
|
||||
|
||||
Uses REQUEST_CHANGES when issues are found, COMMENT otherwise.
|
||||
Never uses APPROVE — the bot shares the PR author's token, so
|
||||
Gitea rejects self-approval. The actual APPROVE must come from
|
||||
the manual review step.
|
||||
"""
|
||||
body = build_review_body(result)
|
||||
event = "REQUEST_CHANGES" if result.has_issues else "COMMENT"
|
||||
comments = result.issues if result.has_issues else []
|
||||
|
||||
return client.create_review(pr_number, event=event, body=body, comments=comments)
|
||||
|
||||
|
||||
def _post_manual_review(
|
||||
client: GiteaClient,
|
||||
pr_number: str,
|
||||
event: str,
|
||||
body: str | None,
|
||||
checklist_confirmed: bool,
|
||||
checklist_categories: str | None,
|
||||
dry_run: bool,
|
||||
owner: str | None = None,
|
||||
repo_name: str | None = None,
|
||||
) -> None:
|
||||
"""Post a manual review with validation for APPROVE events.
|
||||
|
||||
When self-approval is rejected (reviewer token belongs to PR author),
|
||||
falls back to the CI token (different user) if available.
|
||||
"""
|
||||
if not body or len(body) < 50:
|
||||
raise click.ClickException(_("Review body must be at least 50 characters."))
|
||||
|
||||
if event == "APPROVE":
|
||||
if not checklist_confirmed:
|
||||
raise click.ClickException(
|
||||
_("--checklist-confirmed is required for APPROVE events."),
|
||||
)
|
||||
cats = [c.strip() for c in (checklist_categories or "").split(",") if c.strip()]
|
||||
cat_nums: list[int] = []
|
||||
for c in cats:
|
||||
try:
|
||||
cat_nums.append(int(c))
|
||||
except ValueError:
|
||||
raise click.ClickException(
|
||||
_("Invalid checklist category: {cat}. Must be numbers.", cat=c),
|
||||
) from None
|
||||
if len(cat_nums) < 8:
|
||||
raise click.ClickException(
|
||||
_("--checklist-categories must list at least 8 of 13 categories. Got {count}.", count=len(cat_nums)),
|
||||
)
|
||||
|
||||
click.echo(f"Manual review event: {event}")
|
||||
click.echo(f"Body: {body[:80]}...")
|
||||
if checklist_confirmed:
|
||||
click.echo(f"Checklist confirmed: {checklist_categories}")
|
||||
|
||||
if dry_run:
|
||||
click.echo("\n[dry-run] Review not posted.")
|
||||
return
|
||||
|
||||
try:
|
||||
review = client.create_review(pr_number, event=event, body=body)
|
||||
except APIError as e:
|
||||
if "approve" in e.message.lower() or "422" in str(e.status):
|
||||
# Self-approval not allowed (reviewer token belongs to PR author).
|
||||
# Fall back to CI token (different user) if available.
|
||||
ci_token = os.environ.get("CI_GITEA_API_TOKEN", "").strip()
|
||||
if ci_token and owner and repo_name:
|
||||
click.echo(_("Note: Self-approval not allowed with reviewer token. Retrying with CI token."))
|
||||
ci_client = GiteaClient(GITEA_API_URL, ci_token, owner, repo_name)
|
||||
try:
|
||||
review = ci_client.create_review(pr_number, event=event, body=body)
|
||||
except APIError:
|
||||
click.echo(_("Note: CI token also cannot approve. Posting COMMENT instead."))
|
||||
review = client.create_review(pr_number, event="COMMENT", body=body)
|
||||
else:
|
||||
click.echo(_("Note: Self-approval not allowed. Posting COMMENT instead."))
|
||||
review = client.create_review(pr_number, event="COMMENT", body=body)
|
||||
else:
|
||||
raise
|
||||
review_id = review.get("id", "?")
|
||||
click.echo(
|
||||
_(
|
||||
"\nReview #{review_id} posted on PR #{pr_number} with event '{event}'.",
|
||||
review_id=review_id,
|
||||
pr_number=pr_number,
|
||||
event=event,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("pr_number")
|
||||
@click.argument("repo")
|
||||
@click.option("--dry-run", is_flag=True, default=False, help="Print review without posting.")
|
||||
@click.option(
|
||||
"--event",
|
||||
type=click.Choice(["APPROVE", "REQUEST_CHANGES", "COMMENT"], case_sensitive=False),
|
||||
default=None,
|
||||
help="Post a manual review with the given event (skips automated checks).",
|
||||
)
|
||||
@click.option("--body", default=None, help="Review body text (required with --event).")
|
||||
@click.option(
|
||||
"--checklist-confirmed",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help="Attest that REVIEW_CHECKLIST.md categories were checked (required for APPROVE).",
|
||||
)
|
||||
@click.option(
|
||||
"--checklist-categories",
|
||||
default=None,
|
||||
help="Comma-separated checklist category numbers (required for APPROVE, min 8 of 13).",
|
||||
)
|
||||
def main(
|
||||
pr_number: str,
|
||||
repo: str,
|
||||
dry_run: bool,
|
||||
event: str | None,
|
||||
body: str | None,
|
||||
checklist_confirmed: bool,
|
||||
checklist_categories: str | None,
|
||||
) -> None:
|
||||
"""Run automated PR review and post results to Gitea.
|
||||
|
||||
Without --event: runs automated checks and posts COMMENT/REQUEST_CHANGES.
|
||||
With --event: posts a manual review (skips automated checks).
|
||||
"""
|
||||
try:
|
||||
token = get_reviewer_token() if (event and event.upper() == "APPROVE") else get_ci_token()
|
||||
except click.ClickException:
|
||||
raise click.ClickException(_("ERROR: CI_GITEA_TOKEN is not set.")) from None
|
||||
|
||||
owner, repo_name = repo.split("/")
|
||||
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
|
||||
|
||||
if event is not None:
|
||||
_post_manual_review(
|
||||
client,
|
||||
pr_number,
|
||||
event.upper(),
|
||||
body,
|
||||
checklist_confirmed,
|
||||
checklist_categories,
|
||||
dry_run,
|
||||
owner=owner,
|
||||
repo_name=repo_name,
|
||||
)
|
||||
return
|
||||
|
||||
result = run_review(client, pr_number)
|
||||
|
||||
body = build_review_body(result)
|
||||
event = "REQUEST_CHANGES" if result.has_issues else "COMMENT"
|
||||
|
||||
click.echo(f"Review event: {event}")
|
||||
click.echo(f"Issues found: {len(result.issues)}")
|
||||
click.echo("")
|
||||
click.echo(body)
|
||||
|
||||
if dry_run:
|
||||
click.echo("\n[dry-run] Review not posted.")
|
||||
return
|
||||
|
||||
try:
|
||||
review = post_review(client, pr_number, result)
|
||||
except APIError as e:
|
||||
if "approve" in e.message.lower() or "422" in str(e.status):
|
||||
click.echo(_("Note: Self-approval not allowed. Posting COMMENT instead."))
|
||||
review = client.create_review(pr_number, event="COMMENT", body=body)
|
||||
else:
|
||||
raise
|
||||
review_id = review.get("id", "?")
|
||||
click.echo(
|
||||
_(
|
||||
"\nReview #{review_id} posted on PR #{pr_number} with event '{event}' ({num_comments} inline comments).",
|
||||
review_id=review_id,
|
||||
pr_number=pr_number,
|
||||
event=event,
|
||||
num_comments=len(result.issues),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
main()
|
||||
@@ -252,7 +252,7 @@ def commit_and_push(wiki_dir: Path, wiki_url: str, dry_run: bool) -> bool:
|
||||
|
||||
# Push
|
||||
result = subprocess.run( # nosec
|
||||
["git", "push", "--force", wiki_url, "HEAD:main"],
|
||||
["git", "push", "--force", wiki_url, "HEAD:master"],
|
||||
cwd=wiki_dir,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
|
||||
@@ -118,7 +118,7 @@ def main(commit_msg_file: str | None, branch: str | None, from_git: bool) -> Non
|
||||
" Expected: <type>: <description>\n"
|
||||
" Got: {subject}\n"
|
||||
" Allowed types: feat, fix, chore, docs, style, refactor,\n"
|
||||
" perf, test, ci, build, deps, revert, BREAKING CHANGE",
|
||||
" perf, test, ci, build, revert, BREAKING CHANGE",
|
||||
subject=subject,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -1,157 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# Implements: REQ-1
|
||||
"""Validate that a PR has a spec file with required sections and acceptance criteria.
|
||||
|
||||
Spec-driven development gate. Runs in CI before expensive jobs.
|
||||
Used by grm, infra, sso-bridge, and devx itself.
|
||||
|
||||
Validates:
|
||||
1. A spec file exists at ``docs/specs/<TASK-ID>.md`` (TASK-ID extracted from branch).
|
||||
2. The spec contains required sections: Problem, Approach, Test Plan, Deploy Plan, Rollback Plan.
|
||||
3. The spec contains REQ-ID lines (``REQ-N: <description>``).
|
||||
4. The spec contains an Acceptance Criteria checklist with at least one item.
|
||||
5. All acceptance criteria checkboxes are checked (``- [x]``).
|
||||
|
||||
Usage:
|
||||
python -m devx.ci.validate_spec --branch OBL-INFRA-531-fix-foo
|
||||
|
||||
In CI, also pass ``--github-output`` to set ``spec-valid`` and ``spec-path``
|
||||
for downstream steps.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from devx.ci._shared import extract_task_id, write_github_output
|
||||
from devx.i18n import _
|
||||
|
||||
load_dotenv()
|
||||
|
||||
REQUIRED_SECTIONS = [
|
||||
"## Problem",
|
||||
"## Approach",
|
||||
"## Test Plan",
|
||||
"## Deploy Plan",
|
||||
"## Rollback Plan",
|
||||
"## Acceptance Criteria",
|
||||
]
|
||||
|
||||
REQ_ID_RE = re.compile(r"^REQ-\d+:\s+.+", re.MULTILINE)
|
||||
AC_CHECKED_RE = re.compile(r"^\s*- \[x\]\s+.+", re.MULTILINE)
|
||||
AC_UNCHECKED_RE = re.compile(r"^\s*- \[ \]\s+.+", re.MULTILINE)
|
||||
|
||||
|
||||
def find_spec_file(task_id: str, specs_dir: str = "docs/specs") -> Path | None:
|
||||
"""Find the spec file for the given task ID.
|
||||
|
||||
Looks for ``docs/specs/<TASK-ID>.md`` (case-insensitive filename).
|
||||
Returns the Path if found, None otherwise.
|
||||
"""
|
||||
base = Path(specs_dir)
|
||||
if not base.is_dir():
|
||||
return None
|
||||
# Exact match (case-insensitive)
|
||||
for p in base.glob("*.md"):
|
||||
if p.stem.upper() == task_id.upper():
|
||||
return p
|
||||
return None
|
||||
|
||||
|
||||
def validate_spec_content(content: str) -> list[str]:
|
||||
"""Validate spec content and return a list of error messages.
|
||||
|
||||
Returns an empty list if the spec is valid.
|
||||
"""
|
||||
errors: list[str] = []
|
||||
|
||||
# Check required sections
|
||||
for section in REQUIRED_SECTIONS:
|
||||
if section not in content:
|
||||
errors.append(_("Missing required section: {section}", section=section))
|
||||
|
||||
# Check for at least one REQ-ID
|
||||
req_ids = REQ_ID_RE.findall(content)
|
||||
if not req_ids:
|
||||
errors.append(_("No REQ-ID lines found. Each requirement must be labeled (e.g., 'REQ-1: <description>')."))
|
||||
|
||||
# Check acceptance criteria has at least one item
|
||||
checked = AC_CHECKED_RE.findall(content)
|
||||
unchecked = AC_UNCHECKED_RE.findall(content)
|
||||
if not checked and not unchecked:
|
||||
errors.append(_("Acceptance Criteria section has no checklist items. Add at least one '- [ ] item'."))
|
||||
elif unchecked:
|
||||
errors.append(
|
||||
_(
|
||||
"Acceptance Criteria has {count} unchecked item(s). All AC items must be checked (- [x]) before merge.",
|
||||
count=len(unchecked),
|
||||
)
|
||||
)
|
||||
|
||||
return errors
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option("--branch", required=True, help=_("Branch name (e.g., OBL-INFRA-531-fix-foo)"))
|
||||
@click.option("--specs-dir", default="docs/specs", help=_("Directory containing spec files"))
|
||||
@click.option(
|
||||
"--github-output",
|
||||
"github_output",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help=_("Write results to $GITHUB_OUTPUT"),
|
||||
)
|
||||
@click.option("--allow-missing", is_flag=True, default=False, help=_("Allow missing spec (warn only, don't fail)"))
|
||||
def cli(branch: str, specs_dir: str, github_output: bool, allow_missing: bool) -> None:
|
||||
"""Validate that a spec file exists and has required content."""
|
||||
task_id = extract_task_id(branch)
|
||||
if not task_id:
|
||||
msg = _("No task ID found in branch name '{branch}'. Expected format: <PREFIX>-N-description.", branch=branch)
|
||||
if allow_missing:
|
||||
click.echo(f"WARNING: {msg}")
|
||||
if github_output:
|
||||
write_github_output("spec-valid", "false")
|
||||
write_github_output("spec-path", "")
|
||||
return
|
||||
raise click.ClickException(msg)
|
||||
|
||||
spec_path = find_spec_file(task_id, specs_dir)
|
||||
if spec_path is None:
|
||||
msg = _(
|
||||
"No spec file found for task {task_id} in {dir}/. Expected: {dir}/{task_id}.md",
|
||||
task_id=task_id,
|
||||
dir=specs_dir,
|
||||
)
|
||||
if allow_missing:
|
||||
click.echo(f"WARNING: {msg}")
|
||||
if github_output:
|
||||
write_github_output("spec-valid", "false")
|
||||
write_github_output("spec-path", "")
|
||||
return
|
||||
raise click.ClickException(msg)
|
||||
|
||||
content = spec_path.read_text(encoding="utf-8")
|
||||
errors = validate_spec_content(content)
|
||||
|
||||
if github_output:
|
||||
write_github_output("spec-valid", "true" if not errors else "false")
|
||||
write_github_output("spec-path", str(spec_path))
|
||||
|
||||
if errors:
|
||||
click.echo("", err=True)
|
||||
click.echo("=" * 60, err=True)
|
||||
click.echo(f"Spec validation FAILED for {spec_path}:", err=True)
|
||||
click.echo("=" * 60, err=True)
|
||||
for e in errors:
|
||||
click.echo(f" - {e}", err=True)
|
||||
raise click.ClickException(_("Spec validation failed."))
|
||||
|
||||
click.echo(_("[spec-check] Spec validated: {path}", path=spec_path))
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
cli()
|
||||
@@ -116,6 +116,13 @@ def ci_post_merge(args: tuple[str, ...]) -> None:
|
||||
_run_module("devx.ci.post_merge", list(args))
|
||||
|
||||
|
||||
@ci.command("pr-review")
|
||||
@click.argument("args", nargs=-1)
|
||||
def ci_pr_review(args: tuple[str, ...]) -> None:
|
||||
"""Run automated PR review."""
|
||||
_run_module("devx.ci.pr_review", list(args))
|
||||
|
||||
|
||||
@ci.command("publish")
|
||||
@click.argument("args", nargs=-1)
|
||||
def ci_publish(args: tuple[str, ...]) -> None:
|
||||
@@ -294,6 +301,13 @@ def molecule_discover_runners(args: tuple[str, ...]) -> None:
|
||||
_run_module("devx.molecule.discover_runners", list(args))
|
||||
|
||||
|
||||
@molecule.command("guard")
|
||||
@click.argument("args", nargs=-1)
|
||||
def molecule_guard(args: tuple[str, ...]) -> None:
|
||||
"""Run molecule tests sequentially with CI failure polling."""
|
||||
_run_module("devx.molecule.molecule_ci_guard", list(args))
|
||||
|
||||
|
||||
@molecule.command("all")
|
||||
@click.argument("args", nargs=-1)
|
||||
def molecule_all(args: tuple[str, ...]) -> None:
|
||||
|
||||
+1
-1
@@ -93,4 +93,4 @@ RETRY_BACKOFF_BASE = 2 # seconds: 2, 4, 8
|
||||
RETRY_STATUS_CODES = {429, 500, 502, 503, 504}
|
||||
|
||||
# Conventional commit regex — used by validate_commit_msg.py
|
||||
CONVENTIONAL_RE = re.compile(r"^(feat|fix|chore|docs|style|refactor|perf|test|ci|build|revert|deps)(\(.+\))?: .+")
|
||||
CONVENTIONAL_RE = re.compile(r"^(feat|fix|chore|docs|style|refactor|perf|test|ci|build|revert)(\(.+\))?: .+")
|
||||
|
||||
+11
-1
@@ -109,7 +109,7 @@ devx-ensure-venv:
|
||||
fi
|
||||
|
||||
.PHONY: devx-create-task devx-create-pr devx-push devx-push-with-pr devx-check-config
|
||||
.PHONY: devx-pr-status devx-pr-logs devx-pr-label devx-rebase devx-pr-rebase
|
||||
.PHONY: devx-pr-status devx-pr-logs devx-pr-label devx-pr-review devx-rebase devx-pr-rebase
|
||||
.PHONY: devx-configure-gitea-pypi devx-install-tools devx-install-checkmake devx-checkmake
|
||||
.PHONY: devx-workflow-lint devx-workflow-dryrun devx-workflow-dryrun-safe devx-workflow-check
|
||||
.PHONY: devx-notify-failure devx-install-hooks devx-activate-scripts devx-venv devx-ensure-venv
|
||||
@@ -171,6 +171,16 @@ devx-pr-label:
|
||||
$(if $(PR),--pr $(PR)) \
|
||||
--label $(or $(LABEL),ready-to-merge)
|
||||
|
||||
# Usage: make devx-pr-review PR=42 EVENT=APPROVE BODY="..." CHECKLIST=1,2,3,4,5,6,7,8,9,10,11,12,13
|
||||
# make devx-pr-review PR=42 EVENT=REQUEST_CHANGES BODY="..."
|
||||
# make devx-pr-review PR=42 (auto review)
|
||||
devx-pr-review:
|
||||
@$(DEVX_PYTHON) -m devx.ci.pr_review \
|
||||
$(PR) $(DEVX_REPO_OWNER)/$(DEVX_REPO_NAME) \
|
||||
$(if $(EVENT),--event $(EVENT)) \
|
||||
$(if $(BODY),--body "$(BODY)") \
|
||||
$(if $(CHECKLIST),--checklist-confirmed --checklist-categories $(CHECKLIST))
|
||||
|
||||
# Rebase current branch onto origin/master and force-push
|
||||
# Usage: make devx-rebase
|
||||
# make devx-rebase NO_PUSH=1
|
||||
|
||||
@@ -1,88 +0,0 @@
|
||||
"""Emit bounded runner-resource lease env for CI molecule jobs.
|
||||
|
||||
Producer side of the S08 lease contract (consumer: GRM-173
|
||||
``runner-cleanup.sh``). The runner's scoped cleanup honors
|
||||
``org.oblachno.lease-until`` (epoch seconds) and ``org.oblachno.owner``
|
||||
labels on containers, volumes, and networks. Molecule runs on the runner
|
||||
host's rootless Docker socket, so objects it creates must carry the lease
|
||||
labels at creation time — Docker labels cannot be added post-hoc.
|
||||
|
||||
This module computes ``MOLECULE_LEASE_UNTIL`` and ``MOLECULE_OWNER``,
|
||||
which ``molecule.yml`` platform ``labels:`` entries consume via
|
||||
``${MOLECULE_LEASE_UNTIL:-0}`` / ``${MOLECULE_OWNER:-molecule-local}``
|
||||
interpolation, and appends them to ``$GITHUB_ENV`` so every later CI step
|
||||
inherits them. A lease-until of ``0`` (the interpolation default for
|
||||
local runs) is already expired, keeping today's unleased behavior.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import socket
|
||||
import time
|
||||
|
||||
import click
|
||||
|
||||
#: Docker label contract consumed by runner-cleanup.sh (GRM-173).
|
||||
LEASE_UNTIL_LABEL = "org.oblachno.lease-until"
|
||||
OWNER_LABEL = "org.oblachno.owner"
|
||||
|
||||
#: Env vars consumed by molecule.yml platform ``labels:`` entries.
|
||||
LEASE_UNTIL_ENV = "MOLECULE_LEASE_UNTIL"
|
||||
OWNER_ENV = "MOLECULE_OWNER"
|
||||
|
||||
#: Default lease duration — covers the longest molecule job window.
|
||||
DEFAULT_TTL_SECONDS = 4 * 3600
|
||||
|
||||
|
||||
def lease_owner() -> str:
|
||||
"""Identify the owning CI run for the ``org.oblachno.owner`` label.
|
||||
|
||||
act_runner exposes GitHub-compatible ``GITHUB_*`` env vars;
|
||||
``GITEA_*`` names are checked first for forward compatibility.
|
||||
"""
|
||||
run_id = os.environ.get("GITEA_RUN_ID") or os.environ.get("GITHUB_RUN_ID")
|
||||
job = os.environ.get("GITEA_JOB") or os.environ.get("GITHUB_JOB")
|
||||
if run_id:
|
||||
return f"run-{run_id}-{job or 'job'}"
|
||||
return f"local-{socket.gethostname()}"
|
||||
|
||||
|
||||
def lease_env(ttl: int = DEFAULT_TTL_SECONDS, now: int | None = None) -> dict[str, str]:
|
||||
"""Return the lease env dict for the current run."""
|
||||
base = int(time.time()) if now is None else now
|
||||
return {
|
||||
LEASE_UNTIL_ENV: str(base + ttl),
|
||||
OWNER_ENV: lease_owner(),
|
||||
}
|
||||
|
||||
|
||||
def emit_github_env(env: dict[str, str]) -> int:
|
||||
"""Append lease env to ``$GITHUB_ENV``; return lines written (0 if unset)."""
|
||||
github_env = os.environ.get("GITHUB_ENV")
|
||||
if not github_env:
|
||||
return 0
|
||||
with open(github_env, "a", encoding="utf-8") as f:
|
||||
for key, value in env.items():
|
||||
f.write(f"{key}={value}\n")
|
||||
return len(env)
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option(
|
||||
"--ttl",
|
||||
default=DEFAULT_TTL_SECONDS,
|
||||
type=int,
|
||||
help=f"Lease duration in seconds (default: {DEFAULT_TTL_SECONDS}).",
|
||||
)
|
||||
def main(ttl: int) -> None:
|
||||
"""Print lease env vars and export them to $GITHUB_ENV when set."""
|
||||
env = lease_env(ttl)
|
||||
for key, value in env.items():
|
||||
click.echo(f"{key}={value}")
|
||||
if emit_github_env(env):
|
||||
click.echo(f"Exported {len(env)} lease vars to GITHUB_ENV")
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
main()
|
||||
@@ -17,13 +17,8 @@ nextcloud, postgres-upgrade, simple-app), the base target runs all
|
||||
scenarios for that role.
|
||||
|
||||
Playbooks that change also trigger molecule for the roles they include.
|
||||
Shared infrastructure changes (ansible.cfg, requirements.yml, molecule/,
|
||||
group_vars/) trigger all scenarios. Any ``ansible/playbooks/**`` file not
|
||||
in the explicit map fails open to all testable roles — targeted selection
|
||||
must never silently skip coverage.
|
||||
|
||||
Only roles that exist on disk and contain a ``molecule/`` directory are
|
||||
selected, so emitted make targets always resolve.
|
||||
Shared infrastructure changes (ansible.cfg, requirements.yml, molecule/)
|
||||
trigger all scenarios.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -35,29 +30,26 @@ import click
|
||||
|
||||
REPO_ROOT = Path.cwd()
|
||||
|
||||
|
||||
def role_to_target(role: str) -> str:
|
||||
"""Make target for a role: docker_base -> molecule-docker-base."""
|
||||
return f"molecule-{role.replace('_', '-')}"
|
||||
|
||||
# Map role names to make targets.
|
||||
ROLE_TARGET_MAP: dict[str, str] = {
|
||||
"app_container": "molecule-app-container",
|
||||
"app_hardening": "molecule-app-hardening",
|
||||
"crowdsec": "molecule-crowdsec",
|
||||
"disk_cleanup": "molecule-disk-cleanup",
|
||||
"docker_base": "molecule-docker-base",
|
||||
"observability": "molecule-observability",
|
||||
"restore": "molecule-restore",
|
||||
"sso_config": "molecule-sso-config",
|
||||
"storage": "molecule-storage",
|
||||
"zitadel": "molecule-zitadel",
|
||||
}
|
||||
|
||||
# Playbooks that map to molecule scenarios (via roles they include).
|
||||
# Any ansible/playbooks/** file NOT listed here fails open to all testable
|
||||
# roles (REQ-1) — a missing entry must never mean "no coverage".
|
||||
PLAYBOOK_ROLE_MAP: dict[str, list[str]] = {
|
||||
"ansible/playbooks/deploy-observability.yml": ["observability", "docker_base", "zitadel", "crowdsec"],
|
||||
"ansible/playbooks/deploy-customer.yml": ["app_container", "docker_base", "app_hardening"],
|
||||
"ansible/playbooks/deploy-controller.yml": ["deploy_controller"],
|
||||
"ansible/playbooks/deploy-customer.yml": ["app_container", "docker_base", "app_hardening", "sso_config"],
|
||||
"ansible/playbooks/configure-oidc.yml": ["sso_config", "app_container"],
|
||||
"ansible/playbooks/prepare-vms.yml": ["docker_base", "app_hardening", "storage", "disk_cleanup", "crowdsec"],
|
||||
"ansible/playbooks/restore.yml": ["restore"],
|
||||
"ansible/playbooks/upgrade-postgres.yml": ["app_container"],
|
||||
"ansible/playbooks/rolling-update-gitea.yml": ["app_container"],
|
||||
"ansible/playbooks/update-alertmanager.yml": ["observability"],
|
||||
"ansible/playbooks/build-image.yml": ["docker_base", "crowdsec", "disk_cleanup"],
|
||||
# sso_bridge role is checked out from the sso-bridge repo at deploy
|
||||
# time — no molecule coverage exists in the consuming repo, so the
|
||||
# explicit empty list documents "mapped, nothing local to test".
|
||||
"ansible/playbooks/deploy-sso-bridge.yml": [],
|
||||
}
|
||||
|
||||
# Shared infrastructure that affects all molecule tests.
|
||||
@@ -65,7 +57,6 @@ SHARED_PATHS = (
|
||||
"ansible/ansible.cfg",
|
||||
"ansible/requirements.yml",
|
||||
"ansible/molecule/",
|
||||
"ansible/group_vars/",
|
||||
)
|
||||
|
||||
# Minimum path parts for a role file: ansible/roles/<role> (3 parts).
|
||||
@@ -94,30 +85,8 @@ def get_changed_files(base: str) -> list[str]:
|
||||
return []
|
||||
|
||||
|
||||
def _testable_roles(roles_dir: Path) -> set[str]:
|
||||
"""Roles present on disk that carry a molecule/ dir (i.e. have scenarios)."""
|
||||
if not roles_dir.is_dir():
|
||||
return set()
|
||||
return {d.name for d in roles_dir.iterdir() if d.is_dir() and (d / "molecule").is_dir()}
|
||||
|
||||
|
||||
def _is_playbook_file(filepath: str) -> bool:
|
||||
"""True for any file under ansible/playbooks/ (yml tasks included)."""
|
||||
return filepath.startswith("ansible/playbooks/")
|
||||
|
||||
|
||||
def detect_changed_roles(
|
||||
changed_files: list[str],
|
||||
roles_dir: str | Path = "ansible/roles",
|
||||
) -> set[str]:
|
||||
"""Detect which roles have changed files.
|
||||
|
||||
Implements: REQ-1, REQ-2 — fail open on unmapped ansible playbook or
|
||||
shared-path changes; only roles that exist under ``roles_dir`` are
|
||||
returned, so emitted targets always resolve.
|
||||
"""
|
||||
roles_path = Path(roles_dir)
|
||||
all_roles = _testable_roles(roles_path)
|
||||
def detect_changed_roles(changed_files: list[str]) -> set[str]:
|
||||
"""Detect which roles have changed files."""
|
||||
roles: set[str] = set()
|
||||
|
||||
for filepath in changed_files:
|
||||
@@ -130,23 +99,23 @@ def detect_changed_roles(
|
||||
# Check if file is a playbook that maps to roles
|
||||
if filepath in PLAYBOOK_ROLE_MAP:
|
||||
roles.update(PLAYBOOK_ROLE_MAP[filepath])
|
||||
elif _is_playbook_file(filepath):
|
||||
# Unmapped playbook/_tasks file — fail open to all testable roles.
|
||||
return set(all_roles)
|
||||
|
||||
# Check shared infrastructure — triggers all roles
|
||||
for shared in SHARED_PATHS:
|
||||
if filepath.startswith(shared):
|
||||
return set(all_roles)
|
||||
return set(ROLE_TARGET_MAP.keys())
|
||||
|
||||
# REQ-2: drop roles that don't exist on disk (e.g. sso_config after the
|
||||
# role moved to the sso-bridge repo) so targets always resolve.
|
||||
return {r for r in roles if (roles_path / r).is_dir()}
|
||||
return roles
|
||||
|
||||
|
||||
def roles_to_targets(roles: set[str]) -> list[str]:
|
||||
"""Convert role names to make targets (conventional molecule-<role>)."""
|
||||
return [role_to_target(role) for role in sorted(roles)]
|
||||
"""Convert role names to make targets."""
|
||||
targets = []
|
||||
for role in sorted(roles):
|
||||
target = ROLE_TARGET_MAP.get(role)
|
||||
if target:
|
||||
targets.append(target)
|
||||
return targets
|
||||
|
||||
|
||||
@click.command()
|
||||
@@ -165,19 +134,14 @@ def roles_to_targets(roles: set[str]) -> list[str]:
|
||||
is_flag=True,
|
||||
help="Print role names (default if no --print-targets).",
|
||||
)
|
||||
@click.option(
|
||||
"--roles-dir",
|
||||
default="ansible/roles",
|
||||
help="Directory containing Ansible roles (default: ansible/roles).",
|
||||
)
|
||||
def main(base: str, print_targets: bool, print_roles: bool, roles_dir: str) -> None:
|
||||
def main(base: str, print_targets: bool, print_roles: bool) -> None:
|
||||
"""Detect which Ansible roles changed and output molecule scenarios."""
|
||||
changed_files = get_changed_files(base)
|
||||
if not changed_files:
|
||||
click.echo("No changed files detected.", err=True)
|
||||
return
|
||||
|
||||
roles = detect_changed_roles(changed_files, roles_dir)
|
||||
roles = detect_changed_roles(changed_files)
|
||||
if not roles:
|
||||
click.echo("No molecule scenarios affected by changes.", err=True)
|
||||
return
|
||||
|
||||
@@ -0,0 +1,284 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run molecule tests sequentially while polling Gitea for other runner failures.
|
||||
|
||||
Each pair is encoded as one of:
|
||||
|
||||
- **Single-role (4-part):** ``scenario|platform_name|platform_image|platform_command``
|
||||
- **Multi-role (5-part):** ``role|scenario|platform_name|platform_image|platform_command``
|
||||
|
||||
Pairs are executed one at a time (molecule scenarios share temp directories and
|
||||
Docker networks, so parallel execution within a single runner is unsafe).
|
||||
|
||||
A background thread polls the Gitea API. If any other molecule matrix runner
|
||||
reports failure, the current molecule subprocess is killed and this runner
|
||||
exits early with code 1.
|
||||
|
||||
Usage::
|
||||
|
||||
# Single-role
|
||||
python3 -m devx.molecule.molecule_ci_guard pair1 pair2 ...
|
||||
# Multi-role
|
||||
python3 -m devx.molecule.molecule_ci_guard --roles-root ansible/roles pair1 pair2 ...
|
||||
|
||||
Environment variables:
|
||||
GITEA_URL Base URL of the Gitea instance.
|
||||
CI_GITEA_API_TOKEN API token with repo access (CI_GITEA_TOKEN accepted for legacy).
|
||||
RUN_ID Workflow run ID (GITHUB_RUN_ID).
|
||||
JOB_NAME Base job name (GITHUB_JOB), e.g. "molecule-tests".
|
||||
MATRIX_INDEX Current matrix index (runner-index).
|
||||
GITEA_REPOSITORY Repository in "owner/repo" format.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import os
|
||||
import signal
|
||||
import subprocess # nosec B404
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
import requests
|
||||
|
||||
from devx.config import REPO_NAME, REPO_OWNER
|
||||
from devx.i18n import _
|
||||
from devx.tokens import get_ci_token
|
||||
|
||||
POLL_INTERVAL = 10
|
||||
|
||||
|
||||
def get_running_jobs(gitea_url: str, owner: str, repo: str, token: str, run_id: int) -> list[dict]:
|
||||
"""Return jobs for the given workflow run."""
|
||||
url = f"{gitea_url}/api/v1/repos/{owner}/{repo}/actions/runs/{run_id}/jobs"
|
||||
headers = {"Authorization": f"token {token}"}
|
||||
response = requests.get(url, headers=headers, timeout=30)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
return data.get("jobs", [])
|
||||
|
||||
|
||||
def any_other_runner_failed(jobs: list[dict], current_job_name: str, current_index: int) -> bool:
|
||||
"""Return True if any other molecule matrix job has failed."""
|
||||
for job in jobs:
|
||||
name = job.get("name", "")
|
||||
if not name.startswith(current_job_name):
|
||||
continue
|
||||
if name == f"{current_job_name} ({current_index})" or name == current_job_name:
|
||||
continue
|
||||
if job.get("conclusion") == "failure":
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def poll_for_other_failures(
|
||||
gitea_url: str,
|
||||
owner: str,
|
||||
repo: str,
|
||||
token: str,
|
||||
run_id: int,
|
||||
job_name: str,
|
||||
current_index: int,
|
||||
stop_event: threading.Event,
|
||||
failed_event: threading.Event,
|
||||
) -> None:
|
||||
"""Background thread: poll API and signal if another runner fails."""
|
||||
while not stop_event.is_set():
|
||||
try:
|
||||
jobs = get_running_jobs(gitea_url, owner, repo, token, run_id)
|
||||
if any_other_runner_failed(jobs, job_name, current_index):
|
||||
click.echo(_("Another molecule runner failed. Stopping this runner early."))
|
||||
failed_event.set()
|
||||
return
|
||||
except requests.RequestException as exc:
|
||||
click.echo(_("API poll warning: {exc}", exc=exc))
|
||||
stop_event.wait(POLL_INTERVAL)
|
||||
|
||||
|
||||
def build_molecule_cmd(scenario: str) -> list[str]:
|
||||
"""Build the molecule command for a scenario."""
|
||||
cmd = ["molecule", "test"]
|
||||
if scenario != "default":
|
||||
cmd.extend(["-s", scenario])
|
||||
return cmd
|
||||
|
||||
|
||||
def parse_pair(pair: str) -> tuple[str, str, str, str, str]:
|
||||
"""Parse a pair string into (role, scenario, platform_name, platform_image, platform_command).
|
||||
|
||||
Supports both 4-part (single-role) and 5-part (multi-role) formats.
|
||||
For 4-part pairs, role is empty (caller uses default role dir).
|
||||
Spaces in the command field are encoded as ``__SPACE__`` to survive
|
||||
shell word-splitting when ``$TEST_PAIRS`` is expanded unquoted.
|
||||
"""
|
||||
parts = pair.split("|")
|
||||
if len(parts) == 4:
|
||||
return "", parts[0], parts[1], parts[2], parts[3].replace("__SPACE__", " ")
|
||||
if len(parts) == 5:
|
||||
return parts[0], parts[1], parts[2], parts[3], parts[4].replace("__SPACE__", " ")
|
||||
raise click.ClickException(f"Invalid pair format: {pair!r} (expected 4 or 5 pipe-delimited parts)")
|
||||
|
||||
|
||||
def build_env_for_pair(pair: str, base_env: dict[str, str]) -> dict[str, str]:
|
||||
"""Build environment for a single molecule pair."""
|
||||
_role, _scenario, platform_name, platform_image, platform_command = parse_pair(pair)
|
||||
env = base_env.copy()
|
||||
env["MOLECULE_PLATFORM_NAME"] = platform_name
|
||||
env["MOLECULE_PLATFORM_IMAGE"] = platform_image
|
||||
if platform_command:
|
||||
env["MOLECULE_PLATFORM_COMMAND"] = platform_command
|
||||
elif "MOLECULE_PLATFORM_COMMAND" in env:
|
||||
del env["MOLECULE_PLATFORM_COMMAND"]
|
||||
env["ANSIBLE_ALLOW_BROKEN_CONDITIONALS"] = "true"
|
||||
# Use a fresh MOLECULE_HOME per pair to avoid stale config cache
|
||||
# from previous CI runs (causes "Instances missing" errors).
|
||||
if "MOLECULE_HOME" not in env:
|
||||
import tempfile
|
||||
|
||||
env["MOLECULE_HOME"] = tempfile.mkdtemp(prefix="molecule-ci-")
|
||||
return env
|
||||
|
||||
|
||||
def resolve_role_dir(role: str, roles_root: Path | None, repo_root: Path) -> Path:
|
||||
"""Resolve the working directory for a molecule pair.
|
||||
|
||||
For multi-role pairs (role non-empty), uses ``roles_root/role``.
|
||||
For single-role pairs, auto-discovers the first role with a molecule/
|
||||
subdirectory under ``repo_root/ansible/roles/``.
|
||||
"""
|
||||
if role:
|
||||
if roles_root is None:
|
||||
roles_root = repo_root / "ansible" / "roles"
|
||||
return roles_root / role
|
||||
roles_dir = repo_root / "ansible" / "roles"
|
||||
if roles_dir.is_dir():
|
||||
role_dirs = sorted(d for d in roles_dir.iterdir() if (d / "molecule").is_dir())
|
||||
if role_dirs:
|
||||
return role_dirs[0]
|
||||
return roles_dir / "role" # will produce a clear "not found" error
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("pairs", nargs=-1, required=True)
|
||||
@click.option(
|
||||
"--roles-root",
|
||||
type=click.Path(exists=True, file_okay=False, path_type=Path),
|
||||
default=None,
|
||||
help="Root directory for multi-role pairs (e.g. ansible/roles). Required when pairs use 5-part format.",
|
||||
)
|
||||
def cli(pairs: tuple[str, ...], roles_root: Path | None) -> None:
|
||||
"""Run molecule pairs sequentially, stop if another CI runner fails."""
|
||||
gitea_url = os.environ.get("GITEA_URL", "")
|
||||
try:
|
||||
token = get_ci_token()
|
||||
except click.ClickException:
|
||||
token = None
|
||||
run_id = int(os.environ.get("RUN_ID", "0"))
|
||||
job_name = os.environ.get("JOB_NAME", "molecule-tests")
|
||||
current_index = int(os.environ.get("MATRIX_INDEX", "0"))
|
||||
repository = os.environ.get("GITEA_REPOSITORY", "")
|
||||
owner, _sep, repo = repository.partition("/")
|
||||
if not owner or not repo:
|
||||
owner, repo = REPO_OWNER, REPO_NAME
|
||||
|
||||
if not all([gitea_url, token, run_id]):
|
||||
click.echo(_("GITEA_URL/CI_GITEA_TOKEN/RUN_ID not set; running without cross-runner cancellation."))
|
||||
|
||||
# When devx is installed as a pip package, __file__ resolves to the
|
||||
# site-packages directory, not the repo root. Use GITHUB_WORKSPACE
|
||||
# (set by Gitea Actions) or cwd as the repo root.
|
||||
repo_root = Path(os.environ.get("GITHUB_WORKSPACE", os.getcwd())).resolve()
|
||||
|
||||
base_env = os.environ.copy()
|
||||
base_env.setdefault("DOCKER_HOST", f"unix:///run/user/{os.getuid()}/docker.sock")
|
||||
base_env.setdefault("ANSIBLE_INJECT_INVOCATION", "1")
|
||||
|
||||
stop_event = threading.Event()
|
||||
failed_event = threading.Event()
|
||||
|
||||
if gitea_url and token and run_id:
|
||||
poller = threading.Thread(
|
||||
target=poll_for_other_failures,
|
||||
args=(
|
||||
gitea_url,
|
||||
owner,
|
||||
repo,
|
||||
token,
|
||||
run_id,
|
||||
job_name,
|
||||
current_index,
|
||||
stop_event,
|
||||
failed_event,
|
||||
),
|
||||
daemon=True,
|
||||
)
|
||||
poller.start()
|
||||
|
||||
try:
|
||||
for pair in pairs:
|
||||
if failed_event.is_set():
|
||||
sys.exit(1)
|
||||
|
||||
role, scenario, platform_name, _img, _cmd = parse_pair(pair)
|
||||
click.echo(_("Running: {scenario} on {platform}", scenario=scenario, platform=platform_name))
|
||||
|
||||
cmd = build_molecule_cmd(scenario)
|
||||
env = build_env_for_pair(pair, base_env)
|
||||
cwd = resolve_role_dir(role, roles_root, repo_root)
|
||||
|
||||
process = subprocess.Popen( # nosec B603
|
||||
cmd,
|
||||
cwd=str(cwd),
|
||||
env=env,
|
||||
preexec_fn=os.setsid,
|
||||
)
|
||||
|
||||
try:
|
||||
while process.poll() is None:
|
||||
if failed_event.is_set():
|
||||
with contextlib.suppress(ProcessLookupError):
|
||||
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
|
||||
try:
|
||||
process.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
with contextlib.suppress(ProcessLookupError):
|
||||
os.killpg(os.getpgid(process.pid), signal.SIGKILL)
|
||||
process.wait()
|
||||
sys.exit(1)
|
||||
time.sleep(1)
|
||||
except KeyboardInterrupt:
|
||||
with contextlib.suppress(ProcessLookupError):
|
||||
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
|
||||
process.wait()
|
||||
sys.exit(1)
|
||||
|
||||
rc = process.returncode
|
||||
|
||||
if rc != 0:
|
||||
click.echo(_("FAILED: {pair} exited with code {code}", pair=pair, code=rc))
|
||||
sys.exit(rc)
|
||||
|
||||
click.echo(_("PASSED: {pair}", pair=pair))
|
||||
|
||||
# Prune Docker data between scenarios to prevent disk exhaustion
|
||||
# in Docker-in-Docker molecule containers (each scenario pulls
|
||||
# hundreds of MB of images that accumulate across pairs).
|
||||
with contextlib.suppress(subprocess.SubprocessError, OSError):
|
||||
subprocess.run( # nosec B603, B607
|
||||
["docker", "system", "prune", "-af", "--volumes"],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
timeout=60,
|
||||
)
|
||||
|
||||
click.echo(_("All molecule tests passed."))
|
||||
finally:
|
||||
stop_event.set()
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
cli()
|
||||
@@ -10,12 +10,6 @@ If the host socket is not available, it tries the rootless socket, then
|
||||
starts a local ``dockerd`` with the vfs storage driver (requires
|
||||
privileged container).
|
||||
|
||||
When the host socket IS available but has limited disk space (e.g. an
|
||||
inner DinD daemon writing to a 38 GB container overlay), the script
|
||||
prefers a rootless socket that has more available space. This prevents
|
||||
"no space left on device" errors during molecule tests that pull images
|
||||
and create containers via the Docker daemon.
|
||||
|
||||
Usage::
|
||||
|
||||
python3 -m devx.molecule.start_docker [--timeout 30]
|
||||
@@ -23,35 +17,21 @@ Usage::
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import glob
|
||||
import os
|
||||
import shutil
|
||||
import subprocess # nosec B404
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from datetime import UTC, datetime
|
||||
|
||||
import click
|
||||
|
||||
from devx.i18n import _
|
||||
from devx.molecule.lease import LEASE_UNTIL_ENV, OWNER_ENV, emit_github_env, lease_env
|
||||
|
||||
DEFAULT_TIMEOUT = 30
|
||||
DOCKER_SOCK = "/var/run/docker.sock"
|
||||
# Rootless socket fallback (e.g. /run/user/994/docker.sock)
|
||||
ROOTLESS_SOCK = f"/run/user/{os.getuid()}/docker.sock"
|
||||
# Host Docker socket mounted by gitea_runner config (see runner config
|
||||
# ``options: "-v /run/user/<uid>/docker.sock:/run/host-docker.sock"``).
|
||||
# This gives CI containers access to the host's rootless Docker daemon,
|
||||
# which has the full host filesystem (e.g. 455 GB) instead of the
|
||||
# container's limited overlay (e.g. 38 GB).
|
||||
HOST_DOCKER_SOCK = "/run/host-docker.sock"
|
||||
# Minimum free bytes for a Docker daemon to be considered usable.
|
||||
# Below this, image pulls and container creation will fail with ENOSPC.
|
||||
# 20 GB leaves room for molecule-test-base (~500 MB) + a few containers.
|
||||
MIN_FREE_BYTES = 20 * 1024**3 # 20 GB
|
||||
|
||||
|
||||
def is_docker_ready() -> bool:
|
||||
@@ -66,46 +46,6 @@ def is_docker_ready() -> bool:
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def _get_docker_free_bytes() -> int:
|
||||
"""Get free disk space (bytes) at the Docker daemon's data root.
|
||||
|
||||
Returns 0 if the daemon is not reachable or the data root cannot be
|
||||
determined.
|
||||
"""
|
||||
docker_host = os.environ.get("DOCKER_HOST", f"unix://{DOCKER_SOCK}")
|
||||
try:
|
||||
result = subprocess.run( # nosec B603 B607
|
||||
[
|
||||
"docker",
|
||||
"info",
|
||||
"--format",
|
||||
"{{.DockerRootDir}}",
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
check=False,
|
||||
env={**os.environ, "DOCKER_HOST": docker_host},
|
||||
)
|
||||
if result.returncode != 0 or not result.stdout.strip():
|
||||
return 0
|
||||
data_root = result.stdout.strip()
|
||||
if not os.path.exists(data_root):
|
||||
return 0
|
||||
return shutil.disk_usage(data_root).free
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired, OSError):
|
||||
return 0
|
||||
|
||||
|
||||
def _try_socket(sock_path: str) -> bool:
|
||||
"""Set DOCKER_HOST to *sock_path* and check if the daemon is ready.
|
||||
|
||||
Returns ``True`` if the daemon responds, ``False`` otherwise.
|
||||
"""
|
||||
os.environ["DOCKER_HOST"] = f"unix://{sock_path}"
|
||||
return is_docker_ready()
|
||||
|
||||
|
||||
def _diagnose_socket() -> None:
|
||||
"""Print diagnostic info about the Docker socket."""
|
||||
click.echo(f"DOCKER_HOST = {os.environ.get('DOCKER_HOST', '(not set)')}")
|
||||
@@ -157,177 +97,50 @@ def _diagnose_socket() -> None:
|
||||
def start_docker_daemon(timeout: int = DEFAULT_TIMEOUT) -> bool:
|
||||
"""Ensure Docker is ready for molecule tests.
|
||||
|
||||
Tries sockets in this order, preferring ones with enough disk space:
|
||||
|
||||
1. Host rootless socket (``/run/host-docker.sock``) — mounted by the
|
||||
gitea runner config, has access to the host's full filesystem
|
||||
(e.g. 455 GB). Preferred over the inner dockerd.
|
||||
2. Default socket (``/var/run/docker.sock``) — may be an inner dockerd
|
||||
started by the CI image (v29.5.3) with data root on the container's
|
||||
limited overlay (e.g. 38 GB, often 100 % full).
|
||||
3. Other rootless sockets (``/run/user/*/docker.sock``).
|
||||
4. Local ``dockerd`` with vfs storage driver — last resort.
|
||||
First tries the host socket. If that works, sets ``DOCKER_HOST`` and
|
||||
returns immediately. If not, tries the rootless socket. If neither
|
||||
works, starts a local ``dockerd`` with vfs storage driver (requires
|
||||
privileged container).
|
||||
|
||||
Returns ``True`` if Docker is ready, ``False`` if it failed to
|
||||
start within the timeout.
|
||||
"""
|
||||
# Point Docker CLI and Python library to the socket explicitly
|
||||
os.environ["DOCKER_HOST"] = f"unix://{DOCKER_SOCK}"
|
||||
|
||||
# Diagnose socket state
|
||||
click.echo("--- Docker socket diagnostics ---")
|
||||
_diagnose_socket()
|
||||
click.echo("--- End diagnostics ---")
|
||||
|
||||
# Collect candidate sockets in priority order.
|
||||
# The host's rootless Docker socket (mounted at /run/host-docker.sock
|
||||
# by the gitea runner config) is preferred — it has access to the
|
||||
# host's full filesystem instead of the container's limited overlay.
|
||||
candidates: list[str] = []
|
||||
if os.path.exists(HOST_DOCKER_SOCK):
|
||||
candidates.append(HOST_DOCKER_SOCK)
|
||||
if os.path.exists(DOCKER_SOCK):
|
||||
candidates.append(DOCKER_SOCK)
|
||||
if os.path.exists(ROOTLESS_SOCK):
|
||||
candidates.append(ROOTLESS_SOCK)
|
||||
for sock in sorted(glob.glob("/run/user/*/docker.sock")):
|
||||
if sock not in candidates:
|
||||
candidates.append(sock)
|
||||
# Check if host Docker is already available
|
||||
if is_docker_ready():
|
||||
click.echo(_("Docker daemon already running"))
|
||||
return True
|
||||
|
||||
# Try each candidate socket — prefer one with enough free space
|
||||
for sock in candidates:
|
||||
click.echo(f"Trying socket: {sock}")
|
||||
if not _try_socket(sock):
|
||||
# Try rootless socket (e.g. /run/user/994/docker.sock)
|
||||
click.echo(f"Trying rootless socket: {ROOTLESS_SOCK}")
|
||||
os.environ["DOCKER_HOST"] = f"unix://{ROOTLESS_SOCK}"
|
||||
if os.path.exists(ROOTLESS_SOCK) and is_docker_ready():
|
||||
click.echo(_("Docker daemon already running"))
|
||||
return True
|
||||
|
||||
# Scan for any rootless sockets at other UIDs
|
||||
for sock in sorted(glob.glob("/run/user/*/docker.sock")):
|
||||
if sock == ROOTLESS_SOCK:
|
||||
continue
|
||||
free_bytes = _get_docker_free_bytes()
|
||||
free_gb = free_bytes / 1024**3
|
||||
click.echo(f" Docker daemon ready (free space: {free_gb:.1f} GB)")
|
||||
if free_bytes >= MIN_FREE_BYTES:
|
||||
click.echo(f"Trying alternative rootless socket: {sock}")
|
||||
os.environ["DOCKER_HOST"] = f"unix://{sock}"
|
||||
if is_docker_ready():
|
||||
click.echo(_("Docker daemon already running"))
|
||||
return True
|
||||
# If free_bytes is 0, the Docker root dir is on the host filesystem
|
||||
# (not accessible from inside the container). This is expected for
|
||||
# the host's rootless Docker — it has the full host disk.
|
||||
# Only trust this for /run/host-docker.sock (known host socket).
|
||||
# For other sockets (e.g. inner dockerd), free_bytes == 0 means
|
||||
# the data root path doesn't exist inside the container — the
|
||||
# inner dockerd may be using the container's full overlay.
|
||||
if free_bytes == 0 and sock == HOST_DOCKER_SOCK:
|
||||
click.echo("Host rootless Docker root dir not accessible from container, using it")
|
||||
return True
|
||||
# If free_bytes is 0 and there are no dockerd processes inside the
|
||||
# container, the socket is the host's Docker (mounted from outside).
|
||||
# The data root is on the host filesystem and has plenty of space.
|
||||
if free_bytes == 0 and sock == DOCKER_SOCK:
|
||||
has_inner_dockerd = False
|
||||
with contextlib.suppress(Exception):
|
||||
pgrep_result = subprocess.run( # nosec B603 B607
|
||||
["pgrep", "-f", "dockerd"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
has_inner_dockerd = pgrep_result.returncode == 0
|
||||
if not has_inner_dockerd:
|
||||
click.echo("No inner dockerd found, socket is host Docker (data root on host), using it")
|
||||
return True
|
||||
click.echo(f" Insufficient space ({free_gb:.1f} GB), trying next...")
|
||||
|
||||
# No socket with sufficient space found.
|
||||
# Don't fall back to the low-space inner dockerd — it will fail
|
||||
# on image pulls. Instead, kill the inner dockerd, clean up its
|
||||
# data root to free space, and start a new dockerd using the
|
||||
# freed space on the container's overlay.
|
||||
click.echo(_("Host Docker not available, starting local dockerd..."))
|
||||
|
||||
# Kill the inner dockerd (started by the CI image) to free its
|
||||
# data root and socket. The inner dockerd uses the container's
|
||||
# overlay (38G, often 100% full). Killing it frees up the
|
||||
# socket and any space used by its containers/volumes.
|
||||
# Use SIGKILL (-9) since the inner dockerd may not respond to SIGTERM.
|
||||
# Try multiple approaches to ensure the inner dockerd is killed.
|
||||
with contextlib.suppress(Exception):
|
||||
result = subprocess.run( # nosec B603 B607
|
||||
["pgrep", "-af", "dockerd"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
if result.stdout.strip():
|
||||
click.echo(f" dockerd processes before kill: {result.stdout.strip()}")
|
||||
|
||||
for pattern in ["dockerd", "dockerd-entrypoint.sh", "containerd"]:
|
||||
with contextlib.suppress(Exception):
|
||||
subprocess.run( # nosec B603 B607
|
||||
["pkill", "-9", "-f", pattern],
|
||||
check=False,
|
||||
timeout=5,
|
||||
)
|
||||
time.sleep(3)
|
||||
|
||||
# Check if dockerd processes are still alive
|
||||
with contextlib.suppress(Exception):
|
||||
result = subprocess.run( # nosec B603 B607
|
||||
["pgrep", "-af", "dockerd"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
if result.stdout.strip():
|
||||
click.echo(f" dockerd processes after kill: {result.stdout.strip()}")
|
||||
# Try killing by PID directly
|
||||
for pid_str in result.stdout.split("\n"):
|
||||
pid = pid_str.split()[0] if pid_str.strip() else ""
|
||||
if pid:
|
||||
with contextlib.suppress(Exception):
|
||||
os.kill(int(pid), 9)
|
||||
time.sleep(2)
|
||||
|
||||
# Verify the inner dockerd is actually dead. If we can still
|
||||
# connect to /var/run/docker.sock, the old daemon is still running
|
||||
# and we need to use a different socket path.
|
||||
old_daemon_alive = False
|
||||
with contextlib.suppress(Exception):
|
||||
result = subprocess.run( # nosec B603 B607
|
||||
["docker", "info"],
|
||||
env={**os.environ, "DOCKER_HOST": f"unix://{DOCKER_SOCK}"},
|
||||
capture_output=True,
|
||||
timeout=5,
|
||||
)
|
||||
old_daemon_alive = result.returncode == 0
|
||||
|
||||
if old_daemon_alive:
|
||||
click.echo(" Inner dockerd still alive, using alternate socket")
|
||||
local_sock = "/dev/shm/docker.sock" # nosec B108
|
||||
else:
|
||||
local_sock = DOCKER_SOCK
|
||||
|
||||
# Clean up the inner dockerd's data root to free space.
|
||||
# The inner dockerd stores images, containers, and volumes here.
|
||||
# Removing them frees up ~2.4GB on the container's overlay.
|
||||
inner_data_root = "/home/grm-ci-runner-*/.local/share/docker"
|
||||
rm_paths = " ".join(f"{inner_data_root}/{d}" for d in ("overlay2", "image", "volumes", "containers"))
|
||||
with contextlib.suppress(Exception):
|
||||
subprocess.run( # nosec B603 B607
|
||||
["sh", "-c", f"rm -rf {rm_paths}"],
|
||||
check=False,
|
||||
timeout=30,
|
||||
)
|
||||
|
||||
# Use a fresh data root. If the inner dockerd is dead, use the
|
||||
# container's overlay (38G, with freed space). If the inner
|
||||
# dockerd is still alive, use /dev/shm (16G tmpfs) — the overlay
|
||||
# is still full because the inner dockerd's data can't be cleaned.
|
||||
docker_data_root = "/dev/shm/docker" if old_daemon_alive else "/tmp/docker-data" # nosec B108
|
||||
|
||||
# Remove stale socket if present
|
||||
with contextlib.suppress(OSError):
|
||||
os.unlink(local_sock)
|
||||
|
||||
os.environ["DOCKER_HOST"] = f"unix://{local_sock}"
|
||||
# Reset DOCKER_HOST to host socket for local dockerd
|
||||
os.environ["DOCKER_HOST"] = f"unix://{DOCKER_SOCK}"
|
||||
|
||||
# Start local dockerd (requires privileged container)
|
||||
# Use /tmp/docker-data as data root on the container's overlay.
|
||||
# The inner dockerd's data root has been cleaned up, freeing ~2.4GB.
|
||||
# vfs storage driver is used since overlay2 may not work inside
|
||||
# a Docker-in-Docker container without --privileged.
|
||||
log_file = tempfile.NamedTemporaryFile( # noqa: SIM115
|
||||
mode="w", suffix="dockerd.log", delete=False
|
||||
)
|
||||
@@ -337,13 +150,8 @@ def start_docker_daemon(timeout: int = DEFAULT_TIMEOUT) -> bool:
|
||||
"dockerd",
|
||||
"--storage-driver",
|
||||
"vfs",
|
||||
"--data-root",
|
||||
docker_data_root,
|
||||
"--iptables=false",
|
||||
"--ip6tables=false",
|
||||
"--bridge=none",
|
||||
"-H",
|
||||
f"unix://{local_sock}",
|
||||
f"unix://{DOCKER_SOCK}",
|
||||
],
|
||||
stdout=log_file,
|
||||
stderr=subprocess.STDOUT,
|
||||
@@ -386,15 +194,6 @@ def main(timeout: int) -> None:
|
||||
with open(github_env, "a", encoding="utf-8") as f:
|
||||
f.write(f"DOCKER_HOST={os.environ['DOCKER_HOST']}\n")
|
||||
click.echo(f"Exported DOCKER_HOST={os.environ['DOCKER_HOST']} to GITHUB_ENV")
|
||||
# Implements: REQ-2 (DEVX-181) — emit the S08 producer lease env so
|
||||
# molecule-created objects carry org.oblachno.lease-until/owner
|
||||
# labels that runner-cleanup.sh honors.
|
||||
# Implements: REQ-1 (DEVX-182) — one lease_env() call; log the
|
||||
# owner and UTC expiry an operator needs for cleanup debugging.
|
||||
env = lease_env()
|
||||
if emit_github_env(env):
|
||||
until = datetime.fromtimestamp(int(env[LEASE_UNTIL_ENV]), tz=UTC).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
click.echo(f"Exported lease vars to GITHUB_ENV (owner={env[OWNER_ENV]}, until={until})")
|
||||
sys.exit(0)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
+16
-2
@@ -2,16 +2,19 @@
|
||||
|
||||
Centralizes Gitea/Vikunja token discovery with role-based environment
|
||||
variable names and backwards compatibility with the legacy
|
||||
``CI_GITEA_TOKEN`` naming convention.
|
||||
``CI_GITEA_TOKEN`` / ``REVIEW_GITEA_TOKEN`` naming convention.
|
||||
|
||||
Roles:
|
||||
- ``CI_GITEA_API_TOKEN``: CI workflows (read actions, post status, merge, etc.)
|
||||
- ``REVIEWER_GITEA_API_TOKEN``: PR approval reviews (must be a different user
|
||||
from the PR author for Gitea to accept the review as an approval)
|
||||
- ``DEVELOPER_GITEA_API_TOKEN``: local development tools (create-task,
|
||||
create-pr, setup, etc.)
|
||||
|
||||
Fallbacks:
|
||||
- New role names are checked first.
|
||||
- Legacy names (``CI_GITEA_TOKEN``) are accepted for backwards compatibility.
|
||||
- Legacy names (``CI_GITEA_TOKEN``, ``REVIEW_GITEA_TOKEN``) are accepted for
|
||||
backwards compatibility.
|
||||
- If no role-specific token is set, the generic CI tokens are tried last.
|
||||
"""
|
||||
|
||||
@@ -25,6 +28,12 @@ from devx.i18n import _
|
||||
|
||||
# Token environment variable names, in lookup priority order.
|
||||
CI_TOKEN_NAMES = ["CI_GITEA_API_TOKEN", "CI_GITEA_TOKEN"]
|
||||
REVIEWER_TOKEN_NAMES = [
|
||||
"REVIEWER_GITEA_API_TOKEN",
|
||||
# Legacy name used before role-based tokens.
|
||||
"REVIEW_GITEA_TOKEN",
|
||||
*CI_TOKEN_NAMES,
|
||||
]
|
||||
DEVELOPER_TOKEN_NAMES = ["DEVELOPER_GITEA_API_TOKEN", *CI_TOKEN_NAMES]
|
||||
|
||||
VIKUNJA_TOKEN_NAMES = ["VIKUNJA_TOKEN"]
|
||||
@@ -52,6 +61,11 @@ def get_ci_token() -> str:
|
||||
return get_token(*CI_TOKEN_NAMES)
|
||||
|
||||
|
||||
def get_reviewer_token() -> str:
|
||||
"""Resolve the reviewer Gitea API token used for PR approvals."""
|
||||
return get_token(*REVIEWER_TOKEN_NAMES)
|
||||
|
||||
|
||||
def get_developer_token() -> str:
|
||||
"""Resolve the developer Gitea API token used for local tooling."""
|
||||
return get_token(*DEVELOPER_TOKEN_NAMES)
|
||||
|
||||
+10
-168
@@ -40,17 +40,13 @@ and ``CI_GITEA_USERNAME`` environment variables, matching the existing CI workfl
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import subprocess # nosec B404
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
from tenacity import retry, retry_if_exception_type, stop_after_attempt, wait_exponential
|
||||
|
||||
from devx.i18n import _
|
||||
from devx.tokens import get_developer_token
|
||||
@@ -192,188 +188,38 @@ def build_image(
|
||||
return True
|
||||
|
||||
|
||||
def delete_remote_manifest(
|
||||
registry: str,
|
||||
name: str,
|
||||
tag: str,
|
||||
username: str,
|
||||
token: str,
|
||||
*,
|
||||
dry_run: bool = False,
|
||||
) -> bool:
|
||||
"""Delete an existing manifest from the Gitea container registry.
|
||||
|
||||
Gitea 1.27 has a bug (#31964) where pushing a tag that already exists
|
||||
fails with HTTP 500 "package version already exists". This function
|
||||
deletes the existing manifest before the push to work around it.
|
||||
|
||||
Returns True if deleted or not found, False on unexpected errors.
|
||||
"""
|
||||
manifest_url = f"https://{registry}/v2/{name}/manifests/{tag}"
|
||||
if dry_run:
|
||||
click.echo(f"[dry-run] DELETE {manifest_url}")
|
||||
return True
|
||||
|
||||
# First, get the digest via HEAD
|
||||
req = urllib.request.Request(manifest_url, method="HEAD") # nosec B310
|
||||
auth_str = f"{username}:{token}"
|
||||
req.add_header("Authorization", f"Basic {base64.b64encode(auth_str.encode()).decode()}")
|
||||
req.add_header("Accept", "application/vnd.docker.distribution.manifest.v2+json")
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as resp: # nosec B310
|
||||
digest = resp.headers.get("Docker-Content-Digest")
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 404:
|
||||
return True # Tag doesn't exist — nothing to delete
|
||||
if e.code == 405:
|
||||
# HEAD not supported — try GET with a range
|
||||
pass
|
||||
else:
|
||||
click.echo(f" Warning: HEAD {tag} returned {e.code}", err=True)
|
||||
return True # Don't block the push
|
||||
except urllib.error.URLError as e:
|
||||
click.echo(f" Warning: HEAD {tag} failed: {e}", err=True)
|
||||
return True # Don't block the push
|
||||
else:
|
||||
if not digest:
|
||||
return True
|
||||
# Delete by digest
|
||||
del_url = f"https://{registry}/v2/{name}/manifests/{digest}"
|
||||
del_req = urllib.request.Request(del_url, method="DELETE") # nosec B310
|
||||
del_req.add_header("Authorization", f"Basic {base64.b64encode(auth_str.encode()).decode()}")
|
||||
try:
|
||||
with urllib.request.urlopen(del_req, timeout=30) as resp: # nosec B310
|
||||
click.echo(f" Deleted existing {tag} (digest: {digest[:19]}...)")
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 404:
|
||||
return True # Already gone
|
||||
click.echo(f" Warning: DELETE {tag} returned {e.code}", err=True)
|
||||
return True # Don't block the push
|
||||
except urllib.error.URLError as e:
|
||||
click.echo(f" Warning: DELETE {tag} failed: {e}", err=True)
|
||||
return True
|
||||
return True
|
||||
|
||||
|
||||
class PushHTTP500Error(Exception):
|
||||
"""Raised when docker push fails with an HTTP 500 from the registry."""
|
||||
|
||||
|
||||
def _run_push(cmd: list[str]) -> subprocess.CompletedProcess[str]:
|
||||
"""Run a docker push command, raising PushHTTP500Error on registry 500.
|
||||
|
||||
The Gitea container registry (v1.27.x) has a race condition in
|
||||
BlobUploader.Append that causes intermittent HTTP 500 "offset
|
||||
mismatch" errors during concurrent blob uploads. Retrying the
|
||||
push gives the registry time to recover.
|
||||
|
||||
Docker sends push progress/errors to both stdout and stderr depending
|
||||
on the error type, so both streams are checked for the 500 status.
|
||||
Uses stderr=STDOUT to merge both streams into stdout, ensuring all
|
||||
output is captured in one place (docker push output behavior varies
|
||||
depending on TTY detection).
|
||||
"""
|
||||
result = subprocess.run( # nosec B603
|
||||
cmd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0 and "500" in (result.stdout or ""):
|
||||
raise PushHTTP500Error(result.stdout.strip())
|
||||
return result
|
||||
|
||||
|
||||
def push_image(
|
||||
spec: ImageSpec,
|
||||
registry: str,
|
||||
*,
|
||||
dry_run: bool = False,
|
||||
username: str = "",
|
||||
token: str = "",
|
||||
) -> bool:
|
||||
"""Push all tags of a Docker image to the registry.
|
||||
|
||||
Returns True if all pushes succeed, False if any fail.
|
||||
|
||||
Push-first strategy: try pushing directly. Only if the push fails
|
||||
with Gitea #31964 ("package version already exists") do we delete
|
||||
the old manifest and retry. This avoids losing the existing tag
|
||||
when the push fails for unrelated reasons (e.g. HTTP 500).
|
||||
|
||||
HTTP 500 errors from the Gitea registry race condition are retried
|
||||
up to 3 times with exponential backoff (5s, 10s) via tenacity.
|
||||
"""
|
||||
full_tags = [build_full_tag(registry, spec.name, t) for t in spec.tags]
|
||||
all_ok = True
|
||||
for ft, tag in zip(full_tags, spec.tags, strict=False):
|
||||
for ft in full_tags:
|
||||
cmd = ["docker", "push", ft]
|
||||
if dry_run:
|
||||
click.echo(f"[dry-run] {' '.join(cmd)}")
|
||||
continue
|
||||
click.echo(f"Pushing {ft}...")
|
||||
|
||||
@retry(
|
||||
stop=stop_after_attempt(5),
|
||||
wait=wait_exponential(multiplier=10, min=10, max=60),
|
||||
retry=retry_if_exception_type(PushHTTP500Error),
|
||||
before_sleep=lambda retry_state: click.echo(
|
||||
_(
|
||||
" HTTP 500 from registry, retrying in {wait:.0f}s (attempt {attempt}/5)...",
|
||||
wait=retry_state.next_action.sleep if retry_state.next_action else 0,
|
||||
attempt=retry_state.attempt_number + 1,
|
||||
),
|
||||
err=True,
|
||||
),
|
||||
reraise=True,
|
||||
result = subprocess.run( # nosec B603
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
def _attempt(_cmd: list[str] = cmd) -> subprocess.CompletedProcess[str]:
|
||||
return _run_push(_cmd)
|
||||
|
||||
try:
|
||||
result = _attempt()
|
||||
except PushHTTP500Error as e:
|
||||
if result.returncode != 0:
|
||||
click.echo(
|
||||
_("Push failed for {tag}: {error}", tag=ft, error=str(e)),
|
||||
_("Push failed for {tag}: {error}", tag=ft, error=result.stderr.strip()),
|
||||
err=True,
|
||||
)
|
||||
all_ok = False
|
||||
continue
|
||||
|
||||
if result.returncode == 0:
|
||||
else:
|
||||
click.echo(f"Pushed {ft}")
|
||||
continue
|
||||
combined_output = (result.stdout or "").strip()
|
||||
# Gitea #31964: push fails because tag already exists.
|
||||
# Delete the old manifest and retry once.
|
||||
if username and token and "already exists" in combined_output.lower():
|
||||
click.echo(" Tag exists (Gitea #31964), deleting old manifest and retrying...")
|
||||
delete_remote_manifest(
|
||||
registry,
|
||||
spec.name,
|
||||
tag,
|
||||
username,
|
||||
token,
|
||||
dry_run=dry_run,
|
||||
)
|
||||
click.echo(f" Retrying push {ft}...")
|
||||
result = subprocess.run( # nosec B603
|
||||
cmd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
click.echo(f"Pushed {ft} (after retry)")
|
||||
continue
|
||||
combined_output = (result.stdout or "").strip()
|
||||
click.echo(
|
||||
_("Push failed for {tag}: {error}", tag=ft, error=combined_output),
|
||||
err=True,
|
||||
)
|
||||
all_ok = False
|
||||
return all_ok
|
||||
|
||||
|
||||
@@ -474,15 +320,11 @@ def main(
|
||||
raise click.ClickException(_("Registry login failed"))
|
||||
|
||||
failed: list[str] = []
|
||||
push_username = "" # nosec B105
|
||||
push_token = "" # nosec B105
|
||||
if push:
|
||||
push_username, push_token = _get_registry_creds()
|
||||
for spec in specs:
|
||||
if not build_image(spec, registry, dry_run=dry_run, pull=pull):
|
||||
failed.append(spec.name)
|
||||
continue
|
||||
if push and not push_image(spec, registry, dry_run=dry_run, username=push_username, token=push_token): # type: ignore[arg-type]
|
||||
if push and not push_image(spec, registry, dry_run=dry_run): # type: ignore[arg-type]
|
||||
failed.append(spec.name)
|
||||
|
||||
if failed:
|
||||
|
||||
@@ -11,18 +11,6 @@ Usage:
|
||||
The module runs ``make test-unit`` with ``PYTEST_ADDOPTS=--durations=0`` so
|
||||
that pytest emits per-test timing lines alongside the summary. Both the
|
||||
total wall-clock time and individual test durations are parsed and validated.
|
||||
|
||||
CI runner scaling
|
||||
-----------------
|
||||
CI runners (Gitea Actions Docker containers) are typically 5-8x slower than
|
||||
local development machines due to shared CPU, fewer cores, and container
|
||||
overhead. When the ``CI`` environment variable is set (standard CI
|
||||
convention), both the total and per-test limits are multiplied by
|
||||
``CI_SCALE_FACTOR`` (default 6) to account for this. This keeps the local
|
||||
budget strict while preventing false failures on slower CI runners.
|
||||
|
||||
The scale factor can be overridden via the ``DEVX_CI_SCALE_FACTOR``
|
||||
environment variable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -39,12 +27,6 @@ DEFAULT_MAX_SECONDS = 10.0
|
||||
DEFAULT_MAX_SINGLE_SECONDS = 0.5
|
||||
TEST_COMMAND = ["make", "test-unit"]
|
||||
|
||||
# CI runners are typically 5-8x slower than local machines (shared CPU,
|
||||
# fewer cores, container overhead). Scale limits up when running on CI
|
||||
# so the gate catches real regressions, not infrastructure slowness.
|
||||
CI_SCALE_FACTOR = float(os.environ.get("DEVX_CI_SCALE_FACTOR", "6"))
|
||||
_IS_CI = bool(os.environ.get("CI") or os.environ.get("GITEA_ACTIONS"))
|
||||
|
||||
# Matches pytest summary line: "234 passed in 0.70s"
|
||||
_TIMING_RE = re.compile(r"(\d+) passed.* in ([0-9.]+)s")
|
||||
|
||||
@@ -56,13 +38,6 @@ _TIMING_RE = re.compile(r"(\d+) passed.* in ([0-9.]+)s")
|
||||
_DURATION_LINE_RE = re.compile(r"^(\d+\.?\d*)s\s+call\s+(.+)$")
|
||||
|
||||
|
||||
def _ci_scale_limit(limit: float) -> float:
|
||||
"""Scale a time limit by the CI factor when running on CI."""
|
||||
if _IS_CI:
|
||||
return limit * CI_SCALE_FACTOR
|
||||
return limit
|
||||
|
||||
|
||||
def run_tests() -> tuple[str, str]:
|
||||
"""Execute the unit-test suite and return (stdout, stderr).
|
||||
|
||||
@@ -148,38 +123,21 @@ def check_per_test_speed(
|
||||
|
||||
def main(max_seconds: float, max_single_seconds: float) -> None:
|
||||
"""Run tests, parse timings, and enforce both budgets."""
|
||||
# Scale limits for CI runners (slower CPU, fewer workers).
|
||||
effective_max = _ci_scale_limit(max_seconds)
|
||||
effective_single = _ci_scale_limit(max_single_seconds)
|
||||
|
||||
if _IS_CI:
|
||||
click.echo(
|
||||
_(
|
||||
"[check-test-speed] CI environment detected — scaling limits by {factor}x "
|
||||
"(total: {orig}s → {eff}s, per-test: {orig_s}s → {eff_s}s)",
|
||||
factor=CI_SCALE_FACTOR,
|
||||
orig=max_seconds,
|
||||
eff=effective_max,
|
||||
orig_s=max_single_seconds,
|
||||
eff_s=effective_single,
|
||||
)
|
||||
)
|
||||
|
||||
stdout, stderr = run_tests()
|
||||
combined = stdout + "\n" + stderr
|
||||
click.echo(combined, err=False)
|
||||
|
||||
duration = parse_duration(combined)
|
||||
check_speed(duration, effective_max)
|
||||
check_speed(duration, max_seconds)
|
||||
|
||||
if effective_single > 0:
|
||||
if max_single_seconds > 0:
|
||||
per_test = parse_per_test_durations(combined)
|
||||
violations = check_per_test_speed(per_test, effective_single)
|
||||
violations = check_per_test_speed(per_test, max_single_seconds)
|
||||
if violations:
|
||||
msg = _(
|
||||
"Per-test speed check FAILED: {count} test(s) exceed {limit}s limit.",
|
||||
count=len(violations),
|
||||
limit=effective_single,
|
||||
limit=max_single_seconds,
|
||||
)
|
||||
click.echo(f"\n{msg}", err=True)
|
||||
for v in violations:
|
||||
@@ -190,8 +148,8 @@ def main(max_seconds: float, max_single_seconds: float) -> None:
|
||||
_(
|
||||
"Unit tests passed in {duration:.2f}s (under {max}s limit, all tests under {single}s per-test limit).",
|
||||
duration=duration,
|
||||
max=effective_max,
|
||||
single=effective_single,
|
||||
max=max_seconds,
|
||||
single=max_single_seconds,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@@ -150,20 +150,15 @@ def sort_versions_by_date(
|
||||
def select_for_deletion(
|
||||
versions: list[dict[str, Any]],
|
||||
keep: int,
|
||||
protect: frozenset[str] = frozenset(),
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Select versions to delete, keeping the most recent ``keep`` versions.
|
||||
|
||||
Versions named ``latest`` and any version listed in ``protect`` are
|
||||
always preserved. Implements REQ-2 (DEVX-165): tags/digests that a
|
||||
release manifest still pins must survive registry cleanup.
|
||||
Versions named ``latest`` are always preserved.
|
||||
"""
|
||||
sorted_versions = sort_versions_by_date(versions)
|
||||
to_delete = sorted_versions[keep:]
|
||||
# Always preserve 'latest' tag
|
||||
to_delete = [v for v in to_delete if v.get("version") != "latest"]
|
||||
# Preserve explicitly protected versions (e.g., pinned by a release manifest)
|
||||
to_delete = [v for v in to_delete if v.get("version") not in protect]
|
||||
return to_delete
|
||||
|
||||
|
||||
@@ -187,12 +182,6 @@ def select_for_deletion(
|
||||
show_default=True,
|
||||
help="Number of recent versions to keep (excluding 'latest').",
|
||||
)
|
||||
@click.option(
|
||||
"--protect",
|
||||
"protect",
|
||||
multiple=True,
|
||||
help="Version/tag to never delete (e.g., pinned by a release manifest). Can be repeated.",
|
||||
)
|
||||
@click.option(
|
||||
"--dry-run",
|
||||
is_flag=True,
|
||||
@@ -208,7 +197,6 @@ def main(
|
||||
owner: str | None,
|
||||
names: tuple[str, ...],
|
||||
keep: int,
|
||||
protect: tuple[str, ...],
|
||||
dry_run: bool,
|
||||
api_url: str | None,
|
||||
) -> None:
|
||||
@@ -250,7 +238,7 @@ def main(
|
||||
_(" {version} (created: {created})", version=v.get("version", "?"), created=v.get("created_at", "?"))
|
||||
)
|
||||
|
||||
to_delete = select_for_deletion(versions, keep, frozenset(protect))
|
||||
to_delete = select_for_deletion(versions, keep)
|
||||
kept_count = len(versions) - len(to_delete)
|
||||
click.echo(_("\nKeeping {kept}, would delete {count}", kept=kept_count, count=len(to_delete)))
|
||||
|
||||
|
||||
@@ -346,18 +346,11 @@ def collect_quality(repo_root: Path) -> dict[str, str | int]:
|
||||
results: list[bool] = []
|
||||
tool_names: list[str] = []
|
||||
|
||||
# Implements: REQ-1 (DEVX-175) — honor the repo's [tool.bandit] config so
|
||||
# the badge matches the project's actual lint gate (skip lists etc.).
|
||||
bandit_cmd = [sys.executable, "-m", "bandit", "-r", "src/"]
|
||||
pyproject = repo_root / "pyproject.toml"
|
||||
if pyproject.exists() and "[tool.bandit]" in pyproject.read_text():
|
||||
bandit_cmd += ["-c", "pyproject.toml"]
|
||||
|
||||
for cmd, name in [
|
||||
([sys.executable, "-m", "ruff", "check", "src/", "tests/"], "ruff check"),
|
||||
([sys.executable, "-m", "ruff", "format", "--check", "src/", "tests/"], "ruff format"),
|
||||
([sys.executable, "-m", "pyright"], "pyright"),
|
||||
(bandit_cmd, "bandit"),
|
||||
([sys.executable, "-m", "bandit", "-r", "src/"], "bandit"),
|
||||
]:
|
||||
rc, _stdout, stderr = run_command(cmd, cwd=repo_root)
|
||||
if rc == 0:
|
||||
|
||||
@@ -65,13 +65,8 @@ def _ensure_target_dir() -> Path:
|
||||
|
||||
|
||||
def _download(url: str, dest: Path) -> None:
|
||||
"""Download a file from ``url`` to ``dest`` with a 60s timeout.
|
||||
|
||||
A User-Agent header is set because some CDNs (e.g. dl.gitea.com)
|
||||
return 403 to requests with Python's default User-Agent.
|
||||
"""
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "devx/install-tools"})
|
||||
with urllib.request.urlopen(req, timeout=60) as resp, open(dest, "wb") as f: # nosec B310
|
||||
"""Download a file from ``url`` to ``dest`` with a 60s timeout."""
|
||||
with urllib.request.urlopen(url, timeout=60) as resp, open(dest, "wb") as f: # nosec B310
|
||||
shutil.copyfileobj(resp, f)
|
||||
|
||||
|
||||
|
||||
@@ -83,17 +83,6 @@ def _install_in_image(
|
||||
username,
|
||||
token,
|
||||
)
|
||||
# Configure git URL rewrite so git+https dependencies can authenticate
|
||||
subprocess.run( # nosec B603, B607
|
||||
[
|
||||
"git",
|
||||
"config",
|
||||
"--global",
|
||||
f"url.https://{username}:{token}@{gitea_host}/.insteadOf",
|
||||
f"https://{gitea_host}/",
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
|
||||
click.echo(f"[setup-image] Linked {opt_venv}" + (f" with [{extras}]" if extras else "") + ".")
|
||||
subprocess.run(cmd, check=True, env=env) # nosec B603
|
||||
|
||||
+1560
-2024
File diff suppressed because it is too large
Load Diff
@@ -923,12 +923,6 @@ class TestGiteaClientActions:
|
||||
)
|
||||
|
||||
def test_get_repo_variable_returns_value(self) -> None:
|
||||
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
|
||||
client._session.request = MagicMock(return_value=_mock_response({"data": "v0.28.1"}))
|
||||
result = client.get_repo_variable("PRODUCTION_DEPLOY_TAG")
|
||||
assert result == "v0.28.1"
|
||||
|
||||
def test_get_repo_variable_falls_back_to_value(self) -> None:
|
||||
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
|
||||
client._session.request = MagicMock(return_value=_mock_response({"value": "v0.28.1"}))
|
||||
result = client.get_repo_variable("PRODUCTION_DEPLOY_TAG")
|
||||
|
||||
@@ -232,40 +232,6 @@ class TestExtractConventionalMsg:
|
||||
]
|
||||
assert extract_conventional_msg(commits) == "random message"
|
||||
|
||||
def test_skip_ci_subjects_are_ineligible(self) -> None:
|
||||
"""[skip ci] commits must never become the merge title (REQ-1)."""
|
||||
commits = [
|
||||
{"commit": {"message": "ci: retrigger validation"}},
|
||||
{"commit": {"message": "chore: update badge URLs to commit abc123 [skip ci]"}},
|
||||
]
|
||||
assert extract_conventional_msg(commits) == "ci: retrigger validation"
|
||||
|
||||
def test_skip_ci_variants_excluded(self) -> None:
|
||||
"""All skip-token spellings are ineligible."""
|
||||
for token in ("[skip ci]", "[ci skip]", "[skip actions]", "[actions skip]", "[SKIP CI]"):
|
||||
commits = [
|
||||
{"commit": {"message": "feat: real change"}},
|
||||
{"commit": {"message": f"chore: noise {token}"}},
|
||||
]
|
||||
assert extract_conventional_msg(commits) == "feat: real change"
|
||||
|
||||
def test_tie_prefers_newest_commit(self) -> None:
|
||||
"""Equal-priority ties resolve to the newest commit (REQ-2); the
|
||||
API returns commits newest-first."""
|
||||
commits = [
|
||||
{"commit": {"message": "ci: add workflow_dispatch trigger"}},
|
||||
{"commit": {"message": "ci: retrigger validation"}},
|
||||
]
|
||||
assert extract_conventional_msg(commits) == "ci: add workflow_dispatch trigger"
|
||||
|
||||
def test_all_skip_ci_fallback_strips_token(self) -> None:
|
||||
"""When every commit carries [skip ci], the token is stripped so
|
||||
the merge still triggers post-merge."""
|
||||
commits = [
|
||||
{"commit": {"message": "chore: noise [skip ci]"}},
|
||||
]
|
||||
assert extract_conventional_msg(commits) == "chore: noise"
|
||||
|
||||
|
||||
# -- run_cmd --
|
||||
|
||||
@@ -502,138 +468,3 @@ def test_main_module_block() -> None:
|
||||
exec(compile(source, am.__file__, "exec"), namespace)
|
||||
# Verify main is callable
|
||||
assert callable(namespace["main"])
|
||||
|
||||
|
||||
# -- DEVX-164: Vikunja outage resilience + self-approval diagnostics --
|
||||
|
||||
|
||||
class TestVikunjaLookupRetry:
|
||||
"""REQ-1: transient Vikunja API failures are retried, not fatal."""
|
||||
|
||||
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"}, clear=True)
|
||||
@patch("devx.ci.auto_merge.time.sleep")
|
||||
@patch("devx.ci.auto_merge.VikunjaClient")
|
||||
def test_retries_transient_api_error_then_succeeds(self, mock_client_cls: MagicMock, mock_sleep: MagicMock) -> None:
|
||||
"""A 404/502 during a Vikunja restart is retried until tasks list."""
|
||||
mock_client = MagicMock()
|
||||
mock_client.list_project_tasks.side_effect = [
|
||||
APIError(404, "Not Found"),
|
||||
APIError(502, "Bad Gateway"),
|
||||
[{"id": 1, "identifier": "DEVX-19", "title": "Add new feature"}],
|
||||
]
|
||||
mock_client_cls.return_value = mock_client
|
||||
validate_pr_title_matches_vikunja("DEVX-19: Add new feature", "DEVX-19")
|
||||
assert mock_client.list_project_tasks.call_count == 3
|
||||
assert mock_sleep.call_count == 2
|
||||
|
||||
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"}, clear=True)
|
||||
@patch("devx.ci.auto_merge.time.sleep")
|
||||
@patch("devx.ci.auto_merge.VikunjaClient")
|
||||
def test_retry_exhaustion_propagates_error(self, mock_client_cls: MagicMock, _mock_sleep: MagicMock) -> None:
|
||||
"""Persistent outage still fails after the bounded attempt count."""
|
||||
mock_client = MagicMock()
|
||||
mock_client.list_project_tasks.side_effect = APIError(502, "Bad Gateway")
|
||||
mock_client_cls.return_value = mock_client
|
||||
with pytest.raises(APIError, match="Bad Gateway"):
|
||||
validate_pr_title_matches_vikunja("DEVX-19: test", "DEVX-19")
|
||||
assert mock_client.list_project_tasks.call_count == 4
|
||||
|
||||
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"}, clear=True)
|
||||
@patch("devx.ci.auto_merge.time.sleep")
|
||||
@patch("devx.ci.auto_merge.VikunjaClient")
|
||||
def test_retries_connection_error(self, mock_client_cls: MagicMock, mock_sleep: MagicMock) -> None:
|
||||
"""Connection-level failures during restart are also retried."""
|
||||
import requests as req
|
||||
|
||||
mock_client = MagicMock()
|
||||
mock_client.list_project_tasks.side_effect = [
|
||||
req.ConnectionError("refused"),
|
||||
[{"id": 1, "identifier": "DEVX-19", "title": "Found me"}],
|
||||
]
|
||||
mock_client_cls.return_value = mock_client
|
||||
validate_pr_title_matches_vikunja("DEVX-19: Found me", "DEVX-19")
|
||||
assert mock_sleep.call_count == 1
|
||||
|
||||
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"}, clear=True)
|
||||
@patch("devx.ci.auto_merge.VikunjaClient")
|
||||
def test_missing_task_still_fails_without_retry_sleep(self, mock_client_cls: MagicMock) -> None:
|
||||
"""A healthy Vikunja that simply lacks the task fails as before."""
|
||||
mock_client = MagicMock()
|
||||
mock_client.list_project_tasks.return_value = []
|
||||
mock_client_cls.return_value = mock_client
|
||||
with pytest.raises(click.ClickException, match="Could not find"):
|
||||
validate_pr_title_matches_vikunja("DEVX-99: test", "DEVX-99")
|
||||
|
||||
|
||||
class TestMergeApprovalDiagnostics:
|
||||
"""REQ-2: merge 405 reports approval state + self-approval remediation."""
|
||||
|
||||
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok", "VIKUNJA_TOKEN": "tok"}, clear=True)
|
||||
@patch("devx.ci.auto_merge.validate_pr_title_matches_vikunja")
|
||||
@patch("devx.ci.auto_merge.GiteaClient")
|
||||
def test_405_without_approvals_shows_self_approval_hint(
|
||||
self, mock_client_cls: MagicMock, _mock_validate: MagicMock, tmp_path, monkeypatch
|
||||
) -> None: # type: ignore[no-untyped-def]
|
||||
monkeypatch.chdir(tmp_path)
|
||||
mock_client = MagicMock()
|
||||
mock_client.get_pr_commits.return_value = [
|
||||
{"commit": {"message": "fix: resolve timeout"}},
|
||||
]
|
||||
mock_client.merge_pr.side_effect = APIError(405, "Does not have enough approvals")
|
||||
mock_client.get_pr_reviews.return_value = []
|
||||
mock_client_cls.return_value = mock_client
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
main,
|
||||
["DEVX-19-fix-bug", "DEVX-19: Fix timeout", "owner/repo", "7"],
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
assert "Merge failed" in result.output
|
||||
assert "APPROVED" in result.output
|
||||
assert "non-author account" in result.output
|
||||
|
||||
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok", "VIKUNJA_TOKEN": "tok"}, clear=True)
|
||||
@patch("devx.ci.auto_merge.validate_pr_title_matches_vikunja")
|
||||
@patch("devx.ci.auto_merge.GiteaClient")
|
||||
def test_405_with_approvals_omits_hint(
|
||||
self, mock_client_cls: MagicMock, _mock_validate: MagicMock, tmp_path, monkeypatch
|
||||
) -> None: # type: ignore[no-untyped-def]
|
||||
monkeypatch.chdir(tmp_path)
|
||||
mock_client = MagicMock()
|
||||
mock_client.get_pr_commits.return_value = [
|
||||
{"commit": {"message": "fix: resolve timeout"}},
|
||||
]
|
||||
mock_client.merge_pr.side_effect = APIError(405, "Does not have enough approvals")
|
||||
mock_client.get_pr_reviews.return_value = [{"state": "APPROVED", "user": {"login": "kireto"}}]
|
||||
mock_client_cls.return_value = mock_client
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
main,
|
||||
["DEVX-19-fix-bug", "DEVX-19: Fix timeout", "owner/repo", "7"],
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
assert "Merge failed" in result.output
|
||||
assert "non-author account" not in result.output
|
||||
|
||||
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok", "VIKUNJA_TOKEN": "tok"}, clear=True)
|
||||
@patch("devx.ci.auto_merge.validate_pr_title_matches_vikunja")
|
||||
@patch("devx.ci.auto_merge.GiteaClient")
|
||||
def test_405_reviews_fetch_failure_still_raises(
|
||||
self, mock_client_cls: MagicMock, _mock_validate: MagicMock, tmp_path, monkeypatch
|
||||
) -> None: # type: ignore[no-untyped-def]
|
||||
"""If the reviews lookup itself fails, the merge error still surfaces."""
|
||||
monkeypatch.chdir(tmp_path)
|
||||
mock_client = MagicMock()
|
||||
mock_client.get_pr_commits.return_value = [
|
||||
{"commit": {"message": "fix: resolve timeout"}},
|
||||
]
|
||||
mock_client.merge_pr.side_effect = APIError(405, "Does not have enough approvals")
|
||||
mock_client.get_pr_reviews.side_effect = APIError(403, "Forbidden")
|
||||
mock_client_cls.return_value = mock_client
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
main,
|
||||
["DEVX-19-fix-bug", "DEVX-19: Fix timeout", "owner/repo", "7"],
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
assert "Merge failed" in result.output
|
||||
|
||||
@@ -13,9 +13,7 @@ from click.testing import CliRunner
|
||||
import devx.tools.build_image as build_image
|
||||
from devx.tools.build_image import (
|
||||
ImageSpec,
|
||||
PushHTTP500Error,
|
||||
build_full_tag,
|
||||
delete_remote_manifest,
|
||||
load_manifest,
|
||||
push_image,
|
||||
registry_login,
|
||||
@@ -198,7 +196,7 @@ class TestBuildImage:
|
||||
class TestPushImage:
|
||||
def test_success(self) -> None:
|
||||
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest", "1.0"])
|
||||
mock_result = MagicMock(returncode=0, stdout="")
|
||||
mock_result = MagicMock(returncode=0, stderr="", stdout="")
|
||||
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result) as mock_run:
|
||||
assert push_image(spec, "git.example.com") is True
|
||||
assert mock_run.call_count == 2
|
||||
@@ -206,8 +204,8 @@ class TestPushImage:
|
||||
def test_partial_failure(self) -> None:
|
||||
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest", "1.0"])
|
||||
results = [
|
||||
MagicMock(returncode=0, stdout=""),
|
||||
MagicMock(returncode=1, stdout="push failed"),
|
||||
MagicMock(returncode=0, stderr="", stdout=""),
|
||||
MagicMock(returncode=1, stderr="push failed", stdout=""),
|
||||
]
|
||||
with patch("devx.tools.build_image.subprocess.run", side_effect=results):
|
||||
assert push_image(spec, "git.example.com") is False
|
||||
@@ -218,212 +216,6 @@ class TestPushImage:
|
||||
assert push_image(spec, "git.example.com", dry_run=True) is True
|
||||
mock_run.assert_not_called()
|
||||
|
||||
def test_no_delete_on_success_with_creds(self) -> None:
|
||||
"""Push-first: no delete needed when push succeeds."""
|
||||
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest"])
|
||||
mock_result = MagicMock(returncode=0, stdout="")
|
||||
with (
|
||||
patch("devx.tools.build_image.subprocess.run", return_value=mock_result),
|
||||
patch("devx.tools.build_image.delete_remote_manifest") as mock_del,
|
||||
):
|
||||
assert push_image(spec, "git.example.com", username="user", token="tok") is True
|
||||
mock_del.assert_not_called()
|
||||
|
||||
def test_no_delete_without_creds(self) -> None:
|
||||
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest"])
|
||||
mock_result = MagicMock(returncode=0, stdout="")
|
||||
with (
|
||||
patch("devx.tools.build_image.subprocess.run", return_value=mock_result),
|
||||
patch("devx.tools.build_image.delete_remote_manifest") as mock_del,
|
||||
):
|
||||
assert push_image(spec, "git.example.com") is True
|
||||
mock_del.assert_not_called()
|
||||
|
||||
def test_delete_and_retry_on_already_exists(self) -> None:
|
||||
"""Gitea #31964: push fails with 'already exists', delete + retry."""
|
||||
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest"])
|
||||
results = [
|
||||
MagicMock(returncode=1, stdout="package version already exists"),
|
||||
MagicMock(returncode=0, stdout=""),
|
||||
]
|
||||
with (
|
||||
patch("devx.tools.build_image.subprocess.run", side_effect=results),
|
||||
patch("devx.tools.build_image.delete_remote_manifest", return_value=True) as mock_del,
|
||||
):
|
||||
assert push_image(spec, "git.example.com", username="user", token="tok") is True
|
||||
mock_del.assert_called_once_with(
|
||||
"git.example.com",
|
||||
"ci-base",
|
||||
"latest",
|
||||
"user",
|
||||
"tok",
|
||||
dry_run=False,
|
||||
)
|
||||
|
||||
def test_no_delete_on_non_already_exists_failure(self) -> None:
|
||||
"""Push fails for other reasons (non-500) — old manifest preserved."""
|
||||
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest"])
|
||||
mock_result = MagicMock(returncode=1, stdout="denied: requested access to the resource is denied")
|
||||
with (
|
||||
patch("devx.tools.build_image.subprocess.run", return_value=mock_result),
|
||||
patch("devx.tools.build_image.delete_remote_manifest") as mock_del,
|
||||
):
|
||||
assert push_image(spec, "git.example.com", username="user", token="tok") is False
|
||||
mock_del.assert_not_called()
|
||||
|
||||
def test_retry_also_fails(self) -> None:
|
||||
"""Gitea #31964 retry also fails — both pushes fail."""
|
||||
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest"])
|
||||
results = [
|
||||
MagicMock(returncode=1, stdout="package version already exists"),
|
||||
MagicMock(returncode=1, stdout="push failed again"),
|
||||
]
|
||||
with (
|
||||
patch("devx.tools.build_image.subprocess.run", side_effect=results),
|
||||
patch("devx.tools.build_image.delete_remote_manifest", return_value=True),
|
||||
):
|
||||
assert push_image(spec, "git.example.com", username="user", token="tok") is False
|
||||
|
||||
def test_http_500_retries_then_succeeds(self) -> None:
|
||||
"""HTTP 500 from registry race condition — retry succeeds."""
|
||||
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest"])
|
||||
results = [
|
||||
MagicMock(returncode=1, stdout="received unexpected HTTP status: 500 Internal Server Error"),
|
||||
MagicMock(returncode=0, stdout=""),
|
||||
]
|
||||
with (
|
||||
patch("devx.tools.build_image.subprocess.run", side_effect=results),
|
||||
patch("devx.tools.build_image.delete_remote_manifest") as mock_del,
|
||||
patch("time.sleep"),
|
||||
):
|
||||
assert push_image(spec, "git.example.com", username="user", token="tok") is True
|
||||
mock_del.assert_not_called()
|
||||
|
||||
def test_http_500_retries_all_fail(self) -> None:
|
||||
"""HTTP 500 retries exhausted — push fails, no delete attempted."""
|
||||
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest"])
|
||||
mock_result = MagicMock(returncode=1, stdout="received unexpected HTTP status: 500 Internal Server Error")
|
||||
with (
|
||||
patch("devx.tools.build_image.subprocess.run", return_value=mock_result),
|
||||
patch("devx.tools.build_image.delete_remote_manifest") as mock_del,
|
||||
patch("time.sleep"),
|
||||
):
|
||||
assert push_image(spec, "git.example.com", username="user", token="tok") is False
|
||||
mock_del.assert_not_called()
|
||||
|
||||
def test_run_push_raises_on_500(self) -> None:
|
||||
"""_run_push raises PushHTTP500Error when stdout contains 500."""
|
||||
from devx.tools.build_image import _run_push
|
||||
|
||||
mock_result = MagicMock(returncode=1, stdout="received unexpected HTTP status: 500 Internal Server Error")
|
||||
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result):
|
||||
with pytest.raises(PushHTTP500Error, match="500"):
|
||||
_run_push(["docker", "push", "img:latest"])
|
||||
|
||||
def test_run_push_no_raise_on_non_500(self) -> None:
|
||||
"""_run_push returns result when stdout has no 500."""
|
||||
from devx.tools.build_image import _run_push
|
||||
|
||||
mock_result = MagicMock(returncode=1, stdout="denied: access denied")
|
||||
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result):
|
||||
result = _run_push(["docker", "push", "img:latest"])
|
||||
assert result.returncode == 1
|
||||
|
||||
def test_run_push_no_raise_on_success(self) -> None:
|
||||
"""_run_push returns result on success."""
|
||||
from devx.tools.build_image import _run_push
|
||||
|
||||
mock_result = MagicMock(returncode=0, stdout="")
|
||||
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result):
|
||||
result = _run_push(["docker", "push", "img:latest"])
|
||||
assert result.returncode == 0
|
||||
|
||||
|
||||
class TestDeleteRemoteManifest:
|
||||
def test_dry_run(self) -> None:
|
||||
assert delete_remote_manifest("git.example.com", "ci-base", "latest", "u", "t", dry_run=True) is True
|
||||
|
||||
def test_tag_not_found(self) -> None:
|
||||
import urllib.error
|
||||
|
||||
with patch("devx.tools.build_image.urllib.request.urlopen") as mock_urlopen:
|
||||
mock_urlopen.side_effect = urllib.error.HTTPError("url", 404, "Not Found", {}, None)
|
||||
assert delete_remote_manifest("git.example.com", "ci-base", "latest", "u", "t") is True
|
||||
|
||||
def test_delete_success(self) -> None:
|
||||
mock_head_resp = MagicMock()
|
||||
mock_head_resp.__enter__.return_value.headers.get.return_value = "sha256:abc123"
|
||||
mock_del_resp = MagicMock()
|
||||
with patch("devx.tools.build_image.urllib.request.urlopen") as mock_urlopen:
|
||||
mock_urlopen.side_effect = [mock_head_resp, mock_del_resp]
|
||||
assert delete_remote_manifest("git.example.com", "ci-base", "latest", "u", "t") is True
|
||||
|
||||
def test_delete_404_treated_as_success(self) -> None:
|
||||
import urllib.error
|
||||
|
||||
mock_head_resp = MagicMock()
|
||||
mock_head_resp.__enter__.return_value.headers.get.return_value = "sha256:abc123"
|
||||
with patch("devx.tools.build_image.urllib.request.urlopen") as mock_urlopen:
|
||||
mock_urlopen.side_effect = [
|
||||
mock_head_resp,
|
||||
urllib.error.HTTPError("url", 404, "Not Found", {}, None),
|
||||
]
|
||||
assert delete_remote_manifest("git.example.com", "ci-base", "latest", "u", "t") is True
|
||||
|
||||
def test_head_error_does_not_block(self) -> None:
|
||||
import urllib.error
|
||||
|
||||
with patch("devx.tools.build_image.urllib.request.urlopen") as mock_urlopen:
|
||||
mock_urlopen.side_effect = urllib.error.HTTPError("url", 500, "Server Error", {}, None)
|
||||
assert delete_remote_manifest("git.example.com", "ci-base", "latest", "u", "t") is True
|
||||
|
||||
def test_url_error_does_not_block(self) -> None:
|
||||
import urllib.error
|
||||
|
||||
with patch("devx.tools.build_image.urllib.request.urlopen") as mock_urlopen:
|
||||
mock_urlopen.side_effect = urllib.error.URLError("network down")
|
||||
assert delete_remote_manifest("git.example.com", "ci-base", "latest", "u", "t") is True
|
||||
|
||||
def test_no_digest_does_not_block(self) -> None:
|
||||
mock_head_resp = MagicMock()
|
||||
mock_head_resp.__enter__.return_value.headers.get.return_value = None
|
||||
with patch("devx.tools.build_image.urllib.request.urlopen") as mock_urlopen:
|
||||
mock_urlopen.return_value = mock_head_resp
|
||||
assert delete_remote_manifest("git.example.com", "ci-base", "latest", "u", "t") is True
|
||||
|
||||
def test_head_405_passes_through(self) -> None:
|
||||
import urllib.error
|
||||
|
||||
with patch("devx.tools.build_image.urllib.request.urlopen") as mock_urlopen:
|
||||
mock_urlopen.side_effect = urllib.error.HTTPError("url", 405, "Method Not Allowed", {}, None)
|
||||
# 405 falls through with pass, digest never set, returns True
|
||||
assert delete_remote_manifest("git.example.com", "ci-base", "latest", "u", "t") is True
|
||||
assert mock_urlopen.call_count == 1
|
||||
|
||||
def test_delete_500_does_not_block(self) -> None:
|
||||
import urllib.error
|
||||
|
||||
mock_head_resp = MagicMock()
|
||||
mock_head_resp.__enter__.return_value.headers.get.return_value = "sha256:abc123"
|
||||
with patch("devx.tools.build_image.urllib.request.urlopen") as mock_urlopen:
|
||||
mock_urlopen.side_effect = [
|
||||
mock_head_resp,
|
||||
urllib.error.HTTPError("url", 500, "Server Error", {}, None),
|
||||
]
|
||||
assert delete_remote_manifest("git.example.com", "ci-base", "latest", "u", "t") is True
|
||||
|
||||
def test_delete_url_error_does_not_block(self) -> None:
|
||||
import urllib.error
|
||||
|
||||
mock_head_resp = MagicMock()
|
||||
mock_head_resp.__enter__.return_value.headers.get.return_value = "sha256:abc123"
|
||||
with patch("devx.tools.build_image.urllib.request.urlopen") as mock_urlopen:
|
||||
mock_urlopen.side_effect = [
|
||||
mock_head_resp,
|
||||
urllib.error.URLError("network down"),
|
||||
]
|
||||
assert delete_remote_manifest("git.example.com", "ci-base", "latest", "u", "t") is True
|
||||
|
||||
|
||||
class TestSortVersions:
|
||||
def test_sort_by_created_at_desc(self) -> None:
|
||||
@@ -485,38 +277,6 @@ class TestSelectForDeletion:
|
||||
to_delete = select_for_deletion(versions, keep=5)
|
||||
assert len(to_delete) == 0
|
||||
|
||||
def test_protect_exempts_version(self) -> None:
|
||||
"""REQ-2: a manifest-pinned version survives cleanup beyond --keep."""
|
||||
versions = [
|
||||
{"version": "0.1.0", "created_at": "2025-01-01"},
|
||||
{"version": "0.2.0", "created_at": "2025-02-01"},
|
||||
{"version": "0.3.0", "created_at": "2025-03-01"},
|
||||
{"version": "0.4.0", "created_at": "2025-04-01"},
|
||||
]
|
||||
to_delete = select_for_deletion(versions, keep=2, protect=frozenset({"0.1.0"}))
|
||||
deleted = {v["version"] for v in to_delete}
|
||||
assert "0.1.0" not in deleted
|
||||
assert deleted == {"0.2.0"}
|
||||
|
||||
def test_protect_multiple_versions(self) -> None:
|
||||
versions = [
|
||||
{"version": "0.1.0", "created_at": "2025-01-01"},
|
||||
{"version": "0.2.0", "created_at": "2025-02-01"},
|
||||
{"version": "0.3.0", "created_at": "2025-03-01"},
|
||||
{"version": "0.4.0", "created_at": "2025-04-01"},
|
||||
]
|
||||
to_delete = select_for_deletion(versions, keep=1, protect=frozenset({"0.1.0", "0.2.0"}))
|
||||
assert {v["version"] for v in to_delete} == {"0.3.0"}
|
||||
|
||||
def test_protect_default_empty_behaves_as_before(self) -> None:
|
||||
versions = [
|
||||
{"version": "0.1.0", "created_at": "2025-01-01"},
|
||||
{"version": "0.2.0", "created_at": "2025-02-01"},
|
||||
{"version": "0.3.0", "created_at": "2025-03-01"},
|
||||
]
|
||||
to_delete = select_for_deletion(versions, keep=2)
|
||||
assert {v["version"] for v in to_delete} == {"0.1.0"}
|
||||
|
||||
|
||||
class TestCleanImagesAPI:
|
||||
"""Tests for the clean_images module's API functions."""
|
||||
@@ -828,20 +588,11 @@ class TestCLIBuildImage:
|
||||
"devx.tools.build_image.subprocess.run",
|
||||
side_effect=[login_result, build_result, push_result],
|
||||
):
|
||||
with patch("devx.tools.build_image.delete_remote_manifest", return_value=True):
|
||||
result = runner.invoke(
|
||||
build_image.main,
|
||||
[
|
||||
"--dockerfile",
|
||||
str(dockerfile),
|
||||
"--name",
|
||||
"ci-base",
|
||||
"--push",
|
||||
"--registry",
|
||||
"git.example.com",
|
||||
],
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
result = runner.invoke(
|
||||
build_image.main,
|
||||
["--dockerfile", str(dockerfile), "--name", "ci-base", "--push", "--registry", "git.example.com"],
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
|
||||
|
||||
class TestCLICleanImages:
|
||||
|
||||
@@ -1,175 +0,0 @@
|
||||
"""Unit tests for devx.ci.check_pr_size."""
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from click.testing import CliRunner
|
||||
|
||||
from devx.ci.check_pr_size import (
|
||||
check_size,
|
||||
cli,
|
||||
get_diff_stats,
|
||||
has_refactoring_label,
|
||||
is_excluded,
|
||||
)
|
||||
|
||||
|
||||
class TestIsExcluded:
|
||||
def test_excludes_changelog(self) -> None:
|
||||
assert is_excluded("CHANGELOG.md", ["CHANGELOG.md"])
|
||||
|
||||
def test_excludes_svg_glob(self) -> None:
|
||||
assert is_excluded("docs/badges/coverage.svg", ["*.svg"])
|
||||
|
||||
def test_does_not_exclude_source(self) -> None:
|
||||
assert not is_excluded("src/devx/ci/check_pr_size.py", ["CHANGELOG.md", "*.svg"])
|
||||
|
||||
def test_excludes_readme(self) -> None:
|
||||
assert is_excluded("README.md", ["README.md"])
|
||||
|
||||
def test_excludes_plans_glob(self) -> None:
|
||||
assert is_excluded("docs/plans/sso-bridge-full-extraction.md", ["docs/plans/*"])
|
||||
|
||||
def test_does_not_exclude_specs(self) -> None:
|
||||
assert not is_excluded("docs/specs/DEVX-165.md", ["docs/plans/*"])
|
||||
|
||||
|
||||
class TestCheckSize:
|
||||
def test_under_limits_passes(self) -> None:
|
||||
stats = [("src/main.py", 100, 50), ("tests/test_main.py", 80, 20)]
|
||||
ok, detail = check_size(stats, max_lines=500, max_files=10, excluded_patterns=[])
|
||||
assert ok is True
|
||||
assert "250" in detail # 100+50+80+20
|
||||
|
||||
def test_over_lines_fails(self) -> None:
|
||||
stats = [("src/main.py", 300, 300)]
|
||||
ok, detail = check_size(stats, max_lines=500, max_files=10, excluded_patterns=[])
|
||||
assert ok is False
|
||||
assert "600" in detail
|
||||
|
||||
def test_over_files_fails(self) -> None:
|
||||
stats = [(f"src/file{i}.py", 10, 5) for i in range(15)]
|
||||
ok, detail = check_size(stats, max_lines=500, max_files=10, excluded_patterns=[])
|
||||
assert ok is False
|
||||
assert "15" in detail
|
||||
|
||||
def test_excluded_files_not_counted(self) -> None:
|
||||
stats = [("CHANGELOG.md", 500, 500), ("src/main.py", 10, 5)]
|
||||
ok, detail = check_size(stats, max_lines=500, max_files=10, excluded_patterns=["CHANGELOG.md"])
|
||||
assert ok is True
|
||||
assert "15" in detail # only 10+5
|
||||
|
||||
def test_empty_stats_passes(self) -> None:
|
||||
ok, detail = check_size([], max_lines=500, max_files=10, excluded_patterns=[])
|
||||
assert ok is True
|
||||
|
||||
|
||||
class TestGetDiffStats:
|
||||
@patch("devx.ci.check_pr_size.subprocess.run")
|
||||
def test_parses_numstat_output(self, mock_run: MagicMock) -> None:
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout="10\t5\tsrc/main.py\n20\t10\ttests/test_main.py\n",
|
||||
stderr="",
|
||||
)
|
||||
stats = get_diff_stats("origin/master", "HEAD")
|
||||
assert len(stats) == 2
|
||||
assert stats[0] == ("src/main.py", 10, 5)
|
||||
assert stats[1] == ("tests/test_main.py", 20, 10)
|
||||
|
||||
@patch("devx.ci.check_pr_size.subprocess.run")
|
||||
def test_handles_binary_files(self, mock_run: MagicMock) -> None:
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout="-\t-\timage.png\n",
|
||||
stderr="",
|
||||
)
|
||||
stats = get_diff_stats("origin/master", "HEAD")
|
||||
assert len(stats) == 1
|
||||
assert stats[0] == ("image.png", 0, 0)
|
||||
|
||||
@patch("devx.ci.check_pr_size.subprocess.run")
|
||||
def test_empty_output(self, mock_run: MagicMock) -> None:
|
||||
mock_run.return_value = MagicMock(returncode=0, stdout="", stderr="")
|
||||
stats = get_diff_stats("origin/master", "HEAD")
|
||||
assert stats == []
|
||||
|
||||
@patch("devx.ci.check_pr_size.subprocess.run")
|
||||
def test_git_diff_failure_raises(self, mock_run: MagicMock) -> None:
|
||||
import pytest
|
||||
|
||||
mock_run.return_value = MagicMock(returncode=1, stdout="", stderr="fatal: bad ref")
|
||||
with pytest.raises(Exception, match="git diff|bad ref"):
|
||||
get_diff_stats("origin/master", "HEAD")
|
||||
|
||||
@patch("devx.ci.check_pr_size.subprocess.run")
|
||||
def test_malformed_line_skipped(self, mock_run: MagicMock) -> None:
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout="not_a_valid_line\n10\t5\tsrc/main.py\n",
|
||||
stderr="",
|
||||
)
|
||||
stats = get_diff_stats("origin/master", "HEAD")
|
||||
assert len(stats) == 1
|
||||
assert stats[0] == ("src/main.py", 10, 5)
|
||||
|
||||
|
||||
class TestCli:
|
||||
@patch("devx.ci.check_pr_size.subprocess.run")
|
||||
def test_passes_when_small(self, mock_run: MagicMock) -> None:
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout="10\t5\tsrc/main.py\n",
|
||||
stderr="",
|
||||
)
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--base", "origin/master", "--head", "HEAD"])
|
||||
assert result.exit_code == 0
|
||||
assert "OK" in result.output
|
||||
|
||||
@patch("devx.ci.check_pr_size.subprocess.run")
|
||||
def test_fails_when_too_large(self, mock_run: MagicMock) -> None:
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout="300\t300\tsrc/main.py\n",
|
||||
stderr="",
|
||||
)
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--base", "origin/master", "--head", "HEAD", "--max-lines", "500"])
|
||||
assert result.exit_code != 0
|
||||
assert "600" in result.output
|
||||
|
||||
@patch("devx.ci.check_pr_size.subprocess.run")
|
||||
@patch("devx.ci.check_pr_size.has_refactoring_label", return_value=True)
|
||||
def test_bypasses_with_refactoring_label(self, mock_label: MagicMock, mock_run: MagicMock) -> None:
|
||||
mock_run.return_value = MagicMock(
|
||||
returncode=0,
|
||||
stdout="300\t300\tsrc/main.py\n",
|
||||
stderr="",
|
||||
)
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
["--base", "origin/master", "--head", "HEAD", "--repo", "owner/repo", "--pr-number", "42"],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "bypassed" in result.output.lower()
|
||||
|
||||
|
||||
class TestHasRefactoringLabel:
|
||||
@patch("devx.ci.check_pr_size.GiteaClient")
|
||||
@patch("devx.ci.check_pr_size.get_ci_token", return_value="fake-token")
|
||||
def test_returns_true_when_label_present(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.get_pr.return_value = {"labels": [{"name": "refactoring"}, {"name": "bug"}]}
|
||||
assert has_refactoring_label("owner/repo", 42) is True
|
||||
|
||||
@patch("devx.ci.check_pr_size.GiteaClient")
|
||||
@patch("devx.ci.check_pr_size.get_ci_token", return_value="fake-token")
|
||||
def test_returns_false_when_label_absent(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.get_pr.return_value = {"labels": [{"name": "bug"}]}
|
||||
assert has_refactoring_label("owner/repo", 42) is False
|
||||
|
||||
@patch("devx.ci.check_pr_size.get_ci_token", side_effect=Exception("no token"))
|
||||
def test_returns_false_on_error(self, mock_token: MagicMock) -> None:
|
||||
assert has_refactoring_label("owner/repo", 42) is False
|
||||
@@ -10,7 +10,6 @@ from devx.tools.check_test_speed import (
|
||||
DEFAULT_MAX_SECONDS,
|
||||
DEFAULT_MAX_SINGLE_SECONDS,
|
||||
TEST_COMMAND,
|
||||
_ci_scale_limit,
|
||||
check_per_test_speed,
|
||||
check_speed,
|
||||
cli,
|
||||
@@ -145,26 +144,7 @@ def test_main_module_block() -> None:
|
||||
mock_cli.assert_called_once_with([])
|
||||
|
||||
|
||||
class TestCiScaleLimit:
|
||||
def test_no_scaling_when_not_ci(self) -> None:
|
||||
with patch("devx.tools.check_test_speed._IS_CI", False):
|
||||
assert _ci_scale_limit(10.0) == 10.0
|
||||
assert _ci_scale_limit(0.5) == 0.5
|
||||
|
||||
def test_scales_when_ci(self) -> None:
|
||||
with patch("devx.tools.check_test_speed._IS_CI", True):
|
||||
with patch("devx.tools.check_test_speed.CI_SCALE_FACTOR", 4.0):
|
||||
assert _ci_scale_limit(10.0) == 40.0
|
||||
assert _ci_scale_limit(0.5) == 2.0
|
||||
|
||||
def test_custom_scale_factor(self) -> None:
|
||||
with patch("devx.tools.check_test_speed._IS_CI", True):
|
||||
with patch("devx.tools.check_test_speed.CI_SCALE_FACTOR", 2.5):
|
||||
assert _ci_scale_limit(10.0) == 25.0
|
||||
|
||||
|
||||
class TestMain:
|
||||
@patch("devx.tools.check_test_speed._IS_CI", False)
|
||||
@patch("devx.tools.check_test_speed.run_tests")
|
||||
@patch("devx.tools.check_test_speed.parse_duration")
|
||||
@patch("devx.tools.check_test_speed.check_speed")
|
||||
@@ -194,7 +174,6 @@ class TestMain:
|
||||
mock_parse_per.assert_called_once()
|
||||
mock_check_per.assert_called_once_with([], DEFAULT_MAX_SINGLE_SECONDS)
|
||||
|
||||
@patch("devx.tools.check_test_speed._IS_CI", False)
|
||||
@patch("devx.tools.check_test_speed.run_tests")
|
||||
@patch("devx.tools.check_test_speed.parse_duration")
|
||||
def test_slow_total_exits(
|
||||
@@ -210,7 +189,6 @@ class TestMain:
|
||||
assert result.exit_code == 1
|
||||
assert "too slow" in result.output.lower()
|
||||
|
||||
@patch("devx.tools.check_test_speed._IS_CI", False)
|
||||
@patch("devx.tools.check_test_speed.run_tests")
|
||||
@patch("devx.tools.check_test_speed.parse_duration")
|
||||
@patch("devx.tools.check_test_speed.check_speed")
|
||||
@@ -235,7 +213,6 @@ class TestMain:
|
||||
assert "Per-test speed check FAILED" in result.output
|
||||
assert "test_slow" in result.output
|
||||
|
||||
@patch("devx.tools.check_test_speed._IS_CI", False)
|
||||
@patch("devx.tools.check_test_speed.run_tests")
|
||||
def test_parse_failure_exits(
|
||||
self,
|
||||
@@ -248,7 +225,6 @@ class TestMain:
|
||||
assert result.exit_code == 1
|
||||
assert "Could not parse" in result.output
|
||||
|
||||
@patch("devx.tools.check_test_speed._IS_CI", False)
|
||||
@patch("devx.tools.check_test_speed.run_tests")
|
||||
@patch("devx.tools.check_test_speed.parse_duration")
|
||||
@patch("devx.tools.check_test_speed.check_speed")
|
||||
@@ -272,7 +248,6 @@ class TestMain:
|
||||
assert result.exit_code == 0
|
||||
mock_check.assert_called_once_with(0.5, 1.5)
|
||||
|
||||
@patch("devx.tools.check_test_speed._IS_CI", False)
|
||||
@patch("devx.tools.check_test_speed.run_tests")
|
||||
@patch("devx.tools.check_test_speed.parse_duration")
|
||||
@patch("devx.tools.check_test_speed.check_speed")
|
||||
@@ -295,7 +270,6 @@ class TestMain:
|
||||
mock_parse_per.assert_not_called()
|
||||
mock_check_per.assert_not_called()
|
||||
|
||||
@patch("devx.tools.check_test_speed._IS_CI", False)
|
||||
@patch("devx.tools.check_test_speed.run_tests")
|
||||
@patch("devx.tools.check_test_speed.parse_duration")
|
||||
@patch("devx.tools.check_test_speed.check_speed")
|
||||
@@ -318,32 +292,3 @@ class TestMain:
|
||||
result = runner.invoke(cli, ["--max-single-seconds", "1.0"])
|
||||
assert result.exit_code == 0
|
||||
mock_check_per.assert_called_once_with([], 1.0)
|
||||
|
||||
@patch("devx.tools.check_test_speed._IS_CI", True)
|
||||
@patch("devx.tools.check_test_speed.CI_SCALE_FACTOR", 4.0)
|
||||
@patch("devx.tools.check_test_speed.run_tests")
|
||||
@patch("devx.tools.check_test_speed.parse_duration")
|
||||
@patch("devx.tools.check_test_speed.check_speed")
|
||||
@patch("devx.tools.check_test_speed.parse_per_test_durations")
|
||||
@patch("devx.tools.check_test_speed.check_per_test_speed")
|
||||
def test_ci_scales_limits(
|
||||
self,
|
||||
mock_check_per: MagicMock,
|
||||
mock_parse_per: MagicMock,
|
||||
mock_check: MagicMock,
|
||||
mock_parse: MagicMock,
|
||||
mock_run: MagicMock,
|
||||
) -> None:
|
||||
mock_run.return_value = ("out\n", "err\n")
|
||||
mock_parse.return_value = 30.0 # would fail local (10s) but pass CI (40s)
|
||||
mock_parse_per.return_value = []
|
||||
mock_check_per.return_value = []
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, [])
|
||||
assert result.exit_code == 0
|
||||
assert "CI environment detected" in result.output
|
||||
assert "scaling limits by 4.0x" in result.output
|
||||
# check_speed called with scaled limit
|
||||
mock_check.assert_called_once_with(30.0, 40.0)
|
||||
mock_check_per.assert_called_once_with([], 2.0)
|
||||
|
||||
@@ -40,6 +40,7 @@ class TestCliGroups:
|
||||
result = runner.invoke(cli, ["molecule", "--help"])
|
||||
assert result.exit_code == 0
|
||||
assert "distribute" in result.output
|
||||
assert "guard" in result.output
|
||||
assert "all" in result.output
|
||||
|
||||
|
||||
@@ -107,6 +108,13 @@ class TestCiCommands:
|
||||
assert result.exit_code == 0
|
||||
mock_run.assert_called_once_with("devx.ci.post_merge", ["DEVX-1"])
|
||||
|
||||
@patch("devx.cli._run_module")
|
||||
def test_ci_pr_review(self, mock_run: MagicMock) -> None:
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["ci", "pr-review", "42"])
|
||||
assert result.exit_code == 0
|
||||
mock_run.assert_called_once_with("devx.ci.pr_review", ["42"])
|
||||
|
||||
@patch("devx.cli._run_module")
|
||||
def test_ci_publish(self, mock_run: MagicMock) -> None:
|
||||
runner = CliRunner()
|
||||
@@ -223,6 +231,13 @@ class TestMoleculeCommands:
|
||||
assert result.exit_code == 0
|
||||
mock_run.assert_called_once_with("devx.molecule.discover_runners", [])
|
||||
|
||||
@patch("devx.cli._run_module")
|
||||
def test_molecule_guard(self, mock_run: MagicMock) -> None:
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["molecule", "guard"])
|
||||
assert result.exit_code == 0
|
||||
mock_run.assert_called_once_with("devx.molecule.molecule_ci_guard", [])
|
||||
|
||||
@patch("devx.cli._run_module")
|
||||
def test_molecule_all(self, mock_run: MagicMock) -> None:
|
||||
runner = CliRunner()
|
||||
|
||||
@@ -38,7 +38,6 @@ class TestConfigConstants:
|
||||
def test_conventional_re(self) -> None:
|
||||
assert CONVENTIONAL_RE.match("feat: add feature")
|
||||
assert CONVENTIONAL_RE.match("fix(scope): bug fix")
|
||||
assert CONVENTIONAL_RE.match("deps: bump devx from v0.50.2 to v0.51.0")
|
||||
assert not CONVENTIONAL_RE.match("random message")
|
||||
assert not CONVENTIONAL_RE.match("feat:")
|
||||
assert not CONVENTIONAL_RE.match("BREAKING CHANGE: something")
|
||||
|
||||
@@ -1,688 +0,0 @@
|
||||
"""Unit tests for devx.ci.create_dependency_pr."""
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import click
|
||||
import pytest
|
||||
from click.testing import CliRunner
|
||||
|
||||
from devx.ci.create_dependency_pr import (
|
||||
cli,
|
||||
create_vikunja_task,
|
||||
find_existing_pr,
|
||||
find_pinned_version,
|
||||
update_pinned_version,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _mock_subprocess():
|
||||
"""Mock subprocess so CLI tests never run a real git clone."""
|
||||
with patch("devx.ci.create_dependency_pr.subprocess.run") as m:
|
||||
yield m
|
||||
|
||||
|
||||
class TestFindPinnedVersion:
|
||||
def test_finds_pip_git_pin(self, tmp_path: Path) -> None:
|
||||
content = "grm @ git+https://git.example.com/repo.git@v0.5.1"
|
||||
path = tmp_path / "pyproject.toml"
|
||||
path.write_text(content)
|
||||
version = find_pinned_version("grm", str(path))
|
||||
assert version == "0.5.1"
|
||||
|
||||
def test_finds_pyproject_pin(self, tmp_path: Path) -> None:
|
||||
content = 'grm = "0.5.1"'
|
||||
path = tmp_path / "pyproject.toml"
|
||||
path.write_text(content)
|
||||
version = find_pinned_version("grm", str(path))
|
||||
assert version == "0.5.1"
|
||||
|
||||
def test_finds_ansible_var_pin(self, tmp_path: Path) -> None:
|
||||
content = 'grm_version: "0.5.1"'
|
||||
path = tmp_path / "images.yml"
|
||||
path.write_text(content)
|
||||
version = find_pinned_version("grm", str(path))
|
||||
assert version == "0.5.1"
|
||||
|
||||
def test_finds_image_version_pin(self, tmp_path: Path) -> None:
|
||||
content = 'sso_bridge_image_version: "1.2.3"'
|
||||
path = tmp_path / "images.yml"
|
||||
path.write_text(content)
|
||||
version = find_pinned_version("sso_bridge", str(path))
|
||||
assert version == "1.2.3"
|
||||
|
||||
def test_returns_none_when_not_found(self, tmp_path: Path) -> None:
|
||||
path = tmp_path / "pyproject.toml"
|
||||
path.write_text('other = "1.0.0"')
|
||||
assert find_pinned_version("grm", str(path)) is None
|
||||
|
||||
def test_returns_none_when_file_missing(self, tmp_path: Path) -> None:
|
||||
assert find_pinned_version("grm", str(tmp_path / "nonexistent.toml")) is None
|
||||
|
||||
|
||||
class TestUpdatePinnedVersion:
|
||||
def test_updates_pip_git_pin(self, tmp_path: Path) -> None:
|
||||
content = "grm @ git+https://git.example.com/repo.git@v0.5.1"
|
||||
path = tmp_path / "pyproject.toml"
|
||||
path.write_text(content)
|
||||
changed = update_pinned_version(str(path), "grm", "0.5.1", "0.5.2")
|
||||
assert changed is True
|
||||
assert "0.5.2" in path.read_text()
|
||||
assert "0.5.1" not in path.read_text()
|
||||
|
||||
def test_updates_pyproject_pin(self, tmp_path: Path) -> None:
|
||||
content = 'grm = "0.5.1"'
|
||||
path = tmp_path / "pyproject.toml"
|
||||
path.write_text(content)
|
||||
changed = update_pinned_version(str(path), "grm", "0.5.1", "0.5.2")
|
||||
assert changed is True
|
||||
assert 'grm = "0.5.2"' in path.read_text()
|
||||
|
||||
def test_no_change_when_version_not_found(self, tmp_path: Path) -> None:
|
||||
content = 'other = "1.0.0"'
|
||||
path = tmp_path / "pyproject.toml"
|
||||
path.write_text(content)
|
||||
changed = update_pinned_version(str(path), "grm", "0.5.1", "0.5.2")
|
||||
assert changed is False
|
||||
|
||||
def test_no_change_when_file_missing(self, tmp_path: Path) -> None:
|
||||
changed = update_pinned_version(str(tmp_path / "nonexistent"), "grm", "0.5.1", "0.5.2")
|
||||
assert changed is False
|
||||
|
||||
|
||||
class TestFindExistingPr:
|
||||
@patch("devx.tools.create_pr.GiteaClient")
|
||||
def test_returns_pr_when_found(self, mock_client_cls: MagicMock) -> None:
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.list_prs.return_value = [
|
||||
{"head": {"ref": "deps/grm-0.5.2"}, "number": 42},
|
||||
{"head": {"ref": "other-branch"}, "number": 43},
|
||||
]
|
||||
result = find_existing_pr(mock_client, "deps/grm-0.5.2")
|
||||
assert result is not None
|
||||
assert result["number"] == 42
|
||||
|
||||
@patch("devx.tools.create_pr.GiteaClient")
|
||||
def test_returns_none_when_not_found(self, mock_client_cls: MagicMock) -> None:
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.list_prs.return_value = []
|
||||
result = find_existing_pr(mock_client, "deps/grm-0.5.2")
|
||||
assert result is None
|
||||
|
||||
|
||||
class TestCli:
|
||||
@patch("devx.ci.create_dependency_pr.find_pinned_version")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token")
|
||||
def test_same_version_no_pr(self, mock_token: MagicMock, mock_find: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_find.return_value = "0.5.2"
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
[
|
||||
"--package",
|
||||
"grm",
|
||||
"--new-version",
|
||||
"0.5.2",
|
||||
"--source-repo",
|
||||
"oblachno/grm",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "no pr needed" in result.output.lower()
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.find_pinned_version")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token")
|
||||
def test_dry_run(self, mock_token: MagicMock, mock_find: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_find.return_value = "0.5.1"
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
[
|
||||
"--package",
|
||||
"grm",
|
||||
"--new-version",
|
||||
"0.5.2",
|
||||
"--source-repo",
|
||||
"oblachno/grm",
|
||||
"--dry-run",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "DRY RUN" in result.output
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.find_pinned_version")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token")
|
||||
def test_version_not_found_fails(self, mock_token: MagicMock, mock_find: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_find.return_value = None
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
[
|
||||
"--package",
|
||||
"nonexistent",
|
||||
"--new-version",
|
||||
"1.0.0",
|
||||
"--source-repo",
|
||||
"oblachno/test",
|
||||
],
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
|
||||
|
||||
class TestCreateVikunjaTask:
|
||||
def test_returns_none_when_no_token(self) -> None:
|
||||
with patch("devx.ci.create_dependency_pr.get_vikunja_token", side_effect=click.ClickException("no token")):
|
||||
result = create_vikunja_task("Test", "desc")
|
||||
assert result is None
|
||||
|
||||
def test_returns_identifier_on_success(self) -> None:
|
||||
with (
|
||||
patch("devx.ci.create_dependency_pr.get_vikunja_token", return_value="fake-token"),
|
||||
patch("devx.api_clients.VikunjaClient") as mock_client_cls,
|
||||
):
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.create_task.return_value = {"identifier": "OBL-INFRA-999"}
|
||||
result = create_vikunja_task("Test", "desc")
|
||||
assert result == "OBL-INFRA-999"
|
||||
|
||||
def test_explicit_project_id_used(self) -> None:
|
||||
with (
|
||||
patch("devx.ci.create_dependency_pr.get_vikunja_token", return_value="fake-token"),
|
||||
patch("devx.api_clients.VikunjaClient") as mock_client_cls,
|
||||
):
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.create_task.return_value = {"identifier": "OBL-INFRA-1"}
|
||||
create_vikunja_task("Test", "desc", project_id=3)
|
||||
assert mock_client.create_task.call_args.args[0] == 3
|
||||
|
||||
|
||||
def _v2_mocks(digest: str = "sha256:deadbeef") -> list[MagicMock]:
|
||||
"""Token + manifest responses for the registry v2 digest lookup."""
|
||||
tok = MagicMock()
|
||||
tok.raise_for_status = MagicMock()
|
||||
tok.json.return_value = {"token": "bearer-tok"}
|
||||
man = MagicMock()
|
||||
man.raise_for_status = MagicMock()
|
||||
man.headers = {"Docker-Content-Digest": digest}
|
||||
return [tok, man]
|
||||
|
||||
|
||||
class TestResolveContainerDigest:
|
||||
"""REQ-1: existence check via packages API; digest via registry v2."""
|
||||
|
||||
def test_returns_digest_from_registry_v2(self) -> None:
|
||||
from devx.ci.create_dependency_pr import resolve_container_digest
|
||||
|
||||
files = MagicMock()
|
||||
files.raise_for_status = MagicMock()
|
||||
files.json.return_value = [{"name": "manifest.json", "sha256": "blob-not-pullable"}]
|
||||
with patch(
|
||||
"devx.ci.create_dependency_pr.requests.get",
|
||||
side_effect=[files, *_v2_mocks()],
|
||||
) as mock_get:
|
||||
digest = resolve_container_digest(
|
||||
"https://git.example.com/api/v1", "oblachno", "sso-bridge", "0.9.1", "tok"
|
||||
)
|
||||
assert digest == "sha256:deadbeef"
|
||||
man_call = mock_get.call_args_list[2]
|
||||
assert "manifests/0.9.1" in man_call.args[0]
|
||||
assert "oci.image.index" in man_call.kwargs["headers"]["Accept"]
|
||||
tok_call = mock_get.call_args_list[1]
|
||||
assert tok_call.kwargs["params"]["scope"] == "repository:oblachno/sso-bridge:pull"
|
||||
|
||||
def test_raises_when_version_missing(self) -> None:
|
||||
import requests
|
||||
|
||||
from devx.ci.create_dependency_pr import resolve_container_digest
|
||||
|
||||
mock_resp = MagicMock()
|
||||
http_err = requests.HTTPError("404")
|
||||
http_err.response = MagicMock(status_code=404)
|
||||
mock_resp.raise_for_status.side_effect = http_err
|
||||
with patch("devx.ci.create_dependency_pr.requests.get", return_value=mock_resp):
|
||||
with pytest.raises(click.ClickException, match="unpublished artifact"):
|
||||
resolve_container_digest("https://git.example.com/api/v1", "oblachno", "sso-bridge", "9.9.9", "tok")
|
||||
|
||||
def test_raises_when_v2_digest_missing(self) -> None:
|
||||
from devx.ci.create_dependency_pr import resolve_container_digest
|
||||
|
||||
files = MagicMock()
|
||||
files.raise_for_status = MagicMock()
|
||||
files.json.return_value = []
|
||||
tok, man = _v2_mocks(digest="")
|
||||
with patch(
|
||||
"devx.ci.create_dependency_pr.requests.get",
|
||||
side_effect=[files, tok, man],
|
||||
):
|
||||
with pytest.raises(click.ClickException, match="no Docker-Content-Digest"):
|
||||
resolve_container_digest("https://git.example.com/api/v1", "oblachno", "sso-bridge", "0.9.1", "tok")
|
||||
|
||||
def test_raises_when_v2_manifest_request_fails(self) -> None:
|
||||
import requests
|
||||
|
||||
from devx.ci.create_dependency_pr import resolve_container_digest
|
||||
|
||||
files = MagicMock()
|
||||
files.raise_for_status = MagicMock()
|
||||
files.json.return_value = []
|
||||
tok, _ = _v2_mocks()
|
||||
err = requests.HTTPError("500")
|
||||
err.response = MagicMock(status_code=500)
|
||||
man = MagicMock()
|
||||
man.raise_for_status.side_effect = err
|
||||
with patch(
|
||||
"devx.ci.create_dependency_pr.requests.get",
|
||||
side_effect=[files, tok, man],
|
||||
):
|
||||
with pytest.raises(click.ClickException, match="v2 digest lookup failed"):
|
||||
resolve_container_digest("https://git.example.com/api/v1", "oblachno", "sso-bridge", "0.9.1", "tok")
|
||||
|
||||
def test_raises_on_connection_error(self) -> None:
|
||||
import requests
|
||||
|
||||
from devx.ci.create_dependency_pr import resolve_container_digest
|
||||
|
||||
with patch(
|
||||
"devx.ci.create_dependency_pr.requests.get",
|
||||
side_effect=requests.ConnectionError("refused"),
|
||||
):
|
||||
with pytest.raises(click.ClickException, match="Registry lookup failed"):
|
||||
resolve_container_digest("https://git.example.com/api/v1", "oblachno", "sso-bridge", "0.9.1", "tok")
|
||||
|
||||
|
||||
class TestManifestHelpers:
|
||||
"""REQ-1: manifest read/update helpers."""
|
||||
|
||||
def test_read_version(self, tmp_path: Path) -> None:
|
||||
import json
|
||||
|
||||
from devx.ci.create_dependency_pr import read_manifest_version
|
||||
|
||||
p = tmp_path / "m.json"
|
||||
p.write_text(json.dumps({"schema_version": 1, "sso_bridge": {"version": "0.9.0"}}))
|
||||
assert read_manifest_version(str(p), "sso_bridge") == "0.9.0"
|
||||
|
||||
def test_read_version_missing_file(self, tmp_path: Path) -> None:
|
||||
from devx.ci.create_dependency_pr import read_manifest_version
|
||||
|
||||
assert read_manifest_version(str(tmp_path / "nope.json"), "sso_bridge") is None
|
||||
|
||||
def test_read_version_bad_json(self, tmp_path: Path) -> None:
|
||||
from devx.ci.create_dependency_pr import read_manifest_version
|
||||
|
||||
p = tmp_path / "m.json"
|
||||
p.write_text("not json{")
|
||||
assert read_manifest_version(str(p), "sso_bridge") is None
|
||||
|
||||
def test_read_version_missing_section(self, tmp_path: Path) -> None:
|
||||
import json
|
||||
|
||||
from devx.ci.create_dependency_pr import read_manifest_version
|
||||
|
||||
p = tmp_path / "m.json"
|
||||
p.write_text(json.dumps({"schema_version": 1, "other": "x"}))
|
||||
assert read_manifest_version(str(p), "sso_bridge") is None
|
||||
|
||||
def test_update_manifest_fields(self, tmp_path: Path) -> None:
|
||||
import json
|
||||
|
||||
from devx.ci.create_dependency_pr import update_manifest
|
||||
|
||||
p = tmp_path / "m.json"
|
||||
p.write_text(json.dumps({"schema_version": 1, "sso_bridge": {"version": "0.9.0"}}))
|
||||
changed = update_manifest(
|
||||
str(p),
|
||||
"sso_bridge",
|
||||
{"version": "0.9.1", "git_ref": "v0.9.1", "image_digest": "sha256:x"},
|
||||
)
|
||||
assert changed is True
|
||||
data = json.loads(p.read_text())
|
||||
assert data["sso_bridge"]["version"] == "0.9.1"
|
||||
assert data["sso_bridge"]["git_ref"] == "v0.9.1"
|
||||
assert data["sso_bridge"]["image_digest"] == "sha256:x"
|
||||
|
||||
def test_update_manifest_no_change(self, tmp_path: Path) -> None:
|
||||
import json
|
||||
|
||||
from devx.ci.create_dependency_pr import update_manifest
|
||||
|
||||
p = tmp_path / "m.json"
|
||||
p.write_text(json.dumps({"sso_bridge": {"version": "0.9.1"}}))
|
||||
assert update_manifest(str(p), "sso_bridge", {"version": "0.9.1"}) is False
|
||||
|
||||
def test_update_manifest_missing_file(self, tmp_path: Path) -> None:
|
||||
from devx.ci.create_dependency_pr import update_manifest
|
||||
|
||||
with pytest.raises(click.ClickException, match="not found"):
|
||||
update_manifest(str(tmp_path / "nope.json"), "sso_bridge", {"version": "1"})
|
||||
|
||||
def test_update_manifest_bad_json(self, tmp_path: Path) -> None:
|
||||
from devx.ci.create_dependency_pr import update_manifest
|
||||
|
||||
p = tmp_path / "m.json"
|
||||
p.write_text("broken{")
|
||||
with pytest.raises(click.ClickException, match="not valid JSON"):
|
||||
update_manifest(str(p), "sso_bridge", {"version": "1"})
|
||||
|
||||
def test_update_manifest_missing_section(self, tmp_path: Path) -> None:
|
||||
import json
|
||||
|
||||
from devx.ci.create_dependency_pr import update_manifest
|
||||
|
||||
p = tmp_path / "m.json"
|
||||
p.write_text(json.dumps({"schema_version": 1}))
|
||||
with pytest.raises(click.ClickException, match="no object section"):
|
||||
update_manifest(str(p), "sso_bridge", {"version": "1"})
|
||||
|
||||
|
||||
class TestCliManifestMode:
|
||||
@patch("devx.ci.create_dependency_pr.read_manifest_version")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token")
|
||||
def test_manifest_same_version_no_pr(self, mock_token: MagicMock, mock_read: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_read.return_value = "0.9.1"
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
[
|
||||
"--package",
|
||||
"sso_bridge",
|
||||
"--new-version",
|
||||
"0.9.1",
|
||||
"--source-repo",
|
||||
"oblachno/sso-bridge",
|
||||
"--manifest",
|
||||
"deploy/sso-bridge-release.json",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "no pr needed" in result.output.lower()
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.resolve_container_digest")
|
||||
@patch("devx.ci.create_dependency_pr.read_manifest_version")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token")
|
||||
def test_verify_container_runs_before_lookup(
|
||||
self, mock_token: MagicMock, mock_read: MagicMock, mock_digest: MagicMock
|
||||
) -> None:
|
||||
"""Verification failure aborts before the version lookup/PR steps."""
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_digest.side_effect = click.ClickException("unpublished artifact")
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
[
|
||||
"--package",
|
||||
"sso_bridge",
|
||||
"--new-version",
|
||||
"0.9.1",
|
||||
"--source-repo",
|
||||
"oblachno/sso-bridge",
|
||||
"--manifest",
|
||||
"deploy/m.json",
|
||||
"--verify-container",
|
||||
"oblachno/sso-bridge",
|
||||
],
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
mock_read.assert_not_called()
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.resolve_container_digest")
|
||||
@patch("devx.ci.create_dependency_pr.read_manifest_version")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token")
|
||||
def test_verify_container_resolves_digest(
|
||||
self, mock_token: MagicMock, mock_read: MagicMock, mock_digest: MagicMock
|
||||
) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_read.return_value = "0.9.1"
|
||||
mock_digest.return_value = "sha256:abc"
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
[
|
||||
"--package",
|
||||
"sso_bridge",
|
||||
"--new-version",
|
||||
"0.9.1",
|
||||
"--source-repo",
|
||||
"oblachno/sso-bridge",
|
||||
"--manifest",
|
||||
"deploy/m.json",
|
||||
"--verify-container",
|
||||
"oblachno/sso-bridge",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
mock_digest.assert_called_once()
|
||||
args = mock_digest.call_args[0]
|
||||
assert args[1:4] == ("oblachno", "sso-bridge", "0.9.1")
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token")
|
||||
def test_verify_container_invalid_format(self, mock_token: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
[
|
||||
"--package",
|
||||
"sso_bridge",
|
||||
"--new-version",
|
||||
"0.9.1",
|
||||
"--source-repo",
|
||||
"oblachno/sso-bridge",
|
||||
"--verify-container",
|
||||
"no-slash",
|
||||
],
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.resolve_container_digest")
|
||||
@patch("devx.ci.create_dependency_pr.read_manifest_version")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token")
|
||||
def test_container_tag_overrides_new_version(
|
||||
self, mock_token: MagicMock, mock_read: MagicMock, mock_digest: MagicMock
|
||||
) -> None:
|
||||
"""--container-tag selects the image tag when it differs from version."""
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_read.return_value = "0.9.1"
|
||||
mock_digest.return_value = "sha256:abc"
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
[
|
||||
"--package",
|
||||
"sso_bridge",
|
||||
"--new-version",
|
||||
"0.9.1",
|
||||
"--source-repo",
|
||||
"oblachno/sso-bridge",
|
||||
"--manifest",
|
||||
"deploy/m.json",
|
||||
"--verify-container",
|
||||
"oblachno/sso-bridge",
|
||||
"--container-tag",
|
||||
"0.2.4",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
args = mock_digest.call_args[0]
|
||||
assert args[1:4] == ("oblachno", "sso-bridge", "0.2.4")
|
||||
|
||||
|
||||
class TestBranchCreation:
|
||||
"""Branch creation uses the branches API (Gitea lacks POST /git/refs)."""
|
||||
|
||||
def _invoke(self) -> object:
|
||||
runner = CliRunner()
|
||||
return runner.invoke(
|
||||
cli,
|
||||
[
|
||||
"--package",
|
||||
"sso_bridge",
|
||||
"--new-version",
|
||||
"0.9.1",
|
||||
"--source-repo",
|
||||
"oblachno/sso-bridge",
|
||||
"--manifest",
|
||||
"deploy/m.json",
|
||||
],
|
||||
)
|
||||
|
||||
def _client(self, mock_client_cls: MagicMock) -> MagicMock:
|
||||
client = mock_client_cls.return_value
|
||||
client.create_pr.return_value = {"number": 1}
|
||||
return client
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.create_vikunja_task", return_value=None)
|
||||
@patch("devx.ci.create_dependency_pr.update_manifest", return_value=True)
|
||||
@patch("devx.ci.create_dependency_pr.read_manifest_version", return_value="0.9.0")
|
||||
@patch("devx.ci.create_dependency_pr.find_existing_pr", return_value=None)
|
||||
@patch("devx.ci.create_dependency_pr.GiteaClient")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token", return_value="tok")
|
||||
def test_creates_branch_via_branches_api(
|
||||
self,
|
||||
_token: MagicMock,
|
||||
mock_client_cls: MagicMock,
|
||||
_find: MagicMock,
|
||||
_read: MagicMock,
|
||||
_update: MagicMock,
|
||||
_task: MagicMock,
|
||||
) -> None:
|
||||
client = self._client(mock_client_cls)
|
||||
result = self._invoke()
|
||||
assert result.exit_code == 0
|
||||
assert "Created PR" in result.output
|
||||
post = client._request.call_args
|
||||
assert post.args[:2] == ("POST", "/branches")
|
||||
assert post.kwargs["json"] == {
|
||||
"new_branch_name": "deps/sso_bridge-0.9.1",
|
||||
"old_branch_name": "master",
|
||||
}
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.create_vikunja_task", return_value=None)
|
||||
@patch("devx.ci.create_dependency_pr.update_manifest", return_value=True)
|
||||
@patch("devx.ci.create_dependency_pr.read_manifest_version", return_value="0.9.0")
|
||||
@patch("devx.ci.create_dependency_pr.find_existing_pr", return_value=None)
|
||||
@patch("devx.ci.create_dependency_pr.GiteaClient")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token", return_value="tok")
|
||||
def test_existing_branch_tolerated(
|
||||
self,
|
||||
_token: MagicMock,
|
||||
mock_client_cls: MagicMock,
|
||||
_find: MagicMock,
|
||||
_read: MagicMock,
|
||||
_update: MagicMock,
|
||||
_task: MagicMock,
|
||||
) -> None:
|
||||
from devx.exceptions import APIError
|
||||
|
||||
client = self._client(mock_client_cls)
|
||||
client._request.side_effect = APIError(422, "branch already exists")
|
||||
result = self._invoke()
|
||||
assert result.exit_code == 0
|
||||
assert "already exists" in result.output.lower()
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.create_vikunja_task", return_value=None)
|
||||
@patch("devx.ci.create_dependency_pr.update_manifest", return_value=True)
|
||||
@patch("devx.ci.create_dependency_pr.read_manifest_version", return_value="0.9.0")
|
||||
@patch("devx.ci.create_dependency_pr.find_existing_pr", return_value=None)
|
||||
@patch("devx.ci.create_dependency_pr.GiteaClient")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token", return_value="tok")
|
||||
def test_branch_api_error_fails(
|
||||
self,
|
||||
_token: MagicMock,
|
||||
mock_client_cls: MagicMock,
|
||||
_find: MagicMock,
|
||||
_read: MagicMock,
|
||||
_update: MagicMock,
|
||||
_task: MagicMock,
|
||||
) -> None:
|
||||
from devx.exceptions import APIError
|
||||
|
||||
client = self._client(mock_client_cls)
|
||||
client._request.side_effect = APIError(500, "boom")
|
||||
result = self._invoke()
|
||||
assert result.exit_code != 0
|
||||
assert "Failed to create branch" in result.output
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.create_vikunja_task", return_value=None)
|
||||
@patch("devx.ci.create_dependency_pr.update_manifest", return_value=True)
|
||||
@patch("devx.ci.create_dependency_pr.read_manifest_version", return_value="0.9.0")
|
||||
@patch("devx.ci.create_dependency_pr.find_existing_pr", return_value=None)
|
||||
@patch("devx.ci.create_dependency_pr.GiteaClient")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token", return_value="tok")
|
||||
def test_checks_out_remote_tracking_branch(
|
||||
self,
|
||||
_token: MagicMock,
|
||||
mock_client_cls: MagicMock,
|
||||
_find: MagicMock,
|
||||
_read: MagicMock,
|
||||
_update: MagicMock,
|
||||
_task: MagicMock,
|
||||
) -> None:
|
||||
import subprocess
|
||||
|
||||
self._client(mock_client_cls)
|
||||
result = self._invoke()
|
||||
assert result.exit_code == 0
|
||||
calls = [c.args[0] for c in subprocess.run.call_args_list]
|
||||
assert ["git", "fetch", "origin", "deps/sso_bridge-0.9.1:refs/remotes/origin/deps/sso_bridge-0.9.1"] in calls
|
||||
assert ["git", "checkout", "-B", "deps/sso_bridge-0.9.1", "origin/deps/sso_bridge-0.9.1"] in calls
|
||||
|
||||
@patch("devx.ci.create_dependency_pr.create_vikunja_task", return_value="OBL-INFRA-581")
|
||||
@patch("devx.ci.create_dependency_pr.update_manifest", return_value=True)
|
||||
@patch("devx.ci.create_dependency_pr.read_manifest_version", return_value="0.9.0")
|
||||
@patch("devx.ci.create_dependency_pr.find_existing_pr", return_value=None)
|
||||
@patch("devx.ci.create_dependency_pr.GiteaClient")
|
||||
@patch("devx.ci.create_dependency_pr.get_ci_token", return_value="tok")
|
||||
def test_task_id_in_branch_and_spec_committed(
|
||||
self,
|
||||
_token: MagicMock,
|
||||
mock_client_cls: MagicMock,
|
||||
_find: MagicMock,
|
||||
_read: MagicMock,
|
||||
_update: MagicMock,
|
||||
mock_task: MagicMock,
|
||||
) -> None:
|
||||
import subprocess
|
||||
|
||||
client = self._client(mock_client_cls)
|
||||
result = self._invoke()
|
||||
assert result.exit_code == 0
|
||||
# Branch embeds the target-repo task ID
|
||||
post = client._request.call_args
|
||||
assert post.kwargs["json"]["new_branch_name"] == "deps/OBL-INFRA-581-sso_bridge-0.9.1"
|
||||
# Spec file added alongside the manifest change
|
||||
add = next(c for c in subprocess.run.call_args_list if c.args[0][:2] == ["git", "add"])
|
||||
assert "docs/specs/OBL-INFRA-581.md" in add.args[0]
|
||||
# PR title carries the task ID
|
||||
assert client.create_pr.call_args.kwargs["title"] == "OBL-INFRA-581: Bump sso_bridge to 0.9.1"
|
||||
mock_task.assert_called_once()
|
||||
# Commit supplies an explicit identity (fresh clones have none)
|
||||
commit = next(c for c in subprocess.run.call_args_list if "commit" in c.args[0])
|
||||
assert "user.name=gitea-actions-bot" in commit.args[0]
|
||||
assert "user.email=actions@oblachno.fyi" in commit.args[0]
|
||||
|
||||
def test_retries_404_until_published(self) -> None:
|
||||
import requests
|
||||
|
||||
from devx.ci.create_dependency_pr import resolve_container_digest
|
||||
|
||||
err = requests.HTTPError("404")
|
||||
err.response = MagicMock(status_code=404)
|
||||
fail = MagicMock()
|
||||
fail.raise_for_status.side_effect = err
|
||||
ok = MagicMock()
|
||||
ok.raise_for_status = MagicMock()
|
||||
ok.json.return_value = [{"name": "manifest.json", "sha256": "cafe"}]
|
||||
with (
|
||||
patch("devx.ci.create_dependency_pr.requests.get", side_effect=[fail, ok, *_v2_mocks("sha256:cafe")]),
|
||||
patch("devx.ci.create_dependency_pr.time.sleep"),
|
||||
):
|
||||
digest = resolve_container_digest(
|
||||
"https://git.example.com/api/v1", "oblachno", "sso-bridge", "0.9.1", "tok", timeout_s=60
|
||||
)
|
||||
assert digest == "sha256:cafe"
|
||||
@@ -1,83 +0,0 @@
|
||||
"""Unit tests for devx.ci.fast_molecule."""
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from click.testing import CliRunner
|
||||
|
||||
from devx.ci.fast_molecule import (
|
||||
build_molecule_commands,
|
||||
cli,
|
||||
get_molecule_scenarios,
|
||||
)
|
||||
|
||||
|
||||
class TestGetMoleculeScenarios:
|
||||
def test_finds_scenarios(self, tmp_path: Path) -> None:
|
||||
roles_dir = tmp_path / "ansible" / "roles" / "myrole" / "molecule"
|
||||
roles_dir.mkdir(parents=True)
|
||||
(roles_dir / "default").mkdir()
|
||||
(roles_dir / "default" / "molecule.yml").write_text("name: default")
|
||||
(roles_dir / "full").mkdir()
|
||||
(roles_dir / "full" / "molecule.yml").write_text("name: full")
|
||||
(roles_dir / "no_scenario").mkdir() # No molecule.yml
|
||||
|
||||
scenarios = get_molecule_scenarios("myrole", str(tmp_path / "ansible" / "roles"))
|
||||
assert sorted(scenarios) == ["default", "full"]
|
||||
|
||||
def test_returns_empty_when_no_molecule_dir(self, tmp_path: Path) -> None:
|
||||
scenarios = get_molecule_scenarios("nonexistent", str(tmp_path / "ansible" / "roles"))
|
||||
assert scenarios == []
|
||||
|
||||
|
||||
class TestBuildMoleculeCommands:
|
||||
def test_builds_commands_for_roles(self, tmp_path: Path) -> None:
|
||||
roles_dir = tmp_path / "ansible" / "roles"
|
||||
for role in ["role_a", "role_b"]:
|
||||
mol_dir = roles_dir / role / "molecule" / "default"
|
||||
mol_dir.mkdir(parents=True)
|
||||
(mol_dir / "molecule.yml").write_text("name: default")
|
||||
|
||||
commands = build_molecule_commands({"role_a", "role_b"}, str(roles_dir))
|
||||
assert len(commands) == 2
|
||||
# REQ-3: emitted commands are exactly what run_molecule_scenario
|
||||
# executes — full `molecule test -s <scenario>`, no synthetic flags.
|
||||
assert commands == ["molecule test -s default", "molecule test -s default"]
|
||||
assert all("--destroy" not in c and "--platform-name" not in c for c in commands)
|
||||
|
||||
def test_empty_when_no_scenarios(self, tmp_path: Path) -> None:
|
||||
commands = build_molecule_commands({"nonexistent"}, str(tmp_path / "ansible" / "roles"))
|
||||
assert commands == []
|
||||
|
||||
def test_empty_when_no_roles(self) -> None:
|
||||
assert build_molecule_commands(set()) == []
|
||||
|
||||
|
||||
class TestCli:
|
||||
@patch("devx.ci.fast_molecule.get_changed_files")
|
||||
def test_no_changes(self, mock_get: MagicMock) -> None:
|
||||
mock_get.return_value = []
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--base", "origin/master", "--head", "HEAD"])
|
||||
assert result.exit_code == 0
|
||||
assert "No files changed" in result.output
|
||||
|
||||
@patch("devx.ci.fast_molecule.detect_changed_roles")
|
||||
@patch("devx.ci.fast_molecule.get_changed_files")
|
||||
def test_no_ansible_changes(self, mock_get: MagicMock, mock_detect: MagicMock) -> None:
|
||||
mock_get.return_value = ["src/main.py", "README.md"]
|
||||
mock_detect.return_value = set()
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--base", "origin/master", "--head", "HEAD"])
|
||||
assert result.exit_code == 0
|
||||
assert "No Ansible roles changed" in result.output
|
||||
|
||||
@patch("devx.ci.fast_molecule.detect_changed_roles")
|
||||
@patch("devx.ci.fast_molecule.get_changed_files")
|
||||
def test_detects_changed_roles(self, mock_get: MagicMock, mock_detect: MagicMock) -> None:
|
||||
mock_get.return_value = ["ansible/roles/sso_bridge/tasks/main.yml"]
|
||||
mock_detect.return_value = {"sso_bridge"}
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--base", "origin/master", "--head", "HEAD"])
|
||||
assert result.exit_code == 0
|
||||
assert "sso_bridge" in result.output
|
||||
@@ -365,23 +365,6 @@ class TestCollectQuality:
|
||||
badge = collect_quality(tmp_path)
|
||||
assert badge["message"] == "A"
|
||||
|
||||
@patch("devx.tools.generate_badges.run_command")
|
||||
def test_bandit_uses_repo_config_when_present(self, mock_run: MagicMock, tmp_path: Path) -> None: # type: ignore[no-untyped-def]
|
||||
(tmp_path / "pyproject.toml").write_text("[tool.bandit]\nskips = ['B501']\n")
|
||||
mock_run.return_value = (0, "", "")
|
||||
collect_quality(tmp_path)
|
||||
bandit_call = [c for c in mock_run.call_args_list if "bandit" in c.args[0]][0]
|
||||
assert "-c" in bandit_call.args[0]
|
||||
assert "pyproject.toml" in bandit_call.args[0]
|
||||
|
||||
@patch("devx.tools.generate_badges.run_command")
|
||||
def test_bandit_plain_invocation_without_repo_config(self, mock_run: MagicMock, tmp_path: Path) -> None: # type: ignore[no-untyped-def]
|
||||
(tmp_path / "pyproject.toml").write_text("[project]\nname = 'x'\n")
|
||||
mock_run.return_value = (0, "", "")
|
||||
collect_quality(tmp_path)
|
||||
bandit_call = [c for c in mock_run.call_args_list if "bandit" in c.args[0]][0]
|
||||
assert "-c" not in bandit_call.args[0]
|
||||
|
||||
|
||||
class TestGenerateBadges:
|
||||
@patch("devx.tools.generate_badges.collect_quality")
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import platform
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
@@ -67,10 +66,6 @@ class TestDownload:
|
||||
with patch("urllib.request.urlopen", return_value=_FakeResponse()) as mock_urlopen:
|
||||
install_tools._download("https://example.com/file", dest)
|
||||
mock_urlopen.assert_called_once()
|
||||
call_args = mock_urlopen.call_args
|
||||
req = call_args.args[0]
|
||||
assert isinstance(req, urllib.request.Request)
|
||||
assert req.get_header("User-agent") == "devx/install-tools"
|
||||
assert dest.read_bytes() == b"data"
|
||||
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ import subprocess # nosec B404
|
||||
import time
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from click.testing import CliRunner
|
||||
|
||||
from devx.ci.integration_guard import cli
|
||||
@@ -112,8 +111,9 @@ class TestCli:
|
||||
clear=True,
|
||||
),
|
||||
patch("devx.ci.integration_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.ci.integration_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.ci.integration_guard.get_running_jobs", side_effect=get_jobs_side_effect),
|
||||
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
|
||||
patch("os.killpg") as mock_killpg,
|
||||
patch("os.getpgid") as mock_getpgid,
|
||||
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
|
||||
@@ -158,8 +158,9 @@ class TestCli:
|
||||
clear=True,
|
||||
),
|
||||
patch("devx.ci.integration_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.ci.integration_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.ci.integration_guard.get_running_jobs", side_effect=get_jobs_side_effect),
|
||||
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
|
||||
patch("os.killpg", side_effect=ProcessLookupError("no such process")),
|
||||
patch("os.getpgid") as mock_getpgid,
|
||||
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
|
||||
@@ -202,8 +203,9 @@ class TestCli:
|
||||
clear=True,
|
||||
),
|
||||
patch("devx.ci.integration_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.ci.integration_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.ci.integration_guard.get_running_jobs", side_effect=get_jobs_side_effect),
|
||||
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
|
||||
patch("os.killpg") as mock_killpg,
|
||||
patch("os.getpgid") as mock_getpgid,
|
||||
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
|
||||
@@ -287,125 +289,3 @@ def test_main_module_block() -> None:
|
||||
namespace = dict(ig.__dict__)
|
||||
exec(compile(source, ig.__file__, "exec"), namespace)
|
||||
assert callable(namespace["cli"])
|
||||
|
||||
|
||||
class TestGetRunningJobs:
|
||||
def test_returns_jobs(self) -> None:
|
||||
with patch("devx.ci.integration_guard.requests.get") as mock_get:
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"jobs": [
|
||||
{"name": "integration-tests (0)", "conclusion": "success"},
|
||||
{"name": "integration-tests (1)", "conclusion": "failure"},
|
||||
]
|
||||
}
|
||||
mock_response.raise_for_status.return_value = None
|
||||
mock_get.return_value = mock_response
|
||||
|
||||
from devx.ci.integration_guard import get_running_jobs
|
||||
|
||||
jobs = get_running_jobs("https://gitea.example", "owner", "repo", "token", 123)
|
||||
assert len(jobs) == 2
|
||||
mock_get.assert_called_once()
|
||||
|
||||
def test_raises_on_request_error(self) -> None:
|
||||
import requests
|
||||
|
||||
with patch("devx.ci.integration_guard.requests.get") as mock_get:
|
||||
mock_get.side_effect = requests.RequestException("boom")
|
||||
with pytest.raises(requests.RequestException):
|
||||
from devx.ci.integration_guard import get_running_jobs
|
||||
|
||||
get_running_jobs("https://gitea.example", "owner", "repo", "token", 123)
|
||||
|
||||
|
||||
class TestAnyOtherRunnerFailed:
|
||||
def test_detects_other_failure(self) -> None:
|
||||
from devx.ci.integration_guard import any_other_runner_failed
|
||||
|
||||
jobs = [
|
||||
{"name": "integration-tests (0)", "conclusion": "success"},
|
||||
{"name": "integration-tests (1)", "conclusion": "failure"},
|
||||
{"name": "integration-tests (2)", "conclusion": "running"},
|
||||
]
|
||||
assert any_other_runner_failed(jobs, "integration-tests", 0) is True
|
||||
|
||||
def test_ignores_current_runner(self) -> None:
|
||||
from devx.ci.integration_guard import any_other_runner_failed
|
||||
|
||||
jobs = [
|
||||
{"name": "integration-tests (0)", "conclusion": "failure"},
|
||||
{"name": "integration-tests (1)", "conclusion": "success"},
|
||||
]
|
||||
assert any_other_runner_failed(jobs, "integration-tests", 0) is False
|
||||
|
||||
def test_ignores_non_matching_jobs(self) -> None:
|
||||
from devx.ci.integration_guard import any_other_runner_failed
|
||||
|
||||
jobs = [
|
||||
{"name": "quality", "conclusion": "failure"},
|
||||
{"name": "integration-tests (1)", "conclusion": "success"},
|
||||
]
|
||||
assert any_other_runner_failed(jobs, "integration-tests", 0) is False
|
||||
|
||||
|
||||
class TestPollForOtherFailures:
|
||||
def test_sets_failed_event_when_other_runner_fails(self) -> None:
|
||||
from devx.ci.integration_guard import poll_for_other_failures
|
||||
|
||||
stop_event = MagicMock()
|
||||
failed_event = MagicMock()
|
||||
|
||||
def side_effect(*args, **kwargs):
|
||||
if stop_event.wait.call_count < 1:
|
||||
return [
|
||||
{"name": "integration-tests (0)", "conclusion": "success"},
|
||||
{"name": "integration-tests (1)", "conclusion": "failure"},
|
||||
]
|
||||
return []
|
||||
|
||||
with patch("devx.ci.integration_guard.get_running_jobs") as mock_get_jobs:
|
||||
mock_get_jobs.side_effect = side_effect
|
||||
stop_event.is_set.side_effect = [False, False]
|
||||
stop_event.wait.return_value = True
|
||||
|
||||
poll_for_other_failures(
|
||||
"https://gitea.example",
|
||||
"owner",
|
||||
"repo",
|
||||
"token",
|
||||
123,
|
||||
"integration-tests",
|
||||
0,
|
||||
stop_event,
|
||||
failed_event,
|
||||
)
|
||||
|
||||
failed_event.set.assert_called_once()
|
||||
|
||||
def test_poll_warns_on_api_error(self) -> None:
|
||||
import requests
|
||||
|
||||
from devx.ci.integration_guard import poll_for_other_failures
|
||||
|
||||
stop_event = MagicMock()
|
||||
failed_event = MagicMock()
|
||||
|
||||
with patch("devx.ci.integration_guard.get_running_jobs") as mock_get_jobs:
|
||||
mock_get_jobs.side_effect = requests.RequestException("boom")
|
||||
stop_event.is_set.side_effect = [False, True]
|
||||
stop_event.wait.return_value = True
|
||||
|
||||
poll_for_other_failures(
|
||||
"https://gitea.example",
|
||||
"owner",
|
||||
"repo",
|
||||
"token",
|
||||
123,
|
||||
"integration-tests",
|
||||
0,
|
||||
stop_event,
|
||||
failed_event,
|
||||
)
|
||||
|
||||
failed_event.set.assert_not_called()
|
||||
|
||||
@@ -1,195 +1,107 @@
|
||||
"""Unit tests for devx.molecule.molecule_changed.
|
||||
|
||||
Verifies that the script correctly detects changed roles, fails open on
|
||||
unmapped ansible paths, and only emits roles that exist on disk.
|
||||
Verifies that the script correctly detects changed roles and maps
|
||||
them to make targets.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
from click.testing import CliRunner
|
||||
|
||||
from devx.molecule.molecule_changed import (
|
||||
PLAYBOOK_ROLE_MAP,
|
||||
detect_changed_roles,
|
||||
get_changed_files,
|
||||
main,
|
||||
role_to_target,
|
||||
roles_to_targets,
|
||||
)
|
||||
|
||||
TESTABLE_ROLES = ("docker_base", "app_container", "restore", "observability")
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def roles_dir(tmp_path: Path) -> Path:
|
||||
"""Fake roles dir: 4 testable roles (molecule/ present) + 1 untestable."""
|
||||
for role in TESTABLE_ROLES:
|
||||
(tmp_path / role / "molecule" / "default").mkdir(parents=True)
|
||||
(tmp_path / "untested_role").mkdir() # exists but no molecule/ dir
|
||||
return tmp_path
|
||||
|
||||
|
||||
def test_detect_role_change(roles_dir: Path):
|
||||
def test_detect_role_change():
|
||||
"""A file in ansible/roles/<role>/ maps to that role."""
|
||||
roles = detect_changed_roles(["ansible/roles/docker_base/tasks/main.yml"], roles_dir)
|
||||
assert roles == {"docker_base"}
|
||||
files = ["ansible/roles/docker_base/tasks/main.yml"]
|
||||
roles = detect_changed_roles(files)
|
||||
assert "docker_base" in roles
|
||||
|
||||
|
||||
def test_detect_role_not_on_disk_is_dropped(roles_dir: Path):
|
||||
"""Changed role absent from roles_dir selects nothing (REQ-2)."""
|
||||
roles = detect_changed_roles(["ansible/roles/sso_config/tasks/main.yml"], roles_dir)
|
||||
assert roles == set()
|
||||
|
||||
|
||||
def test_detect_playbook_change(roles_dir: Path):
|
||||
"""A mapped playbook maps to its included roles, filtered to disk."""
|
||||
def test_detect_playbook_change():
|
||||
"""A playbook change maps to its included roles."""
|
||||
files = ["ansible/playbooks/deploy-observability.yml"]
|
||||
roles = detect_changed_roles(files, roles_dir)
|
||||
# zitadel + crowdsec are mapped but absent from the fake roles dir.
|
||||
assert roles == {"observability", "docker_base"}
|
||||
roles = detect_changed_roles(files)
|
||||
assert "observability" in roles
|
||||
assert "docker_base" in roles
|
||||
assert "zitadel" in roles
|
||||
|
||||
|
||||
def test_detect_deploy_controller_playbook(roles_dir: Path):
|
||||
"""The deploy-controller playbook maps to the deploy_controller role."""
|
||||
assert PLAYBOOK_ROLE_MAP["ansible/playbooks/deploy-controller.yml"] == ["deploy_controller"]
|
||||
(roles_dir / "deploy_controller" / "molecule" / "default").mkdir(parents=True)
|
||||
roles = detect_changed_roles(["ansible/playbooks/deploy-controller.yml"], roles_dir)
|
||||
assert roles == {"deploy_controller"}
|
||||
def test_detect_shared_infra_triggers_all():
|
||||
"""ansible.cfg change triggers all roles."""
|
||||
files = ["ansible/ansible.cfg"]
|
||||
roles = detect_changed_roles(files)
|
||||
assert len(roles) == 10 # all roles
|
||||
|
||||
|
||||
def test_detect_shared_infra_triggers_all(roles_dir: Path):
|
||||
"""ansible.cfg change triggers all testable roles only."""
|
||||
roles = detect_changed_roles(["ansible/ansible.cfg"], roles_dir)
|
||||
assert roles == set(TESTABLE_ROLES)
|
||||
|
||||
|
||||
def test_detect_molecule_shared_path(roles_dir: Path):
|
||||
"""ansible/molecule/ change triggers all testable roles."""
|
||||
roles = detect_changed_roles(["ansible/molecule/Dockerfile"], roles_dir)
|
||||
assert roles == set(TESTABLE_ROLES)
|
||||
|
||||
|
||||
def test_detect_requirements_yml_triggers_all(roles_dir: Path):
|
||||
"""ansible/requirements.yml change triggers all testable roles."""
|
||||
roles = detect_changed_roles(["ansible/requirements.yml"], roles_dir)
|
||||
assert roles == set(TESTABLE_ROLES)
|
||||
|
||||
|
||||
def test_detect_group_vars_triggers_all(roles_dir: Path):
|
||||
"""ansible/group_vars/ change triggers all testable roles (REQ-1)."""
|
||||
roles = detect_changed_roles(["ansible/group_vars/all/images.yml"], roles_dir)
|
||||
assert roles == set(TESTABLE_ROLES)
|
||||
|
||||
|
||||
def test_unmapped_playbook_fails_open(roles_dir: Path):
|
||||
"""An unmapped playbook selects all testable roles (REQ-1)."""
|
||||
roles = detect_changed_roles(["ansible/playbooks/new-deploy.yml"], roles_dir)
|
||||
assert roles == set(TESTABLE_ROLES)
|
||||
|
||||
|
||||
def test_playbook_tasks_dir_fails_open(roles_dir: Path):
|
||||
"""Shared playbook task files select all testable roles (REQ-1)."""
|
||||
roles = detect_changed_roles(["ansible/playbooks/_tasks/upgrade-postgres-database.yml"], roles_dir)
|
||||
assert roles == set(TESTABLE_ROLES)
|
||||
|
||||
|
||||
def test_mapped_playbooks(roles_dir: Path):
|
||||
"""Each newly mapped playbook selects its roles (REQ-1)."""
|
||||
expectations = {
|
||||
"ansible/playbooks/restore.yml": {"restore"},
|
||||
"ansible/playbooks/upgrade-postgres.yml": {"app_container"},
|
||||
"ansible/playbooks/rolling-update-gitea.yml": {"app_container"},
|
||||
"ansible/playbooks/update-alertmanager.yml": {"observability"},
|
||||
"ansible/playbooks/build-image.yml": {"docker_base"},
|
||||
"ansible/playbooks/deploy-sso-bridge.yml": set(),
|
||||
}
|
||||
for playbook, expected in expectations.items():
|
||||
assert detect_changed_roles([playbook], roles_dir) == expected, playbook
|
||||
|
||||
|
||||
def test_detect_prepare_vms_playbook(roles_dir: Path):
|
||||
"""prepare-vms.yml maps to all base roles present on disk."""
|
||||
roles = detect_changed_roles(["ansible/playbooks/prepare-vms.yml"], roles_dir)
|
||||
assert roles == {"docker_base"}
|
||||
|
||||
|
||||
def test_detect_deploy_customer_playbook(roles_dir: Path):
|
||||
"""deploy-customer.yml maps to its roles present on disk."""
|
||||
roles = detect_changed_roles(["ansible/playbooks/deploy-customer.yml"], roles_dir)
|
||||
assert roles == {"app_container", "docker_base"}
|
||||
|
||||
|
||||
def test_detect_no_ansible_changes(roles_dir: Path):
|
||||
def test_detect_no_ansible_changes():
|
||||
"""Non-Ansible files don't trigger any roles."""
|
||||
roles = detect_changed_roles(["scripts/x.py", "Makefile"], roles_dir)
|
||||
assert roles == set()
|
||||
|
||||
|
||||
def test_detect_environments_not_molecule_covered(roles_dir: Path):
|
||||
"""ansible/environments/ data is not molecule-covered (documented)."""
|
||||
roles = detect_changed_roles(["ansible/environments/staging/customers.yml"], roles_dir)
|
||||
assert roles == set()
|
||||
|
||||
|
||||
def test_detect_missing_roles_dir():
|
||||
"""A nonexistent roles_dir yields no roles (honest: nothing testable)."""
|
||||
roles = detect_changed_roles(["ansible/roles/docker_base/tasks/main.yml"], "/nonexistent")
|
||||
assert roles == set()
|
||||
|
||||
|
||||
def test_role_to_target():
|
||||
"""Role names map to conventional make targets."""
|
||||
assert role_to_target("docker_base") == "molecule-docker-base"
|
||||
assert role_to_target("app_hardening") == "molecule-app-hardening"
|
||||
files = ["scripts/molecule_changed.py", "Makefile"]
|
||||
roles = detect_changed_roles(files)
|
||||
assert len(roles) == 0
|
||||
|
||||
|
||||
def test_roles_to_targets():
|
||||
"""Role names map to make targets."""
|
||||
targets = roles_to_targets({"docker_base", "zitadel"})
|
||||
assert targets == ["molecule-docker-base", "molecule-zitadel"]
|
||||
assert "molecule-docker-base" in targets
|
||||
assert "molecule-zitadel" in targets
|
||||
|
||||
|
||||
def test_roles_to_targets_unknown_role():
|
||||
"""Unknown roles are silently skipped."""
|
||||
targets = roles_to_targets({"docker_base", "unknown_role"})
|
||||
assert targets == ["molecule-docker-base"]
|
||||
|
||||
|
||||
def test_main_no_changes():
|
||||
"""When no files changed, outputs message to stderr."""
|
||||
with patch("devx.molecule.molecule_changed.get_changed_files", return_value=[]):
|
||||
result = CliRunner().invoke(main, ["--print-targets"])
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(main, ["--print-targets"])
|
||||
assert result.exit_code == 0
|
||||
assert "No changed files" in result.output
|
||||
|
||||
|
||||
def test_main_print_targets(roles_dir: Path):
|
||||
"""--print-targets outputs make targets for existing roles."""
|
||||
def test_main_print_targets():
|
||||
"""--print-targets outputs make targets."""
|
||||
with patch(
|
||||
"devx.molecule.molecule_changed.get_changed_files",
|
||||
return_value=["ansible/roles/docker_base/tasks/main.yml"],
|
||||
):
|
||||
result = CliRunner().invoke(main, ["--print-targets", "--roles-dir", str(roles_dir)])
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(main, ["--print-targets"])
|
||||
assert result.exit_code == 0
|
||||
assert "molecule-docker-base" in result.output
|
||||
|
||||
|
||||
def test_main_print_roles(roles_dir: Path):
|
||||
def test_main_print_roles():
|
||||
"""--print-roles outputs role names."""
|
||||
with patch(
|
||||
"devx.molecule.molecule_changed.get_changed_files",
|
||||
return_value=["ansible/roles/restore/tasks/main.yml"],
|
||||
return_value=["ansible/roles/zitadel/tasks/main.yml"],
|
||||
):
|
||||
result = CliRunner().invoke(main, ["--print-roles", "--roles-dir", str(roles_dir)])
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(main, ["--print-roles"])
|
||||
assert result.exit_code == 0
|
||||
assert "restore" in result.output
|
||||
assert "zitadel" in result.output
|
||||
|
||||
|
||||
def test_main_no_ansible_changes(roles_dir: Path):
|
||||
def test_main_no_ansible_changes():
|
||||
"""When only non-Ansible files changed, outputs no scenarios message."""
|
||||
with patch(
|
||||
"devx.molecule.molecule_changed.get_changed_files",
|
||||
return_value=["scripts/molecule_changed.py"],
|
||||
):
|
||||
result = CliRunner().invoke(main, ["--print-targets", "--roles-dir", str(roles_dir)])
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(main, ["--print-targets"])
|
||||
assert result.exit_code == 0
|
||||
assert "No molecule scenarios" in result.output
|
||||
|
||||
@@ -207,6 +119,7 @@ def test_get_changed_files_falls_back_to_master():
|
||||
|
||||
def mock_git(args):
|
||||
calls.append(args)
|
||||
# First call (origin/master) returns empty, second (master) returns files
|
||||
if "origin/master...HEAD" in args[2]:
|
||||
return ""
|
||||
return "ansible/roles/docker_base/tasks/main.yml\n"
|
||||
@@ -224,12 +137,56 @@ def test_get_changed_files_empty():
|
||||
assert files == []
|
||||
|
||||
|
||||
def test_main_default_base(roles_dir: Path):
|
||||
def test_detect_molecule_shared_path():
|
||||
"""ansible/molecule/ change triggers all roles."""
|
||||
files = ["ansible/molecule/Dockerfile"]
|
||||
roles = detect_changed_roles(files)
|
||||
assert len(roles) == 10
|
||||
|
||||
|
||||
def test_detect_requirements_yml_triggers_all():
|
||||
"""ansible/requirements.yml change triggers all roles."""
|
||||
files = ["ansible/requirements.yml"]
|
||||
roles = detect_changed_roles(files)
|
||||
assert len(roles) == 10
|
||||
|
||||
|
||||
def test_detect_configure_oidc_playbook():
|
||||
"""configure-oidc.yml maps to sso_config and app_container."""
|
||||
files = ["ansible/playbooks/configure-oidc.yml"]
|
||||
roles = detect_changed_roles(files)
|
||||
assert "sso_config" in roles
|
||||
assert "app_container" in roles
|
||||
|
||||
|
||||
def test_detect_prepare_vms_playbook():
|
||||
"""prepare-vms.yml maps to all base roles."""
|
||||
files = ["ansible/playbooks/prepare-vms.yml"]
|
||||
roles = detect_changed_roles(files)
|
||||
assert "docker_base" in roles
|
||||
assert "app_hardening" in roles
|
||||
assert "storage" in roles
|
||||
assert "disk_cleanup" in roles
|
||||
assert "crowdsec" in roles
|
||||
|
||||
|
||||
def test_detect_deploy_customer_playbook():
|
||||
"""deploy-customer.yml maps to its roles."""
|
||||
files = ["ansible/playbooks/deploy-customer.yml"]
|
||||
roles = detect_changed_roles(files)
|
||||
assert "app_container" in roles
|
||||
assert "docker_base" in roles
|
||||
assert "app_hardening" in roles
|
||||
assert "sso_config" in roles
|
||||
|
||||
|
||||
def test_main_default_base():
|
||||
"""main() with no --base uses origin/master."""
|
||||
with patch(
|
||||
"devx.molecule.molecule_changed.get_changed_files",
|
||||
return_value=["ansible/roles/restore/tasks/main.yml"],
|
||||
return_value=["ansible/roles/zitadel/tasks/main.yml"],
|
||||
) as mock:
|
||||
result = CliRunner().invoke(main, ["--print-roles", "--roles-dir", str(roles_dir)])
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(main, ["--print-roles"])
|
||||
assert result.exit_code == 0
|
||||
mock.assert_called_once_with("origin/master")
|
||||
|
||||
@@ -0,0 +1,581 @@
|
||||
"""Unit tests for scripts/ci/molecule_ci_guard.py."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess # nosec B404
|
||||
import time
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import click
|
||||
import pytest
|
||||
import requests
|
||||
|
||||
from devx.molecule.molecule_ci_guard import (
|
||||
any_other_runner_failed,
|
||||
build_env_for_pair,
|
||||
build_molecule_cmd,
|
||||
cli,
|
||||
get_running_jobs,
|
||||
parse_pair,
|
||||
poll_for_other_failures,
|
||||
resolve_role_dir,
|
||||
)
|
||||
|
||||
|
||||
class TestGetRunningJobs:
|
||||
def test_returns_jobs(self) -> None:
|
||||
with patch("devx.molecule.molecule_ci_guard.requests.get") as mock_get:
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"jobs": [
|
||||
{"name": "molecule-tests (0)", "conclusion": "success"},
|
||||
{"name": "molecule-tests (1)", "conclusion": "failure"},
|
||||
]
|
||||
}
|
||||
mock_response.raise_for_status.return_value = None
|
||||
mock_get.return_value = mock_response
|
||||
|
||||
jobs = get_running_jobs("https://gitea.example", "owner", "repo", "token", 123)
|
||||
assert len(jobs) == 2
|
||||
mock_get.assert_called_once()
|
||||
|
||||
def test_raises_on_request_error(self) -> None:
|
||||
with patch("devx.molecule.molecule_ci_guard.requests.get") as mock_get:
|
||||
mock_get.side_effect = requests.RequestException("boom")
|
||||
with pytest.raises(requests.RequestException):
|
||||
get_running_jobs("https://gitea.example", "owner", "repo", "token", 123)
|
||||
|
||||
|
||||
class TestAnyOtherRunnerFailed:
|
||||
def test_detects_other_failure(self) -> None:
|
||||
jobs = [
|
||||
{"name": "molecule-tests (0)", "conclusion": "success"},
|
||||
{"name": "molecule-tests (1)", "conclusion": "failure"},
|
||||
{"name": "molecule-tests (2)", "conclusion": "running"},
|
||||
]
|
||||
assert any_other_runner_failed(jobs, "molecule-tests", 0) is True
|
||||
|
||||
def test_ignores_current_runner(self) -> None:
|
||||
jobs = [
|
||||
{"name": "molecule-tests (0)", "conclusion": "failure"},
|
||||
{"name": "molecule-tests (1)", "conclusion": "success"},
|
||||
]
|
||||
assert any_other_runner_failed(jobs, "molecule-tests", 0) is False
|
||||
|
||||
def test_ignores_non_molecule_jobs(self) -> None:
|
||||
jobs = [
|
||||
{"name": "quality", "conclusion": "failure"},
|
||||
{"name": "molecule-tests (1)", "conclusion": "success"},
|
||||
]
|
||||
assert any_other_runner_failed(jobs, "molecule-tests", 0) is False
|
||||
|
||||
|
||||
class TestBuildMoleculeCmd:
|
||||
def test_default_scenario(self) -> None:
|
||||
assert build_molecule_cmd("default") == ["molecule", "test"]
|
||||
|
||||
def test_named_scenario(self) -> None:
|
||||
assert build_molecule_cmd("lifecycle") == ["molecule", "test", "-s", "lifecycle"]
|
||||
|
||||
|
||||
class TestBuildEnvForPair:
|
||||
def test_with_command(self) -> None:
|
||||
env = build_env_for_pair("default|ubuntu-2204|img:latest|/lib/systemd/systemd", {})
|
||||
assert env["MOLECULE_PLATFORM_NAME"] == "ubuntu-2204"
|
||||
assert env["MOLECULE_PLATFORM_IMAGE"] == "img:latest"
|
||||
assert env["MOLECULE_PLATFORM_COMMAND"] == "/lib/systemd/systemd"
|
||||
assert env["ANSIBLE_ALLOW_BROKEN_CONDITIONALS"] == "true"
|
||||
|
||||
def test_without_command(self) -> None:
|
||||
env = build_env_for_pair("default|ubuntu-2204|img:latest|", {})
|
||||
assert env["MOLECULE_PLATFORM_NAME"] == "ubuntu-2204"
|
||||
assert "MOLECULE_PLATFORM_COMMAND" not in env
|
||||
|
||||
def test_without_command_removes_existing(self) -> None:
|
||||
env = build_env_for_pair("default|ubuntu-2204|img:latest|", {"MOLECULE_PLATFORM_COMMAND": "old"})
|
||||
assert "MOLECULE_PLATFORM_COMMAND" not in env
|
||||
|
||||
def test_preserves_existing_molecule_home(self) -> None:
|
||||
"""When MOLECULE_HOME is already set, it is not overridden."""
|
||||
env = build_env_for_pair("default|ubuntu-2204|img:latest|", {"MOLECULE_HOME": "/custom/home"})
|
||||
assert env["MOLECULE_HOME"] == "/custom/home"
|
||||
|
||||
|
||||
class TestPollForOtherFailures:
|
||||
def test_sets_failed_event_when_other_runner_fails(self) -> None:
|
||||
stop_event = MagicMock()
|
||||
failed_event = MagicMock()
|
||||
|
||||
def side_effect(*args, **kwargs):
|
||||
if stop_event.wait.call_count < 1:
|
||||
return [
|
||||
{"name": "molecule-tests (0)", "conclusion": "success"},
|
||||
{"name": "molecule-tests (1)", "conclusion": "failure"},
|
||||
]
|
||||
return []
|
||||
|
||||
with patch("devx.molecule.molecule_ci_guard.get_running_jobs") as mock_get_jobs:
|
||||
mock_get_jobs.side_effect = side_effect
|
||||
stop_event.is_set.side_effect = [False, False]
|
||||
stop_event.wait.return_value = True
|
||||
|
||||
poll_for_other_failures(
|
||||
"https://gitea.example",
|
||||
"owner",
|
||||
"repo",
|
||||
"token",
|
||||
123,
|
||||
"molecule-tests",
|
||||
0,
|
||||
stop_event,
|
||||
failed_event,
|
||||
)
|
||||
|
||||
failed_event.set.assert_called_once()
|
||||
|
||||
def test_poll_warns_on_api_error(self) -> None:
|
||||
stop_event = MagicMock()
|
||||
failed_event = MagicMock()
|
||||
|
||||
with patch("devx.molecule.molecule_ci_guard.get_running_jobs") as mock_get_jobs:
|
||||
mock_get_jobs.side_effect = requests.RequestException("boom")
|
||||
stop_event.is_set.side_effect = [False, True]
|
||||
stop_event.wait.return_value = True
|
||||
|
||||
poll_for_other_failures(
|
||||
"https://gitea.example",
|
||||
"owner",
|
||||
"repo",
|
||||
"token",
|
||||
123,
|
||||
"molecule-tests",
|
||||
0,
|
||||
stop_event,
|
||||
failed_event,
|
||||
)
|
||||
|
||||
failed_event.set.assert_not_called()
|
||||
|
||||
|
||||
class TestCli:
|
||||
def test_all_pass(self) -> None:
|
||||
from click.testing import CliRunner
|
||||
|
||||
with (
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
|
||||
patch("time.sleep"),
|
||||
):
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = 0
|
||||
proc.returncode = 0
|
||||
mock_popen.return_value = proc
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
|
||||
assert result.exit_code == 0
|
||||
assert "All molecule tests passed" in result.output
|
||||
# Verify Docker prune was called between scenarios
|
||||
mock_run.assert_called_once_with(
|
||||
["docker", "system", "prune", "-af", "--volumes"],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
timeout=60,
|
||||
)
|
||||
|
||||
@patch("devx.molecule.molecule_ci_guard.get_ci_token", side_effect=click.ClickException("no token"))
|
||||
def test_missing_token_runs_without_polling(self, mock_token: MagicMock) -> None:
|
||||
"""When no token is available, cross-runner polling is skipped."""
|
||||
from click.testing import CliRunner
|
||||
|
||||
with (
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
|
||||
patch("devx.molecule.molecule_ci_guard.poll_for_other_failures") as mock_poll,
|
||||
patch("time.sleep"),
|
||||
):
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = 0
|
||||
proc.returncode = 0
|
||||
mock_popen.return_value = proc
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
|
||||
assert result.exit_code == 0
|
||||
mock_poll.assert_not_called()
|
||||
|
||||
def test_invalid_pair_format_raises(self) -> None:
|
||||
"""Pair with fewer than 2 parts should raise."""
|
||||
from click.testing import CliRunner
|
||||
|
||||
with (
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen"),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
):
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["invalid_no_pipe"])
|
||||
assert result.exit_code != 0
|
||||
assert "Invalid pair format" in result.output
|
||||
|
||||
def test_failure_exits_nonzero(self) -> None:
|
||||
from click.testing import CliRunner
|
||||
|
||||
with (
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("time.sleep"),
|
||||
):
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = 1
|
||||
proc.returncode = 1
|
||||
mock_popen.return_value = proc
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
|
||||
assert result.exit_code == 1
|
||||
assert "FAILED" in result.output
|
||||
|
||||
def test_exits_before_starting_when_already_failed(self) -> None:
|
||||
from click.testing import CliRunner
|
||||
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"GITEA_URL": "https://gitea.example",
|
||||
"CI_GITEA_TOKEN": "token",
|
||||
"RUN_ID": "123",
|
||||
"JOB_NAME": "molecule-tests",
|
||||
"MATRIX_INDEX": "0",
|
||||
"GITEA_REPOSITORY": "my-org/my-repo",
|
||||
"PATH": os.environ.get("PATH", ""),
|
||||
},
|
||||
clear=True,
|
||||
),
|
||||
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.get_running_jobs") as mock_get_jobs,
|
||||
patch("time.sleep"),
|
||||
):
|
||||
mock_get_jobs.return_value = [
|
||||
{"name": "molecule-tests (0)", "conclusion": "running"},
|
||||
{"name": "molecule-tests (1)", "conclusion": "failure"},
|
||||
]
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = 0
|
||||
proc.returncode = 0
|
||||
mock_popen.return_value = proc
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
|
||||
assert result.exit_code == 1
|
||||
assert "Another molecule runner failed" in result.output
|
||||
|
||||
def test_keyboard_interrupt_kills_process(self) -> None:
|
||||
from click.testing import CliRunner
|
||||
|
||||
with (
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("time.sleep", side_effect=KeyboardInterrupt),
|
||||
patch("os.killpg") as mock_killpg,
|
||||
patch("os.getpgid") as mock_getpgid,
|
||||
):
|
||||
mock_getpgid.return_value = 123
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = None
|
||||
proc.wait.return_value = 0
|
||||
mock_popen.return_value = proc
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
|
||||
assert result.exit_code == 1
|
||||
mock_killpg.assert_called()
|
||||
|
||||
def test_exits_when_other_runner_fails(self) -> None:
|
||||
from click.testing import CliRunner
|
||||
|
||||
real_sleep = time.sleep
|
||||
call_count = [0]
|
||||
|
||||
def get_jobs_side_effect(*args, **kwargs):
|
||||
call_count[0] += 1
|
||||
if call_count[0] < 2:
|
||||
return [{"name": "molecule-tests (1)", "conclusion": "running"}]
|
||||
return [
|
||||
{"name": "molecule-tests (0)", "conclusion": "running"},
|
||||
{"name": "molecule-tests (1)", "conclusion": "failure"},
|
||||
]
|
||||
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"GITEA_URL": "https://gitea.example",
|
||||
"CI_GITEA_TOKEN": "token",
|
||||
"RUN_ID": "123",
|
||||
"JOB_NAME": "molecule-tests",
|
||||
"MATRIX_INDEX": "0",
|
||||
"GITEA_REPOSITORY": "my-org/my-repo",
|
||||
"PATH": os.environ.get("PATH", ""),
|
||||
},
|
||||
clear=True,
|
||||
),
|
||||
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
|
||||
patch("os.killpg") as mock_killpg,
|
||||
patch("os.getpgid") as mock_getpgid,
|
||||
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
|
||||
):
|
||||
mock_getpgid.return_value = 123
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = None
|
||||
proc.wait.return_value = 0
|
||||
mock_popen.return_value = proc
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
|
||||
assert result.exit_code == 1
|
||||
mock_killpg.assert_called()
|
||||
|
||||
def test_default_owner_repo_fallback(self) -> None:
|
||||
from click.testing import CliRunner
|
||||
|
||||
real_sleep = time.sleep
|
||||
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"GITEA_URL": "https://gitea.example",
|
||||
"CI_GITEA_TOKEN": "token",
|
||||
"RUN_ID": "123",
|
||||
"JOB_NAME": "molecule-tests",
|
||||
"MATRIX_INDEX": "0",
|
||||
"GITEA_REPOSITORY": "invalid",
|
||||
"PATH": os.environ.get("PATH", ""),
|
||||
},
|
||||
clear=True,
|
||||
),
|
||||
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
|
||||
patch("devx.molecule.molecule_ci_guard.get_running_jobs") as mock_get_jobs,
|
||||
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
|
||||
):
|
||||
mock_get_jobs.return_value = [{"name": "molecule-tests (1)", "conclusion": "success"}]
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = 0
|
||||
proc.returncode = 0
|
||||
mock_popen.return_value = proc
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
|
||||
assert result.exit_code == 0
|
||||
|
||||
def test_exits_when_other_runner_fails_with_process_lookup_error(self) -> None:
|
||||
from click.testing import CliRunner
|
||||
|
||||
real_sleep = time.sleep
|
||||
call_count = [0]
|
||||
|
||||
def get_jobs_side_effect(*args, **kwargs):
|
||||
call_count[0] += 1
|
||||
if call_count[0] < 2:
|
||||
return [{"name": "molecule-tests (1)", "conclusion": "running"}]
|
||||
return [
|
||||
{"name": "molecule-tests (0)", "conclusion": "running"},
|
||||
{"name": "molecule-tests (1)", "conclusion": "failure"},
|
||||
]
|
||||
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"GITEA_URL": "https://gitea.example",
|
||||
"CI_GITEA_TOKEN": "token",
|
||||
"RUN_ID": "123",
|
||||
"JOB_NAME": "molecule-tests",
|
||||
"MATRIX_INDEX": "0",
|
||||
"GITEA_REPOSITORY": "my-org/my-repo",
|
||||
"PATH": os.environ.get("PATH", ""),
|
||||
},
|
||||
clear=True,
|
||||
),
|
||||
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
|
||||
patch("os.killpg") as mock_killpg,
|
||||
patch("os.getpgid") as mock_getpgid,
|
||||
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
|
||||
):
|
||||
mock_getpgid.return_value = 123
|
||||
mock_killpg.side_effect = ProcessLookupError("no such process")
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = None
|
||||
proc.wait.return_value = 0
|
||||
mock_popen.return_value = proc
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
|
||||
assert result.exit_code == 1
|
||||
|
||||
def test_exits_when_other_runner_fails_with_timeout(self) -> None:
|
||||
from click.testing import CliRunner
|
||||
|
||||
real_sleep = time.sleep
|
||||
call_count = [0]
|
||||
|
||||
def get_jobs_side_effect(*args, **kwargs):
|
||||
call_count[0] += 1
|
||||
if call_count[0] < 2:
|
||||
return [{"name": "molecule-tests (1)", "conclusion": "running"}]
|
||||
return [
|
||||
{"name": "molecule-tests (0)", "conclusion": "running"},
|
||||
{"name": "molecule-tests (1)", "conclusion": "failure"},
|
||||
]
|
||||
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"GITEA_URL": "https://gitea.example",
|
||||
"CI_GITEA_TOKEN": "token",
|
||||
"RUN_ID": "123",
|
||||
"JOB_NAME": "molecule-tests",
|
||||
"MATRIX_INDEX": "0",
|
||||
"GITEA_REPOSITORY": "my-org/my-repo",
|
||||
"PATH": os.environ.get("PATH", ""),
|
||||
},
|
||||
clear=True,
|
||||
),
|
||||
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.get_running_jobs", side_effect=get_jobs_side_effect),
|
||||
patch("os.killpg") as mock_killpg,
|
||||
patch("os.getpgid") as mock_getpgid,
|
||||
patch("time.sleep", side_effect=lambda x: real_sleep(0)),
|
||||
):
|
||||
mock_getpgid.return_value = 123
|
||||
mock_killpg.side_effect = [None, ProcessLookupError("no such process")]
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = None
|
||||
proc.wait.side_effect = [subprocess.TimeoutExpired("cmd", 10)]
|
||||
mock_popen.return_value = proc
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
|
||||
assert result.exit_code == 1
|
||||
|
||||
|
||||
def test_main_module_block() -> None:
|
||||
import devx.molecule.molecule_ci_guard as mg
|
||||
|
||||
with open(mg.__file__) as f:
|
||||
source = f.read()
|
||||
source = source.replace('if __name__ == "__main__":\n cli()\n', "")
|
||||
namespace = dict(mg.__dict__)
|
||||
exec(compile(source, mg.__file__, "exec"), namespace)
|
||||
assert callable(namespace["cli"])
|
||||
|
||||
|
||||
class TestParsePair:
|
||||
def test_single_role_4_part(self) -> None:
|
||||
role, scenario, name, image, cmd = parse_pair("default|ubuntu-2204|ubuntu:22.04|")
|
||||
assert role == ""
|
||||
assert scenario == "default"
|
||||
assert name == "ubuntu-2204"
|
||||
assert image == "ubuntu:22.04"
|
||||
assert cmd == ""
|
||||
|
||||
def test_multi_role_5_part(self) -> None:
|
||||
role, scenario, name, image, cmd = parse_pair("gitea-runner|default|ubuntu-2204|ubuntu:22.04|")
|
||||
assert role == "gitea-runner"
|
||||
assert scenario == "default"
|
||||
assert name == "ubuntu-2204"
|
||||
assert image == "ubuntu:22.04"
|
||||
assert cmd == ""
|
||||
|
||||
def test_multi_role_with_command(self) -> None:
|
||||
role, scenario, name, image, cmd = parse_pair(
|
||||
"docker-base|lifecycle|archlinux|archlinux:latest|/usr/lib/systemd/systemd"
|
||||
)
|
||||
assert role == "docker-base"
|
||||
assert scenario == "lifecycle"
|
||||
assert cmd == "/usr/lib/systemd/systemd"
|
||||
|
||||
def test_invalid_pair_raises(self) -> None:
|
||||
with pytest.raises(click.ClickException, match="Invalid pair format"):
|
||||
parse_pair("only|two|parts")
|
||||
|
||||
def test_too_many_parts_raises(self) -> None:
|
||||
with pytest.raises(click.ClickException, match="Invalid pair format"):
|
||||
parse_pair("a|b|c|d|e|f")
|
||||
|
||||
|
||||
class TestResolveRoleDir:
|
||||
def test_multi_role_with_roles_root(self, tmp_path: Path) -> None:
|
||||
roles_root = tmp_path / "ansible" / "roles"
|
||||
roles_root.mkdir(parents=True)
|
||||
result = resolve_role_dir("gitea-runner", roles_root, tmp_path)
|
||||
assert result == roles_root / "gitea-runner"
|
||||
|
||||
def test_multi_role_default_roles_root(self, tmp_path: Path) -> None:
|
||||
result = resolve_role_dir("docker-base", None, tmp_path)
|
||||
assert result == tmp_path / "ansible" / "roles" / "docker-base"
|
||||
|
||||
def test_single_role_auto_discovers(self, tmp_path: Path) -> None:
|
||||
"""Single-role mode auto-discovers first role with molecule/ dir."""
|
||||
roles_dir = tmp_path / "ansible" / "roles"
|
||||
(roles_dir / "my_role" / "molecule").mkdir(parents=True)
|
||||
result = resolve_role_dir("", None, tmp_path)
|
||||
assert result == roles_dir / "my_role"
|
||||
|
||||
def test_single_role_no_roles_returns_fallback(self, tmp_path: Path) -> None:
|
||||
"""When no roles exist, returns a fallback path (will error at runtime)."""
|
||||
result = resolve_role_dir("", None, tmp_path)
|
||||
assert "roles" in str(result)
|
||||
|
||||
|
||||
class TestCliMultiRole:
|
||||
def test_multi_role_pair_passes(self, tmp_path: Path) -> None:
|
||||
from click.testing import CliRunner
|
||||
|
||||
roles_root = tmp_path / "ansible" / "roles"
|
||||
(roles_root / "gitea-runner").mkdir(parents=True)
|
||||
|
||||
with (
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run"),
|
||||
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
|
||||
patch("time.sleep"),
|
||||
):
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = 0
|
||||
proc.returncode = 0
|
||||
mock_popen.return_value = proc
|
||||
mock_run.return_value = MagicMock(returncode=0)
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
["--roles-root", str(roles_root), "gitea-runner|default|ubuntu-2204|ubuntu:22.04|"],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "All molecule tests passed" in result.output
|
||||
@@ -1,123 +0,0 @@
|
||||
"""Unit tests for devx.molecule.lease."""
|
||||
|
||||
import os
|
||||
from unittest.mock import MagicMock, mock_open, patch
|
||||
|
||||
from click.testing import CliRunner
|
||||
|
||||
from devx.molecule.lease import (
|
||||
DEFAULT_TTL_SECONDS,
|
||||
LEASE_UNTIL_ENV,
|
||||
OWNER_ENV,
|
||||
emit_github_env,
|
||||
lease_env,
|
||||
lease_owner,
|
||||
main,
|
||||
)
|
||||
|
||||
|
||||
class TestLeaseOwner:
|
||||
def test_gitea_env_preferred(self) -> None:
|
||||
with patch.dict(
|
||||
os.environ,
|
||||
{"GITEA_RUN_ID": "123", "GITEA_JOB": "tests", "GITHUB_RUN_ID": "999"},
|
||||
):
|
||||
assert lease_owner() == "run-123-tests"
|
||||
|
||||
def test_github_env_fallback(self) -> None:
|
||||
env = {"GITHUB_RUN_ID": "777", "GITHUB_JOB": "molecule"}
|
||||
with patch.dict(os.environ, env, clear=True):
|
||||
assert lease_owner() == "run-777-molecule"
|
||||
|
||||
def test_job_defaults_when_missing(self) -> None:
|
||||
with patch.dict(os.environ, {"GITHUB_RUN_ID": "5"}, clear=True):
|
||||
assert lease_owner() == "run-5-job"
|
||||
|
||||
def test_local_fallback(self) -> None:
|
||||
with (
|
||||
patch.dict(os.environ, {}, clear=True),
|
||||
patch("devx.molecule.lease.socket.gethostname", return_value="devbox"),
|
||||
):
|
||||
assert lease_owner() == "local-devbox"
|
||||
|
||||
|
||||
class TestLeaseEnv:
|
||||
def test_arithmetic(self) -> None:
|
||||
env = lease_env(ttl=3600, now=1000)
|
||||
assert env[LEASE_UNTIL_ENV] == str(1000 + 3600)
|
||||
assert OWNER_ENV in env
|
||||
|
||||
def test_default_now_uses_time(self) -> None:
|
||||
with patch("devx.molecule.lease.time.time", return_value=2000.9):
|
||||
env = lease_env(ttl=60)
|
||||
assert env[LEASE_UNTIL_ENV] == str(2000 + 60)
|
||||
|
||||
def test_default_ttl(self) -> None:
|
||||
env = lease_env(now=0)
|
||||
assert env[LEASE_UNTIL_ENV] == str(DEFAULT_TTL_SECONDS)
|
||||
|
||||
|
||||
class TestEmitGithubEnv:
|
||||
def test_writes_lines(self) -> None:
|
||||
m = mock_open()
|
||||
with (
|
||||
patch.dict(os.environ, {"GITHUB_ENV": "/tmp/env"}, clear=False),
|
||||
patch("builtins.open", m),
|
||||
):
|
||||
written = emit_github_env({"A": "1", "B": "2"})
|
||||
assert written == 2
|
||||
m.assert_called_once_with("/tmp/env", "a", encoding="utf-8")
|
||||
m().write.assert_any_call("A=1\n")
|
||||
m().write.assert_any_call("B=2\n")
|
||||
|
||||
def test_noop_without_env(self) -> None:
|
||||
with patch.dict(os.environ, {}, clear=True):
|
||||
assert emit_github_env({"A": "1"}) == 0
|
||||
|
||||
|
||||
class TestCli:
|
||||
def test_prints_and_exports(self) -> None:
|
||||
runner = CliRunner()
|
||||
m = mock_open()
|
||||
with (
|
||||
patch.dict(os.environ, {"GITHUB_ENV": "/tmp/env", "GITHUB_RUN_ID": "42"}, clear=True),
|
||||
patch("builtins.open", m),
|
||||
patch("devx.molecule.lease.time.time", return_value=100.0),
|
||||
):
|
||||
result = runner.invoke(main, ["--ttl", "600"])
|
||||
assert result.exit_code == 0
|
||||
assert f"{LEASE_UNTIL_ENV}=700" in result.output
|
||||
assert f"{OWNER_ENV}=run-42-job" in result.output
|
||||
assert "Exported 2 lease vars" in result.output
|
||||
|
||||
def test_prints_without_github_env(self) -> None:
|
||||
runner = CliRunner()
|
||||
with (
|
||||
patch.dict(os.environ, {"GITHUB_RUN_ID": "9", "GITHUB_JOB": "tests"}, clear=True),
|
||||
patch("devx.molecule.lease.time.time", return_value=50.0),
|
||||
):
|
||||
result = runner.invoke(main)
|
||||
assert result.exit_code == 0
|
||||
assert f"{LEASE_UNTIL_ENV}={50 + DEFAULT_TTL_SECONDS}" in result.output
|
||||
assert f"{OWNER_ENV}=run-9-tests" in result.output
|
||||
assert "GITHUB_ENV" not in result.output
|
||||
|
||||
|
||||
class TestStartDockerIntegration:
|
||||
"""start_docker.main() must emit lease env on success (REQ-2)."""
|
||||
|
||||
@patch("devx.molecule.start_docker.start_docker_daemon", return_value=True)
|
||||
def test_emits_lease_on_success(self, _mock: MagicMock) -> None:
|
||||
from devx.molecule.start_docker import main as start_docker_main
|
||||
|
||||
runner = CliRunner()
|
||||
m = mock_open()
|
||||
with (
|
||||
patch.dict(os.environ, {"GITHUB_ENV": "/tmp/env", "GITHUB_RUN_ID": "7"}, clear=True),
|
||||
patch("builtins.open", m),
|
||||
):
|
||||
result = runner.invoke(start_docker_main, ["--timeout", "1"])
|
||||
assert result.exit_code == 0
|
||||
assert "Exported lease vars to GITHUB_ENV" in result.output
|
||||
assert "owner=run-7-job" in result.output
|
||||
assert "until=" in result.output
|
||||
@@ -1,117 +0,0 @@
|
||||
"""Unit tests for devx.ci.nightly_gate."""
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from click.testing import CliRunner
|
||||
|
||||
from devx.ci.nightly_gate import cli, get_nightly_status, set_nightly_status
|
||||
|
||||
|
||||
class TestGetNightlyStatus:
|
||||
@patch("devx.ci.nightly_gate.GiteaClient")
|
||||
def test_returns_value_when_set(self, mock_client_cls: MagicMock) -> None:
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.get_repo_variable.return_value = "passed:12345"
|
||||
result = get_nightly_status(mock_client)
|
||||
assert result == "passed:12345"
|
||||
|
||||
@patch("devx.ci.nightly_gate.GiteaClient")
|
||||
def test_returns_empty_when_not_set(self, mock_client_cls: MagicMock) -> None:
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.get_repo_variable.return_value = None
|
||||
result = get_nightly_status(mock_client)
|
||||
assert result == ""
|
||||
|
||||
|
||||
class TestSetNightlyStatus:
|
||||
@patch("devx.ci.nightly_gate.GiteaClient")
|
||||
def test_sets_passed(self, mock_client_cls: MagicMock) -> None:
|
||||
mock_client = mock_client_cls.return_value
|
||||
set_nightly_status(mock_client, "passed:12345")
|
||||
mock_client.set_repo_variable.assert_called_once_with("NIGHTLY_STATUS", "passed:12345")
|
||||
|
||||
@patch("devx.ci.nightly_gate.GiteaClient")
|
||||
def test_sets_failed(self, mock_client_cls: MagicMock) -> None:
|
||||
mock_client = mock_client_cls.return_value
|
||||
set_nightly_status(mock_client, "failed:99999")
|
||||
mock_client.set_repo_variable.assert_called_once_with("NIGHTLY_STATUS", "failed:99999")
|
||||
|
||||
|
||||
class TestCli:
|
||||
@patch("devx.ci.nightly_gate.GiteaClient")
|
||||
@patch("devx.ci.nightly_gate.get_ci_token")
|
||||
def test_check_bootstrap_allows_deploy(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.get_repo_variable.return_value = None
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "check"])
|
||||
assert result.exit_code == 0
|
||||
assert "bootstrap" in result.output.lower()
|
||||
|
||||
@patch("devx.ci.nightly_gate.GiteaClient")
|
||||
@patch("devx.ci.nightly_gate.get_ci_token")
|
||||
def test_check_passed_allows_deploy(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.get_repo_variable.return_value = "passed:12345"
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "check"])
|
||||
assert result.exit_code == 0
|
||||
assert "passed" in result.output.lower()
|
||||
|
||||
@patch("devx.ci.nightly_gate.GiteaClient")
|
||||
@patch("devx.ci.nightly_gate.get_ci_token")
|
||||
def test_check_failed_blocks_deploy(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.get_repo_variable.return_value = "failed:99999"
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "check"])
|
||||
assert result.exit_code != 0
|
||||
assert "blocked" in result.output.lower()
|
||||
|
||||
@patch("devx.ci.nightly_gate.GiteaClient")
|
||||
@patch("devx.ci.nightly_gate.get_ci_token")
|
||||
def test_check_unknown_status_blocks_deploy(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_client = mock_client_cls.return_value
|
||||
mock_client.get_repo_variable.return_value = "garbage-value"
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "check"])
|
||||
assert result.exit_code != 0
|
||||
assert "fail closed" in result.output.lower()
|
||||
|
||||
@patch("devx.ci.nightly_gate.GiteaClient")
|
||||
@patch("devx.ci.nightly_gate.get_ci_token")
|
||||
def test_set_passed(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_client = mock_client_cls.return_value
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "set-passed", "--run-id", "12345"])
|
||||
assert result.exit_code == 0
|
||||
mock_client.set_repo_variable.assert_called_once_with("NIGHTLY_STATUS", "passed:12345")
|
||||
|
||||
@patch("devx.ci.nightly_gate.GiteaClient")
|
||||
@patch("devx.ci.nightly_gate.get_ci_token")
|
||||
def test_set_failed(self, mock_token: MagicMock, mock_client_cls: MagicMock) -> None:
|
||||
mock_token.return_value = "fake-token"
|
||||
mock_client = mock_client_cls.return_value
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "set-failed", "--run-id", "99999"])
|
||||
assert result.exit_code == 0
|
||||
mock_client.set_repo_variable.assert_called_once_with("NIGHTLY_STATUS", "failed:99999")
|
||||
|
||||
@patch("devx.ci.nightly_gate.get_ci_token")
|
||||
def test_fails_without_token(self, mock_token: MagicMock) -> None:
|
||||
import click as click_mod
|
||||
|
||||
mock_token.side_effect = click_mod.ClickException("No token")
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--repo", "oblachno/infra", "--action", "check"])
|
||||
assert result.exit_code != 0
|
||||
|
||||
def test_fails_with_invalid_repo(self) -> None:
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["--repo", "invalid", "--action", "check"])
|
||||
assert result.exit_code != 0
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user