Compare commits

..
37 Commits
Author SHA1 Message Date
devx-ci-bot 80ca622838 release: v0.9.10 [skip ci] 2026-06-24 18:42:23 +02:00
emil 3f2d19d7ac DEVX-34: fix: retrospective fixes for CI/CD friction 2026-06-24 16:41:22 +00:00
devx-ci-bot c839d49fe3 release: v0.9.9 [skip ci] 2026-06-24 13:17:30 +02:00
emil 93b5d2f926 DEVX-33: fix: use explicit refspecs for git push to avoid tag/branch ambiguity 2026-06-24 11:16:41 +00:00
emil 131c04c9d0 DEVX-32: fix: filter non-version tags in release verification 2026-06-24 11:09:47 +00:00
emil 8e9681cf7d DEVX-31: fix: prefer branch name for task ID extraction + strip heads/ prefix in release 2026-06-24 10:54:58 +00:00
emil 6631525a1d DEVX-30: fix: use DOCKER_HOST env var in is_docker_ready + scan all rootless sockets 2026-06-24 10:35:32 +00:00
devx-ci-bot 6985030a3c release: v0.9.8 [skip ci] 2026-06-24 11:15:20 +02:00
emil 4d073f3beb DEVX-29: fix: use DOCKER_HOST env var in is_docker_ready + scan all rootless sockets 2026-06-24 09:14:27 +00:00
devx-ci-bot 037d7b0d16 release: v0.9.7 [skip ci] 2026-06-24 02:18:19 +00:00
emil 9cb706e387 DEVX-28: fix: add rootless socket fallback and GITHUB_ENV export 2026-06-24 02:17:20 +00:00
devx-ci-bot 5bd6158f2a release: v0.9.6 [skip ci] 2026-06-24 04:00:50 +02:00
emil 05aa2ffe76 DEVX-27: fix: add Docker socket diagnostics to start_docker 2026-06-24 01:59:55 +00:00
emil b9c3b55680 DEVX-27: fix: add Docker socket diagnostics to start_docker 2026-06-24 01:50:46 +00:00
devx-ci-bot d398c8e971 release: v0.9.5 [skip ci] 2026-06-24 01:37:15 +00:00
emil 39526d8e6a DEVX-26: fix: use host Docker socket with DOCKER_HOST fallback to local dockerd 2026-06-24 01:36:21 +00:00
emil 37730e2187 DEVX-25: fix: use host Docker socket with DOCKER_HOST fallback to local dockerd 2026-06-24 01:24:00 +00:00
devx-ci-bot e2f66ca70a release: v0.9.4 [skip ci] 2026-06-24 01:11:08 +00:00
emil 0b88c211f1 DEVX-24: fix: use separate Docker socket for DinD in CI 2026-06-24 01:08:10 +00:00
devx-ci-bot ea4ee0d303 release: v0.9.3 [skip ci] 2026-06-24 02:46:26 +02:00
emil 16fba17b03 DEVX-23: fix: use tempfile for dockerd log to fix CI permission error 2026-06-24 00:45:28 +00:00
devx-ci-bot 011cf3e093 release: v0.9.2 [skip ci] 2026-06-24 02:34:47 +02:00
emil daf99c5fed DEVX-22: fix: use vfs storage driver for Docker-in-Docker in CI 2026-06-24 00:33:48 +00:00
emil 7154e3ad7c DEVX-21: chore: trigger auto-merge after Vikunja title fix 2026-06-24 00:28:18 +00:00
devx-ci-bot 6053fb9fba release: v0.9.1 [skip ci] 2026-06-24 01:50:03 +02:00
emil e76741bfad DEVX-21: fix: always start dockerd in CI runner for molecule tests 2026-06-23 23:49:13 +00:00
devx-ci-bot f206a9cd8d release: v0.9.0 [skip ci] 2026-06-24 01:29:24 +02:00
emil b4b7428f9c DEVX-20: feat: extract Docker daemon start to tested Python module 2026-06-23 23:28:20 +00:00
devx-ci-bot 0d9e76a838 release: v0.8.5 [skip ci] 2026-06-24 01:02:21 +02:00
emil 034cbde2f7 DEVX-19: fix: retry pip install with --ignore-installed only on failure 2026-06-23 23:01:32 +00:00
devx-ci-bot e0abe6f176 release: v0.8.4 [skip ci] 2026-06-23 22:28:29 +00:00
emil 15f6837dc2 DEVX-18: fix: add --ignore-installed to pip in CI to bypass debian packages 2026-06-23 22:27:35 +00:00
devx-ci-bot b4dda91e24 release: v0.8.3 [skip ci] 2026-06-23 21:58:33 +00:00
emil 3e21e774f7 DEVX-17: fix: pass --break-system-packages to pip in CI environments 2026-06-23 21:57:37 +00:00
emil 7c11215e57 DEVX-16: fix: lower check_test_speed threshold to 4 seconds 2026-06-23 20:35:53 +00:00
devx-ci-bot 5384269c83 release: v0.8.2 [skip ci] 2026-06-23 21:44:03 +02:00
emil b3d0dd8ca7 DEVX-15: fix: encode spaces in pair commands to survive shell word-splitting 2026-06-23 19:42:56 +00:00
22 changed files with 1585 additions and 746 deletions
+3 -3
View File
@@ -27,7 +27,7 @@ jobs:
PYTHONPATH: src PYTHONPATH: src
run: | run: |
. .venv/bin/activate . .venv/bin/activate
python3 -m devx.tools.check_test_speed --max-seconds 60 --max-single-seconds 2.0 python3 -m devx.tools.check_test_speed --max-seconds 4 --max-single-seconds 0.5
- name: Documentation coverage check - name: Documentation coverage check
env: env:
PYTHONPATH: src PYTHONPATH: src
@@ -120,8 +120,8 @@ jobs:
auto-merge: auto-merge:
# Auto-merge runs after all CI checks pass. It reads the task ID # Auto-merge runs after all CI checks pass. It reads the task ID
# from .taskid file, validates the PR title, and squash-merges. # from the branch name (falling back to .taskid file), validates
# No manual label or review needed — CI is the quality gate. # the PR title, and squash-merges.
needs: [quality, detect-changes, pr-review] needs: [quality, detect-changes, pr-review]
if: github.event_name == 'pull_request' if: github.event_name == 'pull_request'
runs-on: docker runs-on: docker
+1 -1
View File
@@ -1 +1 @@
DEVX-14 DEVX-30
+35
View File
@@ -285,6 +285,37 @@ setuptools via `dynamic = ["version"]` in `pyproject.toml`.
| PR title | `DEVX-N: <vikunja task title>` | `DEVX-12: Add release automation` | | PR title | `DEVX-N: <vikunja task title>` | `DEVX-12: Add release automation` |
| Merge commit | `DEVX-N <conventional commit>` | `DEVX-12 feat: add release script` | | Merge commit | `DEVX-N <conventional commit>` | `DEVX-12 feat: add release script` |
### Task ID Resolution
`auto_merge` resolves the task ID from the branch name first (e.g.
`DEVX-12-fix-foo` → `DEVX-12`), falling back to the `.taskid` file
for branches without a task ID prefix. If both exist and disagree,
a warning is printed and the branch task ID is preferred.
**When creating a new branch from an existing branch**, the `.taskid`
file may be stale (it contains the old branch's task ID). Either:
1. Update `.taskid` to match the new branch's task ID, or
2. Delete `.taskid` — the branch name is the primary source of truth
### Workflow `auto-merge` Job and `always()`
When `auto-merge` depends on a job that can be skipped (e.g.
`molecule-tests`), the `if:` condition MUST include `always() &&`
at the start. Without it, Gitea Actions skips `auto-merge` when any
dependency is skipped, even if the condition explicitly allows
`result == 'skipped'`.
```yaml
auto-merge:
needs: [quality, detect-changes, pr-review, molecule-tests]
if: >-
always() &&
github.event_name == 'pull_request' &&
needs.quality.result == 'success' &&
needs.pr-review.result == 'success' &&
(needs.molecule-tests.result == 'success' || needs.molecule-tests.result == 'skipped')
```
## Config System ## Config System
devx uses environment variables with `.env` file fallback for configuration. devx uses environment variables with `.env` file fallback for configuration.
@@ -295,6 +326,10 @@ devx uses environment variables with `.env` file fallback for configuration.
|----------|---------|-------------| |----------|---------|-------------|
| `DEVX_GITEA_API_URL` | `https://git.oblachno.oblachno.fyi/api/v1` | Gitea API base URL | | `DEVX_GITEA_API_URL` | `https://git.oblachno.oblachno.fyi/api/v1` | Gitea API base URL |
| `DEVX_VIKUNJA_API_URL` | `https://work.oblachno.oblachno.fyi/api/v1` | Vikunja API base URL | | `DEVX_VIKUNJA_API_URL` | `https://work.oblachno.oblachno.fyi/api/v1` | Vikunja API base URL |
| `DEVX_REPO_OWNER` | **(none — must be set)** | Repository owner for API calls |
| `DEVX_REPO_NAME` | **(none — must be set)** | Repository name (or `owner/repo`) |
| `DEVX_TASK_PREFIX` | `DEVX` | Task ID prefix (GRM, OBL-INFRA, etc.) |
| `DEVX_VIKUNJA_PROJECT_ID` | `6` | Vikunja project ID |
| `DEVX_LANG` | `en` | Language for i18n (en, bg) | | `DEVX_LANG` | `en` | Language for i18n (en, bg) |
| `REPO_TOKEN` | (from .env) | Gitea API token | | `REPO_TOKEN` | (from .env) | Gitea API token |
| `VIKUNJA_TOKEN` | (from .env) | Vikunja API token | | `VIKUNJA_TOKEN` | (from .env) | Vikunja API token |
+96
View File
@@ -2,6 +2,102 @@
All notable changes to this project will be documented in this file. All notable changes to this project will be documented in this file.
## [0.9.10] - 2026-06-24
### Bug Fixes
- Retrospective fixes for CI/CD friction
## [0.9.9] - 2026-06-24
### Bug Fixes
- Use DOCKER_HOST env var in is_docker_ready + scan all rootless sockets
- Prefer branch name for task ID extraction + strip heads/ prefix in release
- Filter non-version tags in release verification
- Use explicit refspecs for git push to avoid tag/branch ambiguity
## [0.9.8] - 2026-06-24
### Bug Fixes
- Use DOCKER_HOST env var in is_docker_ready + scan all rootless sockets
## [0.9.7] - 2026-06-24
### Bug Fixes
- Add rootless socket fallback and GITHUB_ENV export
## [0.9.6] - 2026-06-24
### Bug Fixes
- Add Docker socket diagnostics to start_docker
- Add Docker socket diagnostics to start_docker
## [0.9.5] - 2026-06-24
### Bug Fixes
- Use host Docker socket with DOCKER_HOST fallback to local dockerd
- Use host Docker socket with DOCKER_HOST fallback to local dockerd
## [0.9.4] - 2026-06-24
### Bug Fixes
- Use separate Docker socket for DinD in CI
## [0.9.3] - 2026-06-24
### Bug Fixes
- Use tempfile for dockerd log to fix CI permission error
## [0.9.2] - 2026-06-24
### Bug Fixes
- Use vfs storage driver for Docker-in-Docker in CI
## [0.9.1] - 2026-06-23
### Bug Fixes
- Always start dockerd in CI runner for molecule tests
## [0.9.0] - 2026-06-23
### Features
- Extract Docker daemon start to tested Python module
## [0.8.5] - 2026-06-23
### Bug Fixes
- Retry pip install with --ignore-installed only on failure
## [0.8.4] - 2026-06-23
### Bug Fixes
- Add --ignore-installed to pip in CI to bypass debian packages
## [0.8.3] - 2026-06-23
### Bug Fixes
- Lower check_test_speed threshold to 4 seconds
- Pass --break-system-packages to pip in CI environments
## [0.8.2] - 2026-06-23
### Bug Fixes
- Encode spaces in pair commands to survive shell word-splitting
## [0.8.1] - 2026-06-23 ## [0.8.1] - 2026-06-23
### Bug Fixes ### Bug Fixes
+1 -1
View File
@@ -4,4 +4,4 @@
# Aligned with CI (ci.yml uses same thresholds). # Aligned with CI (ci.yml uses same thresholds).
set -e set -e
export PYTHONPATH=src export PYTHONPATH=src
python3 -m devx.tools.check_test_speed --max-seconds 10 --max-single-seconds 0.5 python3 -m devx.tools.check_test_speed --max-seconds 4 --max-single-seconds 0.5
+1 -1
View File
@@ -1,3 +1,3 @@
"""devx — reusable development and CI/CD tools for oblachno-oss projects.""" """devx — reusable development and CI/CD tools for oblachno-oss projects."""
__version__ = "0.8.1" __version__ = "0.9.10"
+34 -9
View File
@@ -1,8 +1,8 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Auto-merge PR when all CI checks pass. """Auto-merge PR when all CI checks pass.
Runs as the final job in ci.yml. Reads the task ID from ``.taskid`` file Runs as the final job in ci.yml. Reads the task ID from the branch name
(falling back to branch name extraction for backwards compatibility), (falling back to ``.taskid`` file for branches without a task ID prefix),
validates the PR title, and squash-merges with a conventional commit validates the PR title, and squash-merges with a conventional commit
message prefixed by the task ID. message prefixed by the task ID.
@@ -63,20 +63,45 @@ def run_cmd(args: list[str], check: bool = True) -> subprocess.CompletedProcess[
def read_taskid(branch: str) -> str: def read_taskid(branch: str) -> str:
"""Read task ID from .taskid file, falling back to branch name extraction. """Read task ID from branch name, falling back to .taskid file.
The .taskid file is a simple text file containing just the task ID The branch name is the primary source of truth for the task ID
(e.g., ``DEVX-60``). If the file doesn't exist, extract from the (e.g., ``DEVX-31-fix-foo`` → ``DEVX-31``). The ``.taskid`` file
branch name as a backwards-compatibility fallback. is a legacy fallback for branches without a task ID prefix.
If both sources exist and disagree, a warning is printed and the
branch task ID is preferred (it is the current source of truth).
""" """
branch_task_id = extract_task_id(branch)
if branch_task_id:
# Check for stale .taskid file that disagrees with branch name
path = Path(TASKID_FILE)
if path.exists():
file_task_id = path.read_text(encoding="utf-8").strip()
if file_task_id and file_task_id != branch_task_id:
click.echo(
_(
"WARNING: .taskid file ({file_id}) disagrees with branch name ({branch_id}). "
"Using branch task ID. Update or delete .taskid to silence this warning.",
file_id=file_task_id,
branch_id=branch_task_id,
)
)
return branch_task_id
# Fallback: read from .taskid file
path = Path(TASKID_FILE) path = Path(TASKID_FILE)
if path.exists(): if path.exists():
task_id = path.read_text(encoding="utf-8").strip() task_id = path.read_text(encoding="utf-8").strip()
if task_id: if task_id:
click.echo(
_(
"Task ID from .taskid file: {task_id} (not found in branch name '{branch}')",
task_id=task_id,
branch=branch,
)
)
return task_id return task_id
# Fallback: extract from branch name return ""
match = TASK_ID_RE.search(branch)
return match.group(0) if match else ""
def extract_task_id(branch: str) -> str: def extract_task_id(branch: str) -> str:
+13 -6
View File
@@ -136,10 +136,14 @@ def verify_tag_consistency() -> list[str]:
""" """
errors: list[str] = [] errors: list[str] = []
tags = get_all_tags() tags = get_all_tags()
# Sort oldest first to identify the first tag # Filter to version tags (vX.Y.Z) and sort oldest first
sorted_tags = sorted(tags, key=lambda t: [int(x) for x in t.lstrip("v").split(".")]) version_tags = [t for t in tags if re.match(r"^v\d+\.\d+\.\d+$", t)]
sorted_tags = sorted(version_tags, key=lambda t: [int(x) for x in t.lstrip("v").split(".")])
first_tag = sorted_tags[0] if sorted_tags else None first_tag = sorted_tags[0] if sorted_tags else None
for tag in tags: for tag in tags:
# Skip non-version tags (e.g., branch names like "master")
if not re.match(r"^v\d+\.\d+\.\d+$", tag):
continue
tag_version = tag.lstrip("v") tag_version = tag.lstrip("v")
commit_version = get_commit_version(tag) commit_version = get_commit_version(tag)
if commit_version is None: if commit_version is None:
@@ -340,14 +344,14 @@ def create_and_push_tag(new_version: str, changelog: str, dry_run: bool) -> bool
click.echo(_("Tag {tag} already exists and points to HEAD. Skipping creation.", tag=tag)) click.echo(_("Tag {tag} already exists and points to HEAD. Skipping creation.", tag=tag))
if not dry_run: if not dry_run:
# Ensure the existing tag is pushed # Ensure the existing tag is pushed
run_cmd(["git", "push", "origin", tag], check=False) run_cmd(["git", "push", "origin", f"refs/tags/{tag}"], check=False)
return False return False
tag_msg = f"Release v{new_version}\n\n{changelog}" tag_msg = f"Release v{new_version}\n\n{changelog}"
if dry_run: if dry_run:
click.echo(_("[dry-run] Would create tag: {tag}", tag=tag)) click.echo(_("[dry-run] Would create tag: {tag}", tag=tag))
return True return True
run_cmd(["git", "tag", "-a", tag, "-m", tag_msg]) run_cmd(["git", "tag", "-a", tag, "-m", tag_msg])
run_cmd(["git", "push", "origin", tag]) run_cmd(["git", "push", "origin", f"refs/tags/{tag}"])
return True return True
@@ -478,7 +482,7 @@ def verify_alignment() -> int:
) )
if result.returncode == 0 and result.stdout.strip(): if result.returncode == 0 and result.stdout.strip():
all_release_commits = result.stdout.strip().split("\n") all_release_commits = result.stdout.strip().split("\n")
all_tags_set = {t.lstrip("v") for t in get_all_tags()} all_tags_set = {t.lstrip("v") for t in get_all_tags() if re.match(r"^v\d+\.\d+\.\d+$", t)}
truly_untagged: list[str] = [] truly_untagged: list[str] = []
duplicates: list[str] = [] duplicates: list[str] = []
for line in all_release_commits: for line in all_release_commits:
@@ -547,6 +551,8 @@ def main(dry_run: bool, skip_tests: bool, verify: bool) -> None:
# Ensure we're on master (skip this check in dry-run mode for PR validation) # Ensure we're on master (skip this check in dry-run mode for PR validation)
branch = run_cmd(["git", "rev-parse", "--abbrev-ref", "HEAD"]).stdout.strip() branch = run_cmd(["git", "rev-parse", "--abbrev-ref", "HEAD"]).stdout.strip()
# Some git versions return "heads/master" instead of "master"
branch = branch.removeprefix("heads/")
if branch != "master" and not dry_run: if branch != "master" and not dry_run:
raise click.ClickException(_("Release must be run on master, currently on '{branch}'.", branch=branch)) raise click.ClickException(_("Release must be run on master, currently on '{branch}'.", branch=branch))
if branch != "master" and dry_run: if branch != "master" and dry_run:
@@ -694,7 +700,8 @@ def main(dry_run: bool, skip_tests: bool, verify: bool) -> None:
# Pull --rebase before push to handle the case where master # Pull --rebase before push to handle the case where master
# advanced between checkout and commit (e.g., another merge). # advanced between checkout and commit (e.g., another merge).
run_cmd(["git", "pull", "--rebase", "origin", "master"], check=False) run_cmd(["git", "pull", "--rebase", "origin", "master"], check=False)
run_cmd(["git", "push", "origin", "master"]) # Use refs/heads/master to avoid ambiguity with a 'master' tag
run_cmd(["git", "push", "origin", "refs/heads/master:refs/heads/master"])
click.echo(_("Pushed release commit to master.")) click.echo(_("Pushed release commit to master."))
else: else:
click.echo(_("Skipping commit push — no staged changes.")) click.echo(_("Skipping commit push — no staged changes."))
+3 -2
View File
@@ -13,8 +13,9 @@ import re
GITEA_API_URL = os.getenv("DEVX_GITEA_API_URL", "https://git.oblachno.oblachno.fyi/api/v1") GITEA_API_URL = os.getenv("DEVX_GITEA_API_URL", "https://git.oblachno.oblachno.fyi/api/v1")
VIKUNJA_API_URL = os.getenv("DEVX_VIKUNJA_API_URL", "https://work.oblachno.oblachno.fyi/api/v1") VIKUNJA_API_URL = os.getenv("DEVX_VIKUNJA_API_URL", "https://work.oblachno.oblachno.fyi/api/v1")
# Organization defaults # Organization defaults — each project MUST set DEVX_REPO_OWNER explicitly.
REPO_OWNER = os.getenv("DEVX_REPO_OWNER", "oblachno-oss") # No default: prevents silent 404s when the wrong owner is used.
REPO_OWNER = os.getenv("DEVX_REPO_OWNER", "")
# Task prefix for Vikunja task IDs — each project sets its own (GRM, DEVX, INFRA, etc.) # Task prefix for Vikunja task IDs — each project sets its own (GRM, DEVX, INFRA, etc.)
TASK_PREFIX = os.getenv("DEVX_TASK_PREFIX", "DEVX") TASK_PREFIX = os.getenv("DEVX_TASK_PREFIX", "DEVX")
+6 -6
View File
@@ -41,7 +41,8 @@ class TestPair:
def encode(self) -> str: def encode(self) -> str:
"""Serialize to a pipe-delimited string for CI consumption.""" """Serialize to a pipe-delimited string for CI consumption."""
return f"{self.scenario}|{self.platform['name']}|{self.platform['image']}|{self.platform['command']}" cmd = self.platform["command"].replace(" ", "__SPACE__")
return f"{self.scenario}|{self.platform['name']}|{self.platform['image']}|{cmd}"
@staticmethod @staticmethod
def decode(encoded: str) -> TestPair: def decode(encoded: str) -> TestPair:
@@ -49,7 +50,7 @@ class TestPair:
parts = encoded.split("|") parts = encoded.split("|")
return TestPair( return TestPair(
scenario=parts[0], scenario=parts[0],
platform={"name": parts[1], "image": parts[2], "command": parts[3]}, platform={"name": parts[1], "image": parts[2], "command": parts[3].replace("__SPACE__", " ")},
) )
@@ -63,9 +64,8 @@ class MultiRoleTestPair:
def encode(self) -> str: def encode(self) -> str:
"""Serialize to a pipe-delimited string: ``role|scenario|platform_name|image|command``.""" """Serialize to a pipe-delimited string: ``role|scenario|platform_name|image|command``."""
return ( cmd = self.platform["command"].replace(" ", "__SPACE__")
f"{self.role}|{self.scenario}|{self.platform['name']}|{self.platform['image']}|{self.platform['command']}" return f"{self.role}|{self.scenario}|{self.platform['name']}|{self.platform['image']}|{cmd}"
)
@staticmethod @staticmethod
def decode(encoded: str) -> MultiRoleTestPair: def decode(encoded: str) -> MultiRoleTestPair:
@@ -74,7 +74,7 @@ class MultiRoleTestPair:
return MultiRoleTestPair( return MultiRoleTestPair(
role=parts[0], role=parts[0],
scenario=parts[1], scenario=parts[1],
platform={"name": parts[2], "image": parts[3], "command": parts[4]}, platform={"name": parts[2], "image": parts[3], "command": parts[4].replace("__SPACE__", " ")},
) )
+15 -2
View File
@@ -114,12 +114,14 @@ def parse_pair(pair: str) -> tuple[str, str, str, str, str]:
Supports both 4-part (single-role) and 5-part (multi-role) formats. Supports both 4-part (single-role) and 5-part (multi-role) formats.
For 4-part pairs, role is empty (caller uses default role dir). For 4-part pairs, role is empty (caller uses default role dir).
Spaces in the command field are encoded as ``__SPACE__`` to survive
shell word-splitting when ``$TEST_PAIRS`` is expanded unquoted.
""" """
parts = pair.split("|") parts = pair.split("|")
if len(parts) == 4: if len(parts) == 4:
return "", parts[0], parts[1], parts[2], parts[3] return "", parts[0], parts[1], parts[2], parts[3].replace("__SPACE__", " ")
if len(parts) == 5: if len(parts) == 5:
return parts[0], parts[1], parts[2], parts[3], parts[4] return parts[0], parts[1], parts[2], parts[3], parts[4].replace("__SPACE__", " ")
raise click.ClickException(f"Invalid pair format: {pair!r} (expected 4 or 5 pipe-delimited parts)") raise click.ClickException(f"Invalid pair format: {pair!r} (expected 4 or 5 pipe-delimited parts)")
@@ -319,6 +321,17 @@ def cli(pairs: tuple[str, ...], junit_output: str | None, roles_root: Path | Non
click.echo(_("PASSED: {pair}", pair=pair)) click.echo(_("PASSED: {pair}", pair=pair))
# Prune Docker data between scenarios to prevent disk exhaustion
# in Docker-in-Docker molecule containers (each scenario pulls
# hundreds of MB of images that accumulate across pairs).
with contextlib.suppress(subprocess.SubprocessError, OSError):
subprocess.run( # nosec B603, B607
["docker", "system", "prune", "-af", "--volumes"],
check=False,
capture_output=True,
timeout=60,
)
click.echo(_("All molecule tests passed.")) click.echo(_("All molecule tests passed."))
if junit_output: if junit_output:
write_junit_report(junit_output, testcases, current_index) write_junit_report(junit_output, testcases, current_index)
+202
View File
@@ -0,0 +1,202 @@
#!/usr/bin/env python3
"""Ensure Docker is available for molecule tests in CI.
CI runners (e.g. ``gitea/runner-images:ubuntu-latest``) may have the host's
Docker socket mounted. This module verifies Docker is accessible and
sets ``DOCKER_HOST`` explicitly so molecule's Python docker library
connects to the same socket as the Docker CLI.
If the host socket is not available, it tries the rootless socket, then
starts a local ``dockerd`` with the vfs storage driver (requires
privileged container).
Usage::
python3 -m devx.molecule.start_docker [--timeout 30]
"""
from __future__ import annotations
import glob
import os
import subprocess # nosec B404
import sys
import tempfile
import time
import click
from devx.i18n import _
DEFAULT_TIMEOUT = 30
DOCKER_SOCK = "/var/run/docker.sock"
# Rootless socket fallback (e.g. /run/user/994/docker.sock)
ROOTLESS_SOCK = f"/run/user/{os.getuid()}/docker.sock"
def is_docker_ready() -> bool:
"""Check if Docker daemon is responding on the configured socket."""
docker_host = os.environ.get("DOCKER_HOST", f"unix://{DOCKER_SOCK}")
result = subprocess.run( # nosec B603 B607
["docker", "info"],
capture_output=True,
check=False,
env={**os.environ, "DOCKER_HOST": docker_host},
)
return result.returncode == 0
def _diagnose_socket() -> None:
"""Print diagnostic info about the Docker socket."""
click.echo(f"DOCKER_HOST = {os.environ.get('DOCKER_HOST', '(not set)')}")
click.echo(f"Socket path: {DOCKER_SOCK}")
click.echo(f"Socket exists: {os.path.exists(DOCKER_SOCK)}")
if os.path.exists(DOCKER_SOCK):
stat = os.stat(DOCKER_SOCK)
click.echo(f"Socket mode: {oct(stat.st_mode)}")
click.echo(f"Socket uid: {stat.st_uid}, gid: {stat.st_gid}")
# Check if it's a mount point
result = subprocess.run( # nosec B603 B607
["mount"],
capture_output=True,
check=False,
text=True,
)
docker_mounts = [line for line in result.stdout.splitlines() if "docker" in line.lower()]
if docker_mounts:
click.echo("Docker-related mounts:")
for line in docker_mounts:
click.echo(f" {line}")
else:
click.echo("No Docker-related mounts found")
# Check docker context
result = subprocess.run( # nosec B603 B607
["docker", "context", "ls"],
capture_output=True,
check=False,
text=True,
)
click.echo(f"Docker contexts:\n{result.stdout}")
# Try docker info without DOCKER_HOST
result = subprocess.run( # nosec B603 B607
["docker", "info"],
capture_output=True,
check=False,
text=True,
)
click.echo(f"docker info (no DOCKER_HOST): rc={result.returncode}")
if result.returncode != 0:
click.echo(f" stderr: {result.stderr[:500]}")
else:
# Print server version and storage driver
for line in result.stdout.splitlines():
if "Server Version" in line or "Storage Driver" in line or "Docker Root Dir" in line:
click.echo(f" {line.strip()}")
def start_docker_daemon(timeout: int = DEFAULT_TIMEOUT) -> bool:
"""Ensure Docker is ready for molecule tests.
First tries the host socket. If that works, sets ``DOCKER_HOST`` and
returns immediately. If not, tries the rootless socket. If neither
works, starts a local ``dockerd`` with vfs storage driver (requires
privileged container).
Returns ``True`` if Docker is ready, ``False`` if it failed to
start within the timeout.
"""
# Point Docker CLI and Python library to the socket explicitly
os.environ["DOCKER_HOST"] = f"unix://{DOCKER_SOCK}"
# Diagnose socket state
click.echo("--- Docker socket diagnostics ---")
_diagnose_socket()
click.echo("--- End diagnostics ---")
# Check if host Docker is already available
if is_docker_ready():
click.echo(_("Docker daemon already running"))
return True
# Try rootless socket (e.g. /run/user/994/docker.sock)
click.echo(f"Trying rootless socket: {ROOTLESS_SOCK}")
os.environ["DOCKER_HOST"] = f"unix://{ROOTLESS_SOCK}"
if os.path.exists(ROOTLESS_SOCK) and is_docker_ready():
click.echo(_("Docker daemon already running"))
return True
# Scan for any rootless sockets at other UIDs
for sock in sorted(glob.glob("/run/user/*/docker.sock")):
if sock == ROOTLESS_SOCK:
continue
click.echo(f"Trying alternative rootless socket: {sock}")
os.environ["DOCKER_HOST"] = f"unix://{sock}"
if is_docker_ready():
click.echo(_("Docker daemon already running"))
return True
click.echo(_("Host Docker not available, starting local dockerd..."))
# Reset DOCKER_HOST to host socket for local dockerd
os.environ["DOCKER_HOST"] = f"unix://{DOCKER_SOCK}"
# Start local dockerd (requires privileged container)
log_file = tempfile.NamedTemporaryFile( # noqa: SIM115
mode="w", suffix="dockerd.log", delete=False
)
click.echo(f"dockerd log: {log_file.name}")
subprocess.Popen( # nosec B603 B607
[
"dockerd",
"--storage-driver",
"vfs",
"-H",
f"unix://{DOCKER_SOCK}",
],
stdout=log_file,
stderr=subprocess.STDOUT,
start_new_session=True,
)
for _i in range(timeout):
if is_docker_ready():
click.echo(_("Docker daemon started"))
return True
time.sleep(1)
# Print dockerd log on failure
click.echo(_("Docker daemon failed to start"))
click.echo("--- dockerd log ---")
try:
with open(log_file.name) as f:
log_content = f.read()
click.echo(log_content[-3000:] if len(log_content) > 3000 else log_content)
except OSError as e:
click.echo(f"Could not read log: {e}")
click.echo("--- End dockerd log ---")
return False
@click.command()
@click.option(
"--timeout",
default=DEFAULT_TIMEOUT,
type=int,
help="Seconds to wait for Docker daemon to start (default: 30).",
)
def main(timeout: int) -> None:
"""Start Docker daemon for CI molecule tests."""
if start_docker_daemon(timeout):
# Export DOCKER_HOST to GITHUB_ENV for subsequent CI steps
github_env = os.environ.get("GITHUB_ENV")
if github_env and os.environ.get("DOCKER_HOST"):
with open(github_env, "a") as f:
f.write(f"DOCKER_HOST={os.environ['DOCKER_HOST']}\n")
click.echo(f"Exported DOCKER_HOST={os.environ['DOCKER_HOST']} to GITHUB_ENV")
sys.exit(0)
sys.exit(1)
if __name__ == "__main__": # pragma: no cover
main()
+13
View File
@@ -155,6 +155,19 @@ def main(repo: str | None, owner: str | None, branch: str, api_url: str | None)
if not repo: if not repo:
raise click.ClickException(_("ERROR: Repository name not specified. Use --repo or set DEVX_REPO_NAME.")) raise click.ClickException(_("ERROR: Repository name not specified. Use --repo or set DEVX_REPO_NAME."))
# If DEVX_REPO_NAME contains a slash (e.g. "oblachno/infra"), split into owner/repo.
# This prevents 404s when workflows set DEVX_REPO_NAME to the full path.
if "/" in repo and owner is None:
parts = repo.split("/", 1)
owner, repo = parts[0], parts[1]
click.echo(
_(
"Parsed owner={owner}, repo={repo} from DEVX_REPO_NAME",
owner=owner,
repo=repo,
)
)
if owner is None: if owner is None:
owner = REPO_OWNER owner = REPO_OWNER
+18 -3
View File
@@ -26,9 +26,24 @@ def _run(cmd: list[str]) -> None:
def _install_python_deps(bin_dir: str, extras: str = "dev") -> None: def _install_python_deps(bin_dir: str, extras: str = "dev") -> None:
"""Install the project with the specified extras in editable mode.""" """Install the project with the specified extras in editable mode.
In CI (system Python with PIP_BREAK_SYSTEM_PACKAGES=1), a first attempt
uses --break-system-packages. If that fails (e.g. debian-installed
packages without RECORD files), retry with --ignore-installed to skip
uninstalling system packages entirely.
"""
pip = str(Path(bin_dir) / "pip") pip = str(Path(bin_dir) / "pip")
_run([pip, "install", "-e", f".[{extras}]"]) cmd = [pip, "install", "-e", f".[{extras}]"]
if os.environ.get("PIP_BREAK_SYSTEM_PACKAGES") == "1":
cmd.append("--break-system-packages")
result = subprocess.run(cmd, check=False) # nosec B603
if result.returncode != 0 and os.environ.get("PIP_BREAK_SYSTEM_PACKAGES") == "1":
click.echo(" Retrying with --ignore-installed to bypass system packages...")
cmd.append("--ignore-installed")
_run(cmd)
elif result.returncode != 0:
raise subprocess.CalledProcessError(result.returncode, cmd)
def _install_pre_commit_hooks(bin_dir: str) -> None: def _install_pre_commit_hooks(bin_dir: str) -> None:
@@ -40,7 +55,7 @@ def _install_pre_commit_hooks(bin_dir: str) -> None:
def _install_ansible_collections(bin_dir: str) -> None: def _install_ansible_collections(bin_dir: str) -> None:
"""Install required Ansible Galaxy collections if requirements exist.""" """Install required Ansible Galaxy collections if requirements exist."""
galaxy = str(Path(bin_dir) / "ansible-galaxy") galaxy = shutil.which("ansible-galaxy") or str(Path(bin_dir) / "ansible-galaxy")
requirements = Path("ansible/requirements.yml") requirements = Path("ansible/requirements.yml")
if not requirements.exists(): if not requirements.exists():
click.echo(" ansible/requirements.yml not found — skipping collections.") click.echo(" ansible/requirements.yml not found — skipping collections.")
+749 -700
View File
File diff suppressed because it is too large Load Diff
+18 -1
View File
@@ -21,9 +21,16 @@ from devx.exceptions import APIError
class TestReadTaskid: class TestReadTaskid:
def test_reads_from_file(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def] def test_prefers_branch_name_over_file(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path) monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-60\n") (tmp_path / ".taskid").write_text("DEVX-60\n")
# Branch name takes priority over .taskid file
assert read_taskid("DEVX-19-fix-bug") == "DEVX-19"
def test_falls_back_to_file_when_no_branch_match(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-60\n")
# No task ID in branch name → fall back to .taskid
assert read_taskid("some-branch") == "DEVX-60" assert read_taskid("some-branch") == "DEVX-60"
def test_falls_back_to_branch_name(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def] def test_falls_back_to_branch_name(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
@@ -39,6 +46,16 @@ class TestReadTaskid:
(tmp_path / ".taskid").write_text("\n") (tmp_path / ".taskid").write_text("\n")
assert read_taskid("DEVX-42-test") == "DEVX-42" assert read_taskid("DEVX-42-test") == "DEVX-42"
def test_warns_on_stale_taskid_file(self, tmp_path, monkeypatch, capsys) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-60\n")
# Branch name takes priority, but stale .taskid should produce a warning
assert read_taskid("DEVX-19-fix-bug") == "DEVX-19"
captured = capsys.readouterr()
assert "WARNING" in captured.out
assert "DEVX-60" in captured.out
assert "DEVX-19" in captured.out
# -- extract_task_id (legacy fallback) -- # -- extract_task_id (legacy fallback) --
+1 -1
View File
@@ -37,7 +37,7 @@ class TestConfigConstants:
assert DEFAULT_PER_PAGE == 50 assert DEFAULT_PER_PAGE == 50
def test_owner(self) -> None: def test_owner(self) -> None:
assert REPO_OWNER == "oblachno-oss" assert REPO_OWNER == ""
def test_task_prefix(self) -> None: def test_task_prefix(self) -> None:
assert TASK_PREFIX == "DEVX" assert TASK_PREFIX == "DEVX"
+34
View File
@@ -163,3 +163,37 @@ class TestMain:
mock_client.ensure_branch_protection.assert_called_once() mock_client.ensure_branch_protection.assert_called_once()
args = mock_client.ensure_branch_protection.call_args args = mock_client.ensure_branch_protection.call_args
assert args[0][0] == "develop" assert args[0][0] == "develop"
@patch.dict("os.environ", {"REPO_TOKEN": "tok", "DEVX_REPO_NAME": "oblachno/infra"}, clear=True)
@patch("devx.tools.configure_repo.GiteaClient")
def test_main_parses_owner_repo_from_env(self, mock_client_cls: MagicMock) -> None:
"""DEVX_REPO_NAME with 'owner/repo' format should be split."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
# Verify GiteaClient was constructed with parsed owner and repo (positional)
call_args = mock_client_cls.call_args
assert call_args[0][2] == "oblachno" # owner is 3rd positional arg
assert call_args[0][3] == "infra" # repo is 4th positional arg
@patch.dict(
"os.environ",
{"REPO_TOKEN": "tok", "DEVX_REPO_NAME": "infra", "DEVX_REPO_OWNER": "oblachno"},
clear=True,
)
@patch("devx.tools.configure_repo.REPO_OWNER", "oblachno")
@patch("devx.tools.configure_repo.GiteaClient")
def test_main_no_slash_when_owner_set_separately(self, mock_client_cls: MagicMock) -> None:
"""When DEVX_REPO_OWNER is set, DEVX_REPO_NAME should not be split."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
call_args = mock_client_cls.call_args
assert call_args[0][2] == "oblachno" # owner
assert call_args[0][3] == "infra" # repo
+15
View File
@@ -162,17 +162,26 @@ class TestCli:
with ( with (
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen, patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
patch("time.sleep"), patch("time.sleep"),
): ):
proc = MagicMock() proc = MagicMock()
proc.poll.return_value = 0 proc.poll.return_value = 0
proc.returncode = 0 proc.returncode = 0
mock_popen.return_value = proc mock_popen.return_value = proc
mock_run.return_value = MagicMock(returncode=0)
runner = CliRunner() runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"]) result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
assert result.exit_code == 0 assert result.exit_code == 0
assert "All molecule tests passed" in result.output assert "All molecule tests passed" in result.output
# Verify Docker prune was called between scenarios
mock_run.assert_called_once_with(
["docker", "system", "prune", "-af", "--volumes"],
check=False,
capture_output=True,
timeout=60,
)
def test_invalid_pair_format_raises(self) -> None: def test_invalid_pair_format_raises(self) -> None:
"""Pair with fewer than 2 parts should raise.""" """Pair with fewer than 2 parts should raise."""
@@ -324,6 +333,7 @@ class TestCli:
), ),
patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01), patch("devx.molecule.molecule_ci_guard.POLL_INTERVAL", 0.01),
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen, patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
patch("devx.molecule.molecule_ci_guard.get_running_jobs") as mock_get_jobs, patch("devx.molecule.molecule_ci_guard.get_running_jobs") as mock_get_jobs,
patch("time.sleep", side_effect=lambda x: real_sleep(0.05)), patch("time.sleep", side_effect=lambda x: real_sleep(0.05)),
): ):
@@ -332,6 +342,7 @@ class TestCli:
proc.poll.return_value = 0 proc.poll.return_value = 0
proc.returncode = 0 proc.returncode = 0
mock_popen.return_value = proc mock_popen.return_value = proc
mock_run.return_value = MagicMock(returncode=0)
runner = CliRunner() runner = CliRunner()
result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"]) result = runner.invoke(cli, ["default|ubuntu-2204|img:latest|"])
@@ -536,12 +547,14 @@ class TestCliMultiRole:
with ( with (
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen, patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
patch("time.sleep"), patch("time.sleep"),
): ):
proc = MagicMock() proc = MagicMock()
proc.poll.return_value = 0 proc.poll.return_value = 0
proc.returncode = 0 proc.returncode = 0
mock_popen.return_value = proc mock_popen.return_value = proc
mock_run.return_value = MagicMock(returncode=0)
runner = CliRunner() runner = CliRunner()
result = runner.invoke( result = runner.invoke(
@@ -560,12 +573,14 @@ class TestCliMultiRole:
with ( with (
patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen, patch("devx.molecule.molecule_ci_guard.subprocess.Popen") as mock_popen,
patch("devx.molecule.molecule_ci_guard.subprocess.run") as mock_run,
patch("time.sleep"), patch("time.sleep"),
): ):
proc = MagicMock() proc = MagicMock()
proc.poll.return_value = 0 proc.poll.return_value = 0
proc.returncode = 0 proc.returncode = 0
mock_popen.return_value = proc mock_popen.return_value = proc
mock_run.return_value = MagicMock(returncode=0)
runner = CliRunner() runner = CliRunner()
result = runner.invoke( result = runner.invoke(
+11 -2
View File
@@ -270,6 +270,15 @@ class TestVerifyTagConsistency:
mock_tags.return_value = [] mock_tags.return_value = []
assert verify_tag_consistency() == [] assert verify_tag_consistency() == []
@patch("devx.ci.release.get_commit_version")
@patch("devx.ci.release.get_all_tags")
def test_non_version_tags_ignored(self, mock_tags: MagicMock, mock_cv: MagicMock) -> None:
"""Non-version tags like 'master' should be skipped, not crash."""
mock_tags.return_value = ["v0.2.0", "master", "v0.1.0"]
mock_cv.side_effect = ["0.2.0", "0.1.0"] # only version tags get checked
errors = verify_tag_consistency()
assert errors == []
class TestGetInitVersion: class TestGetInitVersion:
def test_returns_version(self, tmp_path, monkeypatch) -> None: def test_returns_version(self, tmp_path, monkeypatch) -> None:
@@ -776,7 +785,7 @@ class TestCreateAndPushTag:
create_and_push_tag("0.2.0", "changelog", dry_run=False) create_and_push_tag("0.2.0", "changelog", dry_run=False)
calls = [c.args[0] for c in mock_run_cmd.call_args_list] calls = [c.args[0] for c in mock_run_cmd.call_args_list]
assert ["git", "tag", "-a", "v0.2.0", "-m", "Release v0.2.0\n\nchangelog"] in calls assert ["git", "tag", "-a", "v0.2.0", "-m", "Release v0.2.0\n\nchangelog"] in calls
assert ["git", "push", "origin", "v0.2.0"] in calls assert ["git", "push", "origin", "refs/tags/v0.2.0"] in calls
@patch("devx.ci.release.tag_exists", return_value=False) @patch("devx.ci.release.tag_exists", return_value=False)
@patch("devx.ci.release.run_cmd") @patch("devx.ci.release.run_cmd")
@@ -803,7 +812,7 @@ class TestCreateAndPushTag:
# Should not create tag, but should ensure it's pushed # Should not create tag, but should ensure it's pushed
calls = [c.args[0] for c in mock_run_cmd.call_args_list] calls = [c.args[0] for c in mock_run_cmd.call_args_list]
assert ["git", "tag", "-a"] not in [c[:3] for c in calls] assert ["git", "tag", "-a"] not in [c[:3] for c in calls]
assert ["git", "push", "origin", "v0.1.0"] in calls assert ["git", "push", "origin", "refs/tags/v0.1.0"] in calls
@patch("devx.ci.release.get_head_commit", return_value="def456") @patch("devx.ci.release.get_head_commit", return_value="def456")
@patch("devx.ci.release.get_tag_commit", return_value="abc123") @patch("devx.ci.release.get_tag_commit", return_value="abc123")
+38 -8
View File
@@ -1,5 +1,6 @@
"""Unit tests for devx.tools.setup.""" """Unit tests for devx.tools.setup."""
import os
import subprocess import subprocess
from pathlib import Path from pathlib import Path
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
@@ -32,20 +33,48 @@ class TestRun:
class TestInstallPythonDeps: class TestInstallPythonDeps:
@patch("devx.tools.setup._run") @patch("devx.tools.setup.subprocess.run")
def test_install_dev(self, mock_run: MagicMock) -> None: def test_install_dev(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(returncode=0)
_install_python_deps(".venv/bin", "dev") _install_python_deps(".venv/bin", "dev")
mock_run.assert_called_once_with([".venv/bin/pip", "install", "-e", ".[dev]"]) mock_run.assert_called_once_with([".venv/bin/pip", "install", "-e", ".[dev]"], check=False)
@patch("devx.tools.setup._run") @patch("devx.tools.setup.subprocess.run")
def test_install_ci(self, mock_run: MagicMock) -> None: def test_install_ci(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(returncode=0)
_install_python_deps(".venv/bin", "ci") _install_python_deps(".venv/bin", "ci")
mock_run.assert_called_once_with([".venv/bin/pip", "install", "-e", ".[ci]"]) mock_run.assert_called_once_with([".venv/bin/pip", "install", "-e", ".[ci]"], check=False)
@patch("devx.tools.setup.subprocess.run")
def test_install_custom_extras(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(returncode=0)
_install_python_deps(".venv/bin", "ci,lint")
mock_run.assert_called_once_with([".venv/bin/pip", "install", "-e", ".[ci,lint]"], check=False)
@patch("devx.tools.setup.subprocess.run")
def test_install_with_break_system_packages(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(returncode=0)
with patch.dict(os.environ, {"PIP_BREAK_SYSTEM_PACKAGES": "1"}):
_install_python_deps(".venv/bin", "ci")
mock_run.assert_called_once_with(
[".venv/bin/pip", "install", "-e", ".[ci]", "--break-system-packages"], check=False
)
@patch("devx.tools.setup._run") @patch("devx.tools.setup._run")
def test_install_custom_extras(self, mock_run: MagicMock) -> None: @patch("devx.tools.setup.subprocess.run")
_install_python_deps(".venv/bin", "ci,lint") def test_install_retry_with_ignore_installed(self, mock_subprocess: MagicMock, mock_run: MagicMock) -> None:
mock_run.assert_called_once_with([".venv/bin/pip", "install", "-e", ".[ci,lint]"]) mock_subprocess.return_value = MagicMock(returncode=1)
with patch.dict(os.environ, {"PIP_BREAK_SYSTEM_PACKAGES": "1"}):
_install_python_deps(".venv/bin", "ci")
mock_run.assert_called_once_with(
[".venv/bin/pip", "install", "-e", ".[ci]", "--break-system-packages", "--ignore-installed"]
)
@patch("devx.tools.setup.subprocess.run")
def test_install_failure_without_break_system(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(returncode=1)
with pytest.raises(subprocess.CalledProcessError):
_install_python_deps(".venv/bin", "ci")
class TestInstallPreCommitHooks: class TestInstallPreCommitHooks:
@@ -60,8 +89,9 @@ class TestInstallPreCommitHooks:
class TestInstallAnsibleCollections: class TestInstallAnsibleCollections:
@patch("devx.tools.setup.shutil.which", return_value="/usr/local/bin/ansible-galaxy")
@patch("devx.tools.setup._run") @patch("devx.tools.setup._run")
def test_installs_from_requirements(self, mock_run: MagicMock, tmp_path: Path) -> None: def test_installs_from_requirements(self, mock_run: MagicMock, mock_which: MagicMock, tmp_path: Path) -> None:
req = tmp_path / "ansible" / "requirements.yml" req = tmp_path / "ansible" / "requirements.yml"
req.parent.mkdir(parents=True) req.parent.mkdir(parents=True)
req.write_text("collections: []") req.write_text("collections: []")
+278
View File
@@ -0,0 +1,278 @@
"""Unit tests for devx.molecule.start_docker."""
import os
from unittest.mock import MagicMock, mock_open, patch
from click.testing import CliRunner
from devx.molecule.start_docker import (
DOCKER_SOCK,
_diagnose_socket,
is_docker_ready,
main,
start_docker_daemon,
)
class TestIsDockerReady:
@patch("devx.molecule.start_docker.subprocess.run")
def test_ready(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(returncode=0)
with patch.dict("os.environ", {"DOCKER_HOST": f"unix://{DOCKER_SOCK}"}, clear=False):
assert is_docker_ready() is True
call_kwargs = mock_run.call_args
assert call_kwargs.args[0] == ["docker", "info"]
assert call_kwargs.kwargs["env"]["DOCKER_HOST"] == f"unix://{DOCKER_SOCK}"
@patch("devx.molecule.start_docker.subprocess.run")
def test_not_ready(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(returncode=1)
assert is_docker_ready() is False
@patch("devx.molecule.start_docker.subprocess.run")
def test_uses_docker_host_env(self, mock_run: MagicMock) -> None:
"""Should check the socket specified by DOCKER_HOST env var."""
mock_run.return_value = MagicMock(returncode=0)
rootless = "unix:///run/user/999/docker.sock"
with patch.dict("os.environ", {"DOCKER_HOST": rootless}, clear=False):
assert is_docker_ready() is True
call_kwargs = mock_run.call_args
assert call_kwargs.kwargs["env"]["DOCKER_HOST"] == rootless
class TestDiagnoseSocket:
@patch("devx.molecule.start_docker.os.stat")
@patch("devx.molecule.start_docker.os.path.exists", return_value=True)
@patch("devx.molecule.start_docker.subprocess.run")
def test_socket_exists(self, mock_run: MagicMock, mock_exists: MagicMock, mock_stat: MagicMock) -> None:
mock_stat.return_value = MagicMock(st_mode=0o660, st_uid=0, st_gid=0)
mock_run.side_effect = [
MagicMock(stdout="/dev/sda1 /var/lib/docker ext4\n", returncode=0, text=""),
MagicMock(stdout="default\n", returncode=0, text=""),
MagicMock(
stdout="Server Version: 29.5.2\nStorage Driver: overlay2\nDocker Root Dir: /var/lib/docker\n",
returncode=0,
text="",
),
]
_diagnose_socket()
mock_exists.assert_called_with(DOCKER_SOCK)
@patch("devx.molecule.start_docker.os.path.exists", return_value=False)
@patch("devx.molecule.start_docker.subprocess.run")
def test_socket_missing(self, mock_run: MagicMock, mock_exists: MagicMock) -> None:
mock_run.side_effect = [
MagicMock(stdout="proc on /proc type proc\n", returncode=0, text=""),
MagicMock(stdout="default\n", returncode=0, text=""),
MagicMock(stdout="", stderr="Cannot connect", returncode=1, text=""),
]
_diagnose_socket()
mock_exists.assert_called_with(DOCKER_SOCK)
class TestStartDockerDaemon:
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.is_docker_ready", return_value=True)
def test_host_socket_available(self, mock_ready: MagicMock, mock_diag: MagicMock) -> None:
"""Should return immediately if host Docker is available."""
assert start_docker_daemon(timeout=5) is True
mock_ready.assert_called_once()
mock_diag.assert_called_once()
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.path.exists", return_value=True)
@patch("devx.molecule.start_docker.is_docker_ready")
def test_rootless_socket_available(
self, mock_ready: MagicMock, mock_exists: MagicMock, mock_diag: MagicMock
) -> None:
"""Should use rootless socket if host socket fails."""
# First check (host) fails, second check (rootless) succeeds
mock_ready.side_effect = [False, True]
with patch("devx.molecule.start_docker.glob.glob", return_value=[]):
assert start_docker_daemon(timeout=5) is True
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.path.exists", return_value=True)
@patch("devx.molecule.start_docker.is_docker_ready")
def test_alt_rootless_socket_found(
self, mock_ready: MagicMock, mock_exists: MagicMock, mock_diag: MagicMock
) -> None:
"""Should find rootless socket at a different UID via glob scan."""
# Host fails, own rootless fails, alt rootless succeeds
mock_ready.side_effect = [False, False, True]
alt_sock = "/run/user/999/docker.sock"
with patch("devx.molecule.start_docker.glob.glob", return_value=[alt_sock]):
assert start_docker_daemon(timeout=5) is True
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.path.exists", return_value=True)
@patch("devx.molecule.start_docker.is_docker_ready")
def test_alt_rootless_socket_skips_own(
self, mock_ready: MagicMock, mock_exists: MagicMock, mock_diag: MagicMock
) -> None:
"""Should skip the own rootless socket in glob scan (already tried)."""
# Host fails, own rootless fails, alt rootless also fails, dockerd fails
mock_ready.side_effect = [False, False, False, False, False, False]
own_sock = f"/run/user/{os.getuid()}/docker.sock"
alt_sock = "/run/user/999/docker.sock"
with (
patch("devx.molecule.start_docker.glob.glob", return_value=[own_sock, alt_sock]),
patch("devx.molecule.start_docker.time.sleep"),
patch("devx.molecule.start_docker.subprocess.Popen"),
patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile") as mock_ntf,
patch("builtins.open", mock_open(read_data="err")),
):
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
assert start_docker_daemon(timeout=2) is False
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.path.exists", return_value=False)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=False)
@patch("devx.molecule.start_docker.time.sleep")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
def test_starts_local_daemon(
self,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_sleep: MagicMock,
mock_ready: MagicMock,
mock_exists: MagicMock,
mock_diag: MagicMock,
mock_glob: MagicMock,
) -> None:
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
# Host fails, rootless doesn't exist, local daemon starts
mock_ready.side_effect = [False, False, False, False, True]
assert start_docker_daemon(timeout=5) is True
mock_popen.assert_called_once()
popen_args = mock_popen.call_args.args[0]
assert "dockerd" in popen_args
assert "--storage-driver" in popen_args
assert "vfs" in popen_args
assert "-H" in popen_args
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.path.exists", return_value=False)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=False)
@patch("devx.molecule.start_docker.time.sleep")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
def test_fails_after_timeout(
self,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_sleep: MagicMock,
mock_ready: MagicMock,
mock_exists: MagicMock,
mock_diag: MagicMock,
mock_glob: MagicMock,
) -> None:
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
with patch("builtins.open", mock_open(read_data="dockerd error log")):
assert start_docker_daemon(timeout=3) is False
mock_popen.assert_called_once()
assert mock_sleep.call_count == 3
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.path.exists", return_value=False)
@patch("devx.molecule.start_docker.is_docker_ready", return_value=False)
@patch("devx.molecule.start_docker.time.sleep")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
def test_fails_log_read_error(
self,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_sleep: MagicMock,
mock_ready: MagicMock,
mock_exists: MagicMock,
mock_diag: MagicMock,
mock_glob: MagicMock,
) -> None:
"""Should handle log read errors gracefully."""
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
with patch("builtins.open", side_effect=OSError("permission denied")):
assert start_docker_daemon(timeout=2) is False
@patch("devx.molecule.start_docker.glob.glob", return_value=[])
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.path.exists", return_value=False)
@patch("devx.molecule.start_docker.is_docker_ready")
@patch("devx.molecule.start_docker.time.sleep")
@patch("devx.molecule.start_docker.subprocess.Popen")
@patch("devx.molecule.start_docker.tempfile.NamedTemporaryFile")
def test_local_daemon_ready_on_first_check(
self,
mock_ntf: MagicMock,
mock_popen: MagicMock,
mock_sleep: MagicMock,
mock_ready: MagicMock,
mock_exists: MagicMock,
mock_diag: MagicMock,
mock_glob: MagicMock,
) -> None:
mock_ntf.return_value = MagicMock(name="/tmp/dockerd.log")
# Host fails, rootless doesn't exist, local ready on first loop check
mock_ready.side_effect = [False, False, True]
assert start_docker_daemon(timeout=5) is True
assert mock_popen.call_count == 1
assert mock_sleep.call_count == 1
@patch("devx.molecule.start_docker._diagnose_socket")
@patch("devx.molecule.start_docker.os.environ")
@patch("devx.molecule.start_docker.is_docker_ready", return_value=True)
def test_sets_docker_host(
self,
mock_ready: MagicMock,
mock_environ: MagicMock,
mock_diag: MagicMock,
) -> None:
"""DOCKER_HOST must be set so molecule connects to correct socket."""
start_docker_daemon(timeout=5)
mock_environ.__setitem__.assert_called_with("DOCKER_HOST", f"unix://{DOCKER_SOCK}")
class TestMain:
@patch("devx.molecule.start_docker.start_docker_daemon", return_value=True)
def test_success(self, mock_start: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
@patch("devx.molecule.start_docker.start_docker_daemon", return_value=False)
def test_failure(self, mock_start: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 1
@patch("devx.molecule.start_docker.start_docker_daemon", return_value=True)
def test_custom_timeout_flag(self, mock_start: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(main, ["--timeout", "60"])
assert result.exit_code == 0
mock_start.assert_called_once_with(60)
@patch("devx.molecule.start_docker.os.environ.get")
@patch("devx.molecule.start_docker.start_docker_daemon", return_value=True)
def test_exports_github_env(self, mock_start: MagicMock, mock_get: MagicMock) -> None:
"""Should write DOCKER_HOST to GITHUB_ENV when available."""
mock_get.side_effect = lambda key, default="": (
"/tmp/github_env" if key == "GITHUB_ENV" else f"unix://{DOCKER_SOCK}" if key == "DOCKER_HOST" else default
)
with patch("builtins.open", mock_open()) as mock_file:
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
mock_file.assert_called_with("/tmp/github_env", "a")
@patch("devx.molecule.start_docker.os.environ.get", return_value="")
@patch("devx.molecule.start_docker.start_docker_daemon", return_value=True)
def test_no_github_env(self, mock_start: MagicMock, mock_get: MagicMock) -> None:
"""Should not crash when GITHUB_ENV is not set."""
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0