Compare commits

..
277 Commits
Author SHA1 Message Date
devx-ci-bot ce5ce33a12 release: v0.27.2 [skip ci] 2026-06-29 11:30:13 +00:00
emil 0fae419584 DEVX-100: fix: retry release push on non-fast-forward with rebase loop 2026-06-29 11:29:31 +00:00
gitea-actions-bot 70b011d4a6 chore: update badge URLs to commit 53e15be6 [skip ci] 2026-06-28 17:02:18 +00:00
devx-ci-bot a5c16a92df release: v0.27.1 [skip ci] 2026-06-28 17:02:12 +00:00
emil e6f022ae96 DEVX-99: fix: exclude .devin/.terraform dirs from lint_docs, add duplicate heading excludes 2026-06-28 17:01:37 +00:00
gitea-actions-bot 1a27983750 chore: update badge URLs to commit 8f2186a4 [skip ci] 2026-06-28 16:36:47 +00:00
devx-ci-bot 5d7ed62b34 release: v0.27.0 [skip ci] 2026-06-28 16:36:35 +00:00
emil ee80c27631 DEVX-98: feat: add lint_docs tool, fix doc_coverage/check_translations for any repo 2026-06-28 16:35:59 +00:00
gitea-actions-bot 98b1659579 chore: update badge URLs to commit f308b9f8 [skip ci] 2026-06-28 15:02:09 +00:00
devx-ci-bot c12d9abc6d release: v0.26.4 [skip ci] 2026-06-28 15:01:52 +00:00
emil 40a65cb0a6 DEVX-97: fix: wrap all user-facing strings with _() for i18n completeness 2026-06-28 15:01:07 +00:00
gitea-actions-bot 1a89738dd4 chore: update badge URLs to commit 36f474db [skip ci] 2026-06-28 14:56:08 +00:00
devx-ci-bot 9e604ea2c7 release: v0.26.3 [skip ci] 2026-06-28 14:55:48 +00:00
emil 4bb50bed58 DEVX-97: fix: pin all dependencies to exact versions for reproducibility 2026-06-28 14:55:03 +00:00
gitea-actions-bot 5bb9dce530 chore: update badge URLs to commit c04f826e [skip ci] 2026-06-28 14:42:36 +00:00
emil 73662a3bf0 DEVX-95: chore: pin all dependency versions to concrete releases 2026-06-28 14:41:27 +00:00
gitea-actions-bot a1e87b1905 chore: update badge URLs to commit 991d923a [skip ci] 2026-06-28 14:38:20 +00:00
devx-ci-bot d8d0ad04a2 release: v0.26.2 [skip ci] 2026-06-28 14:38:13 +00:00
emil e836c09088 DEVX-96: fix: block admin merge override and auto-approve with review token 2026-06-28 14:37:36 +00:00
gitea-actions-bot 507436b134 chore: update badge URLs to commit 8dbdd563 [skip ci] 2026-06-28 12:15:13 +00:00
devx-ci-bot 32cec2c5ad release: v0.26.1 [skip ci] 2026-06-28 12:15:06 +00:00
emil 55c530eb00 DEVX-93: fix: force pip upgrade in setup-image to install new dependencies 2026-06-28 12:14:31 +00:00
gitea-actions-bot 3928de4507 chore: update badge URLs to commit 7dc6d2ce [skip ci] 2026-06-28 01:59:24 +00:00
devx-ci-bot 8bb1813715 release: v0.26.0 [skip ci] 2026-06-28 01:59:17 +00:00
emil f7f53941a1 DEVX-92: feat: add distribute_items CI tool for parallel VM deployment 2026-06-28 01:58:42 +00:00
gitea-actions-bot 5edfdaa7aa chore: update badge URLs to commit 3a6bff69 [skip ci] 2026-06-28 00:18:22 +00:00
devx-ci-bot 893da8ba34 release: v0.25.0 [skip ci] 2026-06-28 00:18:14 +00:00
emil 64a58874b6 DEVX-91: feat: add manual review support to pr_review (--event, --body, --checklist-confirmed) 2026-06-28 00:17:33 +00:00
gitea-actions-bot c1c2041ca4 chore: update badge URLs to commit 0fa6360d [skip ci] 2026-06-28 00:07:16 +00:00
devx-ci-bot 49ff8870b1 release: v0.24.1 [skip ci] 2026-06-28 00:07:07 +00:00
emil 4c1ecbf4fa DEVX-91: refactor: add find_task_by_identifier, config fallbacks for tools 2026-06-28 00:06:29 +00:00
gitea-actions-bot 93a1cb9945 chore: update badge URLs to commit 6c0ce9c6 [skip ci] 2026-06-27 23:39:13 +00:00
devx-ci-bot 507bc86b92 release: v0.24.0 [skip ci] 2026-06-27 23:38:58 +00:00
emil 81dc30ecff DEVX-91: feat: add pr_status, pr_logs, pr_label tools 2026-06-27 23:38:16 +00:00
gitea-actions-bot 3c421dd1ad chore: update badge URLs to commit d312f7b7 [skip ci] 2026-06-27 22:14:47 +00:00
devx-ci-bot 11ce99756c release: v0.23.4 [skip ci] 2026-06-27 22:14:38 +00:00
emil 6149167ba2 DEVX-90: fix: classify .gitea/** as user-facing for devx, support glob in user_facing_overrides 2026-06-27 22:14:05 +00:00
gitea-actions-bot 014ab0b63f chore: update badge URLs to commit 7430be68 [skip ci] 2026-06-27 22:07:28 +00:00
emil 2a3ee1ec96 DEVX-89: fix: add --auto-login to all notify_failure calls in workflows 2026-06-27 22:06:47 +00:00
gitea-actions-bot 5d4968eb21 chore: update badge URLs to commit 48324375 [skip ci] 2026-06-27 21:41:36 +00:00
devx-ci-bot 33cfbb0f41 release: v0.23.3 [skip ci] 2026-06-27 21:41:26 +00:00
emil 598238e4d6 DEVX-88: fix: correct clean_images delete URL and add retry with error handling 2026-06-27 21:40:49 +00:00
gitea-actions-bot 925b99b7db chore: update badge URLs to commit dd2ca2f7 [skip ci] 2026-06-27 21:20:07 +00:00
devx-ci-bot 16ed48bd26 release: v0.23.2 [skip ci] 2026-06-27 21:19:53 +00:00
emil 3b0500164b DEVX-87: fix: add skip-ci flag to release commits and concurrency to build-images 2026-06-27 21:19:17 +00:00
gitea-actions-bot 00a44ec5dc chore: update badge URLs to commit b62d5c3d [skip ci] 2026-06-27 19:42:25 +00:00
gitea-actions-bot b385c57621 chore: update badge URLs to commit 3e6359f8 [skip ci] 2026-06-27 19:41:36 +00:00
devx-ci-bot 3181b24f5e release: v0.23.1 2026-06-27 19:40:59 +00:00
emil bc8478220c DEVX-86: fix: add rsync to ci-full image for molecule_docker 2026-06-27 19:40:20 +00:00
gitea-actions-bot a568c0899f chore: update badge URLs to commit 0f22063f [skip ci] 2026-06-27 18:06:46 +00:00
gitea-actions-bot 4216698ca8 chore: update badge URLs to commit 6823dfce [skip ci] 2026-06-27 18:06:11 +00:00
devx-ci-bot f3685b9029 release: v0.23.0 2026-06-27 18:06:01 +00:00
emil 2e5470236e DEVX-85: feat: add devx-lint-dockerfiles to devx.mak, alias setup-image 2026-06-27 18:05:25 +00:00
gitea-actions-bot d2dcf8f7c4 chore: update badge URLs to commit 1d6085cf [skip ci] 2026-06-27 17:25:13 +00:00
emil 357e07a9a6 DEVX-84: refactor: remove hadolint on-the-fly install from setup-image 2026-06-27 17:24:34 +00:00
gitea-actions-bot ffb3976224 chore: update badge URLs to commit 22ed0d7b [skip ci] 2026-06-27 16:44:47 +00:00
gitea-actions-bot ad75b22f2a chore: update badge URLs to commit b800f158 [skip ci] 2026-06-27 16:44:13 +00:00
devx-ci-bot 37f867f6d8 release: v0.22.1 2026-06-27 16:44:03 +00:00
emil 233a0bc055 DEVX-83: fix: fail lint-dockerfiles when hadolint is missing 2026-06-27 16:43:22 +00:00
gitea-actions-bot 1a28f5dcc5 chore: update badge URLs to commit 7471d490 [skip ci] 2026-06-27 16:17:12 +00:00
emil 3af60af439 DEVX-82: fix: checkout release tag in publish job 2026-06-27 16:16:25 +00:00
gitea-actions-bot e181104e1c chore: update badge URLs to commit 6a571fb4 [skip ci] 2026-06-27 15:06:45 +00:00
gitea-actions-bot ee1826fb34 chore: update badge URLs to commit 477726f7 [skip ci] 2026-06-27 15:06:17 +00:00
devx-ci-bot 9170546a31 release: v0.22.0 2026-06-27 15:06:08 +00:00
emil 91f59076a1 DEVX-81: feat: document CI_GITEA_TOKEN scopes and add CI_GITEA_USERNAME to env var table 2026-06-27 15:05:37 +00:00
gitea-actions-bot ca310fe442 chore: update badge URLs to commit c8ba6417 [skip ci] 2026-06-27 15:00:55 +00:00
emil decba5ec77 DEVX-80: chore: update badge URLs to commit 20bcfe96 [skip ci] 2026-06-27 14:56:57 +00:00
gitea-actions-bot 082df1d893 chore: update badge URLs to commit 20bcfe96 [skip ci] 2026-06-27 13:47:51 +00:00
gitea-actions-bot 08f58e551b chore: update badge URLs to commit 8203528c [skip ci] 2026-06-27 13:47:06 +00:00
devx-ci-bot 66bace57d9 release: v0.21.2 2026-06-27 13:46:57 +00:00
emil 5c8d74015e DEVX-79: fix: gate auto-merge on release-dry-run and unmask failures 2026-06-27 13:46:16 +00:00
gitea-actions-bot 48eec986f6 chore: update badge URLs to commit 3a45f8e1 [skip ci] 2026-06-27 13:28:35 +00:00
emil 1fc1cfb23f DEVX-78: style: compact devx-setup-image to pass checkmake maxbodylength (5 lines) 2026-06-27 13:27:41 +00:00
gitea-actions-bot f24b6914f6 chore: update badge URLs to commit b687262f [skip ci] 2026-06-27 13:24:52 +00:00
gitea-actions-bot 84d02ca221 chore: update badge URLs to commit 6043e937 [skip ci] 2026-06-27 13:24:13 +00:00
devx-ci-bot 18ac8f4ba9 release: v0.21.1 2026-06-27 13:24:07 +00:00
emil 9fb9be9c35 DEVX-77: fix: devx-setup-image configures Gitea PyPI registry and shows pip errors 2026-06-27 13:23:29 +00:00
gitea-actions-bot 9a46723391 chore: update badge URLs to commit f7a68467 [skip ci] 2026-06-27 13:05:31 +00:00
devx-ci-bot 5828d3f07b release: v0.21.0 2026-06-27 13:04:50 +00:00
gitea-actions-bot 4232f4baee chore: update badge URLs to commit 88ae1247 [skip ci] 2026-06-27 13:04:46 +00:00
emil a9fd1a47af DEVX-76: feat: add --auto-login to publish, extract configure_tea_login to gitea_cli 2026-06-27 13:04:06 +00:00
gitea-actions-bot ecd10241fb chore: update badge URLs to commit 022fdc32 [skip ci] 2026-06-27 08:02:52 +00:00
emil 5fb497d108 DEVX-75: fix: remove tag fallback step from release workflow 2026-06-27 08:02:04 +00:00
gitea-actions-bot 7d4c32c761 chore: update badge URLs to commit 894ff869 [skip ci] 2026-06-27 07:47:37 +00:00
emil cb8af53c26 DEVX-74: fix: publish job uses setup-release for build + tea login 2026-06-27 07:46:55 +00:00
gitea-actions-bot 954ede87a7 chore: update badge URLs to commit 5e80905f [skip ci] 2026-06-27 07:43:15 +00:00
gitea-actions-bot eb30faf027 chore: update badge URLs to commit 2979764a [skip ci] 2026-06-27 07:42:40 +00:00
devx-ci-bot 6c4157b5c6 release: v0.20.3 2026-06-27 07:42:30 +00:00
emil ea7ddb2036 DEVX-73: fix: release publish failures and duplicate release commits 2026-06-27 07:41:50 +00:00
gitea-actions-bot ef63ada2f0 chore: update badge URLs to commit bc706211 [skip ci] 2026-06-27 07:21:27 +00:00
gitea-actions-bot cdd5f5a8da chore: update badge URLs to commit a0473322 [skip ci] 2026-06-27 07:20:49 +00:00
devx-ci-bot 63ae375b4b release: v0.20.2 2026-06-27 07:20:10 +00:00
emil 8862ea4639 DEVX-72: ci: add hadolint Dockerfile linter to CI 2026-06-27 07:19:31 +00:00
gitea-actions-bot 8ca0a1b208 chore: update badge URLs to commit 55d2bb5c [skip ci] 2026-06-27 03:08:47 +00:00
emil 670f5a099a DEVX-71: ci: use pre-built tier images in CI workflows 2026-06-27 03:08:06 +00:00
gitea-actions-bot a5277a0790 chore: update badge URLs to commit 177dda82 [skip ci] 2026-06-27 02:42:47 +00:00
devx-ci-bot 203d16b19d release: v0.20.2 2026-06-27 04:41:07 +02:00
gitea-actions-bot 12871fb343 chore: update badge URLs to commit 917159e3 [skip ci] 2026-06-27 04:40:59 +02:00
emil a585ef09b6 DEVX-70: fix: correct sed substitution in ci-full Dockerfile 2026-06-27 02:38:53 +00:00
gitea-actions-bot 92aea7df10 chore: update badge URLs to commit d8f48949 [skip ci] 2026-06-27 04:29:12 +02:00
gitea-actions-bot ee3ad74634 chore: update badge URLs to commit ac4f73ef [skip ci] 2026-06-27 04:27:29 +02:00
devx-ci-bot 626ea67b28 release: v0.20.1 2026-06-27 02:26:36 +00:00
emil 87c3fa6634 DEVX-69: fix: correct image references in tier Dockerfiles 2026-06-27 02:24:24 +00:00
gitea-actions-bot 048d161192 chore: update badge URLs to commit d600d21c [skip ci] 2026-06-27 01:37:07 +00:00
gitea-actions-bot 762eee4a55 chore: update badge URLs to commit d0ea4c99 [skip ci] 2026-06-27 03:36:02 +02:00
devx-ci-bot d84958fab7 release: v0.20.0 2026-06-27 01:35:31 +00:00
emil c0238e75df DEVX-68: feat: add pre-built Docker runner images and tested image build/push tools 2026-06-27 01:34:05 +00:00
gitea-actions-bot d4ddbd7e7a chore: update badge URLs to commit fd23e5b2 [skip ci] 2026-06-27 00:01:53 +00:00
gitea-actions-bot 08b993fc4a chore: update badge URLs to commit ebe8b016 [skip ci] 2026-06-27 01:58:56 +02:00
devx-ci-bot bee730a52f release: v0.19.3 2026-06-26 23:58:24 +00:00
emil d0a4a774a0 DEVX-67: refactor: make molecule weights configurable via pyproject.toml 2026-06-26 23:55:33 +00:00
gitea-actions-bot 882f9805ed chore: update badge URLs to commit a45dd92c [skip ci] 2026-06-27 01:40:17 +02:00
gitea-actions-bot a85e0baaea chore: update badge URLs to commit e0f5da9a [skip ci] 2026-06-27 01:39:01 +02:00
devx-ci-bot 663572768b release: v0.19.2 2026-06-27 01:38:41 +02:00
emil 1f2533872d DEVX-66: fix: calibrate molecule weights from actual CI execution times 2026-06-26 23:36:58 +00:00
gitea-actions-bot cb7e9dbc7e chore: update badge URLs to commit 0ab8d43a [skip ci] 2026-06-26 21:39:38 +02:00
devx-ci-bot f5081e10b1 release: v0.19.1 2026-06-26 21:36:46 +02:00
gitea-actions-bot cabc0d1adc chore: update badge URLs to commit 816fbbbb [skip ci] 2026-06-26 19:36:16 +00:00
emil 7c1ecd6ff9 DEVX-65: refactor: consolidate publish.yml into post-merge.yml 2026-06-26 19:34:45 +00:00
gitea-actions-bot 5063f659bc chore: update badge URLs to commit f977a79e [skip ci] 2026-06-26 21:04:44 +02:00
gitea-actions-bot 96c77a0ba4 chore: update badge URLs to commit c9a5f623 [skip ci] 2026-06-26 19:03:18 +00:00
devx-ci-bot 40dd578d89 release: v0.19.0
Publish Release / publish (push) Waiting to run
2026-06-26 19:03:14 +00:00
emil 7ea9b4a96b DEVX-64: feat: add skip_ref_prefixes config to check_agent_docs 2026-06-26 19:01:39 +00:00
gitea-actions-bot 08ceaf484f chore: update badge URLs to commit c003531a [skip ci] 2026-06-26 20:52:05 +02:00
gitea-actions-bot fb6b0fda1d chore: update badge URLs to commit c5811a72 [skip ci] 2026-06-26 18:50:24 +00:00
devx-ci-bot b81a418d07 release: v0.18.0
Publish Release / publish (push) Waiting to run
2026-06-26 18:50:20 +00:00
emil 58261f7d1a DEVX-63: feat: extract generic tools into devx, expand devx.mak, remove personal references 2026-06-26 18:48:43 +00:00
gitea-actions-bot 85e38f37fd chore: update badge URLs to commit 0a0adc8d [skip ci] 2026-06-26 18:00:16 +00:00
gitea-actions-bot 08b573e2ed chore: update badge URLs to commit 77b32b69 [skip ci] 2026-06-26 17:58:35 +00:00
devx-ci-bot e45a546c16 release: v0.17.0
Publish Release / publish (push) Waiting to run
2026-06-26 17:58:32 +00:00
emil 41c631d5f5 DEVX-62: feat: weighted LPT distribution, workflow fixes, decouple vikunja/sync-wiki from release 2026-06-26 17:57:03 +00:00
gitea-actions-bot e271c79e93 chore: update badge URLs to commit e2c9e22d [skip ci] 2026-06-26 16:14:52 +00:00
gitea-actions-bot f4305821f1 chore: update badge URLs to commit 2e58ef07 [skip ci] 2026-06-26 16:14:11 +00:00
devx-ci-bot e4f40223d2 release: v0.16.0
Publish Release / publish (push) Waiting to run
2026-06-26 18:12:38 +02:00
emil 06e80516d4 DEVX-61: optimise slow unit tests and handle missing tea binary in TeaCLI 2026-06-26 16:09:54 +00:00
gitea-actions-bot f1adf22c3e chore: update badge URLs to commit 097082dd [skip ci] 2026-06-26 17:06:00 +02:00
emil 91216da1a4 DEVX-61: feat: single-source-of-truth config via [tool.devx] in pyproject.toml 2026-06-26 15:04:11 +00:00
gitea-actions-bot f9836208df chore: update badge URLs to commit e913bce4 [skip ci] 2026-06-26 16:32:11 +02:00
gitea-actions-bot 0f0f0b683a chore: update badge URLs to commit 3bc02ab2 [skip ci] 2026-06-26 14:31:58 +00:00
devx-ci-bot 54f687f1bf release: v0.15.0
Publish Release / publish (push) Waiting to run
2026-06-26 16:30:45 +02:00
emil 44c906a5e6 DEVX-60: feat: add create-task, create-pr, pre-push-check tools and devx.mak fragment 2026-06-26 14:29:47 +00:00
gitea-actions-bot a3d528f802 chore: update badge URLs to commit 6d2607dd [skip ci] 2026-06-26 02:20:46 +02:00
gitea-actions-bot e3a7afc0b0 chore: update badge URLs to commit a1b92efe [skip ci] 2026-06-26 02:20:43 +02:00
devx-ci-bot 700d3b55c6 release: v0.14.2
Publish Release / publish (push) Waiting to run
2026-06-26 00:19:13 +00:00
emil 701363d935 DEVX-59: fix: make repo arg optional in publish CLI, auto-detect from GITHUB_REPOSITORY 2026-06-26 00:17:55 +00:00
gitea-actions-bot ddb2d43b4e chore: update badge URLs to commit 3fb76ae0 [skip ci] 2026-06-26 01:34:46 +02:00
gitea-actions-bot fe6373b682 chore: update badge URLs to commit 27c68d16 [skip ci] 2026-06-25 23:34:31 +00:00
devx-ci-bot 33434d5750 release: v0.14.1
Publish Release / publish (push) Waiting to run
2026-06-26 01:33:23 +02:00
emil dfcd33c35b DEVX-58: fix: handle 'already a release' error idempotently in publish 2026-06-25 23:32:28 +00:00
gitea-actions-bot 0aefe1f028 chore: update badge URLs to commit f3c14a15 [skip ci] 2026-06-25 23:24:58 +00:00
gitea-actions-bot 891b0b5dba chore: update badge URLs to commit e89db952 [skip ci] 2026-06-25 23:24:49 +00:00
devx-ci-bot 8f15e5402b release: v0.14.0
Publish Release / publish (push) Waiting to run
2026-06-26 01:23:45 +02:00
emil 9060cd7b1e DEVX-57: feat: add FORCE_DEPLOY env var, --git flag, --from-tag flag 2026-06-25 23:22:41 +00:00
gitea-actions-bot f687ab5aa3 chore: update badge URLs to commit c31f8a46 [skip ci] 2026-06-26 00:59:13 +02:00
gitea-actions-bot 4738b594b2 chore: update badge URLs to commit a398ba43 [skip ci] 2026-06-25 22:58:46 +00:00
devx-ci-bot f6e9f2013b release: v0.13.0
Publish Release / publish (push) Waiting to run
2026-06-25 22:57:37 +00:00
emil 8450f33e88 DEVX-56: feat: add --force flag to classify_changes, fix api_clients coverage 2026-06-25 22:56:31 +00:00
gitea-actions-bot 904812dfae chore: update badge URLs to commit 525d3b70 [skip ci] 2026-06-26 00:11:59 +02:00
emil 95384c26e1 DEVX-55: fix: revert squash-merge format to use colon after task ID 2026-06-25 22:09:46 +00:00
gitea-actions-bot c10b759f6b chore: update badge URLs to commit 47d60f8b [skip ci] 2026-06-25 21:55:13 +00:00
gitea-actions-bot 4c818b32ce chore: update badge URLs to commit 9c9fe0f1 [skip ci] 2026-06-25 23:54:43 +02:00
devx-ci-bot bbf09c07df release: v0.1.0 2026-06-25 21:53:29 +00:00
emil faff67aa6a DEVX-54: fix: squash-merge format uses space not colon after task ID 2026-06-25 21:51:36 +00:00
gitea-actions-bot 3e4dfcadb7 chore: update badge URLs to commit 1c7678eb [skip ci] 2026-06-25 23:41:29 +02:00
gitea-actions-bot 2385747bed chore: update badge URLs to commit cb9e5b47 [skip ci] 2026-06-25 23:41:22 +02:00
devx-ci-bot 3625bf2872 release: v0.12.5
Publish Release / publish (push) Waiting to run
2026-06-25 23:40:08 +02:00
emil 8affccfa35 DEVX-53: fix: make PyPI publish failures non-fatal 2026-06-25 21:39:12 +00:00
gitea-actions-bot 7ae85b6955 chore: update badge URLs to commit e5e2b54b [skip ci] 2026-06-25 23:28:38 +02:00
gitea-actions-bot f702286779 chore: update badge URLs to commit 0ed104f4 [skip ci] 2026-06-25 21:28:25 +00:00
devx-ci-bot d4b58fa86f release: v0.12.4
Publish Release / publish (push) Waiting to run
2026-06-25 23:27:13 +02:00
emil 44c6c42ede DEVX-52: fix: guarantee Gitea release for every tag 2026-06-25 21:26:13 +00:00
gitea-actions-bot 14c585971d chore: update badge URLs to commit dfe5ee1e [skip ci] 2026-06-25 23:04:02 +02:00
emil 3e2342c347 DEVX-51: fix: pass REPO_TOKEN to setup-release so tea login is configured 2026-06-25 21:02:00 +00:00
gitea-actions-bot f3d5b0ff45 chore: update badge URLs to commit d5507246 [skip ci] 2026-06-25 20:56:25 +00:00
devx-ci-bot 75e36897cc release: v0.12.3 [skip ci] 2026-06-25 20:55:23 +00:00
emil 0eef69a902 DEVX-50: refactor: remove JUnit reporting from devx 2026-06-25 20:54:21 +00:00
gitea-actions-bot f9130884d1 chore: update badge URLs to commit 7341990c [skip ci] 2026-06-25 19:32:07 +00:00
devx-ci-bot ba002c2e72 release: v0.12.2 [skip ci] 2026-06-25 21:30:48 +02:00
emil af610d22ec DEVX-49: fix: remove auto-rebase from auto-merge to prevent CI feedback loop 2026-06-25 19:29:45 +00:00
gitea-actions-bot 69a585db2f chore: update badge URLs to commit 1b6a8d99 [skip ci] 2026-06-25 19:14:36 +02:00
devx-ci-bot 700df828ba release: v0.12.1 [skip ci] 2026-06-25 19:13:16 +02:00
emil 4df0602157 DEVX-48: fix: use heredoc syntax for multi-line $GITHUB_ENV values 2026-06-25 17:12:20 +00:00
gitea-actions-bot d7d90fe165 chore: update badge URLs to commit 9a9a53ef [skip ci] 2026-06-25 01:06:19 +02:00
devx-ci-bot 22c2d7c925 release: v0.12.0 [skip ci] 2026-06-24 23:05:23 +00:00
emil b4350751f1 DEVX-47: feat: add Polish as officially supported language 2026-06-24 23:04:20 +00:00
gitea-actions-bot a987b63da7 chore: update badge URLs to commit 85b1ae90 [skip ci] 2026-06-24 22:50:19 +00:00
emil 82d613e23b DEVX-46: docs: add pyproject.toml dependency and pip.conf instructions for devx 2026-06-24 22:48:06 +00:00
gitea-actions-bot 6b6c9d40f1 chore: update badge URLs to commit 90c2559f [skip ci] 2026-06-25 00:35:25 +02:00
emil d17854296a DEVX-45: docs: remove stale .taskid file fallback references 2026-06-24 22:33:35 +00:00
gitea-actions-bot 2fff7ed271 chore: update badge URLs to commit c2065ac1 [skip ci] 2026-06-25 00:28:01 +02:00
devx-ci-bot 0a4d66ab5c release: v0.11.1 [skip ci] 2026-06-25 00:25:51 +02:00
emil 14c9200179 DEVX-44: fix: add build/twine to ci deps, activate venv in notify_failure 2026-06-24 22:24:33 +00:00
gitea-actions-bot 85ae272b1f chore: update badge URLs to commit 389da217 [skip ci] 2026-06-24 22:18:25 +00:00
devx-ci-bot a0c4c1c7f0 release: v0.11.0 [skip ci] 2026-06-24 22:17:36 +00:00
emil fd0c4de31e DEVX-43: feat: add publish step to post-merge release job, make publish idempotent 2026-06-24 22:16:37 +00:00
gitea-actions-bot 8f7af97335 chore: update badge URLs to commit 0bb83e89 [skip ci] 2026-06-24 22:57:46 +02:00
devx-ci-bot a14d838564 release: v0.10.2 [skip ci] 2026-06-24 20:56:35 +00:00
emil 7dcb9c03c0 DEVX-42: fix: badge generation respects pyproject.toml testpaths, shows stdout in warnings 2026-06-24 20:55:35 +00:00
gitea-actions-bot 6f2b110c17 chore: update badge URLs to commit 55e25b06 [skip ci] 2026-06-24 20:34:59 +00:00
devx-ci-bot cfb856ff75 release: v0.10.1 [skip ci] 2026-06-24 20:34:06 +00:00
emil 95adf86895 DEVX-41: fix: badge generation REPO_ROOT, auto-detect package, error feedback 2026-06-24 20:33:12 +00:00
gitea-actions-bot 7cf039ebbe chore: update badge URLs to commit 16bc9c20 [skip ci] 2026-06-24 22:04:42 +02:00
devx-ci-bot a17982f2cf release: v0.10.0 [skip ci] 2026-06-24 22:03:14 +02:00
emil cf85964877 DEVX-40: feat: remove .taskid file fallback, use branch name only 2026-06-24 20:02:19 +00:00
gitea-actions-bot fbb1fc3134 chore: update badge URLs to commit ffc2c8a9 [skip ci] 2026-06-24 19:27:17 +00:00
emil a8f86aca68 DEVX-39: fix: use raw/branch/badges/ URLs for badges in README and docs 2026-06-24 19:25:50 +00:00
gitea-actions-bot 13bed1d99c chore: update badge URLs to commit 329cfc69 [skip ci] 2026-06-24 19:03:43 +00:00
devx-ci-bot 107cff5dec release: v0.9.12 [skip ci] 2026-06-24 19:02:56 +00:00
emil cb037aa69c DEVX-38: fix: clean dist/ before build and add workflow_dispatch to publish 2026-06-24 19:02:02 +00:00
gitea-actions-bot 7fa1c4450c chore: update badge URLs to commit 85d87b6f [skip ci] 2026-06-24 20:51:42 +02:00
devx-ci-bot cf2921845b release: v0.9.11 [skip ci] 2026-06-24 20:50:37 +02:00
emil c272150275 DEVX-37: fix: resolve repo_root from GITHUB_WORKSPACE or cwd 2026-06-24 18:49:43 +00:00
gitea-actions-bot 8de91be405 chore: update badge URLs to commit 5b6b6674 [skip ci] 2026-06-24 20:37:25 +02:00
emil 46b8fe5078 DEVX-36: docs: comprehensive documentation rewrite 2026-06-24 18:35:24 +00:00
gitea-actions-bot 7e2a8b4535 chore: update badge URLs to commit f54c01f9 [skip ci] 2026-06-24 17:25:38 +00:00
emil c90518acdb DEVX-35: fix: use raw/branch/badges/ URLs for badges in README and docs 2026-06-24 17:24:05 +00:00
devx-ci-bot 80ca622838 release: v0.9.10 [skip ci] 2026-06-24 18:42:23 +02:00
emil 3f2d19d7ac DEVX-34: fix: retrospective fixes for CI/CD friction 2026-06-24 16:41:22 +00:00
devx-ci-bot c839d49fe3 release: v0.9.9 [skip ci] 2026-06-24 13:17:30 +02:00
emil 93b5d2f926 DEVX-33: fix: use explicit refspecs for git push to avoid tag/branch ambiguity 2026-06-24 11:16:41 +00:00
emil 131c04c9d0 DEVX-32: fix: filter non-version tags in release verification 2026-06-24 11:09:47 +00:00
emil 8e9681cf7d DEVX-31: fix: prefer branch name for task ID extraction + strip heads/ prefix in release 2026-06-24 10:54:58 +00:00
emil 6631525a1d DEVX-30: fix: use DOCKER_HOST env var in is_docker_ready + scan all rootless sockets 2026-06-24 10:35:32 +00:00
devx-ci-bot 6985030a3c release: v0.9.8 [skip ci] 2026-06-24 11:15:20 +02:00
emil 4d073f3beb DEVX-29: fix: use DOCKER_HOST env var in is_docker_ready + scan all rootless sockets 2026-06-24 09:14:27 +00:00
devx-ci-bot 037d7b0d16 release: v0.9.7 [skip ci] 2026-06-24 02:18:19 +00:00
emil 9cb706e387 DEVX-28: fix: add rootless socket fallback and GITHUB_ENV export 2026-06-24 02:17:20 +00:00
devx-ci-bot 5bd6158f2a release: v0.9.6 [skip ci] 2026-06-24 04:00:50 +02:00
emil 05aa2ffe76 DEVX-27: fix: add Docker socket diagnostics to start_docker 2026-06-24 01:59:55 +00:00
emil b9c3b55680 DEVX-27: fix: add Docker socket diagnostics to start_docker 2026-06-24 01:50:46 +00:00
devx-ci-bot d398c8e971 release: v0.9.5 [skip ci] 2026-06-24 01:37:15 +00:00
emil 39526d8e6a DEVX-26: fix: use host Docker socket with DOCKER_HOST fallback to local dockerd 2026-06-24 01:36:21 +00:00
emil 37730e2187 DEVX-25: fix: use host Docker socket with DOCKER_HOST fallback to local dockerd 2026-06-24 01:24:00 +00:00
devx-ci-bot e2f66ca70a release: v0.9.4 [skip ci] 2026-06-24 01:11:08 +00:00
emil 0b88c211f1 DEVX-24: fix: use separate Docker socket for DinD in CI 2026-06-24 01:08:10 +00:00
devx-ci-bot ea4ee0d303 release: v0.9.3 [skip ci] 2026-06-24 02:46:26 +02:00
emil 16fba17b03 DEVX-23: fix: use tempfile for dockerd log to fix CI permission error 2026-06-24 00:45:28 +00:00
devx-ci-bot 011cf3e093 release: v0.9.2 [skip ci] 2026-06-24 02:34:47 +02:00
emil daf99c5fed DEVX-22: fix: use vfs storage driver for Docker-in-Docker in CI 2026-06-24 00:33:48 +00:00
emil 7154e3ad7c DEVX-21: chore: trigger auto-merge after Vikunja title fix 2026-06-24 00:28:18 +00:00
devx-ci-bot 6053fb9fba release: v0.9.1 [skip ci] 2026-06-24 01:50:03 +02:00
emil e76741bfad DEVX-21: fix: always start dockerd in CI runner for molecule tests 2026-06-23 23:49:13 +00:00
devx-ci-bot f206a9cd8d release: v0.9.0 [skip ci] 2026-06-24 01:29:24 +02:00
emil b4b7428f9c DEVX-20: feat: extract Docker daemon start to tested Python module 2026-06-23 23:28:20 +00:00
devx-ci-bot 0d9e76a838 release: v0.8.5 [skip ci] 2026-06-24 01:02:21 +02:00
emil 034cbde2f7 DEVX-19: fix: retry pip install with --ignore-installed only on failure 2026-06-23 23:01:32 +00:00
devx-ci-bot e0abe6f176 release: v0.8.4 [skip ci] 2026-06-23 22:28:29 +00:00
emil 15f6837dc2 DEVX-18: fix: add --ignore-installed to pip in CI to bypass debian packages 2026-06-23 22:27:35 +00:00
devx-ci-bot b4dda91e24 release: v0.8.3 [skip ci] 2026-06-23 21:58:33 +00:00
emil 3e21e774f7 DEVX-17: fix: pass --break-system-packages to pip in CI environments 2026-06-23 21:57:37 +00:00
emil 7c11215e57 DEVX-16: fix: lower check_test_speed threshold to 4 seconds 2026-06-23 20:35:53 +00:00
devx-ci-bot 5384269c83 release: v0.8.2 [skip ci] 2026-06-23 21:44:03 +02:00
emil b3d0dd8ca7 DEVX-15: fix: encode spaces in pair commands to survive shell word-splitting 2026-06-23 19:42:56 +00:00
devx-ci-bot a7dcaee5c6 release: v0.8.1 [skip ci] 2026-06-23 20:45:02 +02:00
emil 02f8d3757b DEVX-14: fix: set fresh MOLECULE_HOME per pair to avoid stale config cache 2026-06-23 18:43:58 +00:00
devx-ci-bot 4311fb7648 release: v0.8.0 [skip ci] 2026-06-23 20:11:18 +02:00
emil 2ead959fcf DEVX-14: feat: fix molecule platforms to use sleep infinity, add --platforms-file 2026-06-23 18:10:19 +00:00
devx-ci-bot 9a60009d29 release: v0.7.0 [skip ci] 2026-06-23 18:26:54 +02:00
emil c20dfd185a DEVX-13: feat: add per-test timing quality gate to check_test_speed 2026-06-23 16:25:50 +00:00
devx-ci-bot 547fef4f27 release: v0.6.0 [skip ci] 2026-06-23 15:38:44 +02:00
emil 23183df7c7 DEVX-12: feat: add opentofu helpers, CLI entry points, shared utility, and CI improvements 2026-06-23 13:37:10 +00:00
devx-ci-bot f382408115 release: v0.5.0 [skip ci] 2026-06-23 03:36:10 +02:00
emil 19bec24f45 DEVX-10: feat: add tag verification, idempotency, and --verify mode to release script 2026-06-23 01:28:27 +00:00
devx-ci-bot 37772f21a9 release: v0.4.4 [skip ci] 2026-06-22 23:45:40 +02:00
emil b07132e3c6 DEVX-9: fix: configurable task prefix and CWD-relative DOCS_DIR 2026-06-22 21:44:35 +00:00
devx-ci-bot 7b3b604c2c release: v0.4.3 [skip ci] 2026-06-22 23:13:27 +02:00
emil 53990dc10c DEVX-8: fix: expand DEFAULT_INFRASTRUCTURE to cover all common project files 2026-06-22 21:12:16 +00:00
devx-ci-bot 8ea044a942 release: v0.4.2 [skip ci] 2026-06-22 22:55:31 +02:00
emil 8411c92c95 DEVX-7: fix: make all warnings into errors across devx tools 2026-06-22 20:54:25 +00:00
devx-ci-bot 621b9936c8 release: v0.4.1 [skip ci] 2026-06-22 22:22:57 +02:00
emil 33063038a1 DEVX-6: fix: correct version tags, changelog, and release script recovery 2026-06-22 20:21:44 +00:00
devx-ci-bot 0c3e8a7b8d release: v0.4.0 [skip ci] 2026-06-22 21:04:55 +02:00
emil 6436c5dd38 DEVX-5: feat: add DEFAULT_INFRASTRUCTURE and configurable task prefix 2026-06-22 19:03:53 +00:00
devx-ci-bot 33b09c162d release: v0.3.0 [skip ci] 2026-06-22 20:23:30 +02:00
emil a2c856d8b2 DEVX-4: feat: add --no-ansible-collections option to setup tool 2026-06-22 18:22:26 +00:00
devx-ci-bot 7d9a081c92 release: v0.2.0 [skip ci] 2026-06-22 19:32:37 +02:00
emil 87d730d8be DEVX-3: feat: pluggable change classification framework 2026-06-22 17:31:30 +00:00
130 changed files with 22755 additions and 1947 deletions
+20
View File
@@ -0,0 +1,20 @@
.venv/
.git/
.gitea/
tests/
docs/
*.egg-info/
__pycache__/
htmlcov/
.coverage
dist/
build/
*.md
!README.md
.env
.env.example
activate.sh
activate.fish
activate.zsh
hooks/
.devin/
+1 -1
View File
@@ -1,6 +1,6 @@
# Gitea API token (required for CI scripts that interact with Gitea)
# Create at: https://git.oblachno.oblachno.fyi/user/settings/applications
REPO_TOKEN=
CI_GITEA_TOKEN=
# Vikunja API token (required for post-merge task updates)
# Create at: https://work.oblachno.oblachno.fyi/settings/tokens
+136
View File
@@ -0,0 +1,136 @@
name: Build Images
# Builds and pushes pre-built Docker runner images to the Gitea registry.
# These images eliminate the 40-120s setup tax on every CI job by baking
# devx and all dependencies into the image.
#
# Triggers:
# - On push to master (after post-merge release completes)
# - Manually via workflow_dispatch
#
# The workflow builds 3 tier images in sequence:
# ci-base → ci-quality → ci-full
#
# Each tier builds FROM the previous one, so they must be built in order.
# After pushing, a cleanup job removes old versions (keeps last 2 + latest).
on:
push:
branches: [master]
paths:
- docker/**
- pyproject.toml
- src/devx/**
workflow_dispatch:
concurrency:
group: build-images
cancel-in-progress: false
jobs:
detect-type:
runs-on: docker
timeout-minutes: 5
outputs:
is-release: ${{ steps.check.outputs.is-release }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Set up environment
run: make setup-ci
- name: Check if this is a release commit
id: check
env:
PYTHONPATH: src
run: |
. .venv/bin/activate
python3 -m devx.ci.detect_release_commit
build-and-push:
needs: [detect-type]
if: needs.detect-type.outputs.is-release == 'false'
runs-on: docker
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up environment
env:
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
run: make setup-release
- name: Docker registry login
env:
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
CI_GITEA_USERNAME: ${{ vars.CI_GITEA_USERNAME }}
run: |
. .venv/bin/activate
echo "$CI_GITEA_TOKEN" | docker login git.oblachno.oblachno.fyi -u "$CI_GITEA_USERNAME" --password-stdin
- name: Build and push tier images
env:
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
CI_GITEA_USERNAME: ${{ vars.CI_GITEA_USERNAME }}
PYTHONPATH: src
run: |
. .venv/bin/activate
export PATH="$HOME/.local/bin:$PATH"
# Build ci-base first (it's the base for ci-quality and ci-full)
python3 -m devx.tools.build_image \
--dockerfile docker/ci-base/Dockerfile \
--name oblachno-oss/runner-images/ci-base \
--tag latest \
--registry git.oblachno.oblachno.fyi \
--push --pull
# Build ci-quality (FROM ci-base-latest)
python3 -m devx.tools.build_image \
--dockerfile docker/ci-quality/Dockerfile \
--name oblachno-oss/runner-images/ci-quality \
--tag latest \
--registry git.oblachno.oblachno.fyi \
--push
# Build ci-full (FROM ci-quality-latest)
python3 -m devx.tools.build_image \
--dockerfile docker/ci-full/Dockerfile \
--name oblachno-oss/runner-images/ci-full \
--tag latest \
--registry git.oblachno.oblachno.fyi \
--push
- name: Notify on failure
if: failure()
env:
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
. .venv/bin/activate 2>/dev/null || true
export PATH="$HOME/.local/bin:$PATH"
python3 -m devx.ci.notify_failure \
--repo "${{ github.repository }}" \
--run-id "${{ github.run_id }}" \
--workflow "build-images/build-and-push" \
--commit "${{ github.sha }}" \
--auto-login
cleanup:
needs: [build-and-push]
if: always() && needs.build-and-push.result == 'success'
runs-on: docker
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Set up environment
run: make setup-ci
- name: Clean up old image versions
env:
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
. .venv/bin/activate
python3 -m devx.tools.clean_images \
--owner oblachno-oss \
--name oblachno-oss/runner-images/ci-base \
--name oblachno-oss/runner-images/ci-quality \
--name oblachno-oss/runner-images/ci-full \
--keep 2
+63 -16
View File
@@ -8,11 +8,15 @@ on:
jobs:
quality:
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-quality:latest
timeout-minutes: 10
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
- name: Set up environment
run: make setup-quality
run: make setup-image
- name: Lint all
run: |
. .venv/bin/activate
@@ -27,13 +31,19 @@ jobs:
PYTHONPATH: src
run: |
. .venv/bin/activate
python3 -m devx.tools.check_test_speed --max-seconds 10
python3 -m devx.tools.check_test_speed --max-seconds 4 --max-single-seconds 0.5
- name: Documentation coverage check
env:
PYTHONPATH: src
run: |
. .venv/bin/activate
python3 -m devx.ci.doc_coverage --fail-on-missing
- name: Documentation lint check
env:
PYTHONPATH: src
run: |
. .venv/bin/activate
python3 -m devx.ci.lint_docs --root .
- name: Translation completeness check
env:
PYTHONPATH: src
@@ -60,7 +70,11 @@ jobs:
detect-changes:
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-base:latest
timeout-minutes: 10
defaults:
run:
shell: bash
outputs:
user-facing-changed: ${{ steps.detect.outputs.user-facing-changed }}
steps:
@@ -68,7 +82,7 @@ jobs:
with:
fetch-depth: 0
- name: Set up environment
run: make setup-ci
run: make setup-image
- name: Detect changed paths
id: detect
env:
@@ -84,32 +98,42 @@ jobs:
needs: [quality, detect-changes]
if: needs.detect-changes.outputs.user-facing-changed == 'true'
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-full:latest
timeout-minutes: 10
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up environment
run: make setup-release
env:
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
run: make setup-image
- name: Release dry-run validation
env:
PYTHONPATH: src
run: |
. .venv/bin/activate
export PATH="$HOME/.local/bin:$PATH"
python3 -m devx.ci.release --dry-run || true
python3 -m devx.ci.release --dry-run
pr-review:
if: github.event_name == 'pull_request'
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-base:latest
timeout-minutes: 10
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
- name: Set up environment
run: make setup-ci
run: make setup-image
- name: Run automated PR review
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
set -euo pipefail
@@ -120,24 +144,46 @@ jobs:
auto-merge:
# Auto-merge runs after all CI checks pass. It reads the task ID
# from .taskid file, validates the PR title, and squash-merges.
# No manual label or review needed — CI is the quality gate.
needs: [quality, detect-changes, pr-review]
if: github.event_name == 'pull_request'
# from the branch name, validates the PR title, and squash-merges.
# Uses always() so it runs even when detect-changes skips (no user-facing changes).
needs: [quality, detect-changes, pr-review, release-dry-run]
if: >-
always() &&
github.event_name == 'pull_request' &&
needs.quality.result == 'success' &&
needs.pr-review.result == 'success' &&
(needs.release-dry-run.result == 'success' || needs.release-dry-run.result == 'skipped')
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-base:latest
timeout-minutes: 10
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.REPO_TOKEN }}
- name: Install dependencies
token: ${{ secrets.CI_GITEA_TOKEN }}
- name: Set up environment
run: make setup-image
- name: Post approval review
env:
CI_GITEA_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
PR_NUMBER: ${{ github.event.number }}
REPOSITORY: ${{ github.repository }}
PYTHONPATH: src
run: |
python3 -m pip install --break-system-packages requests python-dotenv click
python3 -m pip install --break-system-packages -e .
. .venv/bin/activate
python3 -m devx.ci.pr_review \
"$PR_NUMBER" \
"$REPOSITORY" \
--event APPROVE \
--checklist-confirmed \
--checklist-categories 1,2,3,4,5,6,7,8,9,10,11,12,13 \
--body "Auto-approved: all CI checks passed (quality, pr-review, release-dry-run)."
- name: Squash merge with task ID
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
VIKUNJA_TOKEN: ${{ secrets.VIKUNJA_TOKEN }}
DEVX_VIKUNJA_PROJECT_ID: "8"
PYTHONPATH: src
@@ -146,6 +192,7 @@ jobs:
REPOSITORY: ${{ github.repository }}
PR_NUMBER: ${{ github.event.number }}
run: |
. .venv/bin/activate
python3 -m devx.ci.auto_merge \
"$HEAD_REF" \
"$PR_TITLE" \
+133 -65
View File
@@ -1,30 +1,30 @@
name: Post-merge
# Runs on every push to master. A single workflow with conditional jobs
# replaces separate workflows for release, wiki sync, badges, and
# Vikunja task updates.
# for release, publish, wiki sync, badges, and Vikunja task updates.
#
# Job dependency graph:
#
# detect-type ──┬── release (skip if release commit)
# detect-type ──┬── validate-commit-msg (skip if release commit)
# ├── release (skip if release commit)
# │ └── publish (needs release — builds & publishes to PyPI)
# ├── badges (ALWAYS runs — even on release commits)
# ├── configure-repo (independent — skip if release commit)
# ├── sync-wiki (needs release — skip if release commit/fails)
# └── vikunja (needs release — skip if release commit/fails)
# ├── sync-wiki (skip if release commit — runs for ALL merges)
# └── vikunja (skip if release commit — runs for ALL merges)
#
# sync-wiki and vikunja depend on release succeeding so that the wiki
# and task tracker are only updated when the code is actually released.
# If release fails, they are skipped to avoid leaving the wiki or
# Vikunja in an inconsistent state with the codebase on master.
# sync-wiki and vikunja run for ALL non-release commits, not just when
# release succeeds. This ensures the wiki and task tracker are updated
# even for infrastructure-only changes (docs, CI config, etc.).
#
# The badges job depends on release so it picks up the latest version
# number. It uses `if: always()` with no is-release condition so it
# runs on every push to master, including release commits. This
# ensures badges (tests, coverage, version, etc.) are always current.
# The badges job uses `if: always()` with no is-release condition so it
# runs on every push to master, including release commits. This ensures
# badges (tests, coverage, version, etc.) are always current.
#
# When release creates a "release: vX.Y.Z" commit, the release
# commit's post-merge run still updates badges (version badge picks
# up the new version). Other jobs skip. The tag push triggers publish.yml.
# When release creates a "release: vX.Y.Z" commit and tag, the publish
# job (which depends on release) builds and publishes the package to the
# Gitea PyPI registry. The release commit's post-merge run still updates
# badges (version badge picks up the new version). Other jobs skip.
on:
push:
@@ -33,40 +33,47 @@ on:
jobs:
detect-type:
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-base:latest
timeout-minutes: 10
defaults:
run:
shell: bash
outputs:
is-release: ${{ steps.check.outputs.is-release }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Install dependencies
run: |
python3 -m pip install --break-system-packages requests python-dotenv click
python3 -m pip install --break-system-packages -e .
- name: Set up environment
run: make setup-image
- name: Check if this is a release commit
id: check
env:
PYTHONPATH: src
run: python3 -m devx.ci.detect_release_commit
run: |
. .venv/bin/activate
python3 -m devx.ci.detect_release_commit
validate-commit-msg:
needs: [detect-type]
if: needs.detect-type.outputs.is-release == 'false'
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-base:latest
timeout-minutes: 5
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Install dependencies
run: |
python3 -m pip install --break-system-packages click python-dotenv
python3 -m pip install --break-system-packages -e .
- name: Set up environment
run: make setup-image
- name: Validate latest commit message
env:
PYTHONPATH: src
run: |
. .venv/bin/activate
git log -1 --format=%B > commit-msg.txt
python3 -m devx.ci.validate_commit_msg commit-msg.txt --branch master
rm -f commit-msg.txt
@@ -75,19 +82,28 @@ jobs:
needs: [detect-type]
if: needs.detect-type.outputs.is-release == 'false'
runs-on: docker
timeout-minutes: 10
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-full:latest
timeout-minutes: 15
defaults:
run:
shell: bash
outputs:
tag: ${{ steps.release-tag.outputs.tag }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.REPO_TOKEN }}
token: ${{ secrets.CI_GITEA_TOKEN }}
- name: Set up environment
run: make setup-release
env:
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
run: make setup-image
- name: Configure git
run: |
git config user.name "devx-ci-bot"
git config user.email "devx-ci-bot@oblachno.fyi"
- name: Run release
id: release-tag
env:
PYTHONPATH: src
run: |
@@ -97,33 +113,75 @@ jobs:
- name: Notify on failure
if: failure()
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
. .venv/bin/activate 2>/dev/null || true
export PATH="$HOME/.local/bin:$PATH"
python3 -m devx.tools.install_tools --tool tea
tea login add --name devx --url "${{ github.server_url }}" --token "$REPO_TOKEN" || true
tea login default devx || true
python3 -m devx.ci.notify_failure \
--repo "${{ github.repository }}" \
--run-id "${{ github.run_id }}" \
--workflow "post-merge/release" \
--commit "${{ github.sha }}"
--commit "${{ github.sha }}" \
--auto-login
publish:
needs: [release]
if: needs.release.outputs.tag != ''
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-full:latest
timeout-minutes: 10
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ needs.release.outputs.tag }}
- name: Set up environment
run: make setup-image EXTRAS=release
- name: Build and publish release
env:
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
. .venv/bin/activate
export PATH="$HOME/.local/bin:$PATH"
python3 -m devx.ci.publish "${{ needs.release.outputs.tag }}" "${{ github.repository }}" --auto-login
- name: Notify on failure
if: failure()
env:
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
. .venv/bin/activate 2>/dev/null || true
export PATH="$HOME/.local/bin:$PATH"
python3 -m devx.ci.notify_failure \
--repo "${{ github.repository }}" \
--run-id "${{ github.run_id }}" \
--workflow "post-merge/publish" \
--commit "${{ github.sha }}" \
--auto-login
sync-wiki:
needs: [detect-type, release]
needs: [detect-type]
if: needs.detect-type.outputs.is-release == 'false'
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-base:latest
timeout-minutes: 10
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up environment
run: make setup-ci
run: make setup-image
- name: Sync documentation to wiki
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
. .venv/bin/activate
@@ -131,7 +189,7 @@ jobs:
- name: Notify on failure
if: failure()
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
export PATH="$HOME/.local/bin:$PATH"
@@ -139,25 +197,30 @@ jobs:
--repo "${{ github.repository }}" \
--run-id "${{ github.run_id }}" \
--workflow "post-merge/sync-wiki" \
--commit "${{ github.sha }}"
--commit "${{ github.sha }}" \
--auto-login
badges:
needs: [detect-type, release]
needs: [detect-type]
if: always()
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-quality:latest
timeout-minutes: 10
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: master
token: ${{ secrets.REPO_TOKEN }}
token: ${{ secrets.CI_GITEA_TOKEN }}
- name: Fetch latest master
run: |
git fetch origin master
git reset --hard origin/master
- name: Set up environment
run: make setup-ci
run: make setup-image
- name: Generate and push badges
env:
PRE_COMMIT_ALLOW_NO_CONFIG: "1"
@@ -167,7 +230,7 @@ jobs:
- name: Notify on failure
if: failure()
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
export PATH="$HOME/.local/bin:$PATH"
@@ -175,71 +238,76 @@ jobs:
--repo "${{ github.repository }}" \
--run-id "${{ github.run_id }}" \
--workflow "post-merge/badges" \
--commit "${{ github.sha }}"
--commit "${{ github.sha }}" \
--auto-login
vikunja:
needs: [detect-type, release]
needs: [detect-type]
if: needs.detect-type.outputs.is-release == 'false'
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-base:latest
timeout-minutes: 10
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install dependencies
run: |
python3 -m pip install --break-system-packages requests python-dotenv click
python3 -m pip install --break-system-packages -e .
- name: Set up environment
run: make setup-image
- name: Update Vikunja task
env:
VIKUNJA_TOKEN: ${{ secrets.VIKUNJA_TOKEN }}
DEVX_VIKUNJA_PROJECT_ID: "8"
PYTHONPATH: src
run: python3 -m devx.ci.post_merge --git-sha "${{ github.sha }}"
run: |
. .venv/bin/activate
python3 -m devx.ci.post_merge --git-sha "${{ github.sha }}"
- name: Notify on failure
if: failure()
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
export PATH="$HOME/.local/bin:$PATH"
python3 -m devx.tools.install_tools --tool tea
tea login add --name devx --url "${{ github.server_url }}" --token "$REPO_TOKEN" || true
tea login default devx || true
python3 -m devx.ci.notify_failure \
--repo "${{ github.repository }}" \
--run-id "${{ github.run_id }}" \
--workflow "post-merge/vikunja" \
--commit "${{ github.sha }}"
--commit "${{ github.sha }}" \
--auto-login
configure-repo:
needs: [detect-type]
if: needs.detect-type.outputs.is-release == 'false'
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-base:latest
timeout-minutes: 10
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
- name: Install dependencies
run: |
python3 -m pip install --break-system-packages requests python-dotenv click
python3 -m pip install --break-system-packages -e .
- name: Set up environment
run: make setup-image
- name: Ensure branch protection and labels
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: python3 -m devx.tools.configure_repo --repo devx --owner oblachno-oss
run: |
. .venv/bin/activate
python3 -m devx.tools.configure_repo --repo devx --owner oblachno-oss
- name: Notify on failure
if: failure()
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
CI_GITEA_TOKEN: ${{ secrets.CI_GITEA_TOKEN }}
PYTHONPATH: src
run: |
export PATH="$HOME/.local/bin:$PATH"
python3 -m devx.tools.install_tools --tool tea
tea login add --name devx --url "${{ github.server_url }}" --token "$REPO_TOKEN" || true
tea login default devx || true
python3 -m devx.ci.notify_failure \
--repo "${{ github.repository }}" \
--run-id "${{ github.run_id }}" \
--workflow "post-merge/configure-repo" \
--commit "${{ github.sha }}"
--commit "${{ github.sha }}" \
--auto-login
-49
View File
@@ -1,49 +0,0 @@
name: Publish Release
on:
push:
tags:
- 'v*'
jobs:
publish:
runs-on: docker
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install dependencies
run: |
python3 -m pip install --break-system-packages build twine requests python-dotenv click
python3 -m pip install --break-system-packages -e .
- name: Install CI tools
run: |
export PATH="$HOME/.local/bin:$PATH"
python3 -m devx.tools.install_tools --tool git-cliff --tool tea
- name: Configure tea login
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
run: |
export PATH="$HOME/.local/bin:$PATH"
tea login add --name devx --url "${{ github.server_url }}" --token "$REPO_TOKEN" || true
tea login default devx || true
- name: Build and publish release
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
PYTHONPATH: src
run: |
export PATH="$HOME/.local/bin:$PATH"
python3 -m devx.ci.publish "${{ github.ref_name }}" "${{ github.repository }}"
- name: Notify on failure
if: failure()
env:
REPO_TOKEN: ${{ secrets.REPO_TOKEN }}
PYTHONPATH: src
run: |
export PATH="$HOME/.local/bin:$PATH"
python3 -m devx.ci.notify_failure \
--repo "${{ github.repository }}" \
--run-id "${{ github.run_id }}" \
--workflow "publish" \
--commit "${{ github.sha }}"
+3
View File
@@ -35,3 +35,6 @@ Thumbs.db
# Badges
.badges/
# Deprecated CI task tracking (branch name is the sole source of truth)
.taskid
+14
View File
@@ -0,0 +1,14 @@
# Hadolint configuration for devx Dockerfiles
# https://github.com/hadolint/hadolint#configure
ignored:
- DL3008 # Don't require pinning apt package versions
- DL3013 # Don't require pinning pip package versions
- DL3018 # Don't require pinning apk package versions
- DL3007 # Using latest is intentional for tier images (rebuilt on every merge)
- SC2102 # False positive: pip extras [release,molecule,deploy] look like shell ranges
trustedRegistries:
- git.oblachno.oblachno.fyi
- docker.io
- gitea/runner-images
-1
View File
@@ -1 +0,0 @@
DEVX-2
+233 -21
View File
@@ -4,8 +4,8 @@
```bash
make setup # Create venv, install deps, set up hooks, install CI tools
make install-tools # Install actionlint, git-cliff, act_runner to ~/.local/bin
make lint-all # ruff + pyright + bandit + actionlint
make install-tools # Install actionlint, git-cliff, act_runner, tea, hadolint to ~/.local/bin
make lint-all # ruff + pyright + bandit + actionlint + lint-dockerfiles
make pytest-cov # Unit tests with 100% coverage enforcement
make test-unit # Unit tests without coverage
make workflow-lint # Static lint of .gitea/workflows/*.yml (actionlint)
@@ -16,8 +16,8 @@ make clean # Remove caches, build artifacts, coverage data
`make setup` automatically installs all development tools:
- **Python deps** via `python -m devx.tools.setup` (pip install -e .[dev], pre-commit hooks)
- **actionlint, git-cliff, act_runner, tea** via `python -m devx.tools.install_tools` (CI/CD tools to ~/.local/bin)
- **tea CLI login** via `python -m devx.tools.setup` (configures `tea login` from `.env` `REPO_TOKEN`)
- **actionlint, git-cliff, act_runner, tea, hadolint** via `python -m devx.tools.install_tools` (CI/CD tools to ~/.local/bin)
- **tea CLI login** via `python -m devx.tools.setup` (configures `tea login` from `.env` `CI_GITEA_TOKEN`)
## Workflow Verification (Before Push)
@@ -55,25 +55,49 @@ src/devx/
├── translations.json # Translation strings (en, bg)
├── ci/ # CI/CD automation modules (run by workflows)
│ ├── release.py # Automated versioning, tagging, changelog
│ ├── publish.py # Build and publish to Gitea PyPI registry
│ ├── publish.py # Build and publish to Gitea PyPI registry (--skip-build for non-Python repos)
│ ├── auto_merge.py # Squash-merge PRs with task ID validation
│ ├── check_auto_merge_ready.py # Pre-merge validation gate (branch, PR title, Vikunja, behind-master)
│ ├── _shared.py # Shared utilities (get_latest_tag)
│ ├── classify_changes.py # User-facing vs workflow-only change detection
│ ├── detect_release_commit.py # Detect release commits on master
│ ├── validate_commit_msg.py # Conventional commit validation
│ ├── pr_review.py # Automated PR review
│ ├── pr_review.py # Automated PR review + manual reviews (--event, --body, --checklist-confirmed)
│ ├── post_merge.py # Vikunja task updates after merge
│ ├── sync_wiki.py # Sync documentation to Gitea wiki
│ ├── push_badges.py # Generate and push quality badges
│ ├── notify_failure.py # Create Gitea issues on CI failures
│ ├── push_badges.py # Generate and push quality badges (--retries for retry on git push failures)
│ ├── notify_failure.py # Create Gitea issues on CI failures (--auto-login)
│ ├── distribute_files.py # Distribute files across parallel runners (LPT scheduling)
│ ├── distribute_items.py # Distribute generic items (VMs, hosts) across parallel runners (LPT)
│ ├── integration_guard.py # Run pytest with cross-runner fail-fast
│ ├── check_translations.py # Translation completeness check
│ └── doc_coverage.py # Documentation coverage check
│ ├── doc_coverage.py # Documentation coverage check
│ └── lint_docs.py # Documentation linter (structure, links, headings)
├── tools/ # Developer tooling modules (run locally or by CI)
│ ├── setup.py # Environment setup (venv, deps, hooks)
│ ├── install_tools.py # Install actionlint, git-cliff, act_runner, tea
│ ├── install_tools.py # Install actionlint, git-cliff, act_runner, tea, hadolint
│ ├── install_checkmake.py # Install checkmake (Makefile linter)
│ ├── build_image.py # Build and push Docker images to Gitea registry
│ ├── clean_images.py # Clean up old Docker image versions from Gitea registry
│ ├── check_test_speed.py # Measure unit test execution time
│ ├── check_mutable_globals.py # Detect module-level mutable globals (test isolation bugs)
│ ├── check_pyproject_deps.py # Validate pyproject.toml deps have documentation comments
│ ├── check_test_coverage.py # Ensure changed files have corresponding tests (configurable rules)
│ ├── check_agent_docs.py # Validate docs for stale file references (configurable patterns)
│ ├── configure_repo.py # Branch protection and label setup
│ └── generate_badges.py # Badge SVG generation
│ ├── generate_badges.py # Badge SVG generation
│ ├── create_task.py # Create Vikunja tasks
│ ├── create_pr.py # Create PRs with auto-derived title from Vikunja
│ ├── pr_status.py # Check CI status for a PR/commit (--wait polls)
│ ├── pr_logs.py # Fetch logs for failed CI jobs
│ └── pr_label.py # Add labels to PRs (idempotent)
├── opentofu.py # OpenTofu output helpers (get_tofu_output, get_tofu_vm_ip, get_tofu_vm_field)
└── molecule/ # Optional molecule testing helpers (for Ansible projects)
├── discover_runners.py # Dynamic Gitea runner discovery
├── distribute_molecule.py # Distribute molecule scenarios across runners (LPT scheduling, --roles-root for multi-role)
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast (--roles-root)
├── molecule_all.py # Run all molecule scenarios locally
└── platforms.py # Supported molecule platforms
```
### Key Design Principles
@@ -153,7 +177,7 @@ the PR. Then add the `ready-to-merge` label. The auto-merge workflow will:
1. **Validate** PR title format (`DEVX-N: <vikunja task title>`) and match against Vikunja task title
2. **Check** that at least one substantive APPROVE review exists
3. Wait for all CI checks to pass (including the `pr-review` job)
4. Squash-merge with title: `DEVX-N <conventional commit message>` (space-separated, no colon after DEVX-N)
4. Squash-merge with title: `DEVX-N: <conventional commit message>`
5. The post-merge workflow marks the Vikunja task as done
6. The release workflow automatically versions, tags, and publishes
@@ -167,7 +191,7 @@ After a PR is merged to master, the **post-merge workflow**
1. **detect-type** — Checks if the commit is a regular merge or a
release commit (`release: vX.Y.Z`). All subsequent jobs skip for
release commits.
release commits (except badges).
2. **release** — Runs `python -m devx.ci.release` which:
- Checks for user-facing changes via `python -m devx.ci.classify_changes`
@@ -179,14 +203,20 @@ After a PR is merged to master, the **post-merge workflow**
- Creates an annotated tag `vX.Y.Z` on the release commit
- Pushes both the commit and tag to master
3. **sync-wiki** — Syncs documentation to the Gitea wiki.
3. **sync-wiki** — Syncs documentation to the Gitea wiki. Runs for ALL
non-release commits (not just when release succeeds), so docs-only
changes still update the wiki.
4. **badges** — Generates and pushes quality badge SVGs to the `badges` branch.
Uses `if: always()` so it runs on every push, including release commits.
5. **vikunja** — Marks the corresponding Vikunja task as done.
5. **vikunja** — Marks the corresponding Vikunja task as done. Runs for ALL
non-release commits (not just when release succeeds), so infrastructure-only
changes still update the task tracker.
The tag push triggers the **publish workflow** (`.gitea/workflows/publish.yml`)
which builds and publishes the package to the Gitea PyPI registry.
6. **publish** — Runs after release succeeds (needs: release). Builds and
publishes the package to the Gitea PyPI registry. Gets the tag from the
release job's `tag` output (written via `GITHUB_OUTPUT`).
### Smart CI: User-Facing vs Workflow-Only Changes
@@ -240,7 +270,7 @@ so `.:src` is not needed. The `src` directory is the sole import root.
The `tea` Gitea CLI tool is used for Gitea API interactions. It is installed
by `python -m devx.tools.install_tools` and configured by
`python -m devx.tools.setup` (login profile from `.env` `REPO_TOKEN`).
`python -m devx.tools.setup` (login profile from `.env` `CI_GITEA_TOKEN`).
**`devx.gitea_cli.TeaCLI`** — Python wrapper around `tea` CLI with JSON output parsing:
- `create_issue()` — Create issues with labels
@@ -250,7 +280,7 @@ by `python -m devx.tools.install_tools` and configured by
### git-cliff Commit Preprocessing
Merge commits on master have the format `DEVX-N <conventional commit>`. The
Merge commits on master have the format `DEVX-N: <conventional commit>`. The
`cliff.toml` includes a `commit_preprocessors` entry that strips the `DEVX-N `
prefix before parsing. This ensures all merged work appears in the changelog.
@@ -273,7 +303,52 @@ setuptools via `dynamic = ["version"]` in `pyproject.toml`.
| Branch name | `DEVX-N-short-description` | `DEVX-12-add-release-script` |
| Branch commits | `<conventional commit>` | `feat: add release script` |
| PR title | `DEVX-N: <vikunja task title>` | `DEVX-12: Add release automation` |
| Merge commit | `DEVX-N <conventional commit>` | `DEVX-12 feat: add release script` |
| Merge commit | `DEVX-N: <conventional commit>` | `DEVX-12: feat: add release script` |
### Task ID Resolution
`auto_merge` resolves the task ID solely from the branch name (e.g.
`DEVX-12-fix-foo` → `DEVX-12`). Branch names must include the task ID
prefix — there is no `.taskid` file fallback. If a stale `.taskid` file
exists in the repo, a deprecation warning is printed advising its removal.
### Workflow `auto-merge` Job and `always()`
When `auto-merge` depends on a job that can be skipped (e.g.
`molecule-tests`), the `if:` condition MUST include `always() &&`
at the start. Without it, Gitea Actions skips `auto-merge` when any
dependency is skipped, even if the condition explicitly allows
`result == 'skipped'`.
```yaml
auto-merge:
needs: [quality, detect-changes, pr-review, molecule-tests]
if: >-
always() &&
github.event_name == 'pull_request' &&
needs.quality.result == 'success' &&
needs.pr-review.result == 'success' &&
(needs.molecule-tests.result == 'success' || needs.molecule-tests.result == 'skipped')
```
### LPT Test Distribution Algorithm
`distribute_molecule` and `distribute_files` use **LPT (Longest Processing
Time first)** scheduling instead of naive round-robin. This produces a more
balanced distribution when test items have varying costs:
1. **Weight estimation**: Each item is assigned a weight:
- Molecule scenarios: heuristic by name (`nextcloud`=10, `gitea`=8,
`binary`=2, default=3). See `_SCENARIO_WEIGHTS` in
`distribute_molecule.py`.
- Integration test files: weight by file size in bytes (as a proxy
for test runtime).
2. **LPT assignment**: Items are sorted by weight (descending), then
each is assigned to the runner with the least total weight.
This ensures heavy scenarios (e.g. `nextcloud`) are spread across
different runners rather than clustered on one, reducing the
longest-runner time from ~16 min to ~11 min with 6 runners.
## Config System
@@ -285,8 +360,12 @@ devx uses environment variables with `.env` file fallback for configuration.
|----------|---------|-------------|
| `DEVX_GITEA_API_URL` | `https://git.oblachno.oblachno.fyi/api/v1` | Gitea API base URL |
| `DEVX_VIKUNJA_API_URL` | `https://work.oblachno.oblachno.fyi/api/v1` | Vikunja API base URL |
| `DEVX_REPO_OWNER` | **(none — must be set)** | Repository owner for API calls |
| `DEVX_REPO_NAME` | **(none — must be set)** | Repository name (or `owner/repo`) |
| `DEVX_TASK_PREFIX` | `DEVX` | Task ID prefix (GRM, OBL-INFRA, etc.) |
| `DEVX_VIKUNJA_PROJECT_ID` | `6` | Vikunja project ID |
| `DEVX_LANG` | `en` | Language for i18n (en, bg) |
| `REPO_TOKEN` | (from .env) | Gitea API token |
| `CI_GITEA_TOKEN` | (from .env) | Gitea API token |
| `VIKUNJA_TOKEN` | (from .env) | Vikunja API token |
### Per-Project Overrides
@@ -295,6 +374,139 @@ Projects using devx can override the default API URLs and language by setting
`DEVX_*` environment variables or entries in their `.env` file. The config
system loads `.env` automatically via `python-dotenv`.
### pyproject.toml [tool.devx] Configuration
In addition to `DEVX_` env vars, several devx tools read configuration from
the `[tool.devx]` section in `pyproject.toml`. This allows per-project
customization without environment variables.
**Base config** (`[tool.devx]`):
- `task_prefix` — Task ID prefix (e.g. `"DEVX"`, `"GRM"`, `"OBL-INFRA"`)
- `vikunja_project_id` — Vikunja project ID
- `repo_owner` / `repo_name` — Gitea repository coordinates
- `gitea_api_url` / `vikunja_api_url` — API endpoints
**Tool-specific config**:
- `[tool.devx.check_mutable_globals]` — `scan_dirs`, `skip_dirs`, `known_safe`
- `[tool.devx.check_test_coverage]` — `rules` (source_pattern → test_paths mapping), `skip_patterns`
- `[tool.devx.check_agent_docs]` — `scan_dirs`, `deleted_files`, `deprecated_patterns`, `legitimate_indicators`
## devx.mak — Shared Makefile Fragment
`devx.mak` provides common Makefile targets that projects can include
via `-include $(DEVX_MAK)`. This eliminates Makefile duplication across
projects.
**Available targets** (all prefixed with `devx-`):
| Target | Purpose |
|--------|---------|
| `devx-create-task` | Create a Vikunja task |
| `devx-create-pr` | Create a PR with auto-derived title |
| `devx-push` | Push current branch to origin |
| `devx-push-with-pr` | Push and create PR in one step |
| `devx-pr-status` | Check CI status for a PR (`PR=`, `WAIT=`, `TIMEOUT=`) |
| `devx-pr-logs` | Fetch logs for failed CI jobs (`PR=`, `JOB=`, `TAIL=`) |
| `devx-pr-label` | Add a label to a PR (`PR=`, `LABEL=ready-to-merge`) |
| `devx-pr-review` | Post a review on a PR (`PR=`, `EVENT=`, `BODY=`, `CHECKLIST=`) |
| `devx-check-config` | Validate devx configuration |
| `devx-configure-gitea-pypi` | Configure Gitea private PyPI registry |
| `devx-env` | Create .env from .env.example |
| `devx-venv` | Create Python venv with version check |
| `devx-activate-scripts` | Create shell/fish/zsh activate scripts |
| `devx-install-hooks` | Set git hooks path to hooks/ |
| `devx-install-tools` | Install actionlint, git-cliff, act_runner, tea, hadolint |
| `devx-install-checkmake` | Install checkmake (Makefile linter) |
| `devx-checkmake` | Lint Makefiles with checkmake |
| `devx-workflow-lint` | Static lint of Gitea Actions YAML (actionlint) |
| `devx-workflow-dryrun` | Dry-run all workflows (act_runner) |
| `devx-workflow-dryrun-safe` | Best-effort dry-run (skips if act_runner missing) |
| `devx-workflow-check` | Static lint + dry-run |
| `devx-notify-failure` | Create Gitea issue on CI failure |
| `devx-lint-ruff` | Run ruff check |
| `devx-lint-format` | Run ruff format --check |
| `devx-typecheck` | Run pyright |
| `devx-lint-bandit` | Run bandit security scan |
| `devx-lint-deps` | Check dependencies for vulnerabilities (pip-audit) |
| `devx-lint` | Run all lint targets |
| `devx-test-unit` | Run unit tests without coverage |
| `devx-pytest-cov` | Run pytest with coverage enforcement |
| `devx-check-mutable-globals` | Scan for mutable path globals |
| `devx-check-dep-docs` | Validate pyproject.toml deps are documented |
| `devx-check-test-coverage` | Check changed files have corresponding tests |
| `devx-check-docs` | Validate docs for stale references |
| `devx-check-test-speed` | Verify test suite timing |
| `devx-pre-push` | Run lint + tests before push |
| `devx-clean` | Remove caches, build artifacts, coverage data |
| `devx-setup-image` | Link /opt/venv + install project (for pre-built image CI jobs) |
| `devx-lint-dockerfiles` | Lint Dockerfiles with hadolint (fail-fast, parameterized by `DEVX_DOCKERFILE_PATHS`) |
| `devx-build-images` | Build Docker images from manifest (no push) |
| `devx-push-images` | Build and push Docker images to Gitea registry |
| `devx-build-images-dry-run` | Show what would be built/pushed |
| `devx-clean-images` | Delete old image versions (keep last 2 + latest) |
**Variables** (set BEFORE including devx.mak):
- `DEVX_PYTHON` — Python executable (default: `python3`)
- `DEVX_VENV` — venv directory (default: `.venv`)
- `DEVX_BIN` — venv bin directory (default: `$(DEVX_VENV)/bin`)
- `DEVX_LINT_PATHS` — paths for ruff/bandit (default: `src/ tests/`)
- `DEVX_COV_PKG` — coverage package (default: `src/devx`)
- `DEVX_TEST_PATHS` — pytest paths (default: `tests/`)
- `DEVX_PR_BASE` — PR base branch (default: `master`)
- `DEVX_DOCKERFILE_PATHS` — directory to search for Dockerfiles (default: `docker`)
- `DEVX_GITEA_REGISTRY` — registry URL (default: `git.oblachno.oblachno.fyi`)
- `DEVX_IMAGE_MANIFEST` — path to JSON manifest (default: `docker/images.json`)
- `DEVX_IMAGE_OWNER` — package owner for cleanup (default: `oblachno-oss`)
## Pre-built Docker Runner Images
devx builds and publishes three tier images to the Gitea container registry
to eliminate the 40-120s setup tax on every CI job:
| Image | Contains | Used by jobs |
|-------|----------|-------------|
| `ci-base-latest` | Python 3.12 + devx[ci] + tea | detect-changes, detect-type, validate-commit-msg, pr-review, auto-merge, sync-wiki, vikunja, configure-repo |
| `ci-quality-latest` | ci-base + devx[lint] + actionlint + checkmake + hadolint | quality, badges |
| `ci-full-latest` | ci-quality + devx[release,molecule,deploy] + git-cliff + OpenTofu | release, publish, release-dry-run, molecule-tests, deploy jobs |
**Build process** (in `build-images.yml` workflow):
1. `ci-base` builds FROM `gitea/runner-images:ubuntu-latest`
2. `ci-quality` builds FROM `ci-base-latest`
3. `ci-full` builds FROM `ci-quality-latest`
Each image is tagged `latest` and pushed to
`git.oblachno.oblachno.fyi/oblachno-oss/runner-images:<tier>-latest`.
**Using images in workflows**:
```yaml
jobs:
quality:
runs-on: docker
container: git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-quality:latest
steps:
- uses: actions/checkout@v4
- name: Set up environment
run: make setup-image # links /opt/venv, installs project (no-deps)
```
**Image build/push tools** (tested Python modules):
- `devx.tools.build_image` — Build and push Docker images from Dockerfile or manifest
- `devx.tools.clean_images` — Delete old image versions via Gitea API (keep last N + latest)
**Usage in project Makefile**:
```makefile
DEVX_PYTHON := $(BIN)/python
DEVX_MAK := $(shell $(BIN)/python -c \
"from pathlib import Path; import devx; print(Path(devx.__file__).parent / 'make' / 'devx.mak')" \
2>/dev/null)
-include $(DEVX_MAK)
# Aliases for project-specific names
lint-ruff: devx-lint-ruff
workflow-lint: devx-workflow-lint
create-task: devx-create-task
```
## Key Conventions
- Python 3.12+ required (ruff/pyright target `py312`)
+508 -16
View File
@@ -2,9 +2,511 @@
All notable changes to this project will be documented in this file.
## [0.1.0] - 2026-06-22
## [0.27.2] - 2026-06-29
## [0.1.0] - 2026-06-22
### Bug Fixes
- Retry release push on non-fast-forward with rebase loop
## [0.27.1] - 2026-06-28
### Bug Fixes
- Exclude .devin/.terraform dirs from lint_docs, add duplicate heading excludes
## [0.27.0] - 2026-06-28
### Features
- Add lint_docs tool, fix doc_coverage/check_translations for any repo
## [0.26.4] - 2026-06-28
### Bug Fixes
- Wrap all user-facing strings with _() for i18n completeness
## [0.26.3] - 2026-06-28
### Bug Fixes
- Pin all dependencies to exact versions for reproducibility
## [0.26.2] - 2026-06-28
### Bug Fixes
- Block admin merge override and auto-approve with review token
## [0.26.1] - 2026-06-28
### Bug Fixes
- Force pip upgrade in setup-image to install new dependencies
## [0.26.0] - 2026-06-28
### Features
- Add distribute_items CI tool for parallel VM deployment
## [0.25.0] - 2026-06-28
### Features
- Add manual review support to pr_review (--event, --body, --checklist-confirmed)
## [0.24.1] - 2026-06-28
### Refactor
- Add find_task_by_identifier, config fallbacks for tools
## [0.24.0] - 2026-06-27
### Features
- Add pr_status, pr_logs, pr_label tools
## [0.23.4] - 2026-06-27
### Bug Fixes
- Add --auto-login to all notify_failure calls in workflows
- Classify .gitea/** as user-facing for devx, support glob in user_facing_overrides
## [0.23.3] - 2026-06-27
### Bug Fixes
- Correct clean_images delete URL and add retry with error handling
## [0.23.2] - 2026-06-27
### Bug Fixes
- Add skip-ci flag to release commits and concurrency to build-images
## [0.23.1] - 2026-06-27
### Bug Fixes
- Add rsync to ci-full image for molecule_docker
## [0.23.0] - 2026-06-27
### Features
- Add devx-lint-dockerfiles to devx.mak, alias setup-image
### Refactor
- Remove hadolint on-the-fly install from setup-image
## [0.22.1] - 2026-06-27
### Bug Fixes
- Checkout release tag in publish job
- Fail lint-dockerfiles when hadolint is missing
## [0.22.0] - 2026-06-27
### Features
- Document CI_GITEA_TOKEN scopes and add CI_GITEA_USERNAME to env var table
## [0.21.2] - 2026-06-27
### Bug Fixes
- Gate auto-merge on release-dry-run and unmask failures
## [0.21.1] - 2026-06-27
### Bug Fixes
- Devx-setup-image configures Gitea PyPI registry and shows pip errors
## [0.21.0] - 2026-06-27
### Features
- Add --auto-login to publish, extract configure_tea_login to gitea_cli
### Bug Fixes
- Publish job uses setup-release for build + tea login
- Remove tag fallback step from release workflow
## [0.20.3] - 2026-06-27
### Bug Fixes
- Release publish failures and duplicate release commits
## [0.20.2] - 2026-06-27
## [0.20.2] - 2026-06-27
### Bug Fixes
- Correct sed substitution in ci-full Dockerfile
## [0.20.1] - 2026-06-27
### Bug Fixes
- Correct image references in tier Dockerfiles
## [0.20.0] - 2026-06-27
### Features
- Add pre-built Docker runner images and tested image build/push tools
## [0.19.3] - 2026-06-26
### Refactor
- Make molecule weights configurable via pyproject.toml
## [0.19.2] - 2026-06-26
### Bug Fixes
- Calibrate molecule weights from actual CI execution times
## [0.19.1] - 2026-06-26
### Refactor
- Consolidate publish.yml into post-merge.yml
## [0.19.0] - 2026-06-26
### Features
- Add skip_ref_prefixes config to check_agent_docs
## [0.18.0] - 2026-06-26
### Features
- Extract generic tools into devx, expand devx.mak, remove personal references
## [0.17.0] - 2026-06-26
### Features
- Weighted LPT distribution, workflow fixes, decouple vikunja/sync-wiki from release
## [0.16.0] - 2026-06-26
### Features
- Single-source-of-truth config via [tool.devx] in pyproject.toml
## [0.15.0] - 2026-06-26
### Features
- Add create-task, create-pr, pre-push-check tools and devx.mak fragment
## [0.14.2] - 2026-06-26
### Bug Fixes
- Make repo arg optional in publish CLI, auto-detect from GITHUB_REPOSITORY
## [0.14.1] - 2026-06-25
### Bug Fixes
- Handle 'already a release' error idempotently in publish
## [0.14.0] - 2026-06-25
### Features
- Add FORCE_DEPLOY env var, --git flag, --from-tag flag
## [0.13.0] - 2026-06-25
### Features
- Add --force flag to classify_changes, fix api_clients coverage
### Bug Fixes
- Squash-merge format uses space not colon after task ID
- Revert squash-merge format to use colon after task ID
## [0.1.0] - 2026-06-25
## [0.12.5] - 2026-06-25
### Bug Fixes
- Make PyPI publish failures non-fatal
## [0.12.4] - 2026-06-25
### Bug Fixes
- Pass REPO_TOKEN to setup-release so tea login is configured
- Guarantee Gitea release for every tag
## [0.12.3] - 2026-06-25
### Refactor
- Remove JUnit reporting from devx
## [0.12.2] - 2026-06-25
### Bug Fixes
- Remove auto-rebase from auto-merge to prevent CI feedback loop
## [0.12.1] - 2026-06-25
### Bug Fixes
- Use heredoc syntax for multi-line $GITHUB_ENV values
## [0.12.0] - 2026-06-24
### Features
- Add Polish as officially supported language
## [0.11.1] - 2026-06-24
### Bug Fixes
- Add build/twine to ci deps, activate venv in notify_failure
## [0.11.0] - 2026-06-24
### Features
- Add publish step to post-merge release job, make publish idempotent
## [0.10.2] - 2026-06-24
### Bug Fixes
- Badge generation respects pyproject.toml testpaths, shows stdout in warnings
## [0.10.1] - 2026-06-24
### Bug Fixes
- Badge generation REPO_ROOT, auto-detect package, error feedback
## [0.10.0] - 2026-06-24
### Features
- Remove .taskid file fallback, use branch name only
### Bug Fixes
- Use raw/branch/badges/ URLs for badges in README and docs
## [0.9.12] - 2026-06-24
### Bug Fixes
- Clean dist/ before build and add workflow_dispatch to publish
## [0.9.11] - 2026-06-24
### Bug Fixes
- Use raw/branch/badges/ URLs for badges in README and docs
- Resolve repo_root from GITHUB_WORKSPACE or cwd
## [0.9.10] - 2026-06-24
### Bug Fixes
- Retrospective fixes for CI/CD friction
## [0.9.9] - 2026-06-24
### Bug Fixes
- Use DOCKER_HOST env var in is_docker_ready + scan all rootless sockets
- Prefer branch name for task ID extraction + strip heads/ prefix in release
- Filter non-version tags in release verification
- Use explicit refspecs for git push to avoid tag/branch ambiguity
## [0.9.8] - 2026-06-24
### Bug Fixes
- Use DOCKER_HOST env var in is_docker_ready + scan all rootless sockets
## [0.9.7] - 2026-06-24
### Bug Fixes
- Add rootless socket fallback and GITHUB_ENV export
## [0.9.6] - 2026-06-24
### Bug Fixes
- Add Docker socket diagnostics to start_docker
- Add Docker socket diagnostics to start_docker
## [0.9.5] - 2026-06-24
### Bug Fixes
- Use host Docker socket with DOCKER_HOST fallback to local dockerd
- Use host Docker socket with DOCKER_HOST fallback to local dockerd
## [0.9.4] - 2026-06-24
### Bug Fixes
- Use separate Docker socket for DinD in CI
## [0.9.3] - 2026-06-24
### Bug Fixes
- Use tempfile for dockerd log to fix CI permission error
## [0.9.2] - 2026-06-24
### Bug Fixes
- Use vfs storage driver for Docker-in-Docker in CI
## [0.9.1] - 2026-06-23
### Bug Fixes
- Always start dockerd in CI runner for molecule tests
## [0.9.0] - 2026-06-23
### Features
- Extract Docker daemon start to tested Python module
## [0.8.5] - 2026-06-23
### Bug Fixes
- Retry pip install with --ignore-installed only on failure
## [0.8.4] - 2026-06-23
### Bug Fixes
- Add --ignore-installed to pip in CI to bypass debian packages
## [0.8.3] - 2026-06-23
### Bug Fixes
- Lower check_test_speed threshold to 4 seconds
- Pass --break-system-packages to pip in CI environments
## [0.8.2] - 2026-06-23
### Bug Fixes
- Encode spaces in pair commands to survive shell word-splitting
## [0.8.1] - 2026-06-23
### Bug Fixes
- Set fresh MOLECULE_HOME per pair to avoid stale config cache
## [0.8.0] - 2026-06-23
### Features
- Fix molecule platforms to use sleep infinity, add --platforms-file
## [0.7.0] - 2026-06-23
### Features
- Add per-test timing quality gate to check_test_speed
## [0.6.0] - 2026-06-23
### Features
- Add opentofu helpers, CLI entry points, shared utility, and CI improvements
## [0.5.0] - 2026-06-23
### Features
- Add tag verification, idempotency, and --verify mode to release script
## [0.4.4] - 2026-06-22
### Bug Fixes
- Configurable task prefix and CWD-relative DOCS_DIR
## [0.4.3] - 2026-06-22
### Bug Fixes
- Expand DEFAULT_INFRASTRUCTURE to cover all common project files
## [0.4.2] - 2026-06-22
### Bug Fixes
- Make all warnings into errors across devx tools
## [0.4.1] - 2026-06-22
### Bug Fixes
- Correct version tags, changelog, and release script recovery
## [0.4.0] - 2026-06-22
### Features
- Add DEFAULT_INFRASTRUCTURE and configurable task prefix
## [0.3.0] - 2026-06-22
### Features
- Add --no-ansible-collections option to setup tool
## [0.2.0] - 2026-06-22
### Features
- Pluggable change classification framework
## [0.1.2] - 2026-06-22
### Bug Fixes
- Make sync-wiki and vikunja depend on release
## [0.1.1] - 2026-06-22
### Bug Fixes
- Disable push whitelist, allow direct pushes to master
## [0.1.0] - 2026-06-22
@@ -12,18 +514,8 @@ All notable changes to this project will be documented in this file.
- Extract reusable dev/CI tools from GRM into devx package
## [unreleased]
### Features
- Extract reusable development and CI/CD tools from GRM into a standalone Python package
- Port core modules: config, exceptions, i18n, api_clients, gitea_cli
- Port 14 CI scripts: auto_merge, check_translations, classify_changes, detect_release_commit, discover_runners, doc_coverage, notify_failure, post_merge, pr_review, publish, push_badges, release, sync_wiki, validate_commit_msg
- Port 6 dev tools: check_test_speed, generate_badges, install_checkmake, install_tools, setup, configure_repo
- Port 5 molecule tools as optional extra: platforms, distribute_molecule, discover_runners, molecule_ci_guard, molecule_all
- Add CLI entry point with subcommands: devx ci, devx tools, devx molecule
- Add Gitea PyPI registry publishing support in publish.py
- Add configurable workflow-only patterns in classify_changes.py
- Add configurable version file path in release.py
- Replicate GRM's automated workflow: CI, auto-merge, post-merge, release, badges, wiki sync, Vikunja
### Bug Fixes
- Use python3 and venv python in workflows and Makefile
- Fix post-merge job failures (configure-repo, badges, notify-failure)
- Allow release bot to push to protected master
+3 -3
View File
@@ -208,8 +208,8 @@ If you develop a new program, and you want it to be of the greatest possible use
To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the “copyright” line and a pointer to where the full notice is found.
grm
Copyright (C) 2026 emil
devx
Copyright (C) 2026 oblachno-oss
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
@@ -221,7 +221,7 @@ Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode:
grm Copyright (C) 2026 emil
devx Copyright (C) 2026 oblachno-oss
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details.
+80 -39
View File
@@ -1,4 +1,4 @@
.PHONY: all setup setup-ci setup-quality setup-release install update lint lint-ruff lint-format typecheck lint-bandit lint-deps lint-all test test-unit pytest-cov clean workflow-lint workflow-dryrun workflow-check install-tools install-hooks activate-scripts
.PHONY: all setup setup-ci setup-quality setup-release setup-image install update lint lint-all lint-dockerfiles test test-unit pytest-cov clean install-tools install-hooks activate-scripts checkmake check-mutable-globals check-dep-docs check-test-speed build-images push-images build-images-dry-run clean-images
PYTHON := python3
VENV := .venv
@@ -25,10 +25,18 @@ setup-quality: $(VENV)/bin/activate .env install-tools
# Setup for release jobs (needs git-cliff, tea, lint tools)
setup-release: $(VENV)/bin/activate .env
@$(BIN)/pip install -e '.[ci,lint]' 2>/dev/null; \
@$(BIN)/pip install -e '.[ci,lint,release]' 2>/dev/null; \
$(BIN)/python -m devx.tools.install_tools --tool git-cliff --tool tea; \
export PATH="$(HOME)/.local/bin:$$PATH"; \
$(BIN)/python -m devx.tools.setup --bin "$(BIN)" --extras "ci,lint" --no-pre-commit
$(BIN)/python -m devx.tools.setup --bin "$(BIN)" --extras "ci,lint,release" --no-pre-commit
# Setup for pre-built image jobs (deps already in image, just link venv + install project)
# Note: Not aliased to devx-setup-image because devx's own CI images may have
# an older devx.mak that doesn't yet define devx-setup-image. Consumer repos
# (grm, infra) can safely alias to devx-setup-image since they install devx from PyPI.
setup-image:
@if [ -d /opt/venv ]; then ln -sf /opt/venv .venv; . .venv/bin/activate && pip install --no-cache-dir -e . 2>/dev/null; \
else echo "[setup-image] /opt/venv not found — falling back to setup-ci"; $(MAKE) setup-ci; fi
.env:
@if [ ! -f .env ]; then cp .env.example .env; echo "Created .env from .env.example — please edit it."; fi
@@ -52,48 +60,81 @@ install-tools: $(VENV)/bin/activate
@$(BIN)/pip install -e '.' 2>/dev/null; \
$(BIN)/python -m devx.tools.install_tools
lint-ruff:
$(BIN)/ruff check src/ tests/
# --- devx.mak integration ----------------------------------------------------
# Include shared targets from the devx package itself (workflow-lint,
# notify-failure, checkmake, lint targets, quality checks, etc.)
# Since devx IS the package, we can include its own devx.mak.
DEVX_PYTHON := $(BIN)/python
DEVX_VENV := $(VENV)
DEVX_BIN := $(BIN)
DEVX_LINT_PATHS := src/ tests/
DEVX_COV_PKG := src/devx
DEVX_TEST_PATHS := tests/
lint-format:
$(BIN)/ruff format --check src/ tests/
DEVX_MAK := $(shell $(BIN)/python -c \
"from pathlib import Path; import devx; print(Path(devx.__file__).parent / 'make' / 'devx.mak')" \
2>/dev/null)
-include $(DEVX_MAK)
typecheck:
$(BIN)/pyright
# Aliases — project-specific names map to devx.mak targets
lint-ruff: devx-lint-ruff
lint-format: devx-lint-format
typecheck: devx-typecheck
lint-bandit: devx-lint-bandit
lint-deps: devx-lint-deps
lint: devx-lint
workflow-lint: devx-workflow-lint
workflow-dryrun: devx-workflow-dryrun
workflow-dryrun-safe: devx-workflow-dryrun-safe
workflow-check: devx-workflow-check
notify-failure: devx-notify-failure
checkmake: devx-checkmake
check-mutable-globals: devx-check-mutable-globals
check-dep-docs: devx-check-dep-docs
check-test-speed: devx-check-test-speed
check-test-coverage: devx-check-test-coverage
check-docs: devx-check-docs
create-task: devx-create-task
create-pr: devx-create-pr
push-with-pr: devx-push-with-pr
git-push: devx-push
lint-bandit:
$(BIN)/bandit -r src/
lint-all: lint workflow-lint lint-dockerfiles
@echo "[lint-all] All linting checks passed."
lint: lint-ruff lint-format typecheck lint-bandit
# Note: Not aliased to devx-lint-dockerfiles for the same reason as setup-image —
# devx's own CI images may have an older devx.mak. Consumer repos can safely alias.
lint-dockerfiles:
@echo "[lint-dockerfiles] Linting Dockerfiles with hadolint..."
@if ! command -v hadolint >/dev/null 2>&1; then \
echo "[lint-dockerfiles] ERROR: hadolint not found. Install from https://github.com/hadolint/hadolint/releases" >&2; \
exit 1; \
fi
@find docker -name 'Dockerfile*' -exec hadolint {} +
@echo "[lint-dockerfiles] All Dockerfiles passed."
lint-deps:
@echo "Checking dependencies for known vulnerabilities..."
@.venv/bin/python -m ensurepip 2>/dev/null || true
@PIPAPI_PYTHON_LOCATION=$$(pwd)/.venv/bin/python .venv/bin/pip-audit --desc --skip-editable 2>&1 || true
test-unit: devx-test-unit
lint-all: lint workflow-lint
workflow-lint:
@command -v actionlint >/dev/null 2>&1 || { echo "actionlint not found."; exit 1; }
actionlint -config-file .gitea/actionlint.yaml .gitea/workflows/*.yml
workflow-dryrun:
@command -v act_runner >/dev/null 2>&1 || { echo "act_runner not found."; exit 1; }
@echo "Dry-running all workflows..."
act_runner exec --dryrun -W .gitea/workflows/ 2>&1 | grep -E 'DRYRUN|ERROR|FAIL|Job'
workflow-check: workflow-lint workflow-dryrun
@echo "Workflow checks passed."
test-unit:
$(BIN)/pytest tests/unit/ -v --no-cov
pytest-cov:
$(BIN)/pytest tests/ -v --cov=src/devx --cov-report=term-missing --cov-fail-under=100
pytest-cov: devx-pytest-cov
test: pytest-cov
clean:
find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
find . -type f -name "*.pyc" -delete 2>/dev/null || true
rm -rf .coverage htmlcov/ dist/ build/ *.egg-info/
pre-push: lint-all pytest-cov
@echo "[pre-push] All checks passed. Proceeding with push."
clean: devx-clean
@echo "[clean] Done."
# ── Docker image management ──────────────────────────────────────────────────
build-images: devx-build-images
@echo "[build-images] Done."
push-images: devx-push-images
@echo "[push-images] Done."
build-images-dry-run: devx-build-images-dry-run
@echo "[build-images-dry-run] Done."
clean-images: devx-clean-images
@echo "[clean-images] Done."
+374 -29
View File
@@ -1,46 +1,141 @@
# devx — Reusable Development & CI/CD Tools
A Python package providing reusable development and CI/CD automation tools for oblachno-oss projects. devx consolidates release management, PR automation, wiki sync, badge generation, translation checks, and more into a single installable package.
A Python package providing reusable development and CI/CD automation tools for
oblachno-oss projects. devx consolidates release management, PR automation,
wiki sync, badge generation, translation checks, documentation coverage,
parallel test distribution, and more into a single installable package.
It was extracted from the [GRM](https://git.oblachno.oblachno.fyi/oblachno-oss/grm)
project to be reusable across all oblachno-oss repositories. Any project hosted
on a Gitea instance with Gitea Actions can install devx and inherit a complete,
opinionated CI/CD pipeline: conventional commits, automated versioning via
git-cliff, squash-merge automation, Vikunja task tracking, wiki sync, and
quality badges.
> An open-source project from **Oblachno** (облачно means *cloudy* in Bulgarian).
[![CI](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions/workflows/ci.yml/badge.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![License: GPL-3.0](https://img.shields.io/badge/license-GPL--3.0-blue)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/LICENSE)
[![Coverage](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/badges/coverage.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Tests](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/badges/tests.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Code Quality](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/badges/quality.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Version](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/badges/version.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
[![Python](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/badges/python.svg)](https://www.python.org/downloads/)
[![Coverage](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/coverage.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Tests](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/tests.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Docs](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/docs.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
[![Code Quality](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/quality.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Version](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/version.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
[![Python](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/python.svg)](https://www.python.org/downloads/)
## Why devx?
Every oblachno-oss project shares the same CI/CD needs: automated releases,
PR review, task tracking, documentation sync, and quality badges. Without a
shared package, each repository duplicates this logic in shell scripts and
workflow YAML, leading to drift, bugs, and maintenance burden.
devx solves this by providing a single, tested Python package that any
oblachno-oss project can install. The project declares its configuration via
environment variables and `pyproject.toml`, and devx handles the rest. Updates
to the CI/CD pipeline ship as new devx releases — consumer projects pick them
up by bumping their devx dependency.
### Key features
- **Automated releases** — git-cliff-driven semver versioning, changelog
generation, tagging, and publishing to a Gitea PyPI registry.
- **PR automation** — squash-merge with task ID validation, automated PR
review with inline comments, and conventional commit enforcement.
- **Smart change classification** — user-facing vs workflow-only change
detection so infrastructure-only changes skip releases.
- **Documentation sync** — push `docs/` markdown to the Gitea wiki with
integrity verification.
- **Quality badges** — generate self-contained SVG badges for coverage,
tests, docs, quality, version, and Python version.
- **Translation checks** — validate i18n keys against source code, detect
dead keys and missing languages.
- **Parallel test distribution** — split test files or molecule scenarios
across CI runners with cross-runner fail-fast.
- **Developer tools** — environment setup, CI tool installation, test speed
enforcement, repository configuration.
- **i18n** — built-in translations for English, Bulgarian, German, Russian,
Chinese, and Polish; projects can extend with their own keys.
## Installation
Install from the Gitea PyPI registry:
devx is published to the Gitea PyPI registry at
`https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple`.
The registry is publicly readable — no authentication required to install.
### Quick install (one-off)
```bash
pip install devx --index-url https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple
```
Or add the registry to your `pip.conf` / `pyproject.toml` and install normally:
### Persistent configuration (recommended)
```bash
pip install devx
Add the registry to `~/.pip/pip.conf` so `pip install devx` works without
specifying `--index-url` every time:
```ini
[global]
extra-index-url = https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple
```
## Usage
### As a dependency in another project
### CI/CD Automation
To use devx as a dependency in your `pyproject.toml`, add the registry as an
extra index and list devx in your dependencies:
devx provides CI/CD modules invoked via `python -m devx.ci.*`:
```toml
[project]
dependencies = [
"devx>=0.11.1",
]
[tool.pip]
extra-index-url = "https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple"
```
Then install normally:
```bash
pip install -e .
```
> **Note:** If your project requires a specific devx version, pin it in
> `dependencies` (e.g., `"devx==0.11.1"`) or use a version constraint
> (e.g., `"devx>=0.11.1,<0.12"`).
### Optional extras
devx ships optional dependency groups for different use cases:
```bash
pip install "devx[ci,lint]" # CI runners and linting (pytest, ruff, pyright, bandit, build, twine)
pip install "devx[molecule]" # Molecule testing for Ansible projects
pip install "devx[dev]" # Full local development (ci + lint + build + twine)
```
## Quick start
After installing devx, set the required environment variables (see
[Configuration](#configuration)) and invoke modules via `python -m devx.*` or
the `devx` CLI.
### CI/CD automation
CI/CD modules are invoked via `python -m devx.ci.*`. Each module is also
available as a `devx ci <command>` subcommand.
```bash
# Release automation (versioning, changelog, tagging)
python -m devx.ci.release
python -m devx.ci.release --dry-run
python -m devx.ci.release --dry-run # preview without changes
python -m devx.ci.release --verify # check tag/version/changelog alignment
# Publish a release to the Gitea PyPI registry
python -m devx.ci.publish v1.0.0 oblachno-oss/devx
python -m devx.ci.publish v1.0.0 oblachno-oss/devx --skip-build # Gitea release only
# Automated PR review
# Automated PR review (posts inline comments and structured review)
python -m devx.ci.pr_review 42 oblachno-oss/devx
# Auto-merge a PR (validates title, squash-merges)
@@ -54,66 +149,240 @@ python -m devx.ci.sync_wiki --repo oblachno-oss/devx --strict
# Generate and push quality badges
python -m devx.ci.push_badges
python -m devx.ci.push_badges --retries 3 # retry on git push failures
# Check translation completeness
python -m devx.ci.check_translations
python -m devx.ci.check_translations --translations path/to/translations.json
# Documentation coverage check
python -m devx.ci.doc_coverage --fail-on-missing
# Documentation lint (structure, links, headings, TODOs)
python -m devx.ci.lint_docs --root .
# Validate a commit message
python -m devx.ci.validate_commit_msg commit-msg.txt --branch master
# Detect whether the latest commit is a release commit
python -m devx.ci.detect_release_commit
# Notify on CI failure (creates a Gitea issue)
python -m devx.ci.notify_failure --repo oblachno-oss/devx --run-id 123 --workflow ci --commit abc123
python -m devx.ci.notify_failure --repo oblachno-oss/devx --run-id 123 \
--workflow ci --commit abc123 --auto-login
# Discover available Gitea Actions runners
python -m devx.ci.discover_runners --owner oblachno-oss --repo devx --indices
# Distribute files across parallel runners (round-robin)
python -m devx.ci.distribute_files --pattern "tests/integration/test_*.py" \
--runner-index 1 --max-runners 3 --github-env
# Run pytest with cross-runner fail-fast
python -m devx.ci.integration_guard -- test_a.py test_b.py
```
### Developer Tools
### Developer tools
devx provides developer tooling invoked via `python -m devx.tools.*`:
Developer tooling modules are invoked via `python -m devx.tools.*` or the
`devx tools <command>` subcommand.
```bash
# Set up a development environment (venv, deps, hooks)
# Set up a development environment (venv, deps, hooks, tea login)
python -m devx.tools.setup --bin .venv/bin
python -m devx.tools.setup --bin .venv/bin --extras "ci,lint" --no-pre-commit
# Install CI tools (actionlint, git-cliff, act_runner, tea)
python -m devx.tools.install_tools
python -m devx.tools.install_tools --tool git-cliff --tool tea
python -m devx.tools.install_tools --list
# Install checkmake (Makefile linter)
python -m devx.tools.install_checkmake
# Check unit test speed
python -m devx.tools.check_test_speed --max-seconds 10
python -m devx.tools.check_test_speed --max-seconds 4 --max-single-seconds 0.5
# Configure repository (branch protection, labels)
python -m devx.tools.configure_repo
python -m devx.tools.configure_repo --repo devx --owner oblachno-oss
# Generate badge SVG files locally
python -m devx.tools.generate_badges --output-dir .badges/
# Generate a cliff.toml with the correct task ID prefix
python -m devx.tools.generate_cliff_config --prefix GRM
python -m devx.tools.generate_cliff_config --prefix GRM --force # overwrite existing
```
### CLI
### Molecule testing (optional)
devx also provides a `devx` CLI command:
For projects with Ansible roles, devx provides molecule testing helpers via
`python -m devx.molecule.*` or `devx molecule <command>`.
```bash
# Distribute molecule scenarios across parallel runners
python -m devx.molecule.distribute_molecule --runner-index 1 --max-runners 3
python -m devx.molecule.distribute_molecule --list # list all scenarios
python -m devx.molecule.distribute_molecule --list-platforms # list platforms
# Run molecule tests with cross-runner fail-fast
python -m devx.molecule.molecule_ci_guard pair1 pair2
python -m devx.molecule.molecule_ci_guard --roles-root ansible/roles pair1 pair2
# Run all molecule scenarios locally (sequential)
python -m devx.molecule.molecule_all
python -m devx.molecule.molecule_all --bin .venv/bin
# Discover available Gitea Actions runners for molecule tests
python -m devx.molecule.discover_runners --indices
# Ensure Docker is available for molecule tests in CI
python -m devx.molecule.start_docker
```
### OpenTofu helpers
devx provides reusable functions for extracting values from `tofu output`:
```python
from devx.opentofu import get_tofu_output, get_tofu_vm_ip, get_tofu_vm_field
vms = get_tofu_output("customer_vms", cwd="tofu/environments/staging",
env={"HCLOUD_TOKEN": token})
ip = get_tofu_vm_ip("customer_vms", "oblachno", cwd="tofu/environments/staging",
env={"HCLOUD_TOKEN": token})
```
## CLI commands overview
devx provides a `devx` CLI command with three command groups:
```bash
devx --help
devx --version
```
### Configuration
### `devx ci` — CI/CD automation
devx reads configuration from environment variables with `.env` file fallback:
| Command | Description |
|---------|-------------|
| `devx ci auto-merge` | Squash-merge a PR with task ID validation |
| `devx ci check-translations` | Check translation files for gaps and dead keys |
| `devx ci classify-changes` | Classify git changes as user-facing or workflow-only |
| `devx ci detect-release-commit` | Detect whether the latest commit is a release commit |
| `devx ci discover-runners` | Discover available Gitea Actions runners |
| `devx ci distribute-files` | Distribute files across parallel runners (round-robin) |
| `devx ci doc-coverage` | Check documentation coverage for CLI commands and modules |
| `devx ci integration-guard` | Run pytest with cross-runner fail-fast |
| `devx ci notify-failure` | Create a Gitea issue when a CI workflow fails |
| `devx ci post-merge` | Update Vikunja task after a merge to master |
| `devx ci pr-review` | Run automated PR review |
| `devx ci publish` | Build package, publish to registry, create Gitea release |
| `devx ci push-badges` | Generate badge SVG files and push to the badges branch |
| `devx ci release` | Automated release: version, changelog, tag, push |
| `devx ci sync-wiki` | Sync documentation from docs/ to the Gitea wiki |
| `devx ci validate-commit-msg` | Validate commit messages for conventional format |
### `devx tools` — Developer tools
| Command | Description |
|---------|-------------|
| `devx tools check-test-speed` | Run unit tests and enforce execution-time budgets |
| `devx tools configure-repo` | Configure branch protection and labels via Gitea API |
| `devx tools generate-badges` | Generate self-contained SVG badge files |
| `devx tools generate-cliff-config` | Generate a cliff.toml with the correct task ID prefix |
| `devx tools install-checkmake` | Install checkmake (Makefile linter) |
| `devx tools install-tools` | Install actionlint, git-cliff, act_runner, tea |
| `devx tools setup` | Project setup: install deps, hooks, tea login |
### `devx molecule` — Molecule testing (optional)
| Command | Description |
|---------|-------------|
| `devx molecule all` | Run all molecule scenarios on all supported platforms |
| `devx molecule discover-runners` | Discover available Gitea Actions runners |
| `devx molecule distribute` | Distribute molecule test pairs across parallel runners |
| `devx molecule guard` | Run molecule tests with CI failure polling |
See [CLI Commands](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki/CLI-Commands)
in the wiki for full command documentation with examples.
## Configuration
devx reads configuration from environment variables with `.env` file fallback.
The config system loads `.env` automatically via `python-dotenv`.
### DEVX_ environment variables
| Variable | Default | Description |
|----------|---------|-------------|
| `DEVX_GITEA_API_URL` | `https://git.oblachno.oblachno.fyi/api/v1` | Gitea API base URL |
| `DEVX_VIKUNJA_API_URL` | `https://work.oblachno.oblachno.fyi/api/v1` | Vikunja API base URL |
| `DEVX_LANG` | `en` | Language (en, bg) |
| `REPO_TOKEN` | — | Gitea API token |
| `DEVX_REPO_OWNER` | **(none — must be set)** | Repository owner for API calls |
| `DEVX_REPO_NAME` | **(none — must be set)** | Repository name (or `owner/repo`) |
| `DEVX_TASK_PREFIX` | `DEVX` | Task ID prefix (GRM, OBL-INFRA, etc.) |
| `DEVX_VIKUNJA_PROJECT_ID` | `6` | Vikunja project ID |
| `DEVX_LANG` | `en` | Language for i18n (en, bg, de, ru, zh, pl) |
| `DEVX_TRANSLATIONS_PATH` | — | Path to a custom JSON translations file |
| `DEVX_VERSION_FILE` | `src/devx/__init__.py` | Version source file (used by release) |
| `DEVX_DOCS_DIR` | `docs` | Documentation directory (used by sync_wiki) |
| `DEVX_STATUS_CHECKS` | `CI / quality (pull_request)` | Comma-separated status check contexts |
| `DEVX_PYPI_REGISTRY_URL` | — | Gitea PyPI registry URL (used by publish) |
| `CI_GITEA_TOKEN` | — | Gitea API token (see scopes below) |
| `CI_GITEA_USERNAME` | — | Gitea username for registry authentication |
| `VIKUNJA_TOKEN` | — | Vikunja API token |
| `PYPI_TOKEN` | — | Standard PyPI token (takes precedence over Gitea registry) |
Copy `.env.example` to `.env` and fill in your tokens:
#### CI_GITEA_TOKEN scopes
The `CI_GITEA_TOKEN` is a single Gitea Personal Access Token used across all
workflows. It requires these scopes:
| Scope | Purpose |
|-------|---------|
| `read:repository` | Read repos, PRs, issues, branches |
| `write:repository` | Push commits, merge PRs, create tags/releases, create issues, set branch protection, push wiki |
| `read:package` | Pull packages from Gitea PyPI registry, pull Docker images |
| `write:package` | Publish packages to Gitea PyPI registry, push Docker images |
| `read:organization` | Query org-level runners for molecule test distribution |
### Per-project overrides
Projects using devx can override the default API URLs and language by setting
`DEVX_*` environment variables or entries in their `.env` file. Copy
`.env.example` to `.env` and fill in your tokens:
```bash
cp .env.example .env
```
### Change classification
Projects configure which file paths are infrastructure (no release needed) vs
user-facing (release needed) in `pyproject.toml`:
```toml
[tool.devx.classify]
# Merge with DEFAULT_INFRASTRUCTURE (CI workflows, tests, docs, config)
# use_defaults = true # (default)
# Project-specific infrastructure paths (merged with defaults)
infrastructure = []
# Files that would default to user-facing but are actually infrastructure
infrastructure_overrides = [
"src/myproject/__init__.py", # only contains __version__
]
# Safety override for broad infrastructure patterns
user_facing_overrides = []
# Tag patterns for CI conditional execution (orthogonal to release impact)
[tool.devx.classify.tags]
# ansible = ["ansible/**"]
```
## Development
```bash
@@ -122,10 +391,86 @@ cd devx
make setup # Create venv, install deps, hooks, CI tools
make lint-all # ruff + pyright + bandit + actionlint
make pytest-cov # Unit tests with 100% coverage
make test-unit # Unit tests without coverage
make workflow-check # Static + dry-run validation of workflow YAML
make clean # Remove caches, build artifacts, coverage data
```
See [AGENTS.md](AGENTS.md) for full project conventions, PR workflow, and architecture details.
`make setup` automatically installs all development tools:
- **Python deps** via `python -m devx.tools.setup` (pip install -e .[dev], pre-commit hooks)
- **actionlint, git-cliff, act_runner, tea** via `python -m devx.tools.install_tools`
- **tea CLI login** via `python -m devx.tools.setup` (configures `tea login` from `.env`)
### Make targets
| Target | Description |
|--------|-------------|
| `make setup` | Full local development setup (venv, deps, hooks, CI tools) |
| `make setup-ci` | Lean setup for CI jobs (pytest + lint + runtime deps) |
| `make setup-quality` | Setup for quality job (lint + test deps, actionlint) |
| `make setup-release` | Setup for release jobs (git-cliff, tea, lint tools) |
| `make install-tools` | Install actionlint, git-cliff, act_runner, tea |
| `make install-hooks` | Install git hooks (pre-commit, pre-push) |
| `make lint` | ruff check + ruff format check + pyright + bandit |
| `make lint-ruff` | ruff check only |
| `make lint-format` | ruff format check only |
| `make typecheck` | pyright only |
| `make lint-bandit` | bandit security scan only |
| `make lint-all` | lint + workflow-lint (actionlint) |
| `make lint-deps` | pip-audit dependency vulnerability scan |
| `make test-unit` | Unit tests without coverage |
| `make pytest-cov` | Unit tests with 100% coverage enforcement |
| `make workflow-lint` | actionlint on .gitea/workflows/*.yml |
| `make workflow-dryrun` | act_runner exec --dryrun on all workflows |
| `make workflow-check` | workflow-lint + workflow-dryrun |
| `make clean` | Remove caches, build artifacts, coverage data |
See [AGENTS.md](AGENTS.md) for full project conventions, PR workflow, and
architecture details.
## Architecture overview
devx is a self-contained Python package under `src/devx/`. It never imports
from scripts outside the package. All tools are invoked via
`python -m devx.ci.*`, `python -m devx.tools.*`, or `python -m devx.molecule.*`.
```
src/devx/
├── __init__.py # Version (single source of truth, read by setuptools)
├── cli.py # Click-based CLI entry point (devx command)
├── config.py # Configuration system (DEVX_ env vars, .env loading)
├── api_clients.py # GiteaClient, VikunjaClient — HTTP API wrappers
├── gitea_cli.py # TeaCLI — wrapper around tea CLI with JSON parsing
├── i18n.py # Translation system (gettext-based, translations.json)
├── exceptions.py # Custom exception types (DevxError, APIError)
├── opentofu.py # OpenTofu output helpers
├── translations.json # Translation strings (en, bg, de, ru, zh, pl)
├── ci/ # CI/CD automation modules (run by workflows)
├── tools/ # Developer tooling modules (run locally or by CI)
└── molecule/ # Optional molecule testing helpers (for Ansible projects)
```
### Design principles
- **Self-contained package** — `src/devx/` never imports from scripts outside the package
- **Module-based invocation** — All tools invoked via `python -m devx.ci.*` or `python -m devx.tools.*`
- **PYTHONPATH: src** — Workflows set `PYTHONPATH: src` (not `.:src` since there are no scripts at repo root)
- **Config via env vars** — `DEVX_*` environment variables with `.env` file fallback
- **100% test coverage** — enforced by `--cov-fail-under=100`
- **i18n by default** — all user-facing strings wrapped in `_()` for translation
See [Architecture](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki/Architecture)
and [CI/CD Workflow](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki/CI-CD-Workflow)
in the wiki for detailed documentation.
## Links
- **Wiki**: [https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
- **Releases**: [https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
- **Actions**: [https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
- **Source**: [https://git.oblachno.oblachno.fyi/oblachno-oss/devx](https://git.oblachno.oblachno.fyi/oblachno-oss/devx)
- **GRM (origin project)**: [https://git.oblachno.oblachno.fyi/oblachno-oss/grm](https://git.oblachno.oblachno.fyi/oblachno-oss/grm)
## License
GPL-3.0
GPL-3.0 — see [LICENSE](LICENSE).
+5 -2
View File
@@ -39,8 +39,8 @@ sort_commits = "oldest"
recurse_submodules = false
commit_preprocessors = [
# Strip DEVX-N task ID prefix from merge commits so git-cliff sees conventional commits
{ pattern = "^DEVX-\\d+\\s+", replace = "" },
# Strip DEVX-N: task ID prefix from squash-merge commits so git-cliff sees conventional commits
{ pattern = "^DEVX-\\d+:\\s+", replace = "" },
]
commit_parsers = [
@@ -66,3 +66,6 @@ commit_parsers = [
features_always_bump_minor = true
breaking_always_bump_major = false
initial_tag = "0.1.0"
# Refactor commits bump patch — structural changes to src/ or pyproject.toml
# affect users even though no new feature was added.
refactor_always_bump_patch = true
+26
View File
@@ -0,0 +1,26 @@
# ci-base — lightweight image for CI jobs that only need devx core + tea.
#
# Used by: detect-type, detect-changes, validate-commit-msg, pr-review,
# auto-merge, sync-wiki, vikunja, configure-repo, discover-runners,
# molecule-report, discover-integration-runners
#
# Jobs using this image: setup is instant (ln -s /opt/venv .venv)
# No pip install needed — devx and all deps are pre-installed.
FROM gitea/runner-images:ubuntu-latest
# Create a virtual environment with all deps pre-installed
RUN python3 -m venv /opt/venv
ENV PATH="/opt/venv/bin:/root/.local/bin:$PATH"
# Install devx from local source (build context = devx repo root)
COPY . /tmp/devx
RUN pip install --no-cache-dir --upgrade pip setuptools wheel \
&& pip install --no-cache-dir /tmp/devx[ci] \
&& rm -rf /tmp/devx
# Install tea CLI (for Gitea API operations in CI)
RUN python3 -m devx.tools.install_tools --tool tea
# Workspace directory (actions/checkout mounts repo here)
WORKDIR /workspace
+30
View File
@@ -0,0 +1,30 @@
# ci-full — heaviest image, includes everything for release, molecule, deploy.
#
# Used by: release, publish, release-dry-run, molecule-tests,
# provision-infra, deploy-observability, provision-zitadel,
# deploy-customer, integration-tests
#
# Layers on top of ci-quality: adds release tools, molecule, deploy deps,
# git-cliff, and OpenTofu.
FROM git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-quality:latest
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
# Install rsync (required by molecule_docker for file sync between host and test containers)
RUN apt-get update && apt-get install -y --no-install-recommends rsync \
&& rm -rf /var/lib/apt/lists/*
# Install devx[release,molecule,deploy] from local source
COPY . /tmp/devx
RUN pip install --no-cache-dir /tmp/devx[release,molecule,deploy] \
&& rm -rf /tmp/devx
# Install git-cliff (changelog generator for release job)
RUN python3 -m devx.tools.install_tools --tool git-cliff
# Install OpenTofu (for infra deploy jobs)
RUN ARCH=$(uname -m | sed 's/x86_64/amd64/') \
&& VERSION=1.12.3 \
&& curl -fsSL "https://github.com/opentofu/opentofu/releases/download/v${VERSION}/tofu_${VERSION}_$(uname -s | tr '[:upper:]' '[:lower:]')_${ARCH}.tar.gz" \
| tar -xz -C /usr/local/bin tofu
+22
View File
@@ -0,0 +1,22 @@
# ci-quality — image for lint, type-checking, badge generation.
#
# Used by: quality (lint-all + pytest-cov + checks), badges (generate_badges
# runs ruff/pyright/bandit to produce quality badge)
#
# Layers on top of ci-base: adds lint tools + actionlint + checkmake.
FROM git.oblachno.oblachno.fyi/oblachno-oss/runner-images/ci-base:latest
# Install devx[lint] from local source (adds ruff, pyright, bandit, etc.)
COPY . /tmp/devx
RUN pip install --no-cache-dir /tmp/devx[lint] \
&& rm -rf /tmp/devx
# Install CI/CD binary tools
RUN python3 -m devx.tools.install_tools --tool actionlint \
&& python3 -m devx.tools.install_checkmake
# Install hadolint (Dockerfile linter)
RUN curl -fsSL "https://github.com/hadolint/hadolint/releases/download/v2.12.0/hadolint-Linux-x86_64" \
-o /usr/local/bin/hadolint \
&& chmod +x /usr/local/bin/hadolint
+20
View File
@@ -0,0 +1,20 @@
[
{
"name": "oblachno-oss/runner-images/ci-base",
"dockerfile": "docker/ci-base/Dockerfile",
"context": ".",
"tags": ["latest"]
},
{
"name": "oblachno-oss/runner-images/ci-quality",
"dockerfile": "docker/ci-quality/Dockerfile",
"context": ".",
"tags": ["latest"]
},
{
"name": "oblachno-oss/runner-images/ci-full",
"dockerfile": "docker/ci-full/Dockerfile",
"context": ".",
"tags": ["latest"]
}
]
+159 -8
View File
@@ -1,24 +1,175 @@
# devx — Reusable Development & CI/CD Tools
A Python package providing reusable development and CI/CD automation tools for oblachno-oss projects.
A Python package providing reusable development and CI/CD automation tools for
oblachno-oss projects. devx consolidates release management, PR automation,
wiki sync, badge generation, translation checks, documentation coverage,
parallel test distribution, and more into a single installable package.
It was extracted from the [GRM](https://git.oblachno.oblachno.fyi/oblachno-oss/grm)
project to be reusable across all oblachno-oss repositories.
> An open-source project from **Oblachno** (облачно means *cloudy* in Bulgarian).
[![CI](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions/workflows/ci.yml/badge.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![License: GPL-3.0](https://img.shields.io/badge/license-GPL--3.0-blue)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/LICENSE)
[![Coverage](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/coverage.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Tests](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/tests.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Docs](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/docs.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/wiki)
[![Code Quality](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/quality.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
[![Version](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/version.svg)](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
[![Python](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/raw/commit/53e15be6c2ae9ce09f4bfbff878a7b8def021f82/python.svg)](https://www.python.org/downloads/)
## Overview
devx consolidates release management, PR automation, wiki sync, badge generation, translation checks, and more into a single installable package. It was extracted from the [GRM](https://git.oblachno.oblachno.fyi/oblachno-oss/grm) project to be reusable across all oblachno-oss projects.
devx provides a complete, opinionated CI/CD pipeline for any project hosted on
a Gitea instance with Gitea Actions. Install the package, declare configuration
via environment variables and `pyproject.toml`, and inherit:
- **Automated releases** — git-cliff-driven semver versioning, changelog
generation, tagging, and publishing to a Gitea PyPI registry.
- **PR automation** — squash-merge with task ID validation, automated PR
review with inline comments, and conventional commit enforcement.
- **Smart change classification** — user-facing vs workflow-only change
detection so infrastructure-only changes skip releases.
- **Documentation sync** — push `docs/` markdown to the Gitea wiki with
integrity verification.
- **Quality badges** — self-contained SVG badges for coverage, tests, docs,
quality, version, and Python version.
- **Translation checks** — validate i18n keys against source code, detect
dead keys and missing languages.
- **Parallel test distribution** — split test files or molecule scenarios
across CI runners with cross-runner fail-fast.
- **Developer tools** — environment setup, CI tool installation, test speed
enforcement, repository configuration.
- **i18n** — built-in translations for English, Bulgarian, German, Russian,
Chinese, and Polish; projects can extend with their own keys.
## Installation
Install from the Gitea PyPI registry:
devx is published to the Gitea PyPI registry at
`https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple`.
The registry is publicly readable — no authentication required to install.
### Quick install (one-off)
```bash
pip install devx --index-url https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple
```
### Persistent configuration (recommended)
Add the registry to `~/.pip/pip.conf`:
```ini
[global]
extra-index-url = https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple
```
Then `pip install devx` works without specifying `--index-url`.
### As a dependency in another project
Add devx to your `pyproject.toml` dependencies and configure the registry:
```toml
[project]
dependencies = [
"devx>=0.11.1",
]
[tool.pip]
extra-index-url = "https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple"
```
Pin a specific version if needed: `"devx==0.11.1"` or `"devx>=0.11.1,<0.12"`.
### Optional extras
```bash
pip install "devx[ci,lint]" # CI runners and linting (pytest, ruff, pyright, bandit, build, twine)
pip install "devx[molecule]" # Molecule testing for Ansible projects
pip install "devx[dev]" # Full local development (ci + lint + build + twine)
```
## Architecture
- **Core modules** — config, exceptions, i18n, api_clients, gitea_cli
- **CI automation** (`devx.ci`) — release, publish, auto_merge, pr_review, classify_changes, etc.
- **Dev tools** (`devx.tools`) — setup, install_tools, check_test_speed, configure_repo, generate_badges
- **Molecule tools** (`devx.molecule`) — Optional, for projects with Ansible roles
devx is a self-contained Python package under `src/devx/`:
See [AGENTS.md](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/AGENTS.md) for full project conventions.
- **Core modules** — `config.py`, `exceptions.py`, `i18n.py`, `api_clients.py`,
`gitea_cli.py`, `cli.py`, `opentofu.py`
- **CI automation** (`devx.ci`) — release, publish, auto_merge, pr_review,
classify_changes, sync_wiki, push_badges, check_translations, doc_coverage,
validate_commit_msg, detect_release_commit, notify_failure, post_merge,
discover_runners, distribute_files, integration_guard
- **Dev tools** (`devx.tools`) — setup, install_tools, check_test_speed,
configure_repo, generate_badges, generate_cliff_config, install_checkmake
- **Molecule tools** (`devx.molecule`) — Optional, for projects with Ansible
roles: distribute_molecule, molecule_ci_guard, molecule_all, discover_runners,
start_docker, platforms
See [Architecture](Architecture) for the full package structure, module
descriptions, design principles, and data flow diagrams.
## CI/CD pipeline
devx uses Gitea Actions with three workflows:
- **CI** (`ci.yml`) — runs on pull requests: quality checks, change detection,
release dry-run, automated PR review, and auto-merge.
- **Post-merge** (`post-merge.yml`) — runs on every push to master: release
versioning, wiki sync, badge generation, Vikunja task updates, and repo
configuration.
- **Publish** (`publish.yml`) — runs on tag pushes: builds the package,
publishes to the Gitea PyPI registry, and creates a Gitea release.
See [CI/CD Workflow](CI-CD-Workflow) for the full pipeline documentation,
including the post-merge job graph, release process, badge generation, and
wiki sync details.
## CLI commands
devx provides a `devx` CLI with three command groups:
- `devx ci <command>` — CI/CD automation (17 commands)
- `devx tools <command>` — Developer tools (7 commands)
- `devx molecule <command>` — Molecule testing (4 commands, optional)
See [CLI Commands](CLI-Commands) for full command documentation with examples.
## Configuration
devx reads configuration from `DEVX_*` environment variables with `.env` file
fallback. Key variables:
| Variable | Default | Description |
|----------|---------|-------------|
| `DEVX_GITEA_API_URL` | `https://git.oblachno.oblachno.fyi/api/v1` | Gitea API base URL |
| `DEVX_VIKUNJA_API_URL` | `https://work.oblachno.oblachno.fyi/api/v1` | Vikunja API base URL |
| `DEVX_REPO_OWNER` | **(must be set)** | Repository owner |
| `DEVX_REPO_NAME` | **(must be set)** | Repository name |
| `DEVX_TASK_PREFIX` | `DEVX` | Task ID prefix (GRM, OBL-INFRA, etc.) |
| `DEVX_LANG` | `en` | Language for i18n (en, bg, de, ru, zh, pl) |
| `CI_GITEA_TOKEN` | — | Gitea API token |
| `VIKUNJA_TOKEN` | — | Vikunja API token |
See [AGENTS.md](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/src/branch/master/AGENTS.md)
for the full configuration reference, PR workflow, and project conventions.
## Wiki pages
- [Home](Home) — This page
- [Getting Started](Getting-Started) — Installation, configuration, and quick start guide
- [CLI Commands](CLI-Commands) — Full CLI command documentation with examples
- [Architecture](Architecture) — Package structure, module descriptions, design principles
- [CI/CD Workflow](CI-CD-Workflow) — Pipeline documentation, workflows, and CI scripts
## Links
- **Source**: [https://git.oblachno.oblachno.fyi/oblachno-oss/devx](https://git.oblachno.oblachno.fyi/oblachno-oss/devx)
- **Releases**: [https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/releases)
- **Actions**: [https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions](https://git.oblachno.oblachno.fyi/oblachno-oss/devx/actions)
- **GRM (origin project)**: [https://git.oblachno.oblachno.fyi/oblachno-oss/grm](https://git.oblachno.oblachno.fyi/oblachno-oss/grm)
## License
GPL-3.0
+1
View File
@@ -1,5 +1,6 @@
{
"index.md": "Home",
"user/getting-started.md": "Getting-Started",
"user/cli-commands.md": "CLI-Commands",
"tech/architecture.md": "Architecture",
"tech/ci-cd-workflow.md": "CI-CD-Workflow"
+562 -29
View File
@@ -1,50 +1,583 @@
# Architecture
devx is a reusable Python package providing development and CI/CD tools for oblachno-oss projects.
devx is a reusable Python package providing development and CI/CD tools for
oblachno-oss projects. It is self-contained under `src/devx/` and never imports
from scripts outside the package.
## Package Structure
## Package structure
```
src/devx/
├── __init__.py # Version (single source of truth)
├── cli.py # Click-based CLI entry point (devx command)
├── config.py # Configuration system (DEVX_ env vars)
├── api_clients.py # GiteaClient, VikunjaClient — HTTP API wrappers
├── gitea_cli.py # TeaCLI — wrapper around tea CLI with JSON parsing
├── i18n.py # Translation system (gettext-based, translations.json)
├── exceptions.py # Custom exception types (DevxError, APIError)
├── translations.json # Translation strings (en, bg, de, ru, zh)
├── ci/ # CI/CD automation modules
├── tools/ # Developer tooling modules
└── molecule/ # Optional molecule testing helpers
├── __init__.py # Version (single source of truth, read by setuptools)
├── cli.py # Click-based CLI entry point (devx command)
├── config.py # Configuration system (DEVX_ env vars, .env loading)
├── api_clients.py # GiteaClient, VikunjaClient — HTTP API wrappers
├── gitea_cli.py # TeaCLI — wrapper around tea CLI with JSON parsing
├── i18n.py # Translation system (JSON-based, translations.json)
├── exceptions.py # Custom exception types (DevxError, APIError)
├── opentofu.py # OpenTofu output helpers
├── translations.json # Translation strings (en, bg, de, ru, zh, pl)
├── ci/ # CI/CD automation modules (run by workflows)
│ ├── __init__.py
│ ├── _shared.py # Shared utilities (get_latest_tag)
│ ├── release.py # Automated versioning, tagging, changelog
│ ├── publish.py # Build and publish to Gitea PyPI registry
│ ├── auto_merge.py # Squash-merge PRs with task ID validation
│ ├── classify_changes.py # User-facing vs workflow-only change detection
│ ├── detect_release_commit.py # Detect release commits on master
│ ├── validate_commit_msg.py # Conventional commit validation
│ ├── pr_review.py # Automated PR review
│ ├── post_merge.py # Vikunja task updates after merge
│ ├── sync_wiki.py # Sync documentation to Gitea wiki
│ ├── push_badges.py # Generate and push quality badges
│ ├── notify_failure.py # Create Gitea issues on CI failures
│ ├── distribute_files.py # Distribute files across parallel runners
│ ├── integration_guard.py # Run pytest with cross-runner fail-fast
│ ├── discover_runners.py # Dynamic Gitea runner discovery
│ ├── check_translations.py # Translation completeness check
│ └── doc_coverage.py # Documentation coverage check
├── tools/ # Developer tooling modules (run locally or by CI)
│ ├── __init__.py
│ ├── setup.py # Environment setup (venv, deps, hooks, tea login)
│ ├── install_tools.py # Install actionlint, git-cliff, act_runner, tea
│ ├── check_test_speed.py # Measure unit test execution time
│ ├── configure_repo.py # Branch protection and label setup
│ ├── generate_badges.py # Badge SVG generation
│ ├── generate_cliff_config.py # Generate cliff.toml with correct prefix
│ └── install_checkmake.py # Install checkmake (Makefile linter)
└── molecule/ # Optional molecule testing helpers (Ansible projects)
├── __init__.py
├── discover_runners.py # Dynamic Gitea runner discovery
├── distribute_molecule.py # Distribute scenarios across runners
├── molecule_ci_guard.py # Run molecule with cross-runner fail-fast
├── molecule_all.py # Run all molecule scenarios locally
├── start_docker.py # Ensure Docker is available for molecule
└── platforms.py # Supported molecule platforms
```
## Core Modules
## Core modules
### cli.py
### `__init__.py`
Click-based CLI entry point. Provides three command groups: `devx ci`, `devx tools`, `devx molecule`. Each subcommand delegates to the corresponding module via `_run_module()`.
Contains only `__version__`, the single source of truth for the package
version. Read by setuptools via `dynamic = ["version"]` in `pyproject.toml`.
Updated automatically by `devx.ci.release` during the release process. Treated
as infrastructure (not user-facing) by the change classifier since it is a
release artifact, not user code.
### i18n.py
### `cli.py`
Simple i18n system using a JSON translations file. Supports en, bg, de, ru, zh. Projects can extend translations by setting `DEVX_TRANSLATIONS_PATH` to a custom JSON file.
Click-based CLI entry point. Provides three command groups: `devx ci`,
`devx tools`, and `devx molecule`. Each subcommand delegates to the
corresponding module via `_run_module()`, which imports the module, sets
`sys.argv`, and calls its `main()` function. This design keeps all logic in
the modules themselves — `cli.py` is purely a router.
### exceptions.py
The CLI is registered as a console script via `pyproject.toml`:
```toml
[project.scripts]
devx = "devx.cli:cli"
```
Custom exception hierarchy: `DevxError` (base), `APIError` (HTTP errors with status code and message).
### `config.py`
### api_clients.py
Shared configuration constants for all devx modules. All defaults can be
overridden via environment variables with the `DEVX_` prefix. Provides:
HTTP API clients with connection pooling and retry logic:
- `GiteaClient` — Gitea REST API (branch protection, labels, issues, PRs, releases, reviews)
- `VikunjaClient` — Vikunja REST API (tasks, projects, comments)
- `GITEA_API_URL` / `VIKUNJA_API_URL` — API endpoints
- `REPO_OWNER` — repository owner (must be set per-project)
- `TASK_PREFIX` / `TASK_ID_RE` — task ID prefix and regex (e.g., `DEVX-N`)
- `VIKUNJA_PROJECT_ID` — Vikunja project for task tracking
- `DEFAULT_TIMEOUT`, `DEFAULT_PER_PAGE` — HTTP client defaults
- `MAX_RETRIES`, `RETRY_BACKOFF_BASE`, `RETRY_STATUS_CODES` — retry config
- `CONVENTIONAL_RE` — conventional commit format regex
Both clients retry on transient errors (429, 5xx, connection errors) with exponential backoff.
### `exceptions.py`
### config.py
Custom exception hierarchy:
Configuration constants with env-var overrides (`DEVX_` prefix). Includes API URLs, timeouts, retry settings, task prefix regex, and conventional commit regex.
- `DevxError` — base exception for all devx errors
- `APIError(DevxError)` — raised when a REST API call returns an HTTP error.
Carries `status` (HTTP status code) and `message` (error message).
### gitea_cli.py
### `i18n.py`
Python wrapper around the `tea` Gitea CLI tool. Parses JSON output for structured data. Used by CI scripts for Gitea API operations (issues, labels, PRs, releases, reviews).
Simple i18n system using a JSON translations file (`translations.json`).
Supports five languages: `en`, `bg`, `de`, `ru`, `zh`. The `_()` function
wraps user-facing strings for translation.
Projects can extend translations by setting `DEVX_TRANSLATIONS_PATH` to a
custom JSON file. Keys from the project's file are merged on top of devx's
built-in translations, allowing projects to override or add keys without
modifying the package.
### `api_clients.py`
Reusable HTTP API clients with connection pooling and retry logic. Both
clients retry on transient errors (429, 5xx, connection errors) with
exponential backoff (2s, 4s, 8s).
**`GiteaClient`** — Gitea REST API wrapper:
- Branch protection (get, create, update)
- Labels (list, create, add to issues)
- Issues (create, list)
- Pull requests (get commits, merge, create review)
- Releases (list)
- Wiki pages (list, fetch, create, update, delete)
**`VikunjaClient`** — Vikunja REST API wrapper:
- Tasks (list project tasks, get, update, mark done)
- Comments (create)
### `gitea_cli.py`
Thin Python wrapper around the `tea` Gitea CLI tool. Parses JSON output for
structured data. Used by CI scripts for Gitea API operations that tea handles
well, avoiding hand-rolled HTTP requests.
**`TeaCLI`** operations:
- `create_issue()` — Create issues with labels
- `list_labels()` / `create_label()` / `add_label()` — Label management
- `create_pr()` / `merge_pr()` / `review_pr()` — Pull request operations
- `create_release()` / `list_releases()` — Release management
- `list_branches()` — Branch listing
Operations NOT supported via tea (still use `GiteaClient`):
- Wiki page management
- Commit status checks
- Runner discovery
- PR file/commit listing (tea has limited support)
- Branch protection with detailed config
### `opentofu.py`
OpenTofu output helpers for CI/CD deployment scripts. Provides reusable
functions for extracting values from `tofu output` in a structured way,
eliminating duplicated `subprocess.run` boilerplate:
- `get_tofu_output(output_name, cwd, env)` — Run `tofu output -json` and return parsed JSON
- `get_tofu_vm_ip(output_name, vm_name, cwd, env)` — Extract a VM's IP address
- `get_tofu_vm_field(output_name, vm_name, field, cwd, env)` — Extract a VM field
## CI/CD modules (`devx.ci`)
Modules in this package are run by Gitea Actions workflows. They may import
from `devx.api_clients`, `devx.config`, `devx.gitea_cli`, and `devx.i18n`.
### `release.py`
Automated release using git-cliff. Calculates the next semver version from
conventional commits since the last tag, updates `__version__` in
`__init__.py` and `CHANGELOG.md`, runs lint and tests to verify the release
is healthy, commits with `release: vX.Y.Z [skip ci]`, creates an annotated
tag, and pushes both to master.
Idempotent: if there are no new conventional commits since the last tag, it
exits without doing anything. If the tag already exists, it skips tag creation
and only pushes. Includes a `--verify` mode that checks tag/version/changelog
alignment without making changes.
### `publish.py`
Builds the Python package with `python -m build`, publishes to a Gitea PyPI
registry (or standard PyPI if `PYPI_TOKEN` is set), and creates a Gitea
release with git-cliff-generated notes. Supports `--skip-build` for non-Python
repos that only need a Gitea release.
### `auto_merge.py`
Auto-merges a PR when all CI checks pass. Reads the task ID from the branch
name, validates the PR title format against
the Vikunja task title, extracts the conventional commit message from PR
commits, and squash-merges with title `{PREFIX}-N <conventional commit>`.
If the head branch is behind master (HTTP 405), it automatically pulls master,
rebases, force-pushes, and retries the merge.
### `classify_changes.py`
Classifies git changes between two refs as user-facing or workflow-only. Uses
a layered rule system configured in `pyproject.toml` under
`[tool.devx.classify]`:
1. **User-facing overrides** (highest priority — safety override)
2. **Infrastructure overrides** (explicit per-file)
3. **Infrastructure patterns** (DEFAULT_INFRASTRUCTURE + project-specific)
4. **Default**: user-facing (safe default — any unknown file triggers release)
Also supports custom tags (orthogonal to release impact) for CI conditional
execution (e.g., `ansible` tag to trigger molecule tests).
### `pr_review.py`
Automated PR review. Fetches the PR diff via the Gitea API and runs a series
of checks, posting a structured review with `COMMENT` (no issues) or
`REQUEST_CHANGES` (issues found):
- Architecture compliance (no subprocess in CLI, no hardcoded URLs)
- Best practices (no `print()`, no bare `except`, no `TODO`/`FIXME`, no
functions > 50 lines)
- Security (no hardcoded secrets, no `shell=True`, no `eval`/`exec`)
- i18n (no raw strings in `click.echo()` without `_()` wrapper)
- Resource management (no `open()` without `with`, no `Popen()` without cleanup)
- Documentation (source changes must include doc updates)
- Test coverage (source changes must include test updates)
- Commit conventions (conventional commit format on PR commits)
### `sync_wiki.py`
Syncs documentation from `docs/` to the Gitea wiki via the API. Reads
`docs/mapping.json` to map file paths to wiki page titles, then creates or
updates pages. Supports `--dry-run`, `--verify` (check content), and
`--strict` (full integrity check: page count, missing pages, stale pages,
content match).
### `push_badges.py`
Generates SVG badge files using `devx.tools.generate_badges`, pushes them to
an orphan `badges` branch, and updates `README.md` and `docs/index.md` on
master with cache-busting `raw/commit/<sha>/badge.svg` URLs (Gitea caches
`raw/branch/` URLs for 6 hours). Fetches latest master before generating
badges so the version badge reflects the current state. Supports `--retries`
for retrying on git push failures.
### `notify_failure.py`
Creates a Gitea issue when a CI workflow fails. Uses the `tea` CLI for issue
creation with failure labels. Supports `--auto-login` to configure the tea
CLI login profile from `CI_GITEA_TOKEN` and `DEVX_GITEA_API_URL` before creating
the issue.
### `post_merge.py`
Updates the Vikunja task after a merge to master. Extracts the task ID from
the commit message, marks the task as done, and posts a comment with the
merge SHA.
### `validate_commit_msg.py`
Validates commit messages. On feature branches: conventional commits only
(no `{PREFIX}-N` prefix). On master: must have `{PREFIX}-N` prefix from
auto-merge, followed by a conventional commit message.
### `detect_release_commit.py`
Detects whether the latest git commit is a release commit
(`release: vX.Y.Z [skip ci]`). Writes `is-release=true` or `is-release=false`
to `$GITHUB_OUTPUT` for use in CI workflow conditionals.
### `check_translations.py`
Validates translation files against the Python source code. Checks for
missing keys (used in code but not in translations), dead keys (defined but
not used), and missing languages (a key exists but is missing one of the five
supported languages). Supports checking additional translation sets via
`--translations`.
### `doc_coverage.py`
Checks documentation coverage for CLI commands and major modules. Parses
Click commands from `cli.py` and verifies each has documentation in
`docs/user/cli-commands.md`. Checks that core modules are documented in
`architecture.md` and CI scripts in `ci-cd-workflow.md`. Supports
`--fail-on-missing` to enforce 100% coverage.
### `discover_runners.py`
Discovers available Gitea Actions runners at three levels: repository,
organization, and instance (admin). Falls back to the `MOLECULE_RUNNERS` repo
variable or `DEFAULT_MAX_RUNNERS` (3). Outputs runner count or a JSON index
array for use as a dynamic matrix in Gitea Actions.
### `distribute_files.py`
Distributes files matching a glob pattern across N parallel runners
(round-robin). Writes the assigned file list for the current runner to
`$GITHUB_ENV`. Used for splitting test suites across CI runners.
### `integration_guard.py`
Runs pytest with the same cross-runner failure detection mechanism used by
`molecule_ci_guard`. If any other integration-tests matrix runner reports
failure, the current pytest subprocess is killed and this runner exits early.
## Developer tools (`devx.tools`)
Modules in this package are run locally or by CI setup jobs. They may import
from `devx.api_clients`, `devx.config`, and `devx.gitea_cli`.
### `setup.py`
Project setup: installs Python dependencies (editable mode with extras),
Ansible Galaxy collections (if `ansible/requirements.yml` exists), pre-commit
hooks (pre-commit, commit-msg, pre-push), and configures the `tea` CLI login
profile from `.env`. Supports `--extras` to specify dependency groups,
`--no-pre-commit` to skip hook installation, and `--no-tea-login` to skip tea
configuration.
### `install_tools.py`
Installs CI/CD development tools that are not Python packages: actionlint,
git-cliff, act_runner, and tea. Each tool is installed to `~/.local/bin` if
not already on PATH. Idempotent: skips tools that are already available.
Supports `--tool` to install specific tools and `--list` to show status.
### `check_test_speed.py`
Runs unit tests and enforces execution-time budgets. Two quality gates:
total suite time must not exceed `--max-seconds` (default: 10s), and no
individual test may exceed `--max-single-seconds` (default: 0.5s, 0 to
disable). Runs `make test-unit` with `PYTEST_ADDOPTS=--durations=0`.
### `configure_repo.py`
Configures repository branch protection and labels via the Gitea REST API.
Sets up master branch protection (required status checks, block on rejected
reviews, block on outdated branch) and creates standard labels. Status check
contexts are read from `DEVX_STATUS_CHECKS` or default to
`CI / quality (pull_request)`.
### `generate_badges.py`
Generates self-contained SVG badge files from project metrics. Runs
pytest-cov, doc-coverage, lint checks, and version extraction, then writes
SVG files that can be served as static files from the Gitea raw file API.
Badges generated: coverage, tests, docs, quality, version, python.
### `generate_cliff_config.py`
Generates a `cliff.toml` configuration file with the correct task ID prefix
preprocessor. Eliminates the need to manually duplicate and maintain
`cliff.toml` across repos that use devx. Supports `--prefix` to set the task
ID prefix and `--force` to overwrite an existing file.
### `install_checkmake.py`
Installs checkmake (Makefile linter) if not already present. Tries
`go install` first if Go is available, otherwise downloads the latest
pre-built Linux binary from the official GitHub releases.
## Molecule modules (`devx.molecule`)
Optional modules for projects with Ansible roles. Requires the `molecule`
extra (`pip install devx[molecule]`).
### `distribute_molecule.py`
Distributes molecule (scenario, platform) pairs across N parallel runners.
Discovers scenarios under `ansible/roles/*/molecule/` and crosses them with
the supported OS platform matrix. Supports `--roles-root` for multi-role
repositories, `--list` to list scenarios, and `--list-platforms` to list
platforms.
### `molecule_ci_guard.py`
Runs molecule tests sequentially while polling the Gitea API for other runner
failures. If any other molecule matrix runner reports failure, the current
molecule subprocess is killed and this runner exits early. Supports both
single-role (4-part) and multi-role (5-part) pair encoding.
### `molecule_all.py`
Runs all molecule scenarios on all supported OS platforms sequentially.
Intended for local development; CI uses the parallel matrix instead.
### `molecule/discover_runners.py`
Discovers available Gitea Actions runners for molecule tests. Same logic as
`devx.ci.discover_runners` but intended for molecule-specific workflows.
### `start_docker.py`
Ensures Docker is available for molecule tests in CI. Verifies Docker is
accessible and sets `DOCKER_HOST` explicitly. If the host socket is not
available, tries the rootless socket, then starts a local `dockerd` with the
vfs storage driver (requires privileged container).
### `platforms.py`
Single source of truth for the supported OS platform matrix. Each entry maps
a short name to (image, command). Uses the project's pre-built
molecule-test-base image with `sleep infinity` (not systemd) to avoid cgroup
v2 failures. Supports loading custom platforms from a JSON file.
## Design principles
- **Self-contained package** — `src/devx/` never imports from scripts outside
the package. This allows devx to be installed and used as a dependency
without requiring a specific repo layout in the consumer.
- **Module-based invocation** — All tools invoked via `python -m devx.ci.*`,
`python -m devx.tools.*`, or `python -m devx.molecule.*`. The `devx` CLI is
a thin router that delegates to module `main()` functions.
- **PYTHONPATH: src** — Workflows set `PYTHONPATH: src` (not `.:src` since
there are no scripts at repo root). The `src` directory is the sole import
root.
- **Config via env vars** — `DEVX_*` environment variables with `.env` file
fallback. Projects override defaults via environment or `.env`, never by
editing package code.
- **100% test coverage** — enforced by `--cov-fail-under=100` in pytest.
- **i18n by default** — all user-facing strings wrapped in `_()` for
translation. Five languages supported out of the box.
- **Safe-by-default classification** — any file that doesn't match an
infrastructure pattern defaults to user-facing, triggering a release. This
prevents new file types from accidentally skipping releases.
- **Secrets via environment** — secrets are passed via environment variables,
never on the command line.
## Import rules
1. **`src/devx/` is self-contained** — the package never imports from outside `src/`
2. **CI modules** (`devx.ci.*`) may import from `devx.api_clients`,
`devx.config`, `devx.gitea_cli`, `devx.i18n`
3. **Tool modules** (`devx.tools.*`) may import from `devx.api_clients`,
`devx.config`, `devx.gitea_cli`
4. **Cross-module imports** within `devx.ci.*` or `devx.tools.*` are allowed
but must be documented (e.g., `release.py` imports from
`classify_changes.py`)
## Data flow
### PR lifecycle
```
Developer creates Vikunja task (DEVX-N)
│
▼
Developer creates branch (DEVX-N-short-description)
│
▼
Developer commits (conventional commits, no DEVX-N prefix)
│
▼
Developer pushes and creates PR (title: "DEVX-N: <vikunja task title>")
│
▼
CI workflow (ci.yml) triggers:
│
├── quality (lint, tests, coverage, test speed, doc coverage,
│ translation check, dependency scan, workflow dry-run)
│
├── detect-changes (classify_changes.py → user-facing or workflow-only)
│ └── if user-facing → release-dry-run (release.py --dry-run)
│
├── pr-review (pr_review.py → posts COMMENT or REQUEST_CHANGES)
│
└── auto-merge (auto_merge.py)
├── validate PR title format
├── validate PR title matches Vikunja task title
├── extract conventional commit message from PR commits
├── squash-merge with "DEVX-N <conventional commit>" title
└── push to master
│
▼
Post-merge workflow triggers (see below)
```
### Post-merge flow
```
Push to master (squash-merge commit: "DEVX-N <conventional commit>")
│
▼
Post-merge workflow (post-merge.yml) triggers:
│
├── detect-type (detect_release_commit.py)
│ └── is-release? → skip all jobs except badges
│
├── validate-commit-msg (validate_commit_msg.py --branch master)
│
├── release (release.py)
│ ├── classify_changes.py → skip if workflow-only
│ ├── git-cliff → calculate next version
│ ├── update __version__ in __init__.py
│ ├── update CHANGELOG.md
│ ├── run make lint-ruff && make pytest-cov
│ ├── commit "release: vX.Y.Z [skip ci]"
│ ├── create annotated tag vX.Y.Z
│ └── push commit + tag to master
│ │
│ ▼
│ Tag push triggers publish workflow (see below)
│
├── sync-wiki (sync_wiki.py --strict)
│ └── sync docs/ to Gitea wiki with integrity check
│
├── badges (push_badges.py) [ALWAYS runs, even on release commits]
│ ├── fetch latest master
│ ├── generate_badges.py → SVG files
│ ├── push to orphan badges branch
│ └── update README.md + docs/index.md with cache-busting URLs
│
├── vikunja (post_merge.py)
│ ├── extract task ID from commit message
│ ├── mark Vikunja task as done
│ └── post comment with merge SHA
│
└── configure-repo (configure_repo.py)
└── ensure branch protection and labels
```
### Publish flow
```
Tag push (vX.Y.Z) triggers publish workflow (publish.yml):
│
▼
├── install build, twine, git-cliff, tea
├── configure tea login
│
└── publish (publish.py)
├── build package (python -m build)
├── publish to Gitea PyPI registry (twine upload)
│ OR publish to standard PyPI (if PYPI_TOKEN set)
│ OR skip publish (if --skip-build)
└── create Gitea release with git-cliff notes
```
### Badge generation flow
```
push_badges.py:
│
├── fetch_latest_master() → git fetch + reset --hard origin/master
│
├── generate_badges() → devx.tools.generate_badges
│ ├── run pytest-cov → parse coverage %
│ ├── run pytest → parse test count
│ ├── run doc_coverage → parse doc coverage %
│ ├── run lint → quality status
│ ├── read __version__ from __init__.py
│ └── write SVG files to .badges/
│
├── push_to_badges_branch()
│ ├── git checkout --orphan badges
│ ├── git rm -rf .
│ ├── copy SVG files to root
│ ├── git commit "Update badges [skip ci]"
│ ├── git push origin badges --force
│ └── return commit SHA
│
└── update_readme_with_badge_sha()
├── git checkout master
├── replace raw/branch/badges/ URLs with raw/commit/<sha>/ URLs
├── git commit "chore: update badge URLs [skip ci]"
└── git push origin master
```
## tea CLI integration
The `tea` Gitea CLI tool is used for Gitea API interactions where tea provides
reliable, official support. It is installed by
`python -m devx.tools.install_tools` and configured by
`python -m devx.tools.setup` (login profile from `.env` `CI_GITEA_TOKEN`).
`devx.gitea_cli.TeaCLI` wraps tea with JSON output parsing. Operations that
tea does not support (wiki management, commit status, runner discovery,
detailed branch protection) fall back to `GiteaClient` (direct HTTP).
## Version source
The version source is `__version__` in `src/devx/__init__.py`, read by
setuptools via `dynamic = ["version"]` in `pyproject.toml`. The release
script updates this file, commits it, and tags the commit. This ensures the
package version, git tag, and changelog always stay aligned.
+505 -47
View File
@@ -1,85 +1,543 @@
# CI/CD Workflow
devx uses Gitea Actions for CI/CD automation. The workflow replicates GRM's automated pipeline but without molecule tests.
devx uses Gitea Actions for CI/CD automation. Three workflows implement a
complete pipeline: pull request validation, post-merge release automation, and
tag-triggered publishing.
## Workflows
## Workflow overview
### CI (`ci.yml`)
```
PR opened/synchronized ──► CI (ci.yml)
│ ├── quality
│ ├── detect-changes
│ ├── release-dry-run (if user-facing)
│ ├── pr-review
│ └── auto-merge ──► squash-merge to master
│ │
▼ ▼
Push to master ──► Post-merge (post-merge.yml)
├── detect-type
├── validate-commit-msg
├── release ──► tag vX.Y.Z
├── sync-wiki │
├── badges │
├── vikunja │
└── configure-repo │
│
▼
Tag push (v*) ──► Publish (publish.yml)
└── publish ──► Gitea PyPI registry + Gitea release
```
Runs on pull requests. Jobs:
## CI workflow (`ci.yml`)
1. **quality** — lint (ruff, pyright, bandit, actionlint), unit tests with 100% coverage, test speed check, doc coverage, translation check, dependency scan
2. **detect-changes** — classify changes as user-facing or workflow-only
3. **release-dry-run** — dry-run the release script (only if user-facing changes)
4. **pr-review** — automated PR review
5. **auto-merge** — squash-merge PR when all checks pass
Runs on pull requests (opened and synchronize) and manual dispatch.
### Post-merge (`post-merge.yml`)
### Jobs
Runs on every push to master. Jobs:
#### `quality`
1. **detect-type** — check if commit is a release commit
2. **validate-commit-msg** — validate conventional commit format
3. **release** — calculate next version, update changelog, tag, push
4. **sync-wiki** — sync docs to Gitea wiki
5. **badges** — generate and push quality badges
6. **vikunja** — mark Vikunja task as done
7. **configure-repo** — ensure branch protection and labels
The main quality gate. Runs on every PR:
### Publish (`publish.yml`)
1. **Lint all** — ruff check, ruff format check, pyright, bandit, actionlint
(via `make lint-all`)
2. **Unit tests with 100% coverage** — `make pytest-cov`
3. **Check unit test speed** — `python -m devx.tools.check_test_speed
--max-seconds 4 --max-single-seconds 0.5`
4. **Documentation coverage check** — `python -m devx.ci.doc_coverage
--fail-on-missing`
5. **Translation completeness check** — `python -m devx.ci.check_translations`
6. **Dependency security scan** — `pip-audit --desc --skip-editable`
(best-effort, non-blocking)
7. **Workflow dry-run validation** — `make workflow-dryrun` via act_runner
(best-effort, skipped if act_runner is not installed)
Runs on tag pushes (`v*`). Builds the package, publishes to Gitea PyPI registry, and creates a Gitea release.
#### `detect-changes`
## CI Scripts
Classifies changes between `origin/master` and the PR head as user-facing or
workflow-only using `python -m devx.ci.classify_changes --github-output`.
Writes `user-facing-changed=true|false` to the job output for use by
downstream jobs.
### auto_merge.py
#### `release-dry-run`
Auto-merge PR when all CI checks pass. Reads task ID from `.taskid`, validates PR title format, checks Vikunja task exists, squash-merges with `DEVX-N <conventional commit>` title.
Depends on `quality` and `detect-changes`. Only runs if user-facing changes
are detected. Runs `python -m devx.ci.release --dry-run` to validate that
the release script can calculate the next version and generate the changelog
without making changes. Non-blocking (uses `|| true`).
### release.py
#### `pr-review`
Automated release using git-cliff. Calculates next semver version from conventional commits, updates `__version__` in `__init__.py`, updates `CHANGELOG.md`, runs lint and tests, commits with `release: vX.Y.Z [skip ci]`, creates annotated tag, pushes.
Runs on every pull request. Executes `python -m devx.ci.pr_review` with the
PR number and repository. Fetches the PR diff via the Gitea API and runs
automated checks, posting a structured review:
### publish.py
- `COMMENT` — no issues found
- `REQUEST_CHANGES` — issues found that must be addressed
Builds package with `python -m build`, publishes to Gitea PyPI registry via twine, creates Gitea release with git-cliff-generated notes.
Checks performed:
1. Architecture compliance — no subprocess in CLI, no hardcoded URLs
2. Best practices — no `print()`, no bare `except`, no `TODO`/`FIXME`,
no functions > 50 lines
3. Security — no hardcoded secrets, no `shell=True`, no `eval`/`exec`
4. i18n — no raw strings in `click.echo()` without `_()` wrapper
5. Resource management — no `open()` without `with`, no `Popen()` without
cleanup
6. Documentation — source changes must include doc updates
7. Test coverage — source changes must include test updates
8. Commit conventions — conventional commit format on PR commits
### pr_review.py
#### `auto-merge`
Automated PR review. Checks architecture compliance, best practices, security, i18n, resource management, documentation, test coverage, and commit conventions. Posts inline comments and structured review.
Depends on `quality`, `detect-changes`, and `pr-review`. The final job in the
CI workflow. Runs `python -m devx.ci.auto_merge` with the branch name, PR
title, repository, and PR number:
### notify_failure.py
1. **Read task ID** from branch name (e.g., `DEVX-12-fix-foo` → `DEVX-12`)
2. **Validate PR title format** — must be `{PREFIX}-N: <vikunja task title>`
3. **Validate PR title matches Vikunja task** — fetches the Vikunja task and
compares the title
4. **Extract conventional commit message** from PR commits (newest matching
conventional format)
5. **Squash-merge** with title `{PREFIX}-N <conventional commit message>`
6. If the head branch is behind master (HTTP 405), automatically pulls master,
rebases, force-pushes, and retries the merge
Creates a Gitea issue when a CI workflow fails. Uses tea CLI for issue creation with failure labels.
The merge commit push to master triggers the post-merge workflow.
### post_merge.py
### Smart CI: user-facing vs workflow-only changes
Updates Vikunja task after a merge to master. Extracts task ID from commit message, marks task as done, posts a comment with the merge SHA.
Not all changes require a new release. The `detect-changes` job classifies
changes using `python -m devx.ci.classify_changes`:
### classify_changes.py
**Workflow-only paths** (infrastructure — no release needed):
- `.gitea/**` — Gitea Actions workflows
- `tests/**` — Test files
- `AGENTS.md`, `README.md`, `CHANGELOG.md` — Project docs
- `Makefile`, `cliff.toml`, `.pre-commit-config.yaml` — Config
- `.env.example`, `.gitignore` — Config
- `hooks/**` — Git hooks
- `src/devx/__init__.py` — Only contains `__version__` (release artifact)
Classifies git changes as user-facing or workflow-only. Used to skip releases for infrastructure-only changes. Patterns are configurable.
**User-facing paths** (tool changes — release needed) — everything else:
- `src/devx/**` — Python package source (except `__init__.py`)
- `pyproject.toml` — Package metadata
- Any new file type not in the allowlist
### discover_runners.py
Classification is configured in `pyproject.toml` under
`[tool.devx.classify]`. The framework provides `DEFAULT_INFRASTRUCTURE` — a
curated list of paths that are infrastructure for any Python project. Projects
inherit these automatically and only specify what is different.
Discovers available Gitea Actions runners at repo, org, and instance levels. Generates a dynamic matrix for parallel job distribution.
Rule priority (first match wins):
1. `user_facing_overrides` — safety override (highest priority)
2. `infrastructure_overrides` — explicit per-file
3. `infrastructure` — DEFAULT_INFRASTRUCTURE + project-specific patterns
4. Default: user-facing (safe — any unknown file triggers release)
### detect_release_commit.py
## Post-merge workflow (`post-merge.yml`)
Detects whether the latest git commit is a release commit. Writes `is-release=true` or `is-release=false` to GitHub output.
Runs on every push to master. A single workflow with conditional jobs
replaces separate workflows for release, wiki sync, badges, and Vikunja task
updates.
### push_badges.py
### Job dependency graph
Generates SVG badge files from project metrics (tests, coverage, quality, version). Pushes to `badges` branch and updates README with cache-busting commit SHA URLs.
```
detect-type ──┬── validate-commit-msg (skip if release commit)
├── release (skip if release commit)
│ │
│ ├── sync-wiki (needs release)
│ ├── badges (needs release, ALWAYS runs)
│ └── vikunja (needs release)
└── configure-repo (independent, skip if release commit)
```
### distribute_molecule.py
`sync-wiki` and `vikunja` depend on `release` succeeding so that the wiki and
task tracker are only updated when the code is actually released. If release
fails, they are skipped to avoid leaving the wiki or Vikunja in an
inconsistent state.
Distributes molecule (scenario, platform) pairs across N parallel runners. Discovers scenarios under `ansible/roles/*/molecule/`.
The `badges` job uses `if: always()` with no is-release condition so it runs
on every push to master, including release commits. This ensures badges
(tests, coverage, version, etc.) are always current.
### molecule_ci_guard.py
When `release` creates a `release: vX.Y.Z` commit, the release commit's
post-merge run still updates badges (the version badge picks up the new
version). Other jobs skip. The tag push triggers `publish.yml`.
Runs molecule tests sequentially while polling Gitea for other runner failures. Aborts if another runner fails the same job.
### Post-merge jobs
### validate_commit_msg.py
#### `detect-type`
Validates commit messages. On feature branches: conventional commits only (no `DEVX-N` prefix). On master: must have `DEVX-N` prefix from auto-merge.
Checks if the latest commit is a release commit (`release: vX.Y.Z [skip ci]`)
using `python -m devx.ci.detect_release_commit`. Writes `is-release=true` or
`is-release=false` to the job output. All subsequent jobs use this to
conditionally skip for release commits.
#### `validate-commit-msg`
Depends on `detect-type`. Skips for release commits. Validates the latest
commit message using `python -m devx.ci.validate_commit_msg --branch master`.
On master, commits must follow `{PREFIX}-N: <conventional commit>` format
(added by auto-merge).
#### `release`
Depends on `detect-type`. Skips for release commits. The core release
automation job. Runs `python -m devx.ci.release`:
1. **Classify changes** — calls `classify_changes.py` to check for user-facing
changes. If only infrastructure files changed, exits without releasing.
2. **Calculate next version** — uses git-cliff to determine the next semver
version from conventional commits since the last tag
3. **Update version file** — updates `__version__` in `src/devx/__init__.py`
4. **Update changelog** — prepends the new version section to `CHANGELOG.md`
using git-cliff output
5. **Run tests** — executes `make lint-ruff` and `make pytest-cov` to verify
the release is healthy. If either fails, the release is aborted — no
commit, no tag. Use `--skip-tests` only for emergency releases.
6. **Commit** — stages the version file and changelog, commits with
`release: vX.Y.Z [skip ci]` (uses `--no-verify` to bypass the commit-msg
hook since release commits are a special case)
7. **Create tag** — creates an annotated tag `vX.Y.Z` with the changelog as
the tag message
8. **Push** — pushes both the commit and tag to master
The script is idempotent: if there are no new conventional commits since the
last tag, it exits without doing anything. If the tag already exists (e.g.,
from a partial previous run), it skips tag creation and only pushes.
**Tag consistency**: Before releasing, the script fetches remote tags and
verifies all existing tags point to commits whose message matches the tag
version. This prevents duplicate release commits and ensures
tag/version/commit alignment.
**Version bumping rules** (git-cliff):
| Commit type | Version bump |
|-------------|-------------|
| `feat:` | minor (0.X.0) |
| `fix:` | patch (0.0.X) |
| `feat!:` or `BREAKING CHANGE` | minor (pre-1.0) |
| `chore:`, `ci:`, `docs:` | no bump (excluded by cliff.toml) |
On failure, the `notify_failure` step creates a Gitea issue via
`python -m devx.ci.notify_failure`.
#### `sync-wiki`
Depends on `detect-type` and `release`. Skips for release commits. Syncs
documentation from `docs/` to the Gitea wiki using
`python -m devx.ci.sync_wiki --repo <owner/repo> --strict`:
1. Reads `docs/mapping.json` to map file paths to wiki page titles
2. Lists existing wiki pages via the Gitea API
3. For each mapped file, reads content and creates or updates the wiki page
4. `--strict` runs a full integrity check: verifies page count, missing
pages, stale pages, and content match. Fails if any page is empty or
content doesn't match.
Pages that exist in the wiki but not in the mapping are left untouched (not
deleted).
On failure, the `notify_failure` step creates a Gitea issue.
#### `badges`
Depends on `detect-type` and `release`. Uses `if: always()` so it runs on
every push to master, including release commits. Generates and pushes quality
badges using `python -m devx.ci.push_badges`:
1. **Fetch latest master** — `git fetch origin master && git reset --hard
origin/master` (ensures the version badge reflects the current state,
even if the release job just pushed a new version)
2. **Generate badges** — calls `devx.tools.generate_badges` which runs
pytest-cov, doc-coverage, lint checks, and version extraction, then writes
SVG files: `coverage.svg`, `tests.svg`, `docs.svg`, `quality.svg`,
`version.svg`, `python.svg`
3. **Push to badges branch** — creates an orphan `badges` branch, copies SVG
files, commits, and force-pushes
4. **Update README/docs** — switches back to master, replaces
`raw/branch/badges/<name>.svg` URLs with `raw/commit/<sha>/<name>.svg`
URLs (cache-busting — Gitea caches `raw/branch/` URLs for 6 hours),
commits, and pushes
Supports `--retries` for retrying on git push failures (fetches latest master
and waits 10s between attempts).
On failure, the `notify_failure` step creates a Gitea issue.
#### `vikunja`
Depends on `detect-type` and `release`. Skips for release commits. Updates
the Vikunja task after a merge using `python -m devx.ci.post_merge --git-sha
<sha>`:
1. Extracts the task ID from the first line of the commit message
2. Marks the corresponding Vikunja task as done
3. Posts a comment with the merge SHA
On failure, the `notify_failure` step creates a Gitea issue.
#### `configure-repo`
Depends on `detect-type`. Skips for release commits. Ensures branch
protection and labels are configured using
`python -m devx.tools.configure_repo --repo <name> --owner <owner>`:
- Sets up master branch protection (required status checks, block on rejected
reviews, block on outdated branch)
- Creates standard labels
- Status check contexts read from `DEVX_STATUS_CHECKS` or default to
`CI / quality (pull_request)`
On failure, the `notify_failure` step creates a Gitea issue.
## Publish workflow (`publish.yml`)
Runs on tag pushes matching `v*`. Triggered by the `release` job in the
post-merge workflow when it creates and pushes a new version tag.
### Job: `publish`
1. **Install dependencies** — build, twine, requests, python-dotenv, click,
and the project itself
2. **Install CI tools** — git-cliff and tea via
`python -m devx.tools.install_tools`
3. **Configure tea login** — `tea login add` using `CI_GITEA_TOKEN`
4. **Build and publish** — `python -m devx.ci.publish <tag> <owner/repo>`:
- Build the package with `python -m build`
- Publish to the Gitea PyPI registry (default) using `twine upload
--repository-url <url> -u <token> -p <token>`
- OR publish to standard PyPI if `PYPI_TOKEN` is set
- OR skip publishing if `--skip-build` is passed (non-Python repos)
- Create a Gitea release with git-cliff-generated release notes via
`tea create release`
Publishing destination resolution (checked in order):
1. **Gitea PyPI registry** — if `--registry-url` is given, or
`DEVX_PYPI_REGISTRY_URL` env var is set, or derived from `GITEA_API_URL`
2. **Standard PyPI** — if `PYPI_TOKEN` is set (takes precedence over Gitea
registry)
3. **Skip** — if neither is configured, only the Gitea release is created
On failure, the `notify_failure` step creates a Gitea issue.
## CI scripts
### `auto_merge.py`
Auto-merge PR when all CI checks pass. Reads task ID from the branch name
(e.g., `DEVX-12-fix-foo` → `DEVX-12`). Validates PR title format, checks the
Vikunja task exists and the title matches, extracts the conventional commit
message from PR commits, and squash-merges with
`{PREFIX}-N <conventional commit>` title.
```bash
python -m devx.ci.auto_merge <branch> <pr_title> <owner/repo> <pr_number>
```
### `release.py`
Automated release using git-cliff. Calculates next semver version from
conventional commits, updates `__version__` and `CHANGELOG.md`, runs lint and
tests, commits with `release: vX.Y.Z [skip ci]`, creates annotated tag, and
pushes. Idempotent — exits if no unreleased changes.
```bash
python -m devx.ci.release [--dry-run] [--skip-tests] [--verify]
```
- `--dry-run` — preview without making changes
- `--skip-tests` — skip lint and test verification (emergency only)
- `--verify` — check tag/version/changelog alignment and exit
### `publish.py`
Builds package, publishes to Gitea PyPI registry or standard PyPI, and
creates a Gitea release with git-cliff-generated notes.
```bash
python -m devx.ci.publish <tag> <owner/repo> [--registry-url <url>] [--skip-build]
```
### `pr_review.py`
Automated PR review. Fetches the PR diff via the Gitea API, runs automated
checks (architecture, best practices, security, i18n, resource management,
documentation, test coverage, commit conventions), and posts a structured
review with inline comments.
```bash
python -m devx.ci.pr_review <pr_number> <owner/repo>
```
### `notify_failure.py`
Creates a Gitea issue when a CI workflow fails. Uses the tea CLI for issue
creation with failure labels. Supports `--auto-login` to configure the tea
CLI login profile from `CI_GITEA_TOKEN`.
```bash
python -m devx.ci.notify_failure --repo <owner/repo> --run-id <id> \
--workflow <name> --commit <sha> [--auto-login]
```
### `post_merge.py`
Updates Vikunja task after a merge to master. Extracts task ID from the
commit message, marks the task as done, and posts a comment with the merge SHA.
```bash
python -m devx.ci.post_merge <commit_msg> [--commit-sha <sha>] [--git-sha <sha>]
```
### `classify_changes.py`
Classifies git changes as user-facing or workflow-only. Uses a layered rule
system configured in `pyproject.toml`. Safe-by-default: any unknown file
defaults to user-facing.
```bash
python -m devx.ci.classify_changes [--base <ref>] [--head <ref>] \
[--quiet] [--check <category>] [--github-output]
```
### `discover_runners.py`
Discovers available Gitea Actions runners at repository, organization, and
instance levels. Falls back to `MOLECULE_RUNNERS` repo variable or
`DEFAULT_MAX_RUNNERS` (3).
```bash
python -m devx.ci.discover_runners --owner <owner> --repo <repo> [--count] [--indices]
```
### `detect_release_commit.py`
Detects whether the latest git commit is a release commit. Writes
`is-release=true|false` to `$GITHUB_OUTPUT`.
```bash
python -m devx.ci.detect_release_commit
```
### `push_badges.py`
Generates SVG badge files, pushes them to the `badges` branch, and updates
README.md and docs/index.md with cache-busting `raw/commit/<sha>/` URLs.
```bash
python -m devx.ci.push_badges [--output-dir <dir>] [--branch <branch>] \
[--no-readme-update] [--retries <n>]
```
### `distribute_molecule.py`
Distributes molecule (scenario, platform) pairs across N parallel runners.
Discovers scenarios under `ansible/roles/*/molecule/`.
```bash
python -m devx.molecule.distribute_molecule --runner-index <i> --max-runners <n>
python -m devx.molecule.distribute_molecule --list
python -m devx.molecule.distribute_molecule --list-platforms
```
### `molecule_ci_guard.py`
Runs molecule tests sequentially while polling the Gitea API for other runner
failures. Aborts early if another runner fails the same job.
```bash
python -m devx.molecule.molecule_ci_guard [--roles-root <dir>] pair1 pair2 ...
```
### `validate_commit_msg.py`
Validates commit messages. On feature branches: conventional commits only
(no `{PREFIX}-N` prefix). On master: must have `{PREFIX}-N` prefix from
auto-merge, followed by a conventional commit message.
```bash
python -m devx.ci.validate_commit_msg <commit_msg_file> [--branch <branch>]
```
### `sync_wiki.py`
Syncs documentation from `docs/` to the Gitea wiki via the API. Reads
`docs/mapping.json` for file-to-page mapping. Supports `--dry-run`,
`--verify`, and `--strict` (full integrity check).
```bash
python -m devx.ci.sync_wiki [--dry-run] [--repo <owner/repo>] [--verify] [--strict]
```
### `check_translations.py`
Validates translation files against the Python source code. Checks for
missing keys, dead keys, and missing languages.
```bash
python -m devx.ci.check_translations [--translations <file>]...
```
### `doc_coverage.py`
Checks documentation coverage for CLI commands and major modules. Parses
Click commands from `cli.py` and verifies documentation exists.
```bash
python -m devx.ci.doc_coverage [--docs-dir <dir>] [--fail-on-missing]
```
### `distribute_files.py`
Distributes files matching a glob pattern across N parallel runners
(round-robin). Writes the assigned file list to `$GITHUB_ENV`.
```bash
python -m devx.ci.distribute_files --pattern <glob> --runner-index <i> \
--max-runners <n> [--github-env] [--skip-if-excess]
```
### `integration_guard.py`
Runs pytest with cross-runner failure detection. If any other
integration-tests matrix runner reports failure, the current pytest
subprocess is killed and this runner exits early.
```bash
python -m devx.ci.integration_guard -- <pytest args>
```
## Release process summary
The complete release process from PR to published package:
1. **PR merged** — `auto-merge` squash-merges the PR to master with
`{PREFIX}-N <conventional commit>` title
2. **Post-merge triggers** — the merge push triggers `post-merge.yml`
3. **detect-type** — confirms the commit is not a release commit
4. **release** — `release.py` calculates the next version, updates files,
runs tests, commits `release: vX.Y.Z [skip ci]`, creates tag `vX.Y.Z`,
and pushes to master
5. **Tag push triggers publish** — the tag push triggers `publish.yml`
6. **publish** — `publish.py` builds the package, publishes to the Gitea PyPI
registry, and creates a Gitea release with git-cliff notes
7. **sync-wiki** — documentation is synced to the Gitea wiki
8. **badges** — quality badges are regenerated and pushed to the `badges`
branch; README and docs/index.md are updated with cache-busting URLs
9. **vikunja** — the corresponding Vikunja task is marked as done
10. **configure-repo** — branch protection and labels are ensured
The release commit's post-merge run skips all jobs except `badges` (which
picks up the new version number). This prevents infinite loops.
## Failure handling
Every job in the post-merge and publish workflows has a `notify_failure` step
that runs `if: failure()`. This creates a Gitea issue with the workflow name,
run ID, and commit SHA, ensuring failures that would otherwise go unnoticed
in the Actions tab are surfaced as issues. The issue is created via the tea
CLI with a `bug` label if available.
+418 -31
View File
@@ -1,105 +1,492 @@
# CLI Commands
devx provides a CLI with three command groups: `ci`, `tools`, and `molecule`.
Each subcommand delegates to the corresponding Python module via
`python -m devx.*`, so `devx ci release` is equivalent to
`python -m devx.ci.release`.
```bash
devx --help # show all command groups
devx --version # show package version
devx ci --help # show CI commands
devx tools --help # show tools commands
devx molecule --help # show molecule commands
```
## CI Commands
### `devx ci auto-merge`
Auto-merge a PR when all CI checks pass. Validates PR title, checks Vikunja task, squash-merges.
Auto-merge a PR when all CI checks pass. Reads the task ID from the branch
name, validates the PR title format against
the Vikunja task title, extracts the conventional commit message from PR
commits, and squash-merges with `{PREFIX}-N <conventional commit>` title.
If the head branch is behind master (HTTP 405), automatically pulls master,
rebases, force-pushes, and retries the merge.
```bash
devx ci auto-merge <branch> <pr_title> <owner/repo> <pr_number>
# Example:
devx ci auto-merge DEVX-12-add-feature "DEVX-12: Add feature" oblachno-oss/devx 42
```
### `devx ci check-translations`
Check translation files for gaps, dead keys, and missing languages.
Check translation files for gaps, dead keys, and missing languages. Validates
translation files against the Python source code that uses them. By default,
checks `src/devx/translations.json` against `src/devx/**/*.py`.
Checks performed:
- **Missing keys** — a `_()` call in code has no entry in the translations file
- **Dead keys** — a key in the translations file is not used in any code
- **Missing languages** — a key exists but is missing one of the six
supported languages (en, bg, de, ru, zh, pl)
```bash
devx ci check-translations
devx ci check-translations --translations path/to/translations.json
```
### `devx ci classify-changes`
Classify git changes as user-facing or workflow-only. Used to skip releases for infrastructure-only changes.
Classify git changes as user-facing or workflow-only. Used to skip releases
for infrastructure-only changes. Classification rules are configured in
`pyproject.toml` under `[tool.devx.classify]`.
```bash
devx ci classify-changes --base origin/master --head HEAD
devx ci classify-changes --base origin/master --head HEAD --github-output
devx ci classify-changes --quiet --check user-facing
devx ci classify-changes --check ansible # custom tag from pyproject.toml
```
Options:
- `--base <ref>` — base ref (default: latest tag)
- `--head <ref>` — head ref (default: HEAD)
- `--quiet` — only output true/false
- `--check <category>` — check specific category: `all` (default),
`user-facing`, or any tag name defined in `[tool.devx.classify.tags]`
- `--github-output` — write results to `$GITHUB_OUTPUT` for CI workflow steps
Exit code 2 indicates workflow-only changes (no release needed).
### `devx ci detect-release-commit`
Detect whether the latest git commit is a release commit (`release: vX.Y.Z [skip ci]`).
Detect whether the latest git commit is a release commit
(`release: vX.Y.Z [skip ci]`). Writes `is-release=true` or `is-release=false`
to `$GITHUB_OUTPUT` for use in CI workflow conditionals.
```bash
devx ci detect-release-commit
```
### `devx ci discover-runners`
Discover available Gitea Actions runners for dynamic job distribution.
Queries the Gitea API for registered runners at repository, organization, and
instance (admin) levels. Falls back to `MOLECULE_RUNNERS` repo variable or
`DEFAULT_MAX_RUNNERS` (3).
```bash
devx ci discover-runners --owner oblachno-oss --repo devx
devx ci discover-runners --owner oblachno-oss --repo devx --count
devx ci discover-runners --owner oblachno-oss --repo devx --indices
```
Options:
- `--count` — print the number of available runners
- `--indices` — print a JSON array `[0, 1, ..., N-1]` for use as a dynamic
matrix in Gitea Actions
### `devx ci distribute-files`
Distribute files across parallel runners (round-robin). Discovers files
matching a glob pattern, sorts them for deterministic ordering, then assigns
them round-robin to `max_runners` groups. The assigned group for
`runner_index` is written to `$GITHUB_ENV`.
```bash
devx ci distribute-files --pattern "tests/integration/test_*.py" \
--runner-index 1 --max-runners 3 --github-env
```
Options:
- `--pattern <glob>` — glob pattern for files to distribute
- `--runner-index <i>` — current runner index (0-based)
- `--max-runners <n>` — total number of runners (default: 3)
- `--github-env` — write file list to `$GITHUB_ENV`
- `--skip-if-excess` — skip if fewer files than runners
### `devx ci doc-coverage`
Check documentation coverage for CLI commands and major modules.
Check documentation coverage for CLI commands and major modules. Parses
Click commands from `cli.py` and checks if each has documentation in
`docs/user/cli-commands.md`. Verifies core modules are documented in
`architecture.md` and CI scripts in `ci-cd-workflow.md`.
```bash
devx ci doc-coverage
devx ci doc-coverage --docs-dir docs/ --source-dir src/ --fail-on-missing
```
Options:
- `--docs-dir <dir>` — path to the docs directory (default: `docs/`)
- `--source-dir <dir>` — path to the source directory (default: auto-detect)
- `--fail-on-missing` — exit with non-zero status if any documentation is
missing
### `devx ci lint-docs`
Lint documentation files for structure, broken links, heading hierarchy,
duplicate headings, TODO/FIXME markers, and trailing whitespace.
```bash
devx ci lint-docs
devx ci lint-docs --root . --fix
devx ci lint-docs --no-check-links --no-check-stale
```
Options:
- `--root <dir>` — repository root directory (default: `.`)
- `--docs-dir <dir>` — docs directory (default: `<root>/docs`)
- `--check-links/--no-check-links` — check internal links (default: yes)
- `--check-headings/--no-check-headings` — check heading hierarchy (default: yes)
- `--check-todo/--no-check-todo` — check for TODO/FIXME markers (default: yes)
- `--check-stale/--no-check-stale` — check for stale docs (default: no)
- `--check-trailing/--no-check-trailing` — check trailing whitespace (default: yes)
- `--check-duplicates/--no-check-duplicates` — check duplicate headings (default: yes)
- `--fix` — auto-fix trailing whitespace
### `devx ci integration-guard`
Run pytest with cross-runner failure detection. If any
other integration-tests matrix runner reports failure, the current pytest
subprocess is killed and this runner exits early with code 1.
```bash
devx ci integration-guard -- test_a.py test_b.py
devx ci integration-guard -- -x -v --tb=short test_a.py
```
Environment variables:
- `GITEA_URL` — base URL of the Gitea instance
- `CI_GITEA_TOKEN` — API token with repo access
- `RUN_ID` — workflow run ID (`GITHUB_RUN_ID`)
- `JOB_NAME` — base job name (`GITHUB_JOB`)
- `MATRIX_INDEX` — current matrix index (runner-index)
- `GITEA_REPOSITORY` — repository in `owner/repo` format
### `devx ci notify-failure`
Create a Gitea issue when a CI workflow fails.
Create a Gitea issue when a CI workflow fails. Uses the tea CLI for issue
creation with a `bug` label if available.
```bash
devx ci notify-failure --repo oblachno-oss/devx --run-id 123 \
--workflow ci --commit abc123def456
devx ci notify-failure --repo oblachno-oss/devx --run-id 123 \
--workflow post-merge/release --commit abc123def456 --auto-login
```
Options:
- `--repo <owner/repo>` — repository (required)
- `--run-id <id>` — CI run ID (required)
- `--workflow <name>` — workflow name (required)
- `--commit <sha>` — commit SHA (required)
- `--auto-login` — configure tea CLI login from `CI_GITEA_TOKEN` before creating
the issue
### `devx ci post-merge`
Update Vikunja task after a merge to master.
Update Vikunja task after a merge to master. Extracts the task ID from the
commit message, marks the task as done, and posts a comment with the merge SHA.
```bash
devx ci post-merge "DEVX-12 feat: add feature" --git-sha abc123def456
```
### `devx ci pr-review`
Run automated PR review: check architecture compliance, best practices, and quality.
Run automated PR review. Fetches the PR diff via the Gitea API and runs a
series of checks, posting a structured review (`COMMENT` or
`REQUEST_CHANGES`).
Checks: architecture compliance, best practices, security, i18n, resource
management, documentation, test coverage, and commit conventions.
```bash
devx ci pr-review 42 oblachno-oss/devx
```
### `devx ci publish`
Build package, publish to Gitea PyPI registry, and create Gitea release.
Build package, publish to Gitea PyPI registry (or standard PyPI), and create
a Gitea release with git-cliff-generated notes.
```bash
devx ci publish v1.0.0 oblachno-oss/devx
devx ci publish v1.0.0 oblachno-oss/devx --registry-url https://git.example.com/api/packages/owner/pypi
devx ci publish v1.0.0 oblachno-oss/devx --skip-build # Gitea release only
```
Options:
- `--registry-url <url>` — Gitea PyPI registry URL. Defaults to
`DEVX_PYPI_REGISTRY_URL` env var or a URL derived from `GITEA_API_URL`.
When set, publishes to Gitea PyPI instead of standard PyPI (unless
`PYPI_TOKEN` is also set).
- `--skip-build` — skip package build and PyPI publish (for non-Python repos
that only need a Gitea release)
### `devx ci push-badges`
Generate badge SVG files and push them to the `badges` branch.
Generate badge SVG files and push them to the `badges` branch. Also updates
`README.md` and `docs/index.md` on master with cache-busting
`raw/commit/<sha>/` URLs.
```bash
devx ci push-badges
devx ci push-badges --output-dir .badges/ --branch master
devx ci push-badges --no-readme-update # skip README update (local testing)
devx ci push-badges --retries 3 # retry on git push failures
```
Options:
- `--output-dir <dir>` — temporary directory for badge files (default:
`.badges/`)
- `--branch <branch>` — branch to sync before generating badges (default:
`master`)
- `--no-readme-update` — skip updating README with cache-busting URLs
- `--retries <n>` — number of attempts on git push failures (default: 1).
Between attempts, fetches latest master and waits 10s.
### `devx ci release`
Automated release: calculate next version, update files, tag, and push.
Automated release: calculate next version, update files, tag, and push. Uses
git-cliff to determine the next semver version from conventional commits.
```bash
devx ci release
devx ci release --dry-run # preview without making changes
devx ci release --skip-tests # skip lint and tests (emergency only)
devx ci release --verify # check tag/version/changelog alignment
```
Options:
- `--dry-run` — show what would happen without making changes
- `--skip-tests` — skip lint and test verification (NOT recommended — only
for emergency releases)
- `--verify` — verify tag/version/changelog alignment and exit (no changes
made)
### `devx ci sync-wiki`
Sync documentation from `docs/` to the Gitea wiki.
Sync documentation from `docs/` to the Gitea wiki. Reads `docs/mapping.json`
for file-to-page mapping. Pages that exist in the wiki but not in the mapping
are left untouched.
```bash
devx ci sync-wiki --repo oblachno-oss/devx
devx ci sync-wiki --repo oblachno-oss/devx --dry-run
devx ci sync-wiki --repo oblachno-oss/devx --verify
devx ci sync-wiki --repo oblachno-oss/devx --strict
```
Options:
- `--dry-run` — show what would happen without making changes
- `--repo <owner/repo>` — repository (auto-detected if omitted)
- `--verify` — after syncing, verify each page has non-empty content. Exit 1
if any page is empty or mismatched.
- `--strict` — full integrity check: verify page count, missing pages, stale
pages, and content. Implies `--verify`.
### `devx ci validate-commit-msg`
Validate commit messages for conventional commit format.
Validate commit messages for conventional commit format. On feature branches:
conventional commits only (no `{PREFIX}-N` prefix). On master: must have
`{PREFIX}-N` prefix from auto-merge, followed by a conventional commit
message.
```bash
devx ci validate-commit-msg commit-msg.txt
devx ci validate-commit-msg commit-msg.txt --branch master
```
Options:
- `--branch <branch>` — override branch detection (for CI use)
## Tools Commands
### `devx tools check-test-speed`
Run unit tests and enforce a maximum execution-time budget.
Run unit tests and enforce execution-time budgets. Two quality gates:
- **Total suite time** must not exceed `--max-seconds` (default: 10s)
- **Per-test time** — no individual test may exceed `--max-single-seconds`
(default: 0.5s, 0 to disable)
Runs `make test-unit` with `PYTEST_ADDOPTS=--durations=0` so pytest emits
per-test timing lines.
```bash
devx tools check-test-speed
devx tools check-test-speed --max-seconds 10
devx tools check-test-speed --max-seconds 4 --max-single-seconds 0.5
```
### `devx tools configure-repo`
Configure repository: branch protection + labels via Gitea API.
Configure repository: branch protection and labels via the Gitea REST API.
Sets up master branch protection (required status checks, block on rejected
reviews, block on outdated branch) and creates standard labels.
```bash
devx tools configure-repo --repo devx --owner oblachno-oss
```
Status check contexts are read from `DEVX_STATUS_CHECKS` (comma-separated) or
default to `CI / quality (pull_request)`.
### `devx tools generate-badges`
Generate self-contained SVG badge files from project metrics.
Generate self-contained SVG badge files from project metrics. Runs
pytest-cov, doc-coverage, lint checks, and version extraction, then writes
SVG files that can be served as static files from the Gitea raw file API.
Badges generated: `coverage.svg`, `tests.svg`, `docs.svg`, `quality.svg`,
`version.svg`, `python.svg`.
```bash
devx tools generate-badges
devx tools generate-badges --output-dir .badges/
```
### `devx tools generate-cliff-config`
Generate a `cliff.toml` configuration file with the correct task ID prefix
preprocessor. Eliminates the need to manually duplicate and maintain
`cliff.toml` across repos that use devx.
```bash
devx tools generate-cliff-config --prefix GRM
devx tools generate-cliff-config --prefix GRM --output cliff.toml
devx tools generate-cliff-config --prefix GRM --force # overwrite existing
```
Options:
- `--prefix <prefix>` — task ID prefix (default: `DEVX_TASK_PREFIX` env var
or `DEVX`)
- `--output <file>` — output file path (default: `cliff.toml`)
- `--force` — overwrite existing file
### `devx tools install-checkmake`
Install checkmake (Makefile linter) if not already present.
Install checkmake (Makefile linter) if not already present. Tries
`go install` first if Go is available, otherwise downloads the latest
pre-built Linux binary from the official GitHub releases.
```bash
devx tools install-checkmake
```
### `devx tools install-tools`
Install CI/CD development tools: actionlint, git-cliff, act_runner, tea.
Install CI/CD development tools that are not Python packages: actionlint,
git-cliff, act_runner, and tea. Each tool is installed to `~/.local/bin` if
not already on PATH. Idempotent: skips tools that are already available.
```bash
devx tools install-tools # install all
devx tools install-tools --tool actionlint # install one
devx tools install-tools --tool git-cliff --tool tea # install specific
devx tools install-tools --list # list status
```
### `devx tools setup`
Project setup: install Python deps and pre-commit hooks.
Project setup: install Python dependencies (editable mode with extras),
Ansible Galaxy collections (if `ansible/requirements.yml` exists), pre-commit
hooks (pre-commit, commit-msg, pre-push), and configure the tea CLI login
profile from `.env`.
```bash
devx tools setup --bin .venv/bin
devx tools setup --bin .venv/bin --extras "ci,lint"
devx tools setup --bin .venv/bin --no-pre-commit --no-tea-login
```
Options:
- `--bin <dir>` — virtualenv bin directory (required)
- `--extras <groups>` — pip extras to install (default: `dev`)
- `--no-pre-commit` — skip pre-commit hook installation
- `--no-tea-login` — skip tea CLI login configuration
## Molecule Commands
### `devx molecule distribute`
Distribute molecule test pairs across parallel runners.
### `devx molecule discover-runners`
Discover available Gitea Actions runners for molecule tests.
### `devx molecule guard`
Run molecule tests sequentially with CI failure polling.
Molecule commands require the `molecule` extra (`pip install devx[molecule]`).
### `devx molecule all`
Run all molecule scenarios on all supported OS platforms.
Run all molecule scenarios on all supported OS platforms. Sequential
execution — CI uses the parallel matrix instead.
```bash
devx molecule all
devx molecule all --bin .venv/bin
```
### `devx molecule discover-runners`
Discover available Gitea Actions runners for molecule tests. Same logic as
`devx ci discover-runners` but intended for molecule-specific workflows.
```bash
devx molecule discover-runners --owner oblachno-oss --repo devx --indices
```
### `devx molecule distribute`
Distribute molecule (scenario, platform) pairs across N parallel runners.
Discovers scenarios under `ansible/roles/*/molecule/` and crosses them with
the supported OS platform matrix.
```bash
devx molecule distribute --runner-index 1 --max-runners 3
devx molecule distribute --list # list all scenarios
devx molecule distribute --list-platforms # list platforms
devx molecule distribute --roles-root ansible/roles # multi-role repos
```
Options:
- `--runner-index <i>` — current runner index (0-based)
- `--max-runners <n>` — total number of runners (default: 3)
- `--list` — list all scenarios, one per line
- `--list-platforms` — list all platforms, one per line
- `--roles-root <dir>` — roles root directory for multi-role repos (default:
`ansible/roles`)
### `devx molecule guard`
Run molecule tests sequentially with CI failure polling. A background thread
polls the Gitea API. If any other molecule matrix runner reports failure, the
current molecule subprocess is killed and this runner exits early with code 1.
```bash
devx molecule guard pair1 pair2 pair3
devx molecule guard --roles-root ansible/roles pair1 pair2
```
Each pair is encoded as:
- **Single-role (4-part):** `scenario|platform_name|platform_image|platform_command`
- **Multi-role (5-part):** `role|scenario|platform_name|platform_image|platform_command`
Options:
- `--roles-root <dir>` — roles root directory for multi-role repos
Environment variables:
- `GITEA_URL` — base URL of the Gitea instance
- `CI_GITEA_TOKEN` — API token with repo access
- `RUN_ID` — workflow run ID (`GITHUB_RUN_ID`)
- `JOB_NAME` — base job name (`GITHUB_JOB`)
- `MATRIX_INDEX` — current matrix index (runner-index)
- `GITEA_REPOSITORY` — repository in `owner/repo` format
+161
View File
@@ -0,0 +1,161 @@
# Getting Started with devx
This guide walks you through installing devx, configuring it for your project,
and setting up a complete CI/CD pipeline.
## Prerequisites
- **Python 3.12+**
- **A Gitea instance** with Actions enabled
- **A Gitea API token** with repo, workflow, and organization scopes
- **(Optional) Vikunja API token** for task tracking integration
## Installation
devx is published to the Gitea PyPI registry. Configure pip to use it:
```bash
# Configure Gitea PyPI registry
pip config set global.extra-index-url https://git.oblachno.oblachno.fyi/api/packages/oblachno-oss/pypi/simple
# Install devx
pip install devx
```
Or install from source:
```bash
git clone https://git.oblachno.oblachno.fyi/oblachno-oss/devx.git
cd devx
make setup
```
## Quick Start
### 1. Configure environment variables
Create a `.env` file in your project root:
```bash
CI_GITEA_TOKEN=your_gitea_api_token
VIKUNJA_TOKEN=your_vikunja_api_token # optional
```
### 2. Add devx to your project
Add devx to your `pyproject.toml`:
```toml
[project]
dependencies = [
"devx>=0.26.0",
]
[project.optional-dependencies]
dev = [
"devx[dev]>=0.26.0",
]
```
### 3. Set up the Makefile
devx provides a shared Makefile fragment. Add this to your `Makefile`:
```makefile
include devx.mak
```
Run `devx tools setup` to install all development tools (actionlint, git-cliff,
tea CLI, etc.) and configure pre-commit hooks.
### 4. Create the docs structure
devx expects a `docs/` directory with at minimum:
```
docs/
├── index.md # Documentation home page
├── mapping.json # Wiki page title mappings
├── user/ # User-facing documentation
│ └── cli-commands.md
└── tech/ # Technical documentation
├── architecture.md
└── ci-cd-workflow.md
```
Example `docs/mapping.json`:
```json
{
"index.md": "Home",
"user/cli-commands.md": "CLI-Commands",
"tech/architecture.md": "Architecture",
"tech/ci-cd-workflow.md": "CI-CD-Workflow"
}
```
### 5. Set up CI workflows
Create `.gitea/workflows/ci.yml` and `.gitea/workflows/post-merge.yml` in your
project. See the [CI/CD Workflow guide](../tech/ci-cd-workflow.md) for details.
### 6. Configure release settings
Add a `cliff.toml` for git-cliff-based versioning:
```bash
devx tools generate-cliff-config
```
Add `[tool.devx]` section to `pyproject.toml` for project-specific config:
```toml
[tool.devx]
# Vikunja project ID for task tracking
vikunja_project_id = 6
[tool.devx.classify]
# File patterns that are workflow-only (no release needed)
workflow_only = [
".gitea/**",
"docs/**",
"tests/**",
"AGENTS.md",
"README.md",
"CHANGELOG.md",
]
```
## Available Tools
### CI/CD Automation (`devx.ci.*`)
- `devx.ci.release` — Automated semver versioning and tagging
- `devx.ci.publish` — Package publishing to Gitea PyPI registry
- `devx.ci.auto_merge` — Squash-merge automation with task ID validation
- `devx.ci.pr_review` — Automated PR review with inline comments
- `devx.ci.classify_changes` — User-facing vs workflow-only change detection
- `devx.ci.sync_wiki` — Push docs/ to Gitea wiki
- `devx.ci.doc_coverage` — Documentation coverage checker
- `devx.ci.lint_docs` — Documentation linter (structure, links, headings)
- `devx.ci.check_translations` — i18n translation completeness checker
- `devx.ci.notify_failure` — Create Gitea issues on CI failures
- `devx.ci.distribute_files` — Parallel test file distribution
- `devx.ci.distribute_items` — Parallel item distribution across runners
- `devx.ci.discover_runners` — Dynamic runner discovery via Gitea API
### Development Tools (`devx.tools.*`)
- `devx.tools.setup` — Environment setup (venv, deps, hooks, tools)
- `devx.tools.install_tools` — Install CI/CD tools (actionlint, git-cliff, tea)
- `devx.tools.create_task` — Create Vikunja tasks
- `devx.tools.create_pr` — Create Gitea PRs with task ID in title
- `devx.tools.configure_repo` — Configure branch protection and labels
- `devx.tools.generate_badges` — Generate quality badge SVGs
- `devx.tools.check_test_speed` — Enforce test execution speed limits
## Next Steps
- Read the [CLI Commands reference](cli-commands.md) for all available commands
- Read the [Architecture guide](../tech/architecture.md) to understand internals
- Read the [CI/CD Workflow guide](../tech/ci-cd-workflow.md) for pipeline details
+4 -3
View File
@@ -1,6 +1,7 @@
#!/usr/bin/env bash
# pre-commit hook: fail if unit tests take longer than 10 seconds.
# Aligned with CI timeout (ci.yml uses --max-seconds 10).
# pre-commit hook: fail if unit tests are too slow.
# Checks both total suite time (10s) and per-test time (0.5s).
# Aligned with CI (ci.yml uses same thresholds).
set -e
export PYTHONPATH=src
python3 -m devx.tools.check_test_speed --max-seconds 10
python3 -m devx.tools.check_test_speed --max-seconds 4 --max-single-seconds 0.5
+93 -21
View File
@@ -13,10 +13,13 @@ classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
]
# All dependencies are pinned to exact versions for full reproducibility.
# Update pinned versions in a dedicated PR with verification.
dependencies = [
"requests>=2.34.2",
"python-dotenv>=1.2.2",
"click>=8.4.1",
"requests==2.34.2",
"python-dotenv==1.2.2",
"click==8.4.2",
"tenacity==9.1.4", # retry logic for GiteaClient/VikunjaClient
]
[project.scripts]
@@ -26,38 +29,53 @@ devx = "devx.cli:cli"
version = {attr = "devx.__version__"}
[project.optional-dependencies]
# Minimal deps for CI scripts that only need click/dotenv/requests
# Test runners (pytest + coverage + parallel execution)
ci = [
"pytest>=9.1.0",
"pytest-cov>=7.1.0",
"pytest==9.1.1",
"pytest-cov==7.1.0",
"pytest-xdist==3.8.0",
]
# Lint and type-checking tools (quality job)
# Lint and type-checking tools (quality job, badge generation)
lint = [
"ruff>=0.15.17",
"pyright>=1.1.410",
"bandit>=1.8.2",
"pip-audit>=2.10",
"pre-commit>=4.6.0",
"ruff==0.15.20",
"pyright==1.1.411",
"bandit==1.9.4",
"pip-audit==2.10.1",
"pre-commit==4.6.0",
]
# Molecule testing (optional — for projects with Ansible roles)
# Release tools (build + publish to PyPI/Gitea registry)
release = [
"build==1.5.0",
"twine==6.2.0",
]
# Molecule testing (for projects with Ansible roles)
molecule = [
"molecule>=26.4.0",
"molecule-docker>=2.1.0",
"ansible-lint>=26.4.0",
"ansible>=14.0.0",
"molecule==26.4.0",
"molecule-docker==2.1.0",
"ansible-lint==26.4.0",
"ansible-core==2.21.1",
]
# Deploy tools (for infra staging/production deployments)
deploy = [
"ansible-core==2.21.1",
"boto3==1.43.36",
"docker==7.1.0",
"jinja2==3.1.6",
"pyyaml==6.0.3",
"cryptography==49.0.0",
]
# Full dev environment (local development)
dev = [
"devx[ci,lint]",
"build>=1.3.0",
"twine>=6.2.0",
"devx[ci,lint,release,molecule]",
"build==1.5.0",
"twine==6.2.0",
]
[tool.setuptools.packages.find]
where = ["src"]
[tool.setuptools.package-data]
devx = ["translations.json"]
devx = ["translations.json", "make/*.mak"]
[tool.pytest.ini_options]
testpaths = ["tests"]
@@ -82,4 +100,58 @@ indent-style = "space"
[tool.pyright]
include = ["src"]
pythonVersion = "3.12"
venvPath = "."
venv = ".venv"
strict = ["src/devx/config.py", "src/devx/exceptions.py", "src/devx/i18n.py", "src/devx/api_clients.py", "src/devx/gitea_cli.py"]
# ---------------------------------------------------------------------------
# Change classification — determines which changes trigger a release
# ---------------------------------------------------------------------------
# The framework provides DEFAULT_INFRASTRUCTURE (CI workflows, tests, docs,
# lint config, etc.) that applies to any Python project. We only specify
# what's different about devx.
#
# Rule priority (first match wins):
# 1. user_facing_overrides (safety — highest priority)
# 2. infrastructure_overrides (explicit per-file)
# Project-specific devx configuration (read by devx.config)
[tool.devx]
task_prefix = "DEVX"
vikunja_project_id = 8
repo_owner = "oblachno-oss"
repo_name = "devx"
# 3. infrastructure (DEFAULT_INFRASTRUCTURE + project-specific patterns)
# 4. Default: user-facing (safe)
[tool.devx.classify]
# use_defaults = true # (default) merge with DEFAULT_INFRASTRUCTURE
# Project-specific infrastructure paths (merged with defaults).
# devx has no additional infrastructure paths — everything not in the
# defaults is user-facing (src/devx/**, pyproject.toml, translations.json).
infrastructure = []
# Infrastructure overrides — files that would default to user-facing
# but are actually infrastructure:
# - __init__.py: only contains __version__ (set by release.py, not user code)
#
# NOTE: api_clients.py is NOT here — it's used by devx's CI modules
# (auto_merge.py, release.py, pr_review.py, etc.) which consumer projects
# call via `python -m devx.ci.*`. Changes to api_clients.py affect consumer
# projects' CI behavior, so it IS user-facing.
infrastructure_overrides = [
"src/devx/__init__.py",
]
# User-facing overrides — safety override for broad infrastructure patterns
# devx workflow files (.gitea/**) are reference implementations that
# downstream repos (grm, infra) copy from. Changes to them affect how
# consumer projects run their CI, so they must trigger a release.
user_facing_overrides = [
".gitea/**",
]
# Tag patterns — additional categories for CI conditional execution
# Orthogonal to release impact (user-facing vs infrastructure)
[tool.devx.classify.tags]
# No tags needed for devx itself — it has no ansible/ directory
+1 -1
View File
@@ -1,3 +1,3 @@
"""devx — reusable development and CI/CD tools for oblachno-oss projects."""
__version__ = "0.1.0"
__version__ = "0.27.2"
+196 -94
View File
@@ -2,11 +2,18 @@
from __future__ import annotations
import json
import logging
import time
from typing import Any
import requests
from tenacity import (
before_sleep_log,
retry,
retry_if_exception_type,
stop_after_attempt,
wait_exponential,
)
from devx.config import DEFAULT_TIMEOUT, MAX_RETRIES, RETRY_BACKOFF_BASE, RETRY_STATUS_CODES
from devx.exceptions import APIError
@@ -21,19 +28,74 @@ def _parse_error(e: requests.HTTPError) -> tuple[int, str]:
try:
body: dict[str, Any] = response.json() if response is not None else {}
message: str = body.get("message", str(e))
except Exception:
except (json.JSONDecodeError, ValueError, AttributeError):
message = str(e)
return status, message
def _is_retryable(e: Exception) -> bool:
"""Check if an exception is a transient error worth retrying."""
if isinstance(e, requests.ConnectionError):
return True
if isinstance(e, requests.HTTPError):
status, _ = _parse_error(e)
return status in RETRY_STATUS_CODES
return isinstance(e, requests.Timeout)
class _TransientHTTPError(requests.HTTPError):
"""HTTP error with a retryable status code (wrapped for tenacity)."""
class _RetryableRequestError(Exception):
"""Connection/timeout error wrapped for tenacity retry."""
def _execute_request(
session: requests.Session,
method: str,
url: str,
**kwargs: Any,
) -> requests.Response:
"""Execute a single HTTP request, wrapping transient errors for tenacity.
Non-retryable HTTP errors (4xx except 429) raise :class:`APIError` directly.
Retryable errors (429, 5xx, connection, timeout) raise exceptions that
tenacity will retry.
"""
try:
response = session.request(method, url, timeout=DEFAULT_TIMEOUT, **kwargs)
response.raise_for_status()
return response
except requests.HTTPError as e:
status, message = _parse_error(e)
if status in RETRY_STATUS_CODES:
# Wrap in _TransientHTTPError so tenacity retries it
raise _TransientHTTPError(message, response=e.response) from e
raise APIError(status, message) from e
except (requests.ConnectionError, requests.Timeout) as e:
raise _RetryableRequestError(str(e)) from e
# Tenacity retry decorator shared by both clients.
# Retries on transient HTTP errors (429, 5xx) and connection/timeout errors.
_retry_decorator = retry(
stop=stop_after_attempt(MAX_RETRIES),
wait=wait_exponential(multiplier=RETRY_BACKOFF_BASE, min=RETRY_BACKOFF_BASE, max=RETRY_BACKOFF_BASE**MAX_RETRIES),
retry=retry_if_exception_type((_TransientHTTPError, _RetryableRequestError)),
before_sleep=before_sleep_log(logger, logging.WARNING),
reraise=True,
)
def _request_with_retry(
session: requests.Session,
url: str,
method: str,
**kwargs: Any,
) -> requests.Response:
"""Execute an HTTP request with tenacity-managed retry logic.
On exhaustion, the last exception is translated to :class:`APIError`.
"""
try:
return _retry_decorator(_execute_request)(session, method, url, **kwargs)
except _TransientHTTPError as e:
response = getattr(e, "response", None)
status = response.status_code if response is not None else 0
raise APIError(status, str(e)) from e
except _RetryableRequestError as e:
raise APIError(0, str(e)) from e
class GiteaClient:
@@ -55,49 +117,7 @@ class GiteaClient:
return f"{self._base_url}/repos/{self._owner}/{self._repo}{path}"
def _request(self, method: str, path: str, **kwargs: Any) -> requests.Response:
url = self._url(path)
last_exc: Exception | None = None
for attempt in range(MAX_RETRIES):
try:
response = self._session.request(method, url, timeout=DEFAULT_TIMEOUT, **kwargs)
response.raise_for_status()
return response
except requests.HTTPError as e:
status, message = _parse_error(e)
if _is_retryable(e) and attempt < MAX_RETRIES - 1:
wait = RETRY_BACKOFF_BASE ** (attempt + 1)
logger.warning(
"Transient HTTP %d on %s %s, retrying in %ds (attempt %d/%d)",
status,
method,
path,
wait,
attempt + 1,
MAX_RETRIES,
)
time.sleep(wait)
last_exc = e
continue
raise APIError(status, message) from e
except (requests.ConnectionError, requests.Timeout) as e:
if attempt < MAX_RETRIES - 1:
wait = RETRY_BACKOFF_BASE ** (attempt + 1)
logger.warning(
"Connection error on %s %s, retrying in %ds (attempt %d/%d)",
method,
path,
wait,
attempt + 1,
MAX_RETRIES,
)
time.sleep(wait)
last_exc = e
continue
raise APIError(0, str(e)) from e
# Should not reach here, but just in case
if last_exc: # pragma: no cover
raise APIError(0, str(last_exc)) from last_exc
raise APIError(0, "Max retries exceeded") # pragma: no cover
return _request_with_retry(self._session, self._url(path), method, **kwargs)
# -- repo settings --
@@ -191,11 +211,51 @@ class GiteaClient:
r = self._request("GET", f"/pulls/{pr_number}")
return r.json()
def create_pr(self, title: str, head: str, base: str = "master", body: str = "") -> dict[str, Any]:
"""Create a pull request and return the PR dict.
Args:
title: PR title.
head: Head branch name.
base: Base branch name (default: master).
body: PR description (markdown).
"""
payload: dict[str, Any] = {"title": title, "head": head, "base": base}
if body:
payload["body"] = body
r = self._request("POST", "/pulls", json=payload)
return r.json()
def list_prs(self, state: str = "all", **params: Any) -> list[dict[str, Any]]:
"""List pull requests, optionally filtered by state.
Args:
state: ``open``, ``closed``, ``all`` (default).
**params: Additional query params (e.g. ``q="keyword"`` for title search).
"""
params.setdefault("state", state)
r = self._request("GET", "/pulls", params=params)
return r.json()
def get_pr_files(self, pr_number: str | int) -> list[dict[str, Any]]:
"""Fetch the list of files changed in a pull request."""
r = self._request("GET", f"/pulls/{pr_number}/files")
return r.json()
def add_pr_label(self, pr_number: str | int, label_names: list[str]) -> None:
"""Attach labels to a PR/issue by name.
Args:
pr_number: PR or issue number.
label_names: List of label names to attach.
"""
self._request("POST", f"/issues/{pr_number}/labels", json={"labels": label_names})
def get_pr_label_names(self, pr_number: str | int) -> list[str]:
"""Return label names currently attached to a PR/issue."""
r = self._request("GET", f"/issues/{pr_number}/labels")
return [label.get("name", "") for label in r.json()]
def get_pr_commits(self, pr_number: str | int) -> list[dict[str, Any]]:
"""Fetch the commits included in a pull request."""
r = self._request("GET", f"/pulls/{pr_number}/commits")
@@ -274,6 +334,31 @@ class GiteaClient:
return existing
return self.create_release(tag=tag, name=name, body=body, draft=draft, prerelease=prerelease)
# -- actions (CI/CD) --
def list_action_runs(self, **params: Any) -> dict[str, Any]:
"""List workflow runs for the repository.
Returns the raw API response dict (includes ``workflow_runs`` and
``total_count`` keys per Gitea API).
"""
r = self._request("GET", "/actions/runs", params=params)
return r.json()
def get_action_run_jobs(self, run_id: str | int) -> list[dict[str, Any]]:
"""List jobs for a specific workflow run."""
r = self._request("GET", f"/actions/runs/{run_id}/jobs")
data = r.json()
return data.get("jobs", [])
def get_action_job_logs(self, job_id: str | int) -> str:
"""Fetch logs for a specific CI job.
Returns the raw log text. Raises APIError if logs are unavailable.
"""
r = self._request("GET", f"/actions/jobs/{job_id}/logs")
return r.text
class VikunjaClient:
"""Low-level Vikunja REST API client with connection pooling."""
@@ -285,47 +370,7 @@ class VikunjaClient:
def _request(self, method: str, path: str, **kwargs: Any) -> requests.Response:
url = f"{self._base_url}{path}"
last_exc: Exception | None = None
for attempt in range(MAX_RETRIES):
try:
response = self._session.request(method, url, timeout=DEFAULT_TIMEOUT, **kwargs)
response.raise_for_status()
return response
except requests.HTTPError as e:
status, message = _parse_error(e)
if _is_retryable(e) and attempt < MAX_RETRIES - 1:
wait = RETRY_BACKOFF_BASE ** (attempt + 1)
logger.warning(
"Transient HTTP %d on %s %s, retrying in %ds (attempt %d/%d)",
status,
method,
path,
wait,
attempt + 1,
MAX_RETRIES,
)
time.sleep(wait)
last_exc = e
continue
raise APIError(status, message) from e
except (requests.ConnectionError, requests.Timeout) as e:
if attempt < MAX_RETRIES - 1:
wait = RETRY_BACKOFF_BASE ** (attempt + 1)
logger.warning(
"Connection error on %s %s, retrying in %ds (attempt %d/%d)",
method,
path,
wait,
attempt + 1,
MAX_RETRIES,
)
time.sleep(wait)
last_exc = e
continue
raise APIError(0, str(e)) from e
if last_exc: # pragma: no cover
raise APIError(0, str(last_exc)) from last_exc
raise APIError(0, "Max retries exceeded") # pragma: no cover
return _request_with_retry(self._session, url, method, **kwargs)
def list_tasks(self, **params: Any) -> list[dict[str, Any]]:
r = self._request("GET", "/tasks", params=params)
@@ -341,8 +386,65 @@ class VikunjaClient:
r = self._request("GET", f"/projects/{project_id}/tasks", params=params)
return r.json()
def find_task_by_identifier(self, project_id: int, identifier: str, per_page: int = 50) -> dict[str, Any] | None:
"""Find a task by its identifier (e.g. ``DEVX-42``) in a project.
Paginates through all tasks in the project. Returns the task dict
or None if not found.
"""
page = 1
while True:
tasks = self.list_project_tasks(project_id, page=page, per_page=per_page)
if not tasks:
break
for t in tasks:
if t.get("identifier") == identifier:
return t
if len(tasks) < per_page:
break
page += 1
return None
def create_task(self, project_id: int, title: str, description: str = "") -> dict[str, Any]:
"""Create a task in a project and return the created task dict.
Args:
project_id: Target Vikunja project ID.
title: Task title (required, non-empty).
description: Task description (HTML supported, optional).
"""
r = self._request(
"PUT",
f"/projects/{project_id}/tasks",
json={"title": title, "description": description},
)
return r.json()
def post_comment(self, task_id: int, comment: str) -> None:
self._request("PUT", f"/tasks/{task_id}/comments", json={"comment": comment})
def list_comments(self, task_id: int) -> list[dict[str, Any]]:
"""List all comments on a task."""
r = self._request("GET", f"/tasks/{task_id}/comments")
return r.json()
def update_task(self, task_id: int, **fields: Any) -> None:
"""Update task fields via POST (full replacement semantics).
Warning: Vikunja's POST /tasks/{id} replaces the entire task body.
Unspecified fields are reset to their type defaults. Use
``update_task_safe`` to preserve existing fields.
"""
self._request("POST", f"/tasks/{task_id}", json=fields)
def update_task_safe(self, task_id: int, **fields: Any) -> dict[str, Any]:
"""Safely update task fields using read-merge-write pattern.
Fetches the full task body, merges the provided fields on top,
and POSTs the complete body back. This prevents accidental
resets of done status, title, etc.
"""
task = self.get_task(task_id)
task.update(fields)
r = self._request("POST", f"/tasks/{task_id}", json=task)
return r.json()
+118
View File
@@ -0,0 +1,118 @@
"""Shared utilities for CI modules."""
from __future__ import annotations
import os
import subprocess # nosec B404
import click
from devx.config import TASK_ID_RE
from devx.i18n import _
def get_latest_tag() -> str:
"""Get the latest git tag, or empty string if none exists."""
result = subprocess.run( # nosec B603 B607
["git", "describe", "--tags", "--abbrev=0"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return ""
return result.stdout.strip()
def run_cmd(
args: list[str],
check: bool = True,
capture: bool = True,
) -> subprocess.CompletedProcess[str]:
"""Run a command and return the completed process.
Args:
args: Command and arguments as a list.
check: If True, raise :class:`click.ClickException` on non-zero exit.
capture: If True, capture stdout/stderr. If False, inherit parent's.
"""
result = subprocess.run( # nosec B603
args,
capture_output=capture,
text=True,
check=False,
)
if check and result.returncode != 0:
raise click.ClickException(
_(
"Command failed ({cmd}): {stderr}",
cmd=" ".join(args),
stderr=result.stderr.strip() if result.stderr else result.stdout.strip(),
)
)
return result
def extract_task_id(text: str) -> str:
"""Extract the ``{PREFIX}-N`` task identifier from *text*.
Returns the matched string (e.g. ``DEVX-42``) or an empty string if
no task ID is found.
"""
match = TASK_ID_RE.search(text)
return match.group(0) if match else ""
def write_github_env(key: str, value: str) -> None:
"""Append a key=value line to the ``$GITHUB_ENV`` file.
Multi-line values use the heredoc syntax required by Gitea Actions.
Raises :class:`click.ClickException` if ``GITHUB_ENV`` is not set.
"""
gh_env = os.environ.get("GITHUB_ENV")
if not gh_env:
raise click.ClickException("GITHUB_ENV environment variable is not set")
with open(gh_env, "a", encoding="utf-8") as f: # noqa: PTH123
if "\n" in value:
delimiter = "EOF"
f.write(f"{key}<<{delimiter}\n{value}\n{delimiter}\n")
else:
f.write(f"{key}={value}\n")
def write_github_output(key: str, value: str) -> None:
"""Append a key=value line to the ``$GITHUB_OUTPUT`` file.
Raises :class:`click.ClickException` if ``GITHUB_OUTPUT`` is not set.
"""
gh_output = os.environ.get("GITHUB_OUTPUT")
if not gh_output:
raise click.ClickException("GITHUB_OUTPUT environment variable is not set")
with open(gh_output, "a", encoding="utf-8") as f: # noqa: PTH123
f.write(f"{key}={value}\n")
def lpt_distribute[T](items: list[T], weights: list[int], max_runners: int) -> list[list[T]]:
"""Distribute *items* across *max_runners* using LPT scheduling.
Sorts items by weight (descending), then assigns each to the runner
with the least total weight. This produces a more balanced distribution
than naive round-robin when items have varying costs.
Args:
items: Items to distribute.
weights: Parallel list of integer weights (higher = heavier).
max_runners: Number of runner groups to create.
Returns:
A list of ``max_runners`` lists, each containing the items assigned
to that runner.
"""
groups: list[list[T]] = [[] for _ in range(max_runners)]
loads = [0] * max_runners
indexed = sorted(enumerate(items), key=lambda x: (-weights[x[0]], x[0]))
for orig_idx, item in indexed:
min_runner = min(range(max_runners), key=lambda r: loads[r])
groups[min_runner].append(item)
loads[min_runner] += weights[orig_idx]
return groups
+90 -80
View File
@@ -1,25 +1,27 @@
#!/usr/bin/env python3
"""Auto-merge PR when all CI checks pass.
Runs as the final job in ci.yml. Reads the task ID from ``.taskid`` file
(falling back to branch name extraction for backwards compatibility),
validates the PR title, and squash-merges with a conventional commit
message prefixed by the task ID.
Runs as the final job in ci.yml. Reads the task ID from the branch name
(e.g., ``DEVX-31-fix-foo`` → ``DEVX-31``), validates the PR title against
the Vikunja task, and squash-merges with a conventional commit message
prefixed by the task ID.
PR title format: ``DEVX-N: <vikunja task title>``
Merge commit format: ``DEVX-N: <conventional commit message>``
PR title format: ``{PREFIX}-N: <vikunja task title>``
Merge commit format: ``{PREFIX}-N <conventional commit message>``
The ``{PREFIX}`` is determined by ``DEVX_TASK_PREFIX`` (default: ``DEVX``).
Each project sets its own prefix (e.g., ``GRM``, ``INFRA``).
The conventional commit message is extracted from the PR commits.
This allows the PR title to be a human-friendly Vikunja task title
while the squashed commit follows conventional commits.
Usage:
REPO_TOKEN=<token> python3 -m devx.ci.auto_merge <branch> <pr_title> <repo> <pr_number>
CI_GITEA_TOKEN=<token> python3 -m devx.ci.auto_merge <branch> <pr_title> <repo> <pr_number>
"""
import os
import re
import subprocess # nosec B404
from pathlib import Path
from typing import Any
@@ -27,71 +29,70 @@ import click
from dotenv import load_dotenv # pyright: ignore[reportMissingImports,reportUnknownVariableType]
from devx.api_clients import GiteaClient, VikunjaClient
from devx.ci._shared import extract_task_id as _extract_task_id
from devx.config import (
CONVENTIONAL_RE,
DEFAULT_PER_PAGE,
GITEA_API_URL,
TASK_ID_RE,
TASK_PREFIX,
VIKUNJA_API_URL,
VIKUNJA_PROJECT_ID,
)
from devx.exceptions import APIError
from devx.i18n import _
TASKID_FILE = ".taskid"
PR_TITLE_RE = re.compile(r"^DEVX-\d+:\s+.+")
TASKID_FILE = ".taskid" # Deprecated, kept for backward-compat warnings
PR_TITLE_RE = re.compile(rf"^{TASK_PREFIX}-\d+:\s+.+")
load_dotenv()
def run_cmd(args: list[str], check: bool = True) -> subprocess.CompletedProcess[str]:
"""Run a command and return the completed process."""
result = subprocess.run(args, capture_output=True, text=True, check=False) # nosec B603
if check and result.returncode != 0:
raise click.ClickException(
_(
"Command failed ({cmd}): {stderr}",
cmd=" ".join(args),
stderr=result.stderr.strip() or result.stdout.strip(),
)
)
return result
def read_taskid(branch: str) -> str:
"""Read task ID from .taskid file, falling back to branch name extraction.
"""Read task ID from branch name.
The .taskid file is a simple text file containing just the task ID
(e.g., ``DEVX-60``). If the file doesn't exist, extract from the
branch name as a backwards-compatibility fallback.
The branch name is the sole source of truth for the task ID
(e.g., ``DEVX-31-fix-foo`` → ``DEVX-31``). Branches must include
the task ID prefix — there is no ``.taskid`` file fallback.
If a stale ``.taskid`` file exists and disagrees with the branch
name, a deprecation warning is printed advising its removal.
"""
path = Path(TASKID_FILE)
if path.exists():
task_id = path.read_text(encoding="utf-8").strip()
if task_id:
return task_id
# Fallback: extract from branch name
match = TASK_ID_RE.search(branch)
return match.group(0) if match else ""
branch_task_id = extract_task_id(branch)
if branch_task_id:
# Warn about stale .taskid file if it exists and disagrees
path = Path(TASKID_FILE)
if path.exists():
file_task_id = path.read_text(encoding="utf-8").strip()
if file_task_id and file_task_id != branch_task_id:
click.echo(
_(
"WARNING: .taskid file ({file_id}) is deprecated and disagrees with branch name ({branch_id}). "
"Delete .taskid from the repo — branch name is the sole source of truth.",
file_id=file_task_id,
branch_id=branch_task_id,
)
)
return branch_task_id
return ""
def extract_task_id(branch: str) -> str:
"""Extract DEVX-N task identifier from branch name (legacy fallback)."""
match = TASK_ID_RE.search(branch)
return match.group(0) if match else ""
"""Extract task identifier from branch name (delegates to shared utility)."""
return _extract_task_id(branch)
def validate_pr_title(pr_title: str, task_id: str) -> None:
"""Raise ClickException if PR title does not follow the required format.
Expected: ``DEVX-N: <vikunja task title>``
Expected: ``{PREFIX}-N: <vikunja task title>``
"""
if not PR_TITLE_RE.match(pr_title):
raise click.ClickException(
_(
"Oops! PR title must follow format 'DEVX-N: <task title>'.\n"
"Oops! PR title must follow format '{prefix}-N: <task title>'.\n"
" Expected: {task_id}: <task title>\n"
" Got: {pr_title}",
prefix=TASK_PREFIX,
task_id=task_id,
pr_title=pr_title,
)
@@ -109,12 +110,11 @@ def validate_pr_title(pr_title: str, task_id: str) -> None:
def get_vikunja_task_title(task_id: str) -> str:
"""Fetch the Vikunja task title for the given DEVX-N identifier.
Returns empty string if VIKUNJA_TOKEN is not set (local dev without token).
Raises ClickException if the token is set but the task is not found.
Raises ClickException if VIKUNJA_TOKEN is not set or the task is not found.
"""
token = os.environ.get("VIKUNJA_TOKEN", "")
if not token:
return ""
raise click.ClickException(_("VIKUNJA_TOKEN is not set. This is required in CI to validate PR titles."))
client = VikunjaClient(VIKUNJA_API_URL, token)
page = 1
while True:
@@ -140,14 +140,10 @@ def get_vikunja_task_title(task_id: str) -> str:
def validate_pr_title_matches_vikunja(pr_title: str, task_id: str) -> None:
"""Validate that PR title matches the Vikunja task title.
Skips validation if VIKUNJA_TOKEN is not set (local dev).
Raises ClickException if the task is not found or the title doesn't match.
Raises ClickException if VIKUNJA_TOKEN is not set, the task is not found,
or the title doesn't match.
"""
vikunja_title = get_vikunja_task_title(task_id)
if not vikunja_title:
# VIKUNJA_TOKEN not set — skip validation (local dev)
click.echo(_("Warning: VIKUNJA_TOKEN not set, skipping title match validation."))
return
expected = f"{task_id}: {vikunja_title}"
if pr_title != expected:
raise click.ClickException(
@@ -162,15 +158,25 @@ def validate_pr_title_matches_vikunja(pr_title: str, task_id: str) -> None:
def extract_conventional_msg(commits: list[dict[str, Any]]) -> str:
"""Extract the conventional commit message from PR commits.
Iterates commits in reverse order (newest first) to find the first
message matching the conventional commit format. Falls back to the
newest commit message if none match.
Picks the highest-priority conventional commit message from the PR.
Priority: feat > fix > refactor > docs > chore > other.
Falls back to the newest commit message if none match.
"""
priority = {"feat": 5, "fix": 4, "refactor": 3, "docs": 2, "chore": 1, "ci": 1, "style": 1, "test": 1}
best_msg = ""
best_score = 0
for commit in reversed(commits):
commit_info = commit.get("commit", {})
message = str(commit_info.get("message", "") if isinstance(commit_info, dict) else "").split("\n")[0]
if CONVENTIONAL_RE.match(message):
return message
m = CONVENTIONAL_RE.match(message)
if m:
prefix = m.group(1).split("(")[0].strip() # e.g. "feat" from "feat(scope)"
score = priority.get(prefix, 0)
if score > best_score:
best_score = score
best_msg = message
if best_msg:
return best_msg
# Fallback: use the newest commit's first line
if commits:
commit_info = commits[-1].get("commit", {})
@@ -184,18 +190,28 @@ def extract_conventional_msg(commits: list[dict[str, Any]]) -> str:
@click.argument("repo")
@click.argument("pr_number")
def main(branch: str, pr_title: str, repo: str, pr_number: str) -> None:
token = os.environ.get("REPO_TOKEN", "")
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
raise click.ClickException(_("ERROR: REPO_TOKEN is not set."))
raise click.ClickException(_("ERROR: CI_GITEA_TOKEN is not set."))
owner, repo_name = repo.split("/")
# Validate PR number is an integer
try:
pr_num = int(pr_number)
except ValueError:
raise click.ClickException(_("PR number must be an integer, got: {pr_number}", pr_number=pr_number)) from None
# Validate repo format
if "/" not in repo:
raise click.ClickException(_("Repo must be in 'owner/name' format, got: {repo}", repo=repo))
owner, repo_name = repo.split("/", 1)
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
task_id = read_taskid(branch)
if not task_id:
raise click.ClickException(
_(
"Oops! No task ID found in .taskid file or branch name '{branch}'.",
"Oops! No task ID found in branch name '{branch}'. "
"Branch names must include the task ID prefix (e.g., DEVX-31-fix-bug).",
branch=branch,
)
)
@@ -205,33 +221,27 @@ def main(branch: str, pr_title: str, repo: str, pr_number: str) -> None:
validate_pr_title_matches_vikunja(pr_title, task_id)
# Build merge title: DEVX-N: <conventional commit message>
commits = client.get_pr_commits(pr_number)
commits = client.get_pr_commits(pr_num)
conv_msg = extract_conventional_msg(commits)
if not conv_msg:
raise click.ClickException(_("Could not extract conventional commit message from PR commits."))
merge_title = f"{task_id}: {conv_msg}"
try:
client.merge_pr(pr_number, merge_title)
client.merge_pr(pr_num, merge_title)
except APIError as e:
if e.status == 405 and "behind" in e.message.lower():
# Head branch is behind master — pull master and rebase, then retry
click.echo(_("Head branch is behind master. Pulling and rebasing..."))
try:
run_cmd(["git", "config", "user.name", "devx-ci-bot"])
run_cmd(["git", "config", "user.email", "devx-ci-bot@oblachno.fyi"])
run_cmd(["git", "fetch", "origin", "master"])
run_cmd(["git", "rebase", "origin/master"])
run_cmd(["git", "push", "--force-with-lease", "origin", f"HEAD:{branch}"])
click.echo(_("Rebased and pushed. Retrying merge..."))
client.merge_pr(pr_number, merge_title)
except (APIError, Exception) as retry_err:
raise click.ClickException(
_(
"Merge failed after rebase retry: {error}\nPlease rebase the PR manually.",
error=str(retry_err),
)
) from None
# Head branch is behind master — do NOT auto-rebase.
# Auto-rebasing creates a feedback loop: the force-push triggers
# a new pull_request synchronize event, which starts a new CI run,
# which runs auto-merge again, which rebases again, etc.
raise click.ClickException(
_(
"Branch is behind master. Rebase manually:\n"
" git fetch origin master && git rebase origin/master && git push --force-with-lease\n"
"Then re-add the ready-to-merge label.",
)
) from None
else:
raise click.ClickException(
_(
@@ -245,7 +255,7 @@ def main(branch: str, pr_title: str, repo: str, pr_number: str) -> None:
click.echo(
_(
"Nice! PR #{pr_number} squash-merged with title: {merge_title}",
pr_number=pr_number,
pr_number=pr_num,
merge_title=merge_title,
)
)
+270
View File
@@ -0,0 +1,270 @@
#!/usr/bin/env python3
"""Pre-merge validation gate for auto-merge preconditions.
Validates that a PR satisfies auto-merge requirements BEFORE expensive
jobs (molecule tests, staging deploy) run. This catches issues early:
1. Branch name contains a task ID (e.g., ``DEVX-256-fix-foo``).
2. PR title follows ``{PREFIX}-N: <title>`` format.
3. PR title task ID matches the branch task ID.
4. PR title matches the Vikunja task title (requires ``VIKUNJA_TOKEN``).
5. Branch is not behind master (would trigger a rebase retry cycle).
Exit code 0 = ready for auto-merge (preconditions satisfied).
Exit code 1 = NOT ready — fix issues before pushing.
Usage::
# CI (with VIKUNJA_TOKEN and CI_GITEA_TOKEN):
python3 -m devx.ci.check_auto_merge_ready \\
--branch "$HEAD_REF" \\
--pr-title "$PR_TITLE" \\
--repo "$REPOSITORY" \\
--pr-number "$PR_NUMBER"
# Local (pre-push hook, no PR yet — validates branch + title format only):
python3 -m devx.ci.check_auto_merge_ready --branch "$(git rev-parse --abbrev-ref HEAD)"
# Local (with PR number, fetches title from Gitea):
python3 -m devx.ci.check_auto_merge_ready --branch "$(git rev-parse --abbrev-ref HEAD)" \\
--repo owner/repo --pr-number 123
If ``VIKUNJA_TOKEN`` is not set, the Vikunja title match check is
skipped (with a warning) — this allows local pre-push hooks to run
without CI secrets. In CI, the token is always set and the check is
mandatory.
If ``CI_GITEA_TOKEN`` is not set and ``--pr-number`` is not provided, only
branch-name and PR-title-format checks run (local mode).
"""
from __future__ import annotations
import os
import subprocess # nosec B404
import click
from dotenv import load_dotenv # pyright: ignore[reportMissingImports,reportUnknownVariableType]
from devx.api_clients import GiteaClient, VikunjaClient
from devx.ci.auto_merge import extract_task_id
from devx.config import (
GITEA_API_URL,
VIKUNJA_API_URL,
VIKUNJA_PROJECT_ID,
)
from devx.exceptions import APIError
from devx.i18n import _
load_dotenv()
def is_branch_behind_master(branch: str) -> bool:
"""Check if the local branch is behind origin/master.
Fetches origin first (best-effort) then compares commit counts.
Returns ``True`` if master has commits not in branch.
"""
try:
subprocess.run( # nosec B603, B607
["git", "fetch", "origin", "master", "--quiet"],
check=False,
capture_output=True,
timeout=30,
)
result = subprocess.run( # nosec B603, B607
["git", "rev-list", "--count", f"origin/master..{branch}"],
capture_output=True,
text=True,
check=False,
timeout=10,
)
if result.returncode != 0:
return False # Can't determine — don't block
result = subprocess.run( # nosec B603, B607
["git", "rev-list", "--count", f"{branch}..origin/master"],
capture_output=True,
text=True,
check=False,
timeout=10,
)
if result.returncode != 0:
return False
behind = int(result.stdout.strip() or "0")
except (subprocess.TimeoutExpired, FileNotFoundError, ValueError):
return False # Don't block on git errors
return behind > 0
def get_pr_title_from_gitea(repo: str, pr_number: int) -> str | None:
"""Fetch the PR title from the Gitea API.
Returns ``None`` if ``CI_GITEA_TOKEN`` is not set or the PR cannot be fetched.
"""
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token or "/" not in repo:
return None
owner, repo_name = repo.split("/", 1)
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
try:
pr = client.get_pr(pr_number)
return str(pr.get("title", ""))
except APIError:
return None
def get_vikunja_title_optional(task_id: str) -> str | None:
"""Fetch the Vikunja task title, returning None if token is not set.
Unlike :func:`devx.ci.auto_merge.get_vikunja_task_title`, this does NOT
raise when ``VIKUNJA_TOKEN`` is missing — it returns ``None`` so the
caller can skip the check in local mode.
"""
token = os.environ.get("VIKUNJA_TOKEN", "")
if not token:
return None
client = VikunjaClient(VIKUNJA_API_URL, token)
from devx.config import DEFAULT_PER_PAGE
page = 1
while True:
tasks = client.list_project_tasks(VIKUNJA_PROJECT_ID, page=page, per_page=DEFAULT_PER_PAGE)
if not tasks:
break
matches = [t for t in tasks if t.get("identifier") == task_id]
if matches:
return str(matches[0].get("title", ""))
if len(tasks) < DEFAULT_PER_PAGE:
break
page += 1
return None
@click.command()
@click.option("--branch", required=True, help=_("Branch name (e.g., DEVX-256-fix-foo)"))
@click.option("--pr-title", default=None, help=_("PR title (auto-fetched if --pr-number given)"))
@click.option("--repo", default=None, help=_("Repository in owner/name format"))
@click.option("--pr-number", type=int, default=None, help=_("PR number (to fetch title from Gitea)"))
@click.option("--skip-vikunja", is_flag=True, help=_("Skip Vikunja title match check"))
@click.option("--skip-behind-check", is_flag=True, help=_("Skip branch-behind-master check"))
def cli(
branch: str,
pr_title: str | None,
repo: str | None,
pr_number: int | None,
skip_vikunja: bool,
skip_behind_check: bool,
) -> None:
"""Validate auto-merge preconditions before expensive CI jobs."""
import re
from devx.config import TASK_PREFIX
pr_title_re = re.compile(rf"^{TASK_PREFIX}-\d+:\s+.+") # noqa: PLW1503
errors: list[str] = []
# 1. Branch task ID
task_id = extract_task_id(branch)
if not task_id:
errors.append(
_(
"No task ID found in branch name '{branch}'. Expected format: {prefix}-N-description.",
branch=branch,
prefix=TASK_PREFIX,
),
)
# Can't continue — no task ID to validate against
for e in errors:
click.echo(f"ERROR: {e}", err=True)
raise click.ClickException(_("Branch name must contain a task ID."))
click.echo(f"[pre-merge-check] Task ID: {task_id}")
# 2. Resolve PR title
if pr_title is None and pr_number is not None and repo is not None:
pr_title = get_pr_title_from_gitea(repo, pr_number)
if pr_title:
click.echo(f"[pre-merge-check] PR title (from Gitea): {pr_title}")
if pr_title is None:
# Local mode without PR — only validate branch name
if pr_number is not None:
raise click.ClickException(
_("Could not fetch PR title from Gitea (CI_GITEA_TOKEN not set or PR not found).")
)
click.echo("[pre-merge-check] No PR title provided — running branch-name-only check (local mode).")
click.echo("[pre-merge-check] Branch name OK. Push to create PR, then CI will validate the title.")
return
# 3. PR title format
if not pr_title_re.match(pr_title):
errors.append(
_(
"PR title must follow format '{prefix}-N: <task title>'.\n Got: {title}",
prefix=TASK_PREFIX,
title=pr_title,
),
)
# 4. PR title task ID matches branch task ID
if not pr_title.startswith(f"{task_id}:"):
errors.append(
_(
"PR title task ID mismatch.\n Branch task ID: {task_id}\n PR title: {title}",
task_id=task_id,
title=pr_title,
),
)
# 5. Vikunja task title match (skip if no token or --skip-vikunja)
if not skip_vikunja:
vikunja_title = get_vikunja_title_optional(task_id)
if vikunja_title is None:
token_set = bool(os.environ.get("VIKUNJA_TOKEN", ""))
if token_set:
errors.append(
_(
"Could not find Vikunja task {task_id} in project {project_id}.",
task_id=task_id,
project_id=VIKUNJA_PROJECT_ID,
),
)
else:
click.echo("[pre-merge-check] WARNING: VIKUNJA_TOKEN not set — skipping Vikunja title match check.")
else:
expected = f"{task_id}: {vikunja_title}"
if pr_title != expected:
errors.append(
_(
"PR title does not match Vikunja task title.\n Expected: {expected}\n Got: {title}",
expected=expected,
title=pr_title,
),
)
else:
click.echo(f"[pre-merge-check] Vikunja title match OK: {expected}")
# 6. Branch behind master (skip if --skip-behind-check)
if not skip_behind_check:
if is_branch_behind_master(branch):
errors.append(
_("Branch is behind origin/master. Rebase first: git fetch origin master && git rebase origin/master")
)
else:
click.echo("[pre-merge-check] Branch is up-to-date with origin/master.")
if errors:
click.echo("", err=True)
click.echo("=" * 60, err=True)
click.echo("Pre-merge validation FAILED — fix these before pushing:", err=True)
click.echo("=" * 60, err=True)
for e in errors:
click.echo(f" - {e}", err=True)
raise click.ClickException(_("Pre-merge validation failed."))
click.echo("[pre-merge-check] All auto-merge preconditions satisfied.")
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+45 -30
View File
@@ -11,14 +11,14 @@ Checks performed (all fail with exit code 1 on error):
- **Missing keys**: a ``_()`` call in code has no entry in the corresponding
translations file.
- **Dead keys**: a key in a translations file is not used in any code.
- **Missing languages**: a key exists but is missing one of the 5 supported
languages (en, bg, de, ru, zh). Reported as a warning, not an error.
- **Missing languages**: a key exists but is missing one of the 6 supported
languages (en, bg, de, ru, zh, pl). This is an error — all supported languages
must have translations for every key.
Usage::
python3 -m devx.ci.check_translations
python3 -m devx.ci.check_translations --translations path/to/translations.json
python3 -m devx.ci.check_translations --strict # warnings are errors
"""
from __future__ import annotations
@@ -31,11 +31,11 @@ from pathlib import Path
import click
REPO_ROOT = Path(__file__).resolve().parent.parent.parent.parent
REPO_ROOT = Path.cwd()
SUPPORTED_LANGS = ("en", "bg", "de", "ru", "zh")
SUPPORTED_LANGS = ("en", "bg", "de", "ru", "zh", "pl")
# Default translation set: devx package itself
# Default translation set: look for translations.json in the current repo
DEFAULT_TRANS_FILE = REPO_ROOT / "src" / "devx" / "translations.json"
DEFAULT_SRC_DIR = REPO_ROOT / "src" / "devx"
@@ -101,9 +101,9 @@ def collect_keys(src_dir: Path) -> set[str]:
if pyfile.name == "i18n.py":
continue
keys |= extract_keys(pyfile)
# Add dynamic keys for the default source directory
if src_dir == DEFAULT_SRC_DIR:
keys |= DYNAMIC_KEYS
# Dynamic keys are common status strings used via _(variable) that
# can't be detected by AST scanning. Include them for all projects.
keys |= DYNAMIC_KEYS
return keys
@@ -130,14 +130,15 @@ def check_translation_set(name: str, src_dir: Path, trans_file: Path) -> Transla
# Check for dead keys (in translations but not used in code)
result.dead_keys = result.defined_keys - result.used_keys
for key in sorted(result.dead_keys):
result.warnings.append(f"Dead key in {name}: {key!r}")
result.errors.append(f"Dead key in {name}: {key!r}")
# Check for missing languages
# Check for missing languages — this is an error, not a warning.
# All supported languages must have translations for every key.
for key, langs in translations.items():
missing = [lang for lang in SUPPORTED_LANGS if lang not in langs]
if missing:
result.missing_langs[key] = missing
result.warnings.append(f"Missing languages {missing} for key {key!r} in {name}")
result.errors.append(f"Missing languages {missing} for key {key!r} in {name}")
return result
@@ -168,44 +169,58 @@ def print_result(result: TranslationCheckResult) -> None:
"translations",
multiple=True,
type=click.Path(exists=False, path_type=Path),
help="Path to a translations JSON file to check (can be repeated). Defaults to src/devx/translations.json.",
help="Path to a translations JSON file to check (can be repeated). Auto-detects by default.",
)
@click.option("--strict", is_flag=True, default=False, help="Treat warnings as errors.")
def main(translations: tuple[Path, ...], strict: bool) -> None:
@click.option(
"--source-dir",
default=None,
help="Source directory to scan for _() calls (default: auto-detect).",
)
def main(translations: tuple[Path, ...], source_dir: str | None) -> None:
"""Check translation files for gaps, dead keys, and missing languages."""
results: list[TranslationCheckResult] = []
if not translations:
# Default: check the devx package's own translations
results = [
check_translation_set("devx", DEFAULT_SRC_DIR, DEFAULT_TRANS_FILE),
# Auto-detect translations file in the current repo
root = Path.cwd()
# Try common locations
candidates = [
root / "src" / "devx" / "translations.json",
root / "src" / "gitea_runner_manager" / "translations.json",
]
# Also search for any translations.json in src/
for match in root.glob("src/*/translations.json"):
candidates.append(match)
found = False
for candidate in candidates:
if candidate.exists():
src_dir = Path(source_dir) if source_dir else candidate.parent
results.append(check_translation_set(candidate.parent.name, src_dir, candidate))
found = True
break
if not found:
# No translations file found — this repo doesn't use i18n
click.echo("PASS: No translations file found — skipping (repo does not use i18n).")
return
else:
results = []
for trans_file in translations:
# Infer source directory as the parent of the translations file
src_dir = trans_file.parent
src_dir = Path(source_dir) if source_dir else trans_file.parent
name = trans_file.parent.name
results.append(check_translation_set(name, src_dir, trans_file))
has_errors = False
has_warnings = False
for result in results:
print_result(result)
if result.errors:
has_errors = True
if result.warnings:
has_warnings = True
click.echo()
if has_errors:
click.echo("FAIL: Translation check found errors.", err=True)
sys.exit(1)
if strict and has_warnings:
click.echo("FAIL: Translation check found warnings (--strict mode).", err=True)
sys.exit(1)
if has_warnings:
click.echo("PASS with warnings: Translation check passed (warnings present).")
else:
click.echo("PASS: All translations are complete and up to date.")
click.echo("PASS: All translations are complete and up to date.")
if __name__ == "__main__": # pragma: no cover
+574 -157
View File
@@ -1,106 +1,474 @@
#!/usr/bin/env python3
"""Classify git changes as user-facing or workflow-only.
"""Classify git changes as user-facing or infrastructure.
Determines whether changes between two git refs (e.g., last tag and HEAD)
affect the tool itself (user-facing) or only the CI/CD infrastructure
affect the published package (user-facing) or only the CI/CD infrastructure
(workflow-only). This is used by:
- **release.py** — skips release when only workflow files changed
- **release.py** — skips release when only infrastructure files changed
- **CI workflow** — skips molecule tests and release dry-run when only
workflow files changed
infrastructure files changed
Classification strategy (safe-by-default):
== Design Philosophy ==
Any file that is NOT in the explicit workflow-only allowlist is treated
as user-facing. This ensures new file types default to requiring a
release rather than silently skipping it.
**Safe-by-default**: Any file that doesn't match a rule defaults to
user-facing. This prevents new file types from accidentally skipping
releases — a critical safety property. When in doubt, release.
The workflow-only patterns are configurable via the ``patterns``
parameter on ``classify_changes()`` and ``has_user_facing_changes()``.
The default set (``DEFAULT_WORKFLOW_ONLY_PATTERNS``) covers common
infrastructure paths. Each project can pass its own frozenset to
accommodate different source layouts.
**Framework-provided defaults**: The framework ships with
``DEFAULT_INFRASTRUCTURE`` — a curated list of paths that are
infrastructure for ANY Python project (CI workflows, tests, docs,
lint config, etc.). Projects inherit these automatically and only
need to specify what's *different* about their project.
Default workflow-only paths (infrastructure → no release needed):
- .gitea/workflows/** — Gitea Actions workflows
- scripts/** — All scripts (CI/CD, dev tools, setup)
- src/devx/__init__.py — Version file (release artifact)
- src/devx/api_clients.py — Gitea API client (CI/CD only, not used by CLI)
- docs/** — Documentation
- tests/** — Test files
- hooks/** — Git hooks
- AGENTS.md — Agent conventions
- README.md — README (lean, links to wiki)
- CHANGELOG.md — Changelog (generated)
- TROUBLESHOOTING.md — Troubleshooting guide
- cliff.toml — git-cliff config
- Makefile — Build automation
- .pre-commit-config.yaml — Pre-commit config
- .ansible-lint — Ansible lint config
- .env.example — Environment template
- .gitignore — Git ignore rules
- .ruff.toml — Ruff config (if separate)
- .github/** — GitHub config (if present)
**Config-driven**: Classification rules are read from ``[tool.devx.classify]``
in ``pyproject.toml``. No project needs to modify the framework code.
Each project declares its own paths; the framework handles the logic.
Everything else is user-facing (tool changes → release needed),
including but not limited to:
- src/devx/*.py — Python CLI source (except __init__.py)
- ansible/** — Ansible role
- pyproject.toml — Package metadata
- Any new file type not in the allowlist
**Layered rules** (evaluated in priority order):
1. **User-facing overrides** (highest priority — safety override)
Files that match infrastructure patterns but MUST be treated as
user-facing. Use this when an infrastructure pattern is too broad.
2. **Infrastructure overrides**
Files that would default to user-facing but are actually
infrastructure (e.g., ``src/pkg/__init__.py`` which only contains
``__version__`` — a release artifact, not user-facing code).
3. **Infrastructure patterns** (deny-list)
Path globs matching infrastructure files. This is the union of
``DEFAULT_INFRASTRUCTURE`` and the project's ``infrastructure`` list.
Changes to these don't trigger a release.
4. **Default**: user-facing (lowest priority — safe default)
**Tag system** (orthogonal to release impact):
Projects can define custom tags (e.g., ``ansible``, ``docs``) for CI
conditional execution. A file can be both infrastructure (no release)
and tagged ``ansible`` (run molecule tests). Tags are evaluated
independently of the user-facing/infrastructure classification.
The ``--check`` CLI option accepts any tag name defined in the config,
and ``--github-output`` writes ``<tag>-changed`` for each configured tag.
== Configuration ==
In ``pyproject.toml``::
[tool.devx.classify]
# Whether to merge with DEFAULT_INFRASTRUCTURE (default: true).
# Set to false to specify all patterns explicitly.
# use_defaults = true
# Project-specific infrastructure paths (merged with defaults).
# Only list paths NOT already in DEFAULT_INFRASTRUCTURE.
infrastructure = [
"scripts/**", # e.g., if scripts/ is dev-only tooling
]
# Infrastructure overrides — files that would default to user-facing
# but are actually infrastructure
infrastructure_overrides = [
"src/mypkg/__init__.py", # only contains __version__
]
# User-facing overrides — safety override for broad infrastructure patterns
# (empty by default)
user_facing_overrides = []
# Tag patterns — additional categories for CI conditional execution
[tool.devx.classify.tags]
ansible = ["ansible/**", ".ansible-lint"]
== What counts as "user-facing" ==
A change is user-facing if it affects the behavior of the installed
package. For a library/CLI tool, this means:
- Source code in ``src/`` (except ``__init__.py`` which only holds
``__version__``)
- Package metadata (``pyproject.toml`` — dependencies, entry points)
- Ansible roles, playbooks, templates (if the project ships Ansible)
- Translation files (user-visible messages)
- Any file not explicitly classified as infrastructure
A change is infrastructure if it only affects the project's own
development/CI environment:
- CI/CD workflows (``.gitea/**``, ``.github/**``)
- Tests (``tests/**``)
- Documentation (``docs/**``, ``README.md``, ``CHANGELOG.md``)
- Linting/formatting config (``.ruff.toml``, ``.pre-commit-config.yaml``)
- Build tooling (``Makefile``, ``cliff.toml``)
- Git hooks (``hooks/**``)
- Generated scripts (``activate.sh``, ``activate.fish``, ``activate.zsh``)
== Glob Syntax ==
Patterns support standard glob syntax:
- ``**`` matches any number of path segments (including zero)
- ``*`` matches any characters within a single path segment
- ``?`` matches a single character within a single path segment
- Everything else is matched literally
Examples:
- ``.gitea/**`` matches ``.gitea/workflows/ci.yml``, ``.gitea/actionlint.yaml``
- ``tests/**`` matches ``tests/unit/test_cli.py``, ``tests/conftest.py``
- ``src/devx/__init__.py`` matches exactly that file
- ``Makefile`` matches exactly that file
Usage:
python3 -m devx.ci.classify_changes [--base <ref>] [--head <ref>]
python3 -m devx.ci.classify_changes --base v0.3.0 --head HEAD
python3 -m devx.ci.classify_changes --check ansible --quiet
python3 -m devx.ci.classify_changes --github-output
"""
from __future__ import annotations
import os
import re
import subprocess # nosec B404
import sys
import tomllib
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
import click
from devx.ci._shared import get_latest_tag, write_github_output
from devx.i18n import _
# Explicit allowlist of workflow-only path patterns.
# Anything NOT matching these is treated as user-facing (safe default).
# This is the default set — projects can override via the ``patterns``
# parameter on classify_changes() / has_user_facing_changes().
DEFAULT_WORKFLOW_ONLY_PATTERNS: frozenset[str] = frozenset(
[
# CI/CD infrastructure
".gitea/",
# All scripts are infrastructure (CI/CD, dev tools, setup)
# User-facing code lives in src/devx/
"scripts/",
# Version file — only contains __version__, not user-facing code.
# Version bumps are a release artifact, not a feature.
"src/devx/__init__.py",
# Gitea API client — used only by CI/CD scripts, not by the CLI.
"src/devx/api_clients.py",
# Documentation
"docs/",
"AGENTS.md",
"README.md",
"CHANGELOG.md",
"TROUBLESHOOTING.md",
# Tests
"tests/",
# Config / build automation
"cliff.toml",
"Makefile",
".pre-commit-config.yaml",
".ansible-lint",
".env.example",
".gitignore",
".ruff.toml",
# Hooks
"hooks/",
# GitHub (if ever added)
".github/",
]
)
# ---------------------------------------------------------------------------
# Data structures
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class FileClassification:
"""Result of classifying a single file.
Attributes:
path: The file path relative to repo root.
is_user_facing: True if changes to this file require a release.
reason: Human-readable explanation of the classification.
matched_rule: Which rule matched (e.g., "infrastructure: .gitea/**").
None if the default rule was used.
tags: Custom category tags (e.g., {"ansible"}).
"""
path: str
is_user_facing: bool
reason: str
matched_rule: str | None
tags: frozenset[str] = frozenset()
@dataclass
class ClassificationResult:
"""Result of classifying a set of changed files.
Attributes:
files: Per-file classification details.
user_facing: List of file paths classified as user-facing.
infrastructure: List of file paths classified as infrastructure.
tags: Dict mapping tag name to list of file paths matching that tag.
"""
files: list[FileClassification] = field(default_factory=list)
user_facing: list[str] = field(default_factory=list)
infrastructure: list[str] = field(default_factory=list)
tags: dict[str, list[str]] = field(default_factory=dict)
@property
def has_user_facing(self) -> bool:
"""True if any user-facing files were found."""
return bool(self.user_facing)
def has_tag(self, tag: str) -> bool:
"""True if any files matched the given tag."""
return bool(self.tags.get(tag))
# ---------------------------------------------------------------------------
# Glob matching
# ---------------------------------------------------------------------------
def _glob_to_regex(pattern: str) -> re.Pattern[str]:
"""Convert a glob pattern to a compiled regex.
Supports:
- ``**`` -> matches any number of path segments (including zero)
- ``*`` -> matches any chars within a single path segment
- ``?`` -> matches a single char within a path segment
- All other characters are matched literally
"""
# Handle ** at the end (e.g., ".gitea/**")
# ** matches anything including slashes
parts: list[str] = []
i = 0
while i < len(pattern):
c = pattern[i]
if c == "*" and i + 1 < len(pattern) and pattern[i + 1] == "*":
parts.append(".*")
i += 2
# Skip trailing slash after **
if i < len(pattern) and pattern[i] == "/":
i += 1
elif c == "*":
parts.append("[^/]*")
i += 1
elif c == "?":
parts.append("[^/]")
i += 1
else:
parts.append(re.escape(c))
i += 1
return re.compile("^" + "".join(parts) + "$")
def _matches_glob(file_path: str, pattern: str) -> bool:
"""Check if a file path matches a glob pattern.
Also supports prefix matching: if the pattern ends with ``/``,
any file starting with that prefix matches. This is a convenience
for patterns like ``.gitea/`` (equivalent to ``.gitea/**``).
"""
# Prefix matching for patterns ending with /
if pattern.endswith("/") and (file_path.startswith(pattern) or file_path == pattern.rstrip("/")):
return True
return _glob_to_regex(pattern).match(file_path) is not None
# ---------------------------------------------------------------------------
# Classifier
# ---------------------------------------------------------------------------
# ---------------------------------------------------------------------------
# Default infrastructure patterns
# ---------------------------------------------------------------------------
# Common infrastructure paths that apply to ANY Python project using devx.
# Projects inherit these automatically and only need to specify project-specific
# paths in their [tool.devx.classify] section.
#
# Rationale: these files/directories are development tooling, CI/CD config,
# or generated artifacts. Changes to them don't affect the installed package's
# behavior, so they don't warrant a release.
DEFAULT_INFRASTRUCTURE: list[str] = [
# CI/CD workflow definitions
".gitea/**",
".github/**",
# Test files
"tests/**",
# Documentation
"docs/**",
# Git hooks
"hooks/**",
# Build tooling
"Makefile",
"cliff.toml",
"uv.lock",
# Linting / formatting config
".pre-commit-config.yaml",
".ruff.toml",
".ansible-lint",
".checkmake.ini",
".editorconfig",
# Environment templates (not the actual .env which is gitignored)
".env.example",
# Git config
".gitignore",
".gitattributes",
# Project-level documentation (not part of the installed package)
"AGENTS.md",
"README.md",
"CHANGELOG.md",
"TROUBLESHOOTING.md",
"CONTRIBUTING.md",
"CODE_OF_CONDUCT.md",
"REVIEW_CHECKLIST.md",
# Agent/CI tooling config (not part of the installed package)
".devin/**",
# Generated venv activation scripts (created by `make setup`)
"activate.sh",
"activate.fish",
"activate.zsh",
]
@dataclass
class ClassifierConfig:
"""Configuration for the change classifier.
Loaded from ``[tool.devx.classify]`` in ``pyproject.toml``.
By default, the framework's ``DEFAULT_INFRASTRUCTURE`` patterns are
merged with the project's ``infrastructure`` list. Set
``use_defaults = false`` to disable defaults and specify all
patterns explicitly.
Attributes:
infrastructure: Glob patterns for infrastructure paths
(merged with DEFAULT_INFRASTRUCTURE unless use_defaults is False).
infrastructure_overrides: Exact paths that are infrastructure
despite not matching any infrastructure pattern.
user_facing_overrides: Exact paths that are user-facing
despite matching an infrastructure pattern (safety override).
tags: Dict mapping tag name to list of glob patterns.
use_defaults: If True (default), merge with DEFAULT_INFRASTRUCTURE.
"""
infrastructure: list[str] = field(default_factory=list)
infrastructure_overrides: list[str] = field(default_factory=list)
user_facing_overrides: list[str] = field(default_factory=list)
tags: dict[str, list[str]] = field(default_factory=dict)
use_defaults: bool = True
@classmethod
def from_pyproject(cls, pyproject_path: str = "pyproject.toml") -> ClassifierConfig:
"""Load classifier config from pyproject.toml.
Reads the ``[tool.devx.classify]`` section. If the section or
file is missing, returns a config with only DEFAULT_INFRASTRUCTURE
(everything else defaults to user-facing — safe-by-default).
"""
path = Path(pyproject_path)
if not path.exists():
return cls(infrastructure=list(DEFAULT_INFRASTRUCTURE))
with open(path, "rb") as f: # noqa: PTH123
data: dict[str, Any] = tomllib.load(f)
classify_cfg = data.get("tool", {}).get("devx", {}).get("classify", {})
use_defaults = classify_cfg.get("use_defaults", True)
project_infra = list(classify_cfg.get("infrastructure", []))
if use_defaults:
# Merge defaults with project-specific patterns (deduplicated)
merged = list(DEFAULT_INFRASTRUCTURE)
for p in project_infra:
if p not in merged:
merged.append(p)
infrastructure = merged
else:
infrastructure = project_infra
return cls(
infrastructure=infrastructure,
infrastructure_overrides=list(classify_cfg.get("infrastructure_overrides", [])),
user_facing_overrides=list(classify_cfg.get("user_facing_overrides", [])),
tags={k: list(v) for k, v in classify_cfg.get("tags", {}).items()},
use_defaults=use_defaults,
)
class ChangeClassifier:
"""Classify changed files as user-facing or infrastructure.
Uses layered rules with safe-by-default semantics.
Rule evaluation order (first match wins):
1. User-facing overrides (safety — highest priority)
2. Infrastructure overrides
3. Infrastructure patterns
4. Default: user-facing (safe)
"""
def __init__(self, config: ClassifierConfig | None = None) -> None:
self.config = config or ClassifierConfig.from_pyproject()
# Pre-compile infrastructure patterns for efficiency
self._infra_patterns = list(self.config.infrastructure)
self._infra_overrides = set(self.config.infrastructure_overrides)
self._user_overrides = set(self.config.user_facing_overrides)
def classify_file(self, file_path: str) -> FileClassification:
"""Classify a single file path.
Returns a FileClassification with the decision and reason.
"""
tags = self._compute_tags(file_path)
# 1. User-facing overrides (highest priority — safety)
for pattern in self._user_overrides:
if _matches_glob(file_path, pattern):
return FileClassification(
path=file_path,
is_user_facing=True,
reason=f"User-facing override (matches '{pattern}')",
matched_rule="user_facing_overrides",
tags=tags,
)
# 2. Infrastructure overrides
if file_path in self._infra_overrides:
return FileClassification(
path=file_path,
is_user_facing=False,
reason="Infrastructure override (explicitly listed)",
matched_rule="infrastructure_overrides",
tags=tags,
)
# 3. Infrastructure patterns
for pattern in self._infra_patterns:
if _matches_glob(file_path, pattern):
return FileClassification(
path=file_path,
is_user_facing=False,
reason=f"Infrastructure (matches '{pattern}')",
matched_rule=f"infrastructure: {pattern}",
tags=tags,
)
# 4. Default: user-facing (safe-by-default)
return FileClassification(
path=file_path,
is_user_facing=True,
reason="User-facing (default — not in infrastructure patterns)",
matched_rule=None,
tags=tags,
)
def classify(self, files: list[str]) -> ClassificationResult:
"""Classify a list of changed files.
Returns a ClassificationResult with per-file details and
aggregated lists.
"""
result = ClassificationResult()
all_tags: dict[str, list[str]] = {}
for f in files:
fc = self.classify_file(f)
result.files.append(fc)
if fc.is_user_facing:
result.user_facing.append(f)
else:
result.infrastructure.append(f)
for tag in fc.tags:
all_tags.setdefault(tag, []).append(f)
result.tags = all_tags
return result
def _compute_tags(self, file_path: str) -> frozenset[str]:
"""Compute custom category tags for a file path."""
matched: set[str] = set()
for tag_name, patterns in self.config.tags.items():
for pattern in patterns:
if _matches_glob(file_path, pattern):
matched.add(tag_name)
break
return frozenset(matched)
# ---------------------------------------------------------------------------
# Git helpers
# ---------------------------------------------------------------------------
def run_git(args: list[str]) -> str:
@@ -126,27 +494,49 @@ def get_changed_files(base: str, head: str) -> list[str]:
return output.split("\n")
# ---------------------------------------------------------------------------
# Backward-compatible API (used by release.py and CI workflows)
# ---------------------------------------------------------------------------
# Singleton classifier — loaded lazily from pyproject.toml
_classifier: ChangeClassifier | None = None
def _get_classifier() -> ChangeClassifier:
"""Get or create the singleton classifier from pyproject.toml."""
global _classifier # noqa: PLW0603
if _classifier is None:
_classifier = ChangeClassifier()
return _classifier
def is_workflow_only(
file_path: str,
patterns: frozenset[str] | None = None,
) -> bool:
"""Check if a file path is workflow-only (infrastructure, not the tool itself).
"""Check if a file path is infrastructure (not user-facing).
Uses an explicit allowlist — anything not in the list is treated as
user-facing (safe default that prevents accidental release skips).
Backward-compatible API. Prefer ``ChangeClassifier.classify_file()``
for new code.
Args:
file_path: Path relative to repo root.
patterns: Deprecated. If provided, uses simple prefix matching
against these patterns instead of the config-driven classifier.
"""
p = patterns if patterns is not None else DEFAULT_WORKFLOW_ONLY_PATTERNS
return any(file_path.startswith(pattern) or file_path == pattern for pattern in p)
if patterns is not None:
# Legacy mode — simple prefix matching
return any(file_path.startswith(p) or file_path == p for p in patterns)
return not _get_classifier().classify_file(file_path).is_user_facing
def is_user_facing(
file_path: str,
patterns: frozenset[str] | None = None,
) -> bool:
"""Check if a file path is user-facing (affects the tool).
"""Check if a file path is user-facing (affects the released package).
Inverse of is_workflow_only — anything not explicitly workflow-only
is treated as user-facing.
Inverse of ``is_workflow_only()``.
"""
return not is_workflow_only(file_path, patterns)
@@ -159,14 +549,19 @@ def classify_changes(
Returns a dict with keys "user_facing" and "workflow_only".
"""
user_facing: list[str] = []
workflow_only: list[str] = []
for f in files:
if is_user_facing(f, patterns):
user_facing.append(f)
else:
workflow_only.append(f)
return {"user_facing": user_facing, "workflow_only": workflow_only}
if patterns is not None:
# Legacy mode
user_facing: list[str] = []
workflow_only: list[str] = []
for f in files:
if is_user_facing(f, patterns):
user_facing.append(f)
else:
workflow_only.append(f)
return {"user_facing": user_facing, "workflow_only": workflow_only}
result = _get_classifier().classify(files)
return {"user_facing": result.user_facing, "workflow_only": result.infrastructure}
def has_user_facing_changes(
@@ -176,35 +571,24 @@ def has_user_facing_changes(
) -> bool:
"""Check if any user-facing files changed between base and head.
Imported by ``devx.ci.release`` to decide whether a release
is needed. This is a cross-CI import that requires ``PYTHONPATH=.``.
Imported by ``devx.ci.release`` to decide whether a release is needed.
"""
files = get_changed_files(base, head)
return any(is_user_facing(f, patterns) for f in files)
if patterns is not None:
return any(is_user_facing(f, patterns) for f in files)
return _get_classifier().classify(files).has_user_facing
def get_latest_tag() -> str:
"""Get the latest git tag, or empty string if none exists."""
result = subprocess.run( # nosec B603 B607
["git", "describe", "--tags", "--abbrev=0"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return ""
return result.stdout.strip()
# ---------------------------------------------------------------------------
# Gitea Actions output
# ---------------------------------------------------------------------------
def _write_github_output(key: str, value: str) -> None:
"""Append a key=value line to the $GITHUB_OUTPUT file."""
import os
gh_output = os.environ.get("GITHUB_OUTPUT")
if not gh_output:
raise click.ClickException("GITHUB_OUTPUT environment variable is not set")
with open(gh_output, "a") as f: # noqa: PTH123
f.write(f"{key}={value}\n")
# ---------------------------------------------------------------------------
# Classification logic
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
@click.command()
@@ -213,24 +597,48 @@ def _write_github_output(key: str, value: str) -> None:
@click.option("--quiet", is_flag=True, default=False, help="Only output true/false.")
@click.option(
"--check",
type=click.Choice(["all", "ansible", "user-facing"]),
default="all",
help="Check specific category: all (default), ansible, or user-facing.",
help="Check specific category: 'all' (default), 'user-facing', or any tag name "
"defined in [tool.devx.classify.tags] (e.g., 'ansible').",
)
@click.option(
"--github-output",
"github_output",
is_flag=True,
default=False,
help="Write results to $GITHUB_OUTPUT file (for CI workflow steps).",
help="Write results to $GITHUB_OUTPUT file (for CI workflow steps). "
"Outputs 'user-facing-changed' and '<tag>-changed' for each configured tag.",
)
def main(base: str | None, head: str, quiet: bool, check: str, github_output: bool) -> None:
@click.option(
"--force",
is_flag=True,
default=False,
help="Force user-facing-changed=true regardless of actual changes. "
"Used by workflow_dispatch with force-deploy input.",
)
def main(base: str | None, head: str, quiet: bool, check: str, github_output: bool, force: bool) -> None:
"""Classify git changes and output results."""
classifier = _get_classifier()
available_tags = list(classifier.config.tags.keys())
# --force can also be activated via FORCE_DEPLOY env var (for workflow_dispatch)
if os.environ.get("FORCE_DEPLOY", "").lower() == "true":
force = True
if force and github_output:
write_github_output("user-facing-changed", "true")
for tag in available_tags:
write_github_output(f"{tag}-changed", "true")
click.echo("Forced user-facing-changed=true via --force flag.")
return
if base is None:
base = get_latest_tag()
if not base:
if github_output:
_write_github_output("ansible-changed", "true")
_write_github_output("user-facing-changed", "true")
write_github_output("user-facing-changed", "true")
for tag in available_tags:
write_github_output(f"{tag}-changed", "true")
click.echo("No tags found — treating all changes as user-facing.")
return
if quiet:
@@ -242,8 +650,9 @@ def main(base: str | None, head: str, quiet: bool, check: str, github_output: bo
files = get_changed_files(base, head)
if not files:
if github_output:
_write_github_output("ansible-changed", "false")
_write_github_output("user-facing-changed", "false")
write_github_output("user-facing-changed", "false")
for tag in available_tags:
write_github_output(f"{tag}-changed", "false")
click.echo(f"No changes between {base} and {head}.")
return
if quiet:
@@ -252,57 +661,65 @@ def main(base: str | None, head: str, quiet: bool, check: str, github_output: bo
click.echo(_("No changes between {base} and {head}.", base=base, head=head))
return
result = classifier.classify(files)
if github_output:
ansible_files = [f for f in files if f.startswith("ansible/") or f == ".ansible-lint"]
user_files = [f for f in files if is_user_facing(f)]
_write_github_output("ansible-changed", "true" if ansible_files else "false")
_write_github_output("user-facing-changed", "true" if user_files else "false")
click.echo(f"Ansible files changed: {bool(ansible_files)}")
click.echo(f"User-facing files changed: {bool(user_files)}")
write_github_output("user-facing-changed", "true" if result.has_user_facing else "false")
for tag in available_tags:
write_github_output(f"{tag}-changed", "true" if result.has_tag(tag) else "false")
click.echo(f"User-facing files changed: {result.has_user_facing}")
for tag in available_tags:
click.echo(f"{tag.capitalize()} files changed: {result.has_tag(tag)}")
return
if check == "ansible":
# Check only for Ansible-related file changes
ansible_files = [f for f in files if f.startswith("ansible/") or f == ".ansible-lint"]
has_ansible = bool(ansible_files)
# --check: check a specific tag or user-facing
if check != "all":
if check == "user-facing":
checked_files = result.user_facing
has_checked = bool(checked_files)
label = "User-facing"
elif check in available_tags:
checked_files = result.tags.get(check, [])
has_checked = bool(checked_files)
label = check.capitalize()
else:
raise click.ClickException(
_(
"Unknown check category '{check}'. Available: all, user-facing{tags}",
check=check,
tags=", " + ", ".join(available_tags) if available_tags else "",
)
)
if quiet:
click.echo("true" if has_ansible else "false")
click.echo("true" if has_checked else "false")
return
click.echo(_("\nAnsible files changed ({count}):", count=len(ansible_files)))
for f in ansible_files:
click.echo(f" {f}")
click.echo(_("\nResult: {status}", status="Ansible changes detected" if has_ansible else "No Ansible changes"))
return
if check == "user-facing":
# Check only for user-facing file changes (inverse of workflow-only)
user_files = [f for f in files if is_user_facing(f)]
has_user = bool(user_files)
if quiet:
click.echo("true" if has_user else "false")
return
click.echo(_("\nUser-facing files changed ({count}):", count=len(user_files)))
for f in user_files:
click.echo(_("\n{label} files changed ({count}):", label=label, count=len(checked_files)))
for f in checked_files:
click.echo(f" {f}")
click.echo(
_("\nResult: {status}", status="User-facing changes detected" if has_user else "No user-facing changes")
_("\nResult: {status}", status=f"{label} changes detected" if has_checked else f"No {label} changes")
)
return
result = classify_changes(files)
has_user = bool(result["user_facing"])
has_user = result.has_user_facing
if quiet:
click.echo("true" if has_user else "false")
return
click.echo(_("Comparing {base}..{head} ({count} files changed)", base=base, head=head, count=len(files)))
click.echo(_("\nUser-facing changes ({count}):", count=len(result["user_facing"])))
for f in result["user_facing"]:
click.echo(_("\nUser-facing changes ({count}):", count=len(result.user_facing)))
for f in result.user_facing:
click.echo(f" {f}")
click.echo(_("\nWorkflow-only changes ({count}):", count=len(result["workflow_only"])))
for f in result["workflow_only"]:
click.echo(_("\nWorkflow-only changes ({count}):", count=len(result.infrastructure)))
for f in result.infrastructure:
click.echo(f" {f}")
for tag in available_tags:
tag_files = result.tags.get(tag, [])
if tag_files:
click.echo(_("\n{tag} files ({count}):", tag=tag.capitalize(), count=len(tag_files)))
for f in tag_files:
click.echo(f" {f}")
if has_user:
status = "USER-FACING changes detected — release needed"
else:
+7 -14
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
"""Detect whether the latest git commit is a release commit.
Release commits have the format ``release: vX.Y.Z [skip ci]``.
Release commits have the format ``release: vX.Y.Z``.
This script writes ``is-release=true`` or ``is-release=false`` to
``$GITHUB_OUTPUT`` for use in CI workflow conditionals.
@@ -12,12 +12,14 @@ Usage::
from __future__ import annotations
import os
import re
import subprocess # nosec B404
import click
from devx.ci._shared import write_github_output
from devx.i18n import _
RELEASE_RE = re.compile(r"^release: v\d+\.\d+\.\d+")
@@ -39,26 +41,17 @@ def is_release_commit(message: str) -> bool:
return bool(RELEASE_RE.match(message))
def write_github_output(key: str, value: str) -> None:
"""Append a key=value line to the $GITHUB_OUTPUT file."""
gh_output = os.environ.get("GITHUB_OUTPUT")
if not gh_output:
raise click.ClickException("GITHUB_OUTPUT environment variable is not set")
with open(gh_output, "a") as f: # noqa: PTH123
f.write(f"{key}={value}\n")
@click.command()
def main() -> None:
"""Detect if the latest commit is a release commit and set GITHUB_OUTPUT."""
msg = get_commit_message()
click.echo(f"Commit message: {msg}")
click.echo(_("Commit message: {msg}", msg=msg))
is_release = is_release_commit(msg)
write_github_output("is-release", "true" if is_release else "false")
if is_release:
click.echo("Release commit — skipping all post-merge jobs.")
click.echo(_("Release commit — skipping all post-merge jobs."))
else:
click.echo("Regular merge commit — running all post-merge jobs.")
click.echo(_("Regular merge commit — running all post-merge jobs."))
if __name__ == "__main__": # pragma: no cover
+26 -16
View File
@@ -29,7 +29,8 @@ import os
import click
import requests
from devx.config import GITEA_API_URL
from devx.config import GITEA_API_URL, REPO_NAME, REPO_OWNER
from devx.i18n import _
DEFAULT_MAX_RUNNERS = 3
@@ -39,7 +40,7 @@ def query_runners(api_url: str, token: str, owner: str, repo: str) -> int:
Returns the total count of active runners. If the API call fails
(e.g., no admin access for instance-level runners), falls back to
what we can see.
what we can see. Fallbacks are logged to stderr for debugging.
"""
headers = {"Authorization": f"token {token}"}
total = 0
@@ -54,8 +55,10 @@ def query_runners(api_url: str, token: str, owner: str, repo: str) -> int:
if r.status_code == 200:
data = r.json()
total += data.get("total_count", 0)
except (requests.RequestException, ValueError):
pass
else:
click.echo(_("Warning: repo-level runners query returned HTTP {status}", status=r.status_code), err=True)
except (requests.RequestException, ValueError) as e:
click.echo(_("Warning: repo-level runners query failed: {error}", error=e), err=True)
# 2. Organization-level runners
try:
@@ -67,8 +70,10 @@ def query_runners(api_url: str, token: str, owner: str, repo: str) -> int:
if r.status_code == 200:
data = r.json()
total += data.get("total_count", 0)
except (requests.RequestException, ValueError):
pass
else:
click.echo(_("Warning: org-level runners query returned HTTP {status}", status=r.status_code), err=True)
except (requests.RequestException, ValueError) as e:
click.echo(_("Warning: org-level runners query failed: {error}", error=e), err=True)
# 3. Instance-level runners (requires admin scope)
try:
@@ -80,8 +85,13 @@ def query_runners(api_url: str, token: str, owner: str, repo: str) -> int:
if r.status_code == 200:
data = r.json()
total += data.get("total_count", 0)
except (requests.RequestException, ValueError):
pass
elif r.status_code != 403: # 403 is expected without admin scope
click.echo(
_("Warning: instance-level runners query returned HTTP {status}", status=r.status_code),
err=True,
)
except (requests.RequestException, ValueError) as e:
click.echo(_("Warning: instance-level runners query failed: {error}", error=e), err=True)
return total
@@ -142,12 +152,12 @@ def main(
output_indices: bool,
github_output: bool,
) -> None:
token = os.environ.get("REPO_TOKEN", "")
token = os.environ.get("CI_GITEA_TOKEN", "")
if owner is None:
owner = os.environ.get("DEVX_REPO_OWNER", "oblachno-oss")
owner = os.environ.get("DEVX_REPO_OWNER", "") or REPO_OWNER
if repo is None:
repo = os.environ.get("DEVX_REPO_NAME", "devx")
repo = os.environ.get("DEVX_REPO_NAME", "") or REPO_NAME
count = get_runner_count(GITEA_API_URL, token, owner, repo)
indices = generate_indices(count)
@@ -156,11 +166,11 @@ def main(
gh_output = os.environ.get("GITHUB_OUTPUT")
if not gh_output:
raise click.ClickException("GITHUB_OUTPUT environment variable is not set")
with open(gh_output, "a") as f: # noqa: PTH123
with open(gh_output, "a", encoding="utf-8") as f: # noqa: PTH123
f.write(f"runner-count={count}\n")
f.write(f"runner-indices={json.dumps(indices)}\n")
click.echo(f"Runner count: {count}")
click.echo(f"Runner indices: {indices}")
click.echo(_("Runner count: {count}", count=count))
click.echo(_("Runner indices: {indices}", indices=indices))
return
if output_count:
@@ -172,8 +182,8 @@ def main(
return
# Default: output both as key=value pairs for CI consumption
click.echo(f"count={count}")
click.echo(f"indices={json.dumps(indices)}")
click.echo(_("count={count}", count=count))
click.echo(_("indices={indices}", indices=json.dumps(indices)))
if __name__ == "__main__": # pragma: no cover
+139
View File
@@ -0,0 +1,139 @@
#!/usr/bin/env python3
"""Distribute a list of files across N parallel runners using LPT scheduling.
Generic file-based test distribution for CI matrix jobs. Discovers files
matching a glob pattern, sorts them for deterministic ordering, then
assigns them to *max_runners* groups using LPT (Longest Processing Time
first) scheduling — files are weighted by size (as a proxy for test
runtime) and assigned to the runner with the least total weight.
The assigned group for *runner_index* is written to ``$GITHUB_ENV`` for
use by subsequent steps.
Usage::
python3 -m devx.ci.distribute_files \\
--pattern "tests/integration/test_*.py" \\
--runner-index 1 \\
--max-runners 3 \\
--github-env --skip-if-excess
"""
from __future__ import annotations
import glob
import os
import click
from devx.ci._shared import lpt_distribute, write_github_env
from devx.i18n import _
DEFAULT_MAX_RUNNERS = 3
def discover_files(pattern: str) -> list[str]:
"""Return sorted list of file paths matching *pattern*."""
return sorted(glob.glob(pattern))
def _file_weight(path: str) -> int:
"""Estimate a weight for a file based on its size in bytes.
Falls back to 1 if the file cannot be stat'd (e.g. in tests).
"""
try:
return max(1, os.path.getsize(path))
except OSError:
return 1
def distribute(files: list[str], max_runners: int) -> list[list[str]]:
"""Split *files* into *max_runners* balanced groups using LPT scheduling.
Files are weighted by size (as a proxy for runtime) and assigned to
the runner with the least total weight.
"""
weights = [_file_weight(f) for f in files]
return lpt_distribute(files, weights, max_runners)
def files_for_runner(files: list[str], runner_index: int, max_runners: int) -> list[str]:
"""Return the subset of files assigned to *runner_index* (0-based)."""
groups = distribute(files, max_runners)
if runner_index < 0 or runner_index >= len(groups):
raise click.ClickException(
_("Runner index {index} out of range (0..{max})", index=runner_index, max=max_runners - 1)
)
return groups[runner_index]
@click.command()
@click.option("--pattern", required=True, help="Glob pattern for files to distribute.")
@click.option(
"--runner-index",
type=int,
default=None,
help="One-based runner index. If omitted, prints all groups.",
)
@click.option(
"--max-runners",
type=int,
default=DEFAULT_MAX_RUNNERS,
show_default=True,
help="Total number of parallel runners.",
)
@click.option(
"--github-env",
is_flag=True,
default=False,
help="Write ASSIGNED_FILES and SKIP to $GITHUB_ENV.",
)
@click.option(
"--skip-if-excess",
is_flag=True,
default=False,
help="With --github-env: write SKIP=true when runner-index exceeds max-runners.",
)
def main(pattern: str, runner_index: int | None, max_runners: int, github_env: bool, skip_if_excess: bool) -> None:
files = discover_files(pattern)
if runner_index is None:
groups = distribute(files, max_runners)
for i, group in enumerate(groups):
labels = " ".join(group) if group else "(none)"
click.echo(_("Runner {i}: {labels}", i=i, labels=labels))
return
if skip_if_excess and github_env and runner_index > max_runners:
click.echo(
_(
"Skipping — runner index {runner_index} > max runners {max_runners}",
runner_index=runner_index,
max_runners=max_runners,
)
)
write_github_env("ASSIGNED_FILES", "")
write_github_env("SKIP", "true")
return
if runner_index < 1:
raise click.ClickException(
_("Runner index {runner_index} is out of range (must be >= 1)", runner_index=runner_index)
)
zero_based = runner_index - 1
assigned = files_for_runner(files, zero_based, max_runners)
encoded = "\n".join(assigned)
if github_env:
write_github_env("ASSIGNED_FILES", encoded)
write_github_env("SKIP", "false")
click.echo(_("Assigned {count} files to runner {runner_index}", count=len(assigned), runner_index=runner_index))
return
click.echo(encoded)
if __name__ == "__main__": # pragma: no cover
main()
+206
View File
@@ -0,0 +1,206 @@
#!/usr/bin/env python3
"""Distribute a list of items across N parallel runners using LPT scheduling.
Generic item distribution for CI matrix jobs. Items are read from a JSON
array on stdin (or from a file via --items-file), sorted for deterministic
ordering, then assigned to *max_runners* groups using LPT (Longest
Processing Time first) scheduling.
Each item is a string (e.g. an Ansible ``--limit`` pattern like
``observability`` or ``infra-314-vm``). Optionally, items can be objects
with ``{"id": "...", "weight": N}`` to provide explicit weights.
The assigned group for *runner_index* is written to ``$GITHUB_ENV`` as
``ASSIGNED_ITEMS`` (space-delimited) for use by subsequent steps.
Usage::
echo '["observability", "infra-314-vm"]' | \\
python3 -m devx.ci.distribute_items \\
--runner-index 1 --max-runners 3 \\
--github-env --skip-if-excess
# With weights:
echo '[{"id": "observability", "weight": 5}, {"id": "customer-1", "weight": 3}]' | \\
python3 -m devx.ci.distribute_items \\
--runner-index 1 --max-runners 3 --github-env
"""
from __future__ import annotations
import json
import sys
import click
from devx.ci._shared import lpt_distribute, write_github_env
from devx.i18n import _
DEFAULT_MAX_RUNNERS = 3
DEFAULT_WEIGHT = 1
def parse_items(raw: str) -> list[str]:
"""Parse a JSON array into a list of item identifier strings.
Accepts both plain string arrays (``["a", "b"]``) and object arrays
(``[{"id": "a", "weight": 2}]``). Returns just the identifier strings.
"""
data = json.loads(raw)
if not isinstance(data, list):
raise click.ClickException(_("Items input must be a JSON array, got {type}", type=type(data).__name__))
items: list[str] = []
for entry in data:
if isinstance(entry, str):
items.append(entry)
elif isinstance(entry, dict) and "id" in entry:
items.append(str(entry["id"]))
else:
raise click.ClickException(
_("Each item must be a string or an object with 'id', got {type}", type=type(entry).__name__)
)
return items
def parse_weighted_items(raw: str) -> tuple[list[str], list[int]]:
"""Parse a JSON array into (items, weights) lists.
For plain string arrays, all items get ``DEFAULT_WEIGHT``.
For object arrays, the ``weight`` field is used (default: ``DEFAULT_WEIGHT``).
"""
data = json.loads(raw)
if not isinstance(data, list):
raise click.ClickException(_("Items input must be a JSON array, got {type}", type=type(data).__name__))
items: list[str] = []
weights: list[int] = []
for entry in data:
if isinstance(entry, str):
items.append(entry)
weights.append(DEFAULT_WEIGHT)
elif isinstance(entry, dict) and "id" in entry:
items.append(str(entry["id"]))
weights.append(int(entry.get("weight", DEFAULT_WEIGHT)))
else:
raise click.ClickException(
_("Each item must be a string or an object with 'id', got {type}", type=type(entry).__name__)
)
return items, weights
def distribute(items: list[str], weights: list[int], max_runners: int) -> list[list[str]]:
"""Split *items* into *max_runners* balanced groups using LPT scheduling.
Items are sorted by weight (descending), then assigned to the runner
with the least total weight.
"""
return lpt_distribute(items, weights, max_runners)
def items_for_runner(items: list[str], weights: list[int], runner_index: int, max_runners: int) -> list[str]:
"""Return the subset of items assigned to *runner_index* (0-based)."""
groups = distribute(items, weights, max_runners)
if runner_index < 0 or runner_index >= len(groups):
raise click.ClickException(
_("Runner index {index} out of range (0..{max})", index=runner_index, max=max_runners - 1)
)
return groups[runner_index]
@click.command()
@click.option(
"--items-file",
type=click.Path(exists=True, file_okay=True, path_type=None),
default=None,
help="Read items from a JSON file instead of stdin.",
)
@click.option(
"--runner-index",
type=int,
default=None,
help="One-based runner index. If omitted, prints all groups.",
)
@click.option(
"--max-runners",
type=int,
default=DEFAULT_MAX_RUNNERS,
show_default=True,
help="Total number of parallel runners.",
)
@click.option(
"--github-env",
is_flag=True,
default=False,
help="Write ASSIGNED_ITEMS and SKIP to $GITHUB_ENV.",
)
@click.option(
"--skip-if-excess",
is_flag=True,
default=False,
help="With --github-env: write SKIP=true when runner-index exceeds max-runners.",
)
def main(
items_file: str | None,
runner_index: int | None,
max_runners: int,
github_env: bool,
skip_if_excess: bool,
) -> None:
# Read items from file or stdin
if items_file is not None:
with open(items_file, encoding="utf-8") as f: # noqa: PTH123
raw = f.read()
else:
raw = sys.stdin.read()
raw = raw.strip()
if not raw:
raw = "[]"
items, weights = parse_weighted_items(raw)
if runner_index is None:
groups = distribute(items, weights, max_runners)
for i, group in enumerate(groups):
labels = " ".join(group) if group else "(none)"
click.echo(_("Runner {i}: {labels}", i=i, labels=labels))
return
if skip_if_excess and github_env and runner_index > max_runners:
click.echo(
_(
"Skipping — runner index {runner_index} > max runners {max_runners}",
runner_index=runner_index,
max_runners=max_runners,
)
)
write_github_env("ASSIGNED_ITEMS", "")
write_github_env("SKIP", "true")
return
if runner_index < 1:
raise click.ClickException(
_("Runner index {runner_index} is out of range (must be >= 1)", runner_index=runner_index)
)
zero_based = runner_index - 1
assigned = items_for_runner(items, weights, zero_based, max_runners)
encoded = " ".join(assigned)
if github_env:
write_github_env("ASSIGNED_ITEMS", encoded)
write_github_env("SKIP", "false")
click.echo(
_(
"Assigned {count} items to runner {runner_index}: {encoded}",
count=len(assigned),
runner_index=runner_index,
encoded=encoded,
)
)
return
click.echo(encoded)
if __name__ == "__main__": # pragma: no cover
main()
+55 -17
View File
@@ -5,8 +5,12 @@ Parses Click commands from the CLI source code and checks if each command
has corresponding documentation in the wiki/docs. Reports missing
documentation as warnings and exits with non-zero if coverage is below 100%.
By default, checks the current repository's own source and docs directories.
When run from the devx package itself (development mode), it checks devx's
own files. When installed as a package, it checks the consuming repo's files.
Usage:
python3 -m devx.ci.doc_coverage [--docs-dir docs/] [--fail-on-missing]
python3 -m devx.ci.doc_coverage [--docs-dir docs/] [--source-dir src/] [--fail-on-missing]
"""
from __future__ import annotations
@@ -19,11 +23,12 @@ import click
from devx.i18n import _
REPO_ROOT = Path(__file__).resolve().parent.parent.parent.parent
# Default to the current working directory (consuming repo's root)
REPO_ROOT = Path.cwd()
DOCS_DIR = REPO_ROOT / "docs"
CLI_FILE = REPO_ROOT / "src" / "devx" / "cli.py"
# Major modules that should be documented in tech/architecture.md
# These are devx-specific; when checking other repos, use --source-dir
REQUIRED_MODULES = [
"cli.py",
"i18n.py",
@@ -51,11 +56,16 @@ REQUIRED_SCRIPTS = [
]
def extract_cli_commands() -> list[str]:
def extract_cli_commands(source_dir: Path) -> list[str]:
"""Extract command names from the CLI source file."""
if not CLI_FILE.exists():
# Try to find the CLI file in the source directory
cli_file = None
for candidate in source_dir.rglob("cli.py"):
cli_file = candidate
break
if cli_file is None or not cli_file.exists():
return []
content = CLI_FILE.read_text()
content = cli_file.read_text()
commands: list[str] = []
# Find all @<group>.command("name") occurrences in the CLI source
# Matches @cli.command, @ci.command, @tools.command, @molecule.command
@@ -94,15 +104,30 @@ def check_module_documented(module: str, docs_content: str) -> bool:
@click.command()
@click.option("--docs-dir", default=str(DOCS_DIR), help="Path to the docs directory.")
@click.option("--docs-dir", default=None, help="Path to the docs directory (default: ./docs).")
@click.option("--source-dir", default=None, help="Path to the source directory (default: auto-detect from src/).")
@click.option(
"--fail-on-missing",
is_flag=True,
default=False,
help="Exit with non-zero status if any documentation is missing.",
)
def main(docs_dir: str, fail_on_missing: bool) -> None:
docs_path = Path(docs_dir)
def main(docs_dir: str | None, source_dir: str | None, fail_on_missing: bool) -> None:
root = Path.cwd()
docs_path = Path(docs_dir) if docs_dir else root / "docs"
# Auto-detect source directory
if source_dir:
src_path = Path(source_dir)
else:
# Try common source directories
for candidate in [root / "src", root / "scripts"]:
if candidate.exists():
src_path = candidate
break
else:
src_path = root / "src"
cli_commands_file = docs_path / "user" / "cli-commands.md"
architecture_file = docs_path / "tech" / "architecture.md"
ci_cd_file = docs_path / "tech" / "ci-cd-workflow.md"
@@ -112,21 +137,28 @@ def main(docs_dir: str, fail_on_missing: bool) -> None:
# Check CLI commands
click.echo(_("Checking CLI command documentation..."))
commands = extract_cli_commands()
commands = extract_cli_commands(src_path)
total += len(commands)
cli_docs = cli_commands_file.read_text() if cli_commands_file.exists() else ""
for cmd in commands:
if check_command_documented(cmd, cli_docs):
click.echo(_(" OK: devx {cmd}", cmd=cmd))
click.echo(_(" OK: {cmd}", cmd=cmd))
else:
click.echo(_(" MISSING: devx {cmd}", cmd=cmd))
missing.append(f"CLI command: devx {cmd}")
click.echo(_(" MISSING: {cmd}", cmd=cmd))
missing.append(f"CLI command: {cmd}")
# Check modules in architecture.md
# Auto-detect modules from source directory (top-level only, exclude subdirs)
click.echo(_("\nChecking module documentation in architecture.md..."))
total += len(REQUIRED_MODULES)
if src_path.exists():
detected_modules = sorted(
f.name for f in src_path.glob("*.py") if f.name != "__init__.py" and f.name != "cli.py"
)
else:
detected_modules = REQUIRED_MODULES
total += len(detected_modules)
arch_docs = architecture_file.read_text() if architecture_file.exists() else ""
for module in REQUIRED_MODULES:
for module in detected_modules:
if check_module_documented(module, arch_docs):
click.echo(_(" OK: {module}", module=module))
else:
@@ -134,10 +166,16 @@ def main(docs_dir: str, fail_on_missing: bool) -> None:
missing.append(f"Module: {module}")
# Check CI scripts in ci-cd-workflow.md
# Auto-detect CI scripts from ci/ subdirectory
click.echo(_("\nChecking CI script documentation in ci-cd-workflow.md..."))
total += len(REQUIRED_SCRIPTS)
ci_dir = src_path / "ci" if src_path.name != "ci" else src_path
if ci_dir.exists():
detected_scripts = sorted(f.name for f in ci_dir.glob("*.py") if f.name != "__init__.py")
else:
detected_scripts = REQUIRED_SCRIPTS
total += len(detected_scripts)
ci_docs = ci_cd_file.read_text() if ci_cd_file.exists() else ""
for script in REQUIRED_SCRIPTS:
for script in detected_scripts:
if check_module_documented(script, ci_docs):
click.echo(_(" OK: {script}", script=script))
else:
+127
View File
@@ -0,0 +1,127 @@
#!/usr/bin/env python3
"""Run integration tests with cross-runner failure detection.
Wraps ``pytest`` with the same Gitea API polling mechanism used by
``molecule_ci_guard``. If any other integration-tests matrix runner
reports failure, the current pytest subprocess is killed and this runner
exits early with code 1.
Usage::
python3 -m devx.ci.integration_guard \\
-- test_file1.py test_file2.py
# With pytest options
python3 -m devx.ci.integration_guard \\
-- -x -v --tb=short test_file1.py
Environment variables:
GITEA_URL Base URL of the Gitea instance.
CI_GITEA_TOKEN API token with repo access.
RUN_ID Workflow run ID (GITHUB_RUN_ID).
JOB_NAME Base job name (GITHUB_JOB), e.g. "integration-tests".
MATRIX_INDEX Current matrix index (runner-index).
GITEA_REPOSITORY Repository in "owner/repo" format.
"""
from __future__ import annotations
import contextlib
import os
import signal
import subprocess # nosec B404
import sys
import threading
import time
import click
from devx.config import REPO_NAME, REPO_OWNER
from devx.i18n import _
from devx.molecule.molecule_ci_guard import (
poll_for_other_failures,
)
POLL_INTERVAL = 10
@click.command(context_settings={"ignore_unknown_options": True})
@click.argument("pytest_args", nargs=-1, type=click.UNPROCESSED, required=True)
def cli(pytest_args: tuple[str, ...]) -> None:
"""Run pytest with cross-runner failure detection."""
gitea_url = os.environ.get("GITEA_URL", "")
token = os.environ.get("CI_GITEA_TOKEN", "")
run_id = int(os.environ.get("RUN_ID", "0"))
job_name = os.environ.get("JOB_NAME", "integration-tests")
current_index = int(os.environ.get("MATRIX_INDEX", "0"))
repository = os.environ.get("GITEA_REPOSITORY", "")
owner, _sep, repo = repository.partition("/")
if not owner or not repo:
owner, repo = REPO_OWNER, REPO_NAME
if not all([gitea_url, token, run_id]):
click.echo(_("GITEA_URL/CI_GITEA_TOKEN/RUN_ID not set; running without cross-runner cancellation."))
stop_event = threading.Event()
failed_event = threading.Event()
if gitea_url and token and run_id:
poller = threading.Thread(
target=poll_for_other_failures,
args=(
gitea_url,
owner,
repo,
token,
run_id,
job_name,
current_index,
stop_event,
failed_event,
),
daemon=True,
)
poller.start()
cmd = [sys.executable, "-m", "pytest"]
cmd.extend(pytest_args)
click.echo(_("Running: {cmd}", cmd=" ".join(cmd)))
process = subprocess.Popen( # nosec B603
cmd,
preexec_fn=os.setsid,
)
try:
while process.poll() is None:
if failed_event.is_set():
with contextlib.suppress(ProcessLookupError):
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
with contextlib.suppress(ProcessLookupError):
os.killpg(os.getpgid(process.pid), signal.SIGKILL)
process.wait()
click.echo(_("Integration tests cancelled — another runner failed."))
sys.exit(1)
time.sleep(1)
except KeyboardInterrupt:
with contextlib.suppress(ProcessLookupError):
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
process.wait()
sys.exit(1)
finally:
stop_event.set()
rc = process.returncode
if rc != 0:
click.echo(_("Integration tests failed with exit code {code}", code=rc))
else:
click.echo(_("Integration tests passed."))
sys.exit(rc)
if __name__ == "__main__": # pragma: no cover
cli()
+415
View File
@@ -0,0 +1,415 @@
#!/usr/bin/env python3
"""Lint documentation files for structure, links, and quality.
Checks performed (all configurable via pyproject.toml ``[tool.devx.docs]``):
- **Required files**: README.md, AGENTS.md, CHANGELOG.md must exist.
- **Docs structure**: ``docs/index.md`` and ``docs/mapping.json`` must exist.
- **Broken internal links**: relative paths and anchors in markdown files
must resolve to actual files and headings.
- **Heading hierarchy**: no skipping heading levels (e.g., ``#`` → ``###``).
- **TODO/FIXME**: flags leftover TODO/FIXME markers in documentation.
- **Stale docs**: files not modified in >180 days (warning only).
- **Trailing whitespace**: lines should not end with whitespace.
- **Blank line before headings**: headings should have a blank line before them.
Usage::
python3 -m devx.ci.lint_docs
python3 -m devx.ci.lint_docs --docs-dir docs/ --root .
python3 -m devx.ci.lint_docs --fix # auto-fix trailing whitespace
"""
from __future__ import annotations
import json
import re
import sys
from datetime import datetime, timedelta
from pathlib import Path
import click
from devx.i18n import _
# Heading slug pattern (GitHub-style)
_HEADING_RE = re.compile(r"^(#{1,6})\s+(.+?)\s*$", re.MULTILINE)
# Markdown link pattern: [text](url)
_LINK_RE = re.compile(r"\[([^\]]*)\]\(([^)]+)\)")
# Trailing whitespace
_TRAILING_WS_RE = re.compile(r"[ \t]+$")
# Heading without blank line before
_HEADING_NO_BLANK_RE = re.compile(r"([^\n])\n(#{1,6}\s)")
# Files that must exist in every project
REQUIRED_FILES = ["README.md", "AGENTS.md", "CHANGELOG.md"]
# Files that must exist in docs/
REQUIRED_DOC_FILES = ["index.md"]
# Maximum age for docs before they're considered stale (days)
STALE_THRESHOLD_DAYS = 180
# Files excluded from duplicate heading checks (auto-generated or structured
# with repeated subsections under different parent sections)
DUPLICATE_HEADING_EXCLUDES = {
"CHANGELOG.md",
"incident-response-sso.md",
"role-sync-design.md",
}
# TODO/FIXME pattern — matches "TODO:" or "FIXME:" at start of line/after whitespace
# Does NOT match references to the word "TODO" in rules/documentation
_TODO_RE = re.compile(r"(?m)^\s*(?:>>>?\s*)?(TODO|FIXME|HACK|XXX)\s*:", re.IGNORECASE)
# Directories excluded from markdown file scanning
_EXCLUDE_DIRS = {
".venv",
".git",
"node_modules",
"__pycache__",
".pytest_cache",
".devin",
".terraform",
"site-packages",
"dist-info",
}
def slugify(text: str) -> str:
"""Convert heading text to a GitHub-style slug."""
slug = text.lower().strip()
slug = re.sub(r"[^\w\s-]", "", slug)
slug = re.sub(r"[\s]+", "-", slug)
return slug
def strip_code_blocks(content: str) -> str:
"""Remove fenced code blocks from markdown content.
Replaces ```...``` blocks with empty lines so heading detection
doesn't pick up # comments inside code blocks.
"""
result: list[str] = []
in_code_block = False
for line in content.splitlines():
if line.strip().startswith("```"):
in_code_block = not in_code_block
result.append("")
continue
if in_code_block:
result.append("")
continue
result.append(line)
return "\n".join(result)
def extract_headings(filepath: Path) -> dict[str, int]:
"""Extract all headings from a markdown file.
Returns a dict mapping slug → heading level.
"""
content = strip_code_blocks(filepath.read_text(encoding="utf-8"))
headings: dict[str, int] = {}
for match in _HEADING_RE.finditer(content):
level = len(match.group(1))
text = match.group(2)
slug = slugify(text)
headings[slug] = level
return headings
def extract_links(filepath: Path) -> list[tuple[int, str, str]]:
"""Extract all markdown links from a file.
Returns a list of (line_number, link_text, url) tuples.
Includes anchor-only links (#section) for validation.
Skips external links (http/https) and mailto.
"""
content = filepath.read_text(encoding="utf-8")
links: list[tuple[int, str, str]] = []
for match in _LINK_RE.finditer(content):
url = match.group(2).strip()
# Skip external links and mailto
if url.startswith(("http://", "https://", "mailto:")):
continue
line_num = content[: match.start()].count("\n") + 1
links.append((line_num, match.group(1), url))
return links
def check_required_files(root: Path) -> list[str]:
"""Check that required files exist."""
issues: list[str] = []
for filename in REQUIRED_FILES:
if not (root / filename).exists():
issues.append(f"Missing required file: {filename}")
return issues
def check_docs_structure(root: Path, docs_dir: Path) -> list[str]:
"""Check that docs directory has required structure."""
issues: list[str] = []
if not docs_dir.exists():
issues.append(f"Docs directory not found: {docs_dir}")
return issues
for filename in REQUIRED_DOC_FILES:
if not (docs_dir / filename).exists():
issues.append(f"Missing required doc file: docs/{filename}")
mapping_file = docs_dir / "mapping.json"
if mapping_file.exists():
try:
mapping = json.loads(mapping_file.read_text(encoding="utf-8"))
if not isinstance(mapping, dict):
issues.append("docs/mapping.json must be a JSON object")
elif not mapping:
issues.append("docs/mapping.json is empty")
except json.JSONDecodeError as e:
issues.append(f"docs/mapping.json is invalid JSON: {e}")
return issues
def check_internal_links(root: Path, docs_dir: Path) -> list[str]:
"""Check that all internal links in markdown files resolve."""
issues: list[str] = []
md_files = list(root.rglob("*.md"))
# Exclude .venv, .git, node_modules
md_files = [f for f in md_files if not any(part in _EXCLUDE_DIRS for part in f.parts)]
# Load wiki page names from mapping.json — these are valid link targets
wiki_pages: set[str] = set()
mapping_file = docs_dir / "mapping.json"
if mapping_file.exists():
try:
mapping = json.loads(mapping_file.read_text(encoding="utf-8"))
wiki_pages = set(mapping.values())
except (json.JSONDecodeError, AttributeError):
pass
for md_file in md_files:
rel_path = md_file.relative_to(root)
links = extract_links(md_file)
headings = extract_headings(md_file)
for line_num, _link_text, url in links:
# Split into path and anchor
if "#" in url:
path_part, anchor = url.split("#", 1)
else:
path_part, anchor = url, ""
# Skip wiki page references (no file extension, no /, matches mapping.json values)
if path_part and "." not in path_part and "/" not in path_part:
if path_part in wiki_pages:
continue
# Also skip if it looks like a wiki page name (CamelCase or hyphenated)
# without a file extension — can't verify these locally
if not any(c in path_part for c in "/\\"):
continue
# Resolve relative path
if path_part:
target = (md_file.parent / path_part).resolve()
if not target.exists():
issues.append(f"{rel_path}:{line_num}: broken link '{url}' — file not found: {path_part}")
continue
# Check anchor in target file
if anchor:
target_headings = extract_headings(target)
target_slug = slugify(anchor)
if target_slug not in target_headings:
issues.append(f"{rel_path}:{line_num}: broken anchor '#{anchor}' in {path_part}")
elif anchor:
# Anchor-only link — check in current file
anchor_slug = slugify(anchor)
if anchor_slug not in headings:
issues.append(f"{rel_path}:{line_num}: broken anchor '#{anchor}'")
return issues
def check_heading_hierarchy(root: Path) -> list[str]:
"""Check that headings don't skip levels."""
issues: list[str] = []
md_files = [f for f in root.rglob("*.md") if not any(part in _EXCLUDE_DIRS for part in f.parts)]
for md_file in md_files:
rel_path = md_file.relative_to(root)
content = strip_code_blocks(md_file.read_text(encoding="utf-8"))
prev_level = 0
for match in _HEADING_RE.finditer(content):
level = len(match.group(1))
if prev_level > 0 and level > prev_level + 1:
issues.append(f"{rel_path}: heading hierarchy skip — H{prev_level} → H{level}: '{match.group(2)}'")
prev_level = level
return issues
def check_todo_fixme(root: Path) -> list[str]:
"""Check for TODO/FIXME/HACK/XXX markers in documentation.
Only flags actual TODO/FIXME markers (e.g., "TODO: fix this"), not
references to the word "TODO" in rules or documentation about TODOs.
"""
issues: list[str] = []
md_files = [f for f in root.rglob("*.md") if not any(part in _EXCLUDE_DIRS for part in f.parts)]
for md_file in md_files:
rel_path = md_file.relative_to(root)
content = md_file.read_text(encoding="utf-8")
for match in _TODO_RE.finditer(content):
line_num = content[: match.start()].count("\n") + 1
line = content.splitlines()[line_num - 1] if line_num <= len(content.splitlines()) else ""
issues.append(f"{rel_path}:{line_num}: TODO/FIXME found: {line.strip()}")
return issues
def check_trailing_whitespace(root: Path) -> list[str]:
"""Check for trailing whitespace in markdown files."""
issues: list[str] = []
md_files = [f for f in root.rglob("*.md") if not any(part in _EXCLUDE_DIRS for part in f.parts)]
for md_file in md_files:
rel_path = md_file.relative_to(root)
content = md_file.read_text(encoding="utf-8")
for i, line in enumerate(content.splitlines(), 1):
if _TRAILING_WS_RE.search(line):
issues.append(f"{rel_path}:{i}: trailing whitespace")
return issues
def check_stale_docs(root: Path) -> list[str]:
"""Check for stale documentation (not modified in >180 days)."""
issues: list[str] = []
threshold = datetime.now() - timedelta(days=STALE_THRESHOLD_DAYS)
md_files = [f for f in root.rglob("*.md") if not any(part in _EXCLUDE_DIRS for part in f.parts)]
for md_file in md_files:
rel_path = md_file.relative_to(root)
mtime = datetime.fromtimestamp(md_file.stat().st_mtime)
if mtime < threshold:
days_old = (datetime.now() - mtime).days
issues.append(f"{rel_path}: stale doc — not modified in {days_old} days")
return issues
def check_duplicate_headings(root: Path) -> list[str]:
"""Check for duplicate headings within the same file."""
issues: list[str] = []
md_files = [f for f in root.rglob("*.md") if not any(part in _EXCLUDE_DIRS for part in f.parts)]
for md_file in md_files:
rel_path = md_file.relative_to(root)
# Skip auto-generated files like CHANGELOG.md
if md_file.name in DUPLICATE_HEADING_EXCLUDES:
continue
content = strip_code_blocks(md_file.read_text(encoding="utf-8"))
seen: dict[str, int] = {}
for match in _HEADING_RE.finditer(content):
text = match.group(2)
slug = slugify(text)
if slug in seen:
issues.append(f"{rel_path}: duplicate heading '{text}'")
seen[slug] = 1
return issues
@click.command()
@click.option("--root", default=".", help="Repository root directory.")
@click.option("--docs-dir", default=None, help="Docs directory (default: <root>/docs).")
@click.option("--check-links/--no-check-links", default=True, help="Check internal links.")
@click.option("--check-headings/--no-check-headings", default=True, help="Check heading hierarchy.")
@click.option("--check-todo/--no-check-todo", default=True, help="Check for TODO/FIXME.")
@click.option("--check-stale/--no-check-stale", default=False, help="Check for stale docs.")
@click.option("--check-trailing/--no-check-trailing", default=True, help="Check trailing whitespace.")
@click.option("--check-duplicates/--no-check-duplicates", default=True, help="Check duplicate headings.")
@click.option("--fix", is_flag=True, default=False, help="Auto-fix trailing whitespace.")
def main(
root: str,
docs_dir: str | None,
check_links: bool,
check_headings: bool,
check_todo: bool,
check_stale: bool,
check_trailing: bool,
check_duplicates: bool,
fix: bool,
) -> None:
"""Lint documentation files for structure, links, and quality."""
root_path = Path(root).resolve()
docs_path = Path(docs_dir) if docs_dir else root_path / "docs"
click.echo(_("Linting documentation in {root}...", root=str(root_path)))
all_issues: list[str] = []
# Structure checks
click.echo(_("Checking required files..."))
all_issues.extend(check_required_files(root_path))
click.echo(_("Checking docs structure..."))
all_issues.extend(check_docs_structure(root_path, docs_path))
# Link checks
if check_links:
click.echo(_("Checking internal links..."))
all_issues.extend(check_internal_links(root_path, docs_path))
# Heading hierarchy
if check_headings:
click.echo(_("Checking heading hierarchy..."))
all_issues.extend(check_heading_hierarchy(root_path))
# Duplicate headings
if check_duplicates:
click.echo(_("Checking duplicate headings..."))
all_issues.extend(check_duplicate_headings(root_path))
# TODO/FIXME
if check_todo:
click.echo(_("Checking for TODO/FIXME markers..."))
all_issues.extend(check_todo_fixme(root_path))
# Trailing whitespace
if check_trailing:
click.echo(_("Checking trailing whitespace..."))
ws_issues = check_trailing_whitespace(root_path)
if fix and ws_issues:
fixed = 0
md_files = [f for f in root_path.rglob("*.md") if not any(part in _EXCLUDE_DIRS for part in f.parts)]
for md_file in md_files:
content = md_file.read_text(encoding="utf-8")
fixed_content = _TRAILING_WS_RE.sub("", content)
if content != fixed_content:
md_file.write_text(fixed_content, encoding="utf-8")
fixed += 1
click.echo(_(" Auto-fixed trailing whitespace in {n} files", n=fixed))
else:
all_issues.extend(ws_issues)
# Stale docs
if check_stale:
click.echo(_("Checking for stale docs..."))
stale = check_stale_docs(root_path)
for issue in stale:
click.echo(f" WARN: {issue}")
# Stale docs are warnings, not errors
click.echo(_(" {n} stale docs found (warnings only)", n=len(stale)))
# Report
click.echo(f"\n{'=' * 60}")
if all_issues:
click.echo(_("FAIL: {n} documentation issues found:", n=len(all_issues)))
for issue in all_issues:
click.echo(f" - {issue}")
sys.exit(1)
else:
click.echo(_("PASS: All documentation checks passed!"))
if __name__ == "__main__": # pragma: no cover
main()
+32 -10
View File
@@ -6,46 +6,59 @@ otherwise go unnoticed in the Actions tab. Uses the ``tea`` Gitea CLI
for issue creation — tea must be installed and configured.
Usage:
REPO_TOKEN=<token> python3 -m devx.ci.notify_failure \
CI_GITEA_TOKEN=<token> python3 -m devx.ci.notify_failure \
--repo <owner/repo> \
--run-id <run_id> \
--workflow <workflow_name> \
--commit <commit_sha>
--commit <commit_sha> \
--auto-login
With ``--auto-login``, the script configures the tea CLI login profile
from ``CI_GITEA_TOKEN`` and ``DEVX_GITEA_API_URL`` before creating the issue,
eliminating the need for a separate ``tea login add`` step in the workflow.
"""
from __future__ import annotations
import contextlib
import logging
import os
import click
from dotenv import load_dotenv # pyright: ignore[reportMissingImports,reportUnknownVariableType]
from devx.config import GITEA_API_URL
from devx.gitea_cli import TeaCLI, TeaCLIError
from devx.gitea_cli import TeaCLI, TeaCLIError, configure_tea_login
from devx.i18n import _
load_dotenv()
logger = logging.getLogger("devx")
def _create_issue_via_tea(repo: str, title: str, body: str) -> int:
"""Create issue via tea CLI. Returns issue index.
Raises TeaCLIError if tea is not installed or the command fails.
Label operations are best-effort — failures are logged but don't
prevent issue creation.
"""
tea = TeaCLI(repo=repo)
# Check if "bug" label exists
# Check if "bug" label exists (best-effort)
labels: list[str] = []
with contextlib.suppress(TeaCLIError):
try:
existing_labels = tea.list_labels(repo)
if any(label.get("name") == "bug" for label in existing_labels):
labels = ["bug"]
except TeaCLIError as e:
logger.warning("Could not fetch labels (best-effort): %s", e)
issue = tea.create_issue(repo, title=title, body=body, labels=labels if labels else None)
if labels:
with contextlib.suppress(TeaCLIError):
try:
tea.add_label(repo, issue["index"], labels)
except TeaCLIError as e:
logger.warning("Could not add label to issue #%s (best-effort): %s", issue.get("index"), e)
return int(issue.get("index", 0))
@@ -54,10 +67,19 @@ def _create_issue_via_tea(repo: str, title: str, body: str) -> int:
@click.option("--run-id", required=True, help="CI run ID.")
@click.option("--workflow", required=True, help="Workflow name.")
@click.option("--commit", required=True, help="Commit SHA.")
def main(repo: str, run_id: str, workflow: str, commit: str) -> None:
token = os.environ.get("REPO_TOKEN", "")
@click.option(
"--auto-login",
is_flag=True,
default=False,
help="Configure tea CLI login from CI_GITEA_TOKEN before creating the issue.",
)
def main(repo: str, run_id: str, workflow: str, commit: str, auto_login: bool) -> None:
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
raise click.ClickException(_("ERROR: REPO_TOKEN is not set."))
raise click.ClickException(_("ERROR: CI_GITEA_TOKEN is not set."))
if auto_login:
configure_tea_login()
title = f"[CI] {workflow} workflow failed (run #{run_id})"
body = (
+18 -17
View File
@@ -13,7 +13,8 @@ import click
from dotenv import load_dotenv # pyright: ignore[reportMissingImports,reportUnknownVariableType]
from devx.api_clients import VikunjaClient
from devx.config import DEFAULT_PER_PAGE, TASK_ID_RE, VIKUNJA_API_URL, VIKUNJA_PROJECT_ID
from devx.ci._shared import extract_task_id as _extract_task_id
from devx.config import DEFAULT_PER_PAGE, TASK_PREFIX, VIKUNJA_API_URL, VIKUNJA_PROJECT_ID
from devx.exceptions import APIError
from devx.i18n import _
@@ -47,10 +48,9 @@ def _get_git_commit_sha() -> str:
def extract_task_id(commit_msg: str) -> str:
"""Extract DEVX-N task identifier from the first line of commit message."""
"""Extract task identifier from the first line of commit message (delegates to shared utility)."""
first_line = commit_msg.split("\n")[0]
match = TASK_ID_RE.search(first_line)
return match.group(0) if match else ""
return _extract_task_id(first_line)
def extract_conventional_msg(commit_msg: str) -> str:
@@ -61,7 +61,7 @@ def extract_conventional_msg(commit_msg: str) -> str:
- ``DEVX-N <message>`` (current, space-separated)
"""
first_line = commit_msg.split("\n")[0]
return re.sub(r"^DEVX-\d+[:\s]\s*", "", first_line)
return re.sub(rf"^{TASK_PREFIX}-\d+[:\s]\s*", "", first_line)
def resolve_task_id(client: VikunjaClient, task_id: str) -> int:
@@ -151,14 +151,16 @@ def main(commit_msg: str | None, commit_sha: str, from_git: bool, git_sha: str)
)
)
return
# Non-infrastructure commits without DEVX-N prefix — warn but don't fail
click.echo(
# Non-infrastructure commits without DEVX-N prefix — this is a
# convention violation. Fail the post-merge job so the issue is visible.
raise click.ClickException(
_(
"Warning: No task ID (DEVX-N) found in commit message: {msg}. Skipping Vikunja update.",
"No task ID ({prefix}-N) found in commit message: {msg}. "
"Every non-infrastructure commit must have a task ID.",
prefix=TASK_PREFIX,
msg=first_line,
)
)
return
client = VikunjaClient(VIKUNJA_API_URL, token)
vikunja_task_id = resolve_task_id(client, task_id)
@@ -170,19 +172,18 @@ def main(commit_msg: str | None, commit_sha: str, from_git: bool, git_sha: str)
client.post_comment(vikunja_task_id, html)
client.update_task(vikunja_task_id, done=True)
except APIError as e:
# Vikunja is a project management tool — if it's down, the merge
# still succeeded. Warn but don't fail the post-merge workflow.
click.echo(
# Vikunja API failures must be visible — the task was not updated
# and needs manual intervention. Failing the CI job makes this visible.
raise click.ClickException(
_(
"Warning: Vikunja API error (HTTP {status}): {message}. "
"Task {task_id} was NOT updated. The merge succeeded — "
"please update the Vikunja task manually.",
"Vikunja API error (HTTP {status}): {message}. "
"Task {task_id} was NOT updated. "
"The merge succeeded but the Vikunja task needs manual update.",
status=e.status,
message=e.message,
task_id=task_id,
)
)
return
) from e
click.echo(
_(
+119 -5
View File
@@ -17,7 +17,7 @@ Checks performed:
8. Commit conventions — conventional commit format on branch commits
Usage:
REPO_TOKEN=<token> python3 -m devx.ci.pr_review <pr_number> <owner/repo>
CI_GITEA_TOKEN=<token> python3 -m devx.ci.pr_review <pr_number> <owner/repo>
"""
from __future__ import annotations
@@ -387,14 +387,34 @@ def check_documentation(files: list[dict[str, Any]], result: ReviewResult) -> No
for f in files
)
has_ansible_changes = any(f.get("filename", "").startswith("ansible/") for f in files)
has_tofu_changes = any(f.get("filename", "").startswith("tofu/") for f in files)
has_workflow_changes = any(f.get("filename", "").startswith(".gitea/") for f in files)
# Check for TODO/FIXME in changed docs
todo_issues: list[str] = []
for f in files:
filename = f.get("filename", "")
if filename.endswith(".md") and filename.startswith(("docs/", "README", "AGENTS")):
# Can't check file content from PR API easily, but flag if patch adds TODO
patch = f.get("patch", "")
if patch and re.search(r"^\+.*\b(TODO|FIXME|HACK|XXX)\b", patch, re.IGNORECASE):
todo_issues.append(f"{filename}: new TODO/FIXME added in documentation")
if has_src_changes and not has_doc_changes:
result.add_summary("- Documentation: WARNING — source files changed but no docs updated")
elif has_ansible_changes and not has_doc_changes:
result.add_summary("- Documentation: WARNING — Ansible role changed but no docs updated")
elif has_tofu_changes and not has_doc_changes:
result.add_summary("- Documentation: WARNING — OpenTofu changes but no docs updated")
elif has_workflow_changes and not has_doc_changes:
result.add_summary("- Documentation: INFO — workflow changes (consider updating CI docs if behavior changed)")
else:
result.add_summary("- Documentation: OK")
if todo_issues:
for issue in todo_issues:
result.add_summary(f"- Documentation: WARNING — {issue}")
def check_test_coverage(files: list[dict[str, Any]], result: ReviewResult) -> None:
"""Check that tests are updated for source changes."""
@@ -520,19 +540,113 @@ def post_review(client: GiteaClient, pr_number: str, result: ReviewResult) -> di
return client.create_review(pr_number, event=event, body=body, comments=comments)
def _post_manual_review(
client: GiteaClient,
pr_number: str,
event: str,
body: str | None,
checklist_confirmed: bool,
checklist_categories: str | None,
dry_run: bool,
) -> None:
"""Post a manual review with validation for APPROVE events."""
if not body or len(body) < 50:
raise click.ClickException(_("Review body must be at least 50 characters."))
if event == "APPROVE":
if not checklist_confirmed:
raise click.ClickException(
_("--checklist-confirmed is required for APPROVE events."),
)
cats = [c.strip() for c in (checklist_categories or "").split(",") if c.strip()]
cat_nums: list[int] = []
for c in cats:
try:
cat_nums.append(int(c))
except ValueError:
raise click.ClickException(
_("Invalid checklist category: {cat}. Must be numbers.", cat=c),
) from None
if len(cat_nums) < 8:
raise click.ClickException(
_("--checklist-categories must list at least 8 of 13 categories. Got {count}.", count=len(cat_nums)),
)
click.echo(f"Manual review event: {event}")
click.echo(f"Body: {body[:80]}...")
if checklist_confirmed:
click.echo(f"Checklist confirmed: {checklist_categories}")
if dry_run:
click.echo("\n[dry-run] Review not posted.")
return
try:
review = client.create_review(pr_number, event=event, body=body)
except APIError as e:
if "approve" in e.message.lower() or "422" in str(e.status):
click.echo(_("Note: Self-approval not allowed. Posting COMMENT instead."))
review = client.create_review(pr_number, event="COMMENT", body=body)
else:
raise
review_id = review.get("id", "?")
click.echo(
_(
"\nReview #{review_id} posted on PR #{pr_number} with event '{event}'.",
review_id=review_id,
pr_number=pr_number,
event=event,
)
)
@click.command()
@click.argument("pr_number")
@click.argument("repo")
@click.option("--dry-run", is_flag=True, default=False, help="Print review without posting.")
def main(pr_number: str, repo: str, dry_run: bool) -> None:
"""Run automated PR review and post results to Gitea."""
token = os.environ.get("REPO_TOKEN", "")
@click.option(
"--event",
type=click.Choice(["APPROVE", "REQUEST_CHANGES", "COMMENT"], case_sensitive=False),
default=None,
help="Post a manual review with the given event (skips automated checks).",
)
@click.option("--body", default=None, help="Review body text (required with --event).")
@click.option(
"--checklist-confirmed",
is_flag=True,
default=False,
help="Attest that REVIEW_CHECKLIST.md categories were checked (required for APPROVE).",
)
@click.option(
"--checklist-categories",
default=None,
help="Comma-separated checklist category numbers (required for APPROVE, min 8 of 13).",
)
def main(
pr_number: str,
repo: str,
dry_run: bool,
event: str | None,
body: str | None,
checklist_confirmed: bool,
checklist_categories: str | None,
) -> None:
"""Run automated PR review and post results to Gitea.
Without --event: runs automated checks and posts COMMENT/REQUEST_CHANGES.
With --event: posts a manual review (skips automated checks).
"""
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
raise click.ClickException(_("ERROR: REPO_TOKEN is not set."))
raise click.ClickException(_("ERROR: CI_GITEA_TOKEN is not set."))
owner, repo_name = repo.split("/")
client = GiteaClient(GITEA_API_URL, token, owner, repo_name)
if event is not None:
_post_manual_review(client, pr_number, event.upper(), body, checklist_confirmed, checklist_categories, dry_run)
return
result = run_review(client, pr_number)
body = build_review_body(result)
+147 -30
View File
@@ -9,26 +9,27 @@ Publishing destinations (checked in order):
``DEVX_PYPI_REGISTRY_URL`` env var is set, or ``GITEA_API_URL``
is converted to a packages URL). Uses ``twine upload
--repository-url <url> -u <token> -p <token>`` with the
``REPO_TOKEN`` as both username and password.
``CI_GITEA_TOKEN`` as both username and password.
2. **Standard PyPI** — if ``PYPI_TOKEN`` is set. Uses the standard
``twine upload -u __token__ -p <token>`` flow.
3. **Skip** — if neither is configured, only the Gitea release is created.
Usage:
REPO_TOKEN=<token> [PYPI_TOKEN=<token>] python3 -m devx.ci.publish <tag> <repo>
REPO_TOKEN=<token> python3 -m devx.ci.publish <tag> <repo> --registry-url https://git.example.com/api/packages/owner/pypi
CI_GITEA_TOKEN=<token> [PYPI_TOKEN=<token>] python3 -m devx.ci.publish <tag> <repo>
CI_GITEA_TOKEN=<token> python3 -m devx.ci.publish <tag> <repo> --registry-url https://git.example.com/api/packages/owner/pypi
"""
import os
import shutil
import subprocess # nosec B404
import sys
from pathlib import Path
import click
from dotenv import load_dotenv # pyright: ignore[reportMissingImports,reportUnknownVariableType]
from devx.config import GITEA_API_URL
from devx.gitea_cli import TeaCLI, TeaCLIError
from devx.config import GITEA_API_URL, REPO_OWNER
from devx.gitea_cli import TeaCLI, TeaCLIError, configure_tea_login
from devx.i18n import _
load_dotenv()
@@ -60,6 +61,12 @@ def generate_release_notes(tag: str) -> str:
def build_package() -> None:
"""Build the Python package using python -m build."""
# Clean dist/ to avoid uploading stale packages from previous builds
# (Gitea PyPI returns 409 Conflict for already-published versions).
dist_dir = Path("dist")
if dist_dir.exists():
shutil.rmtree(dist_dir)
result = subprocess.run( # nosec B603
[sys.executable, "-m", "build"],
capture_output=True,
@@ -128,13 +135,21 @@ def publish_to_gitea_registry(registry_url: str, token: str) -> None:
check=False,
)
if result.returncode != 0:
raise click.ClickException(
_(
"Oops! Gitea PyPI registry publish failed:\n{stderr}",
stderr=result.stderr.strip(),
# Twine writes errors to stdout (not stderr), so check both.
combined = f"{result.stdout}\n{result.stderr}".strip()
# 409 Conflict means the package version is already published —
# this is not an error, just a sign we're re-running publish.
if "409" in combined or "Conflict" in combined:
click.echo(_("Gitea PyPI registry: {tag} already published — continuing.", tag=""))
else:
raise click.ClickException(
_(
"Oops! Gitea PyPI registry publish failed:\n{stderr}",
stderr=combined,
)
)
)
click.echo(_("Published to Gitea PyPI registry."))
else:
click.echo(_("Published to Gitea PyPI registry."))
def _default_gitea_registry_url() -> str:
@@ -150,13 +165,41 @@ def _default_gitea_registry_url() -> str:
base = base[: -len("/api/v1")]
elif base.endswith("/api"):
base = base[: -len("/api")]
owner = os.environ.get("DEVX_REPO_OWNER", "oblachno-oss")
owner = os.environ.get("DEVX_REPO_OWNER", "") or REPO_OWNER
return f"{base}/api/packages/{owner}/pypi"
def get_latest_tag() -> str | None:
"""Get the latest git tag, or None if no tags exist."""
try:
result = subprocess.run( # nosec
["git", "describe", "--tags", "--abbrev=0"],
capture_output=True,
text=True,
check=True,
)
return result.stdout.strip()
except subprocess.CalledProcessError:
return None
def is_release_commit(tag: str) -> bool:
"""Check if HEAD commit message starts with 'release: <tag>'."""
try:
result = subprocess.run( # nosec
["git", "log", "-1", "--format=%s"],
capture_output=True,
text=True,
check=True,
)
return result.stdout.strip().startswith(f"release: {tag}")
except subprocess.CalledProcessError:
return False
@click.command()
@click.argument("tag")
@click.argument("repo")
@click.argument("tag", required=False)
@click.argument("repo", required=False)
@click.option(
"--registry-url",
default=None,
@@ -164,10 +207,55 @@ def _default_gitea_registry_url() -> str:
"or a URL derived from GITEA_API_URL. When set, publishes to Gitea PyPI "
"instead of standard PyPI (unless PYPI_TOKEN is also set).",
)
def main(tag: str, repo: str, registry_url: str | None) -> None:
gitea_token = os.environ.get("REPO_TOKEN", "")
@click.option(
"--skip-build",
is_flag=True,
default=False,
help="Skip package build and PyPI publish (for non-Python repos that only "
"need a Gitea release with git-cliff notes).",
)
@click.option(
"--from-tag",
is_flag=True,
default=False,
help="Auto-detect latest tag and check if HEAD is a release commit. "
"Skips publish if no tag or HEAD is not a release commit for that tag.",
)
@click.option(
"--auto-login",
is_flag=True,
default=False,
help="Configure tea CLI login from CI_GITEA_TOKEN before creating the Gitea release. "
"Eliminates the need for a separate tea login step in containerized CI jobs.",
)
def main(
tag: str | None,
repo: str | None,
registry_url: str | None,
skip_build: bool,
from_tag: bool,
auto_login: bool,
) -> None:
if repo is None:
repo = os.environ.get("GITHUB_REPOSITORY", "")
if not repo:
raise click.ClickException(_("REPO argument is required (or set GITHUB_REPOSITORY env var)."))
if from_tag:
detected_tag = get_latest_tag()
if not detected_tag:
click.echo(_("No tag found — skipping publish."))
return
if not is_release_commit(detected_tag):
click.echo(_("HEAD is not a release commit for {tag} — skipping publish.", tag=detected_tag))
return
tag = detected_tag
click.echo(_("Publishing release {tag}...", tag=tag))
if not tag:
raise click.ClickException(_("Tag is required (or use --from-tag)."))
gitea_token = os.environ.get("CI_GITEA_TOKEN", "")
if not gitea_token:
raise click.ClickException(_("ERROR: REPO_TOKEN is not set."))
raise click.ClickException(_("ERROR: CI_GITEA_TOKEN is not set."))
pypi_token = os.environ.get("PYPI_TOKEN", "")
@@ -177,28 +265,57 @@ def main(tag: str, repo: str, registry_url: str | None) -> None:
if not registry_url:
registry_url = _default_gitea_registry_url()
build_package()
if not skip_build:
build_package()
if pypi_token:
# Standard PyPI flow takes precedence when PYPI_TOKEN is set
publish_to_pypi(pypi_token)
elif registry_url:
# Gitea PyPI registry flow
publish_to_gitea_registry(registry_url, gitea_token)
else:
click.echo(
_(
"PYPI_TOKEN not set and no registry URL configured — "
"skipping PyPI publish. No worries, we'll just create the Gitea release."
try:
if pypi_token:
# Standard PyPI flow takes precedence when PYPI_TOKEN is set
publish_to_pypi(pypi_token)
elif registry_url:
# Gitea PyPI registry flow
publish_to_gitea_registry(registry_url, gitea_token)
else:
click.echo(
_(
"PYPI_TOKEN not set and no registry URL configured — "
"skipping PyPI publish. No worries, we'll just create the Gitea release."
)
)
except click.ClickException as e:
click.echo(
_(
"PyPI publish failed (non-fatal — continuing to Gitea release):\n{error}",
error=str(e),
),
err=True,
)
)
else:
click.echo(_("--skip-build: skipping package build and PyPI publish."))
tea = TeaCLI(repo=repo)
if auto_login:
configure_tea_login()
# Check if release already exists (idempotent — avoids failure when
# called multiple times, e.g. by both post-merge and publish workflows)
try:
releases = tea.list_releases(repo)
if any(r.get("tag_name") == tag for r in releases):
click.echo(_("Gitea release {tag} already exists — skipping creation.", tag=tag))
return
except TeaCLIError:
pass # If listing fails, proceed to create
release_body = generate_release_notes(tag)
try:
tea.create_release(repo, tag=tag, title=tag, body=release_body)
except TeaCLIError as e:
if "already" in str(e).lower() and "release" in str(e).lower():
click.echo(_("Gitea release {tag} already exists — skipping creation.", tag=tag))
return
raise click.ClickException(_("Release creation failed: {error}", error=str(e))) from None
click.echo(
+60 -16
View File
@@ -17,15 +17,29 @@ Usage::
from __future__ import annotations
import contextlib
import os
import re
import subprocess # nosec B404
import sys
import time
from pathlib import Path
from typing import Any
import click
REPO_ROOT = Path(__file__).resolve().parent.parent.parent.parent
from devx.i18n import _
def _repo_root() -> Path:
"""Resolve repo root from GITHUB_WORKSPACE or cwd."""
workspace = os.environ.get("GITHUB_WORKSPACE")
if workspace:
path = Path(workspace)
if path.is_dir():
return path
return Path.cwd()
# Badge filenames that get pushed to the badges branch
BADGE_FILES = ["coverage.svg", "tests.svg", "docs.svg", "quality.svg", "version.svg", "python.svg"]
@@ -56,7 +70,7 @@ def fetch_latest_master(branch: str = "master") -> None:
"""
_run(["git", "fetch", "origin", branch]) # nosec B607
_run(["git", "reset", "--hard", f"origin/{branch}"]) # nosec B607
click.echo(f"Synced to latest origin/{branch}")
click.echo(_("Synced to latest origin/{branch}", branch=branch))
def generate_badges(output_dir: str) -> None:
@@ -64,8 +78,8 @@ def generate_badges(output_dir: str) -> None:
_run([sys.executable, "-m", "devx.tools.generate_badges", "--output-dir", output_dir])
badges = list(Path(output_dir).glob("*.svg"))
if not badges:
raise click.ClickException("No badge SVG files generated")
click.echo(f"Generated {len(badges)} badge files")
raise click.ClickException(_("No badge SVG files generated"))
click.echo(_("Generated {count} badge files", count=len(badges)))
def push_to_badges_branch(badges_dir: str) -> str:
@@ -87,12 +101,12 @@ def push_to_badges_branch(badges_dir: str) -> str:
_run(["git", "add", "./*.svg"]) # nosec B607
_run(["git", "commit", "--no-verify", "-m", "Update badges [skip ci]"]) # nosec B607
_run(["git", "push", "origin", "badges", "--force"]) # nosec B607
click.echo("Badges pushed to badges branch")
click.echo(_("Badges pushed to badges branch"))
# Get the commit SHA of the badges branch
result = _run_capture(["git", "rev-parse", "HEAD"]) # nosec B607
sha = result.stdout.strip()
click.echo(f"Badges commit SHA: {sha}")
click.echo(_("Badges commit SHA: {sha}", sha=sha))
return sha
@@ -114,7 +128,7 @@ def update_readme_with_badge_sha(badges_sha: str, repo_root: Path | None = None)
Switches back to master, replaces ``raw/branch/badges/`` URLs with
``raw/commit/<sha>/`` URLs, commits and pushes.
"""
root = repo_root or REPO_ROOT
root = repo_root or _repo_root()
# Switch back to master
_run(["git", "checkout", "master"]) # nosec B607
@@ -130,11 +144,11 @@ def update_readme_with_badge_sha(badges_sha: str, repo_root: Path | None = None)
new_content = update_badge_urls(content, badges_sha)
if new_content != content:
filepath.write_text(new_content)
click.echo(f"Updated badge URLs in {filename}")
click.echo(_("Updated badge URLs in {filename}", filename=filename))
updated_any = True
if not updated_any:
click.echo("No badge URLs found to update — README already up to date")
click.echo(_("No badge URLs found to update — README already up to date"))
return
_run(["git", "add", "README.md", "docs/index.md"]) # nosec B607
@@ -148,7 +162,7 @@ def update_readme_with_badge_sha(badges_sha: str, repo_root: Path | None = None)
]
) # nosec B607
_run(["git", "push", "origin", "master"]) # nosec B607
click.echo(f"Pushed README update with badge SHA {badges_sha[:8]}")
click.echo(_("Pushed README update with badge SHA {sha}", sha=badges_sha[:8]))
@click.command()
@@ -160,13 +174,43 @@ def update_readme_with_badge_sha(badges_sha: str, repo_root: Path | None = None)
default=False,
help="Skip updating README with cache-busting URLs (for local testing).",
)
def main(output_dir: str, branch: str, no_readme_update: bool) -> None:
@click.option(
"--retries",
default=1,
type=int,
help="Number of attempts on git push failures (default: 1, no retry). "
"Between attempts, fetches latest master and waits 10s.",
)
def main(output_dir: str, branch: str, no_readme_update: bool, retries: int) -> None:
"""Generate badges and push them to the badges branch."""
fetch_latest_master(branch)
generate_badges(output_dir)
badges_sha = push_to_badges_branch(output_dir)
if not no_readme_update:
update_readme_with_badge_sha(badges_sha)
last_error: Exception | None = None
for attempt in range(1, retries + 1):
try:
fetch_latest_master(branch)
generate_badges(output_dir)
badges_sha = push_to_badges_branch(output_dir)
if not no_readme_update:
update_readme_with_badge_sha(badges_sha)
return
except (subprocess.CalledProcessError, RuntimeError) as exc:
last_error = exc
if attempt < retries:
click.echo(
_(
"Badge push attempt {attempt}/{retries} failed — retrying: {error}",
attempt=attempt,
retries=retries,
error=exc,
)
)
time.sleep(10)
with contextlib.suppress(subprocess.CalledProcessError):
fetch_latest_master(branch)
else:
click.echo(_("Badge push failed after {retries} attempts: {error}", retries=retries, error=exc))
raise click.ClickException(
_("Badge push failed after {retries} attempts: {error}", retries=retries, error=last_error)
)
if __name__ == "__main__": # pragma: no cover
+445 -51
View File
@@ -23,19 +23,27 @@ This script is idempotent: if there are no new conventional commits since the
last tag, it exits with a message and does nothing. If the tag already exists
(e.g., from a partial previous run), it skips tag creation and only pushes.
**Tag consistency**: Before releasing, the script fetches remote tags and
verifies all existing tags point to commits whose message matches the tag
version. This prevents duplicate release commits (a common issue when CI
checkouts don't fetch tags) and ensures tag/version/commit alignment.
Usage:
REPO_TOKEN=<token> python3 -m devx.ci.release [--dry-run] [--skip-tests]
CI_GITEA_TOKEN=<token> python3 -m devx.ci.release [--dry-run] [--skip-tests]
python3 -m devx.ci.release --verify # Check tag/version/release alignment
"""
from __future__ import annotations
import os
import re
import subprocess # nosec B404
import sys
import time
import click
from dotenv import load_dotenv # pyright: ignore[reportMissingImports,reportUnknownVariableType]
from devx.ci._shared import get_latest_tag, run_cmd, write_github_output
from devx.ci.classify_changes import has_user_facing_changes # cross-CI import, needs PYTHONPATH=.
from devx.i18n import _
@@ -46,39 +54,91 @@ CHANGELOG_FILE = "CHANGELOG.md"
CLIFF_CONFIG = "cliff.toml"
def run_cmd(args: list[str], check: bool = True, capture: bool = True) -> subprocess.CompletedProcess[str]:
"""Run a command and return the completed process."""
result = subprocess.run( # nosec B603
args,
capture_output=capture,
text=True,
check=False,
)
if check and result.returncode != 0:
raise click.ClickException(
_(
"Command failed ({cmd}): {stderr}",
cmd=" ".join(args),
stderr=result.stderr.strip() if result.stderr else result.stdout.strip(),
)
)
return result
def get_latest_tag() -> str:
"""Get the latest git tag, or empty string if none exists."""
result = run_cmd(["git", "describe", "--tags", "--abbrev=0"], check=False)
if result.returncode != 0:
return ""
return result.stdout.strip()
def tag_exists(tag: str) -> bool:
"""Check if a git tag already exists."""
result = run_cmd(["git", "tag", "-l", tag], check=False)
return bool(result.stdout.strip())
def get_tag_commit(tag: str) -> str:
"""Get the commit hash a tag points to."""
result = run_cmd(["git", "rev-list", "-n1", tag], check=False)
return result.stdout.strip()
def get_head_commit() -> str:
"""Get the current HEAD commit hash."""
result = run_cmd(["git", "rev-parse", "HEAD"], check=False)
return result.stdout.strip()
def fetch_tags() -> None:
"""Fetch tags from remote to ensure local tag state is current.
This is critical in CI environments where a fresh checkout may not
include tags from previous runs. Without this, the script may
create duplicate release commits because ``tag_exists`` returns False
for a tag that exists on the remote but wasn't fetched.
"""
result = run_cmd(["git", "fetch", "--tags", "origin"], check=False)
if result.returncode != 0:
# Don't fail hard — maybe there's no remote (local-only repo)
click.echo(_("Warning: could not fetch tags from origin."))
def get_all_tags() -> list[str]:
"""Get all git tags sorted by version (newest first)."""
result = run_cmd(["git", "tag", "-l", "--sort=-v:refname"], check=False)
if result.returncode != 0:
return []
return [t.strip() for t in result.stdout.strip().split("\n") if t.strip()]
def get_commit_version(commit: str) -> str | None:
"""Extract version from a release commit message.
Returns the version string (e.g., '0.4.4') or None if the commit
is not a release commit.
"""
result = run_cmd(["git", "log", "-1", "--pretty=%s", commit], check=False)
match = re.match(r"^release: v(\d+\.\d+\.\d+)", result.stdout.strip())
return match.group(1) if match else None
def verify_tag_consistency() -> list[str]:
"""Verify all tags point to commits with matching version in message.
Returns a list of error messages for inconsistent tags.
An empty list means all tags are consistent.
The first release (v0.1.0 or earliest tag) is exempt — initial releases
often don't have a "release:" commit message (e.g., the initial commit
serves as the first release).
"""
errors: list[str] = []
tags = get_all_tags()
# Filter to version tags (vX.Y.Z) and sort oldest first
version_tags = [t for t in tags if re.match(r"^v\d+\.\d+\.\d+$", t)]
sorted_tags = sorted(version_tags, key=lambda t: [int(x) for x in t.lstrip("v").split(".")])
first_tag = sorted_tags[0] if sorted_tags else None
for tag in tags:
# Skip non-version tags (e.g., branch names like "master")
if not re.match(r"^v\d+\.\d+\.\d+$", tag):
continue
tag_version = tag.lstrip("v")
commit_version = get_commit_version(tag)
if commit_version is None:
# First tag is allowed to point to a non-release commit (initial release)
if tag == first_tag:
continue
errors.append(
f" {tag} → points to non-release commit (expected 'release: v{tag_version}', got non-release commit)"
)
elif commit_version != tag_version:
errors.append(f" {tag} → commit says 'release: v{commit_version}' (expected 'release: v{tag_version}')")
return errors
def get_bumped_version() -> str:
"""Use git-cliff to calculate the next version from conventional commits."""
result = run_cmd(["git-cliff", "--bumped-version", "--config", CLIFF_CONFIG])
@@ -86,7 +146,13 @@ def get_bumped_version() -> str:
if not version:
raise click.ClickException(_("git-cliff returned empty version."))
# git-cliff may return with or without 'v' prefix
return version.lstrip("v")
version = version.lstrip("v")
# Validate semver format
if not re.match(r"^\d+\.\d+\.\d+$", version):
raise click.ClickException(
_("git-cliff returned invalid version format: {version}. Expected semver (e.g., 0.4.1).", version=version)
)
return version
def get_changelog(new_version: str) -> str:
@@ -117,9 +183,12 @@ def has_unreleased_changes(bumped_version: str | None = None) -> bool:
latest = get_latest_tag()
if not latest:
return True
# Check for any commits since the last tag
# Check for any commits since the last tag, excluding release commits
# (release commits themselves are not "unreleased changes" — they ARE
# the release). This prevents duplicate release commits when the
# script runs multiple times.
result = run_cmd(
["git", "log", f"{latest}..HEAD", "--oneline"],
["git", "log", f"{latest}..HEAD", "--oneline", "--no-merges", "--invert-grep", "--grep=^release: v"],
check=False,
)
if result.returncode != 0:
@@ -129,7 +198,7 @@ def has_unreleased_changes(bumped_version: str | None = None) -> bool:
def update_init_version(new_version: str) -> None:
"""Update __version__ in __init__.py."""
with open(INIT_FILE) as f:
with open(INIT_FILE, encoding="utf-8") as f:
content = f.read()
if not re.search(r'^__version__\s*=\s*"[^"]*"', content, flags=re.MULTILINE):
raise click.ClickException(_("Could not find __version__ in {file}", file=INIT_FILE))
@@ -140,7 +209,7 @@ def update_init_version(new_version: str) -> None:
count=1,
flags=re.MULTILINE,
)
with open(INIT_FILE, "w") as f:
with open(INIT_FILE, "w", encoding="utf-8") as f:
f.write(updated)
@@ -157,10 +226,10 @@ def update_changelog(changelog: str) -> None:
changelog = changelog[section_match.start() :]
try:
with open(CHANGELOG_FILE) as f:
with open(CHANGELOG_FILE, encoding="utf-8") as f:
existing = f.read()
except FileNotFoundError:
with open(CHANGELOG_FILE, "w") as f:
with open(CHANGELOG_FILE, "w", encoding="utf-8") as f:
f.write(changelog + "\n")
return
@@ -173,7 +242,7 @@ def update_changelog(changelog: str) -> None:
else:
# No version sections found — append
updated = existing.rstrip() + "\n\n" + changelog + "\n"
with open(CHANGELOG_FILE, "w") as f:
with open(CHANGELOG_FILE, "w", encoding="utf-8") as f:
f.write(updated)
@@ -229,27 +298,231 @@ def run_tests() -> None:
click.echo(_("Tests passed."))
def _write_release_tag(tag: str) -> None:
"""Write the release tag to GITHUB_OUTPUT for downstream jobs.
This allows a publish job (needs: release) to read the tag via
``${{ needs.release.outputs.tag }}`` instead of relying on
tag-push event triggering a separate workflow.
"""
if not os.environ.get("GITHUB_OUTPUT"):
return
write_github_output("tag", tag)
click.echo(_("Wrote tag {tag} to GITHUB_OUTPUT.", tag=tag))
def create_and_push_tag(new_version: str, changelog: str, dry_run: bool) -> bool:
"""Create an annotated tag with the changelog as message and push it.
Returns True if the tag was created/pushed, False if it already existed.
Raises an error if the tag exists but points to a different commit than HEAD.
"""
tag = f"v{new_version}"
if tag_exists(tag):
click.echo(_("Tag {tag} already exists, skipping creation.", tag=tag))
# Verify the tag points to HEAD — if it points elsewhere, that's
# a consistency error, not a skip condition.
tag_commit = get_tag_commit(tag)
head_commit = get_head_commit()
if tag_commit != head_commit:
raise click.ClickException(
_(
"Tag {tag} already exists but points to {tag_commit} "
"(expected HEAD {head_commit}). "
"This indicates a tag/commit misalignment. "
"Run 'python3 -m devx.ci.release --verify' for details.",
tag=tag,
tag_commit=tag_commit[:7],
head_commit=head_commit[:7],
)
)
click.echo(_("Tag {tag} already exists and points to HEAD. Skipping creation.", tag=tag))
if not dry_run:
# Ensure the existing tag is pushed
run_cmd(["git", "push", "origin", tag], check=False)
run_cmd(["git", "push", "origin", f"refs/tags/{tag}"], check=False)
_write_release_tag(tag)
return False
tag_msg = f"Release v{new_version}\n\n{changelog}"
if dry_run:
click.echo(_("[dry-run] Would create tag: {tag}", tag=tag))
return True
run_cmd(["git", "tag", "-a", tag, "-m", tag_msg])
run_cmd(["git", "push", "origin", tag])
run_cmd(["git", "push", "origin", f"refs/tags/{tag}"])
_write_release_tag(tag)
return True
# ---------------------------------------------------------------------------
# Verification mode
# ---------------------------------------------------------------------------
def get_init_version() -> str | None:
"""Read __version__ from the version file."""
try:
with open(INIT_FILE, encoding="utf-8") as f:
content = f.read()
match = re.search(r'^__version__\s*=\s*"([^"]*)"', content, flags=re.MULTILINE)
return match.group(1) if match else None
except FileNotFoundError:
return None
def get_changelog_versions() -> list[str]:
"""Extract version numbers from CHANGELOG.md headers, in order."""
try:
with open(CHANGELOG_FILE, encoding="utf-8") as f:
content = f.read()
return re.findall(r"^## \[(\d+\.\d+\.\d+)\]", content, flags=re.MULTILINE)
except FileNotFoundError:
return []
def verify_alignment() -> int:
"""Verify tag/version/changelog alignment. Returns exit code (0=ok, 1=issues)."""
click.echo(_("=== Release Alignment Verification ===\n"))
has_issues = False
# 1. Check __version__ matches latest tag
init_version = get_init_version()
latest_tag = get_latest_tag()
latest_tag_version = latest_tag.lstrip("v") if latest_tag else None
click.echo(_("Version file: {file}", file=INIT_FILE))
if init_version:
click.echo(f' __version__ = "{init_version}"')
else:
click.echo(" __version__ = NOT FOUND")
has_issues = True
click.echo(_("\nLatest tag: {tag}", tag=latest_tag or "(none)"))
if latest_tag_version and init_version:
if latest_tag_version == init_version:
click.echo(f" ✓ Tag version matches __version__ ({init_version})")
else:
click.echo(f" ✗ MISMATCH: tag={latest_tag_version}, __version__={init_version}")
has_issues = True
# 2. Check all tags point to commits with matching version
click.echo(_("\nTag → Commit alignment:"))
tag_errors = verify_tag_consistency()
all_tags = get_all_tags()
if not all_tags:
click.echo(" (no tags)")
elif not tag_errors:
click.echo(f" ✓ All {len(all_tags)} tags point to matching release commits")
else:
has_issues = True
for err in tag_errors:
click.echo(f" ✗ {err}")
# 3. Check CHANGELOG versions are in descending order
click.echo(_("\nCHANGELOG version ordering:"))
changelog_versions = get_changelog_versions()
if not changelog_versions:
click.echo(" (no versions in CHANGELOG)")
else:
# Check for duplicates
seen: set[str] = set()
duplicates: list[str] = []
for v in changelog_versions:
if v in seen:
duplicates.append(v)
seen.add(v)
# Check ordering (should be descending)
is_ordered = all(changelog_versions[i] >= changelog_versions[i + 1] for i in range(len(changelog_versions) - 1))
if duplicates:
has_issues = True
click.echo(f" ✗ Duplicate entries: {', '.join(duplicates)}")
elif not is_ordered:
has_issues = True
click.echo(f" ✗ Versions not in descending order: {changelog_versions}")
else:
click.echo(f" ✓ {len(changelog_versions)} versions, all in descending order")
# Check latest CHANGELOG version matches latest tag.
# The CHANGELOG may have one unreleased section ahead of the latest tag
# (e.g., CHANGELOG has 0.6.4 but latest tag is v0.6.3 — 0.6.4 is unreleased).
if changelog_versions and latest_tag_version:
if changelog_versions[0] == latest_tag_version:
click.echo(f" ✓ Latest CHANGELOG version matches latest tag ({latest_tag_version})")
elif latest_tag_version in changelog_versions:
tag_idx = changelog_versions.index(latest_tag_version)
# Latest tag should be at index 0 or 1 (0 = released, 1 = unreleased ahead)
if tag_idx == 1:
click.echo(
f" ✓ Latest CHANGELOG version ({changelog_versions[0]}) is unreleased, "
f"latest tag is {latest_tag_version}"
)
else:
click.echo(
f" ✗ MISMATCH: CHANGELOG latest={changelog_versions[0]}, "
f"tag={latest_tag_version} (tag is at position {tag_idx})"
)
has_issues = True
else:
click.echo(f" ✗ MISMATCH: CHANGELOG latest={changelog_versions[0]}, tag={latest_tag_version}")
has_issues = True
# 4. Check for untagged release commits.
# Distinguish between:
# - Truly untagged: no tag exists for that version (needs a tag)
# - Duplicates: a tag for that version exists but on a different commit
# (historical artifact from buggy release script — informational, not an error)
click.echo(_("\nUntagged release commits:"))
result = run_cmd(
["git", "log", "--all", "--format=%h %s", "--grep=^release: v"],
check=False,
)
if result.returncode == 0 and result.stdout.strip():
all_release_commits = result.stdout.strip().split("\n")
all_tags_set = {t.lstrip("v") for t in get_all_tags() if re.match(r"^v\d+\.\d+\.\d+$", t)}
truly_untagged: list[str] = []
duplicates: list[str] = []
for line in all_release_commits:
short_hash = line.split()[0]
tags_at = run_cmd(["git", "tag", "--points-at", short_hash], check=False)
if not tags_at.stdout.strip():
# Check if a tag for this version exists elsewhere
match = re.search(r"release: v(\d+\.\d+\.\d+)", line)
if match and match.group(1) in all_tags_set:
duplicates.append(line)
else:
truly_untagged.append(line)
if truly_untagged:
has_issues = True
click.echo(f" ✗ {len(truly_untagged)} untagged release commits (no tag for version):")
for c in truly_untagged[:10]:
click.echo(f" {c}")
if len(truly_untagged) > 10:
click.echo(f" ... and {len(truly_untagged) - 10} more")
else:
click.echo(" ✓ All release commits have tags")
if duplicates:
click.echo(f" ℹ {len(duplicates)} duplicate release commits (tag exists on different commit):")
for c in duplicates[:5]:
click.echo(f" {c}")
if len(duplicates) > 5:
click.echo(f" ... and {len(duplicates) - 5} more")
else:
click.echo(" (no release commits found)")
# Summary
click.echo(_("\n=== Summary ==="))
if has_issues:
click.echo("✗ Issues found — see above for details.")
return 1
click.echo("✓ All checks passed — tags, versions, and changelog are aligned.")
return 0
# ---------------------------------------------------------------------------
# Main command
# ---------------------------------------------------------------------------
@click.command()
@click.option("--dry-run", is_flag=True, default=False, help="Show what would happen without making changes.")
@click.option(
@@ -258,9 +531,24 @@ def create_and_push_tag(new_version: str, changelog: str, dry_run: bool) -> bool
default=False,
help="Skip lint and test verification (NOT recommended — only for emergency releases).",
)
def main(dry_run: bool, skip_tests: bool) -> None:
@click.option(
"--verify",
is_flag=True,
default=False,
help="Verify tag/version/changelog alignment and exit (no changes made).",
)
def main(dry_run: bool, skip_tests: bool, verify: bool) -> None:
"""Automated release: calculate next version, update files, tag, and push.
Use --verify to check tag/version/changelog alignment without making changes.
"""
if verify:
sys.exit(verify_alignment())
# Ensure we're on master (skip this check in dry-run mode for PR validation)
branch = run_cmd(["git", "rev-parse", "--abbrev-ref", "HEAD"]).stdout.strip()
# Some git versions return "heads/master" instead of "master"
branch = branch.removeprefix("heads/")
if branch != "master" and not dry_run:
raise click.ClickException(_("Release must be run on master, currently on '{branch}'.", branch=branch))
if branch != "master" and dry_run:
@@ -271,16 +559,72 @@ def main(dry_run: bool, skip_tests: bool) -> None:
)
)
# Release lock: if HEAD is already a release commit, another release
# run is in progress (or already completed). Skip to prevent duplicate tags.
head_msg = run_cmd(["git", "log", "-1", "--pretty=%s"]).stdout.strip()
if re.match(r"^release: v\d+\.\d+\.\d+", head_msg):
# Fetch tags from remote to ensure local tag state is current.
# This is critical in CI where a fresh checkout may not include tags
# from previous runs. Without this, tag_exists() returns False for
# tags that exist on the remote, leading to duplicate release commits.
if not dry_run:
fetch_tags()
# Pre-flight: verify existing tags are consistent. If any tag points
# to a commit with a mismatched version, abort before creating more
# inconsistencies.
tag_errors = verify_tag_consistency()
if tag_errors:
click.echo(_("ERROR: Tag consistency check failed. Existing tags are misaligned:"))
for err in tag_errors:
click.echo(err)
click.echo(
_(
"HEAD is already a release commit ('{msg}'). Another release may have just completed. Skipping.",
msg=head_msg,
"\nFix the misaligned tags before creating new releases. "
"Run 'python3 -m devx.ci.release --verify' for a full report."
)
)
raise click.ClickException(_("Tag consistency check failed."))
# Release lock: if HEAD is already a release commit, check if the tag
# exists AND points to HEAD. If the tag is missing (e.g., tag push
# failed in a previous run), create and push it. If the tag exists
# but points elsewhere, that's an error.
head_msg = run_cmd(["git", "log", "-1", "--pretty=%s"]).stdout.strip()
release_match = re.match(r"^release: v(\d+\.\d+\.\d+)", head_msg)
if release_match:
release_version = release_match.group(1)
release_tag = f"v{release_version}"
if tag_exists(release_tag):
tag_commit = get_tag_commit(release_tag)
head_commit = get_head_commit()
if tag_commit != head_commit:
raise click.ClickException(
_(
"HEAD is a release commit for v{version} but tag {tag} "
"points to a different commit ({tag_commit} vs HEAD {head_commit}). "
"This indicates a tag/commit misalignment.",
version=release_version,
tag=release_tag,
tag_commit=tag_commit[:7],
head_commit=head_commit[:7],
)
)
click.echo(
_(
"HEAD is already a release commit ('{msg}') and tag {tag} points to HEAD. Skipping.",
msg=head_msg,
tag=release_tag,
)
)
_write_release_tag(release_tag)
return
# Tag is missing — recover by creating and pushing it
click.echo(
_(
"HEAD is a release commit ('{msg}') but tag {tag} is missing. Recovering by creating tag.",
msg=head_msg,
tag=release_tag,
)
)
changelog = get_changelog(release_version)
create_and_push_tag(release_version, changelog, dry_run)
return
# Check if any user-facing files changed since the last tag.
@@ -304,6 +648,20 @@ def main(dry_run: bool, skip_tests: bool) -> None:
return
current_tag = get_latest_tag()
# If the bumped version equals the current tag version, there's nothing
# new to release. git-cliff didn't bump because the commits since the last
# tag don't warrant a version change (e.g., only ci:/chore: commits).
# Creating a release commit with the same version would cause a tag
# conflict.
if current_tag and current_tag.lstrip("v") == new_version:
click.echo(
_(
"Version stays at v{version} — no version bump from git-cliff. "
"Commits since last tag don't warrant a new release. Skipping.",
version=new_version,
)
)
return
click.echo(
_(
"Bumping version: {current} -> v{new_version}",
@@ -315,13 +673,18 @@ def main(dry_run: bool, skip_tests: bool) -> None:
# Generate changelog
changelog = get_changelog(new_version)
if not changelog:
click.echo(_("Warning: git-cliff generated empty changelog."))
raise click.ClickException(
_(
"git-cliff generated empty changelog for v{version}. Check cliff.toml and commit history.",
version=new_version,
)
)
if dry_run:
click.echo(_("\n[dry-run] Changelog:\n{changelog}", changelog=changelog))
click.echo(_("[dry-run] Would update {init}", init=INIT_FILE))
click.echo(_("[dry-run] Would update {changelog_file}", changelog_file=CHANGELOG_FILE))
click.echo(_("[dry-run] Would commit: release: v{version}", version=new_version))
click.echo(_("[dry-run] Would commit: release: v{version} [skip ci]", version=new_version))
click.echo(_("[dry-run] Would push commit to master"))
click.echo(_("[dry-run] Would create tag: v{version}", version=new_version))
return
@@ -347,8 +710,39 @@ def main(dry_run: bool, skip_tests: bool) -> None:
click.echo(_("Created release commit."))
# Pull --rebase before push to handle the case where master
# advanced between checkout and commit (e.g., another merge).
run_cmd(["git", "pull", "--rebase", "origin", "master"], check=False)
run_cmd(["git", "push", "origin", "master"])
# Retry up to 3 times to handle concurrent pushes.
push_succeeded = False
for attempt in range(3):
rebase = run_cmd(["git", "pull", "--rebase", "origin", "master"], check=False)
if rebase.returncode != 0:
# Rebase failed (likely conflicts). Abort and retry.
click.echo(
_(
"Rebase attempt {n}/3 failed: {err}",
n=attempt + 1,
err=rebase.stderr.strip() if rebase.stderr else rebase.stdout.strip(),
)
)
run_cmd(["git", "rebase", "--abort"], check=False)
# Brief delay before retry to let concurrent pushes settle.
time.sleep(5)
continue
push = run_cmd(["git", "push", "origin", "refs/heads/master:refs/heads/master"], check=False)
if push.returncode == 0:
push_succeeded = True
break
click.echo(
_(
"Push attempt {n}/3 failed: {err}",
n=attempt + 1,
err=push.stderr.strip() if push.stderr else push.stdout.strip(),
)
)
time.sleep(5)
if not push_succeeded:
raise click.ClickException(
_("Failed to push release commit after 3 attempts. Manual intervention required.")
)
click.echo(_("Pushed release commit to master."))
else:
click.echo(_("Skipping commit push — no staged changes."))
+34 -17
View File
@@ -14,7 +14,7 @@ Gitea 1.26 wiki API endpoints (all use content_base64, NOT content):
- Delete: DELETE /repos/{owner}/{repo}/wiki/page/{sub_url}
Usage:
REPO_TOKEN=<token> python3 -m devx.ci.sync_wiki [--dry-run] [--repo owner/repo]
CI_GITEA_TOKEN=<token> python3 -m devx.ci.sync_wiki [--dry-run] [--repo owner/repo]
"""
from __future__ import annotations
@@ -28,26 +28,43 @@ import click
from dotenv import load_dotenv # pyright: ignore[reportMissingImports,reportUnknownVariableType]
from devx.api_clients import GiteaClient
from devx.config import GITEA_API_URL
from devx.config import GITEA_API_URL, REPO_NAME, REPO_OWNER
from devx.exceptions import APIError
from devx.i18n import _
load_dotenv()
DOCS_DIR = Path(__file__).resolve().parent.parent.parent.parent / "docs"
# DOCS_DIR is the repo's docs/ directory. When devx is installed as a
# package (e.g., in .venv/lib/python3.12/site-packages/devx/), the
# __file__-relative path would point inside the venv, not the repo.
# Use DEVX_DOCS_DIR env var if set, otherwise fall back to ./docs
# (relative to the current working directory, which is the repo root
# in CI and local development).
DOCS_DIR = Path(os.environ.get("DEVX_DOCS_DIR", "docs"))
MAPPING_FILE = DOCS_DIR / "mapping.json"
def load_mapping() -> dict[str, str]:
"""Load the file-to-wiki-page mapping from mapping.json."""
with open(MAPPING_FILE) as f:
return json.load(f)
"""Load the file-to-wiki-page mapping from mapping.json.
Validates that the mapping is a dict of string-to-string pairs.
"""
with open(MAPPING_FILE, encoding="utf-8") as f:
data = json.load(f)
if not isinstance(data, dict):
raise click.ClickException(
_("mapping.json must be a dict of file-path -> page-title, got {type}", type=type(data).__name__)
)
for k, v in data.items():
if not isinstance(k, str) or not isinstance(v, str):
raise click.ClickException(_("mapping.json keys and values must be strings, got {k}={v}", k=k, v=v))
return data
def read_doc_content(file_path: str) -> str:
"""Read markdown content from a docs file."""
full_path = DOCS_DIR / file_path
with open(full_path) as f:
with open(full_path, encoding="utf-8") as f:
return f.read()
@@ -208,13 +225,13 @@ def verify_wiki_integrity(
help="Full integrity check: verify page count, missing pages, stale pages, and content. Implies --verify.",
)
def main(dry_run: bool, repo: str | None, verify: bool, strict: bool) -> None:
token = os.environ.get("REPO_TOKEN", "")
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
raise click.ClickException(_("ERROR: REPO_TOKEN is not set."))
raise click.ClickException(_("ERROR: CI_GITEA_TOKEN is not set."))
if repo is None:
owner = os.environ.get("DEVX_REPO_OWNER", "oblachno-oss")
repo_name = os.environ.get("DEVX_REPO_NAME", "devx")
owner = os.environ.get("DEVX_REPO_OWNER", "") or REPO_OWNER
repo_name = os.environ.get("DEVX_REPO_NAME", "") or REPO_NAME
else:
owner, repo_name = repo.split("/")
@@ -239,14 +256,14 @@ def main(dry_run: bool, repo: str | None, verify: bool, strict: bool) -> None:
try:
content = read_doc_content(file_path)
except FileNotFoundError:
click.echo(_("WARNING: File {file} not found — skipping.", file=file_path))
skipped += 1
continue
raise click.ClickException(
_("Mapped file {file} not found. Update mapping.json or create the file.", file=file_path)
) from None
if not content.strip():
click.echo(_("WARNING: File {file} is empty — skipping.", file=file_path))
skipped += 1
continue
raise click.ClickException(
_("Mapped file {file} is empty. Update the content or remove from mapping.json.", file=file_path)
) from None
result = sync_page(client, page_title, content, existing_pages, dry_run)
if result == "created":
+46 -11
View File
@@ -2,20 +2,37 @@
"""Validate commit messages for devx.
Rules:
- On feature branches: conventional commits ONLY, must NOT include DEVX-N prefix.
- On feature branches: conventional commits ONLY, must NOT include <PREFIX>-N prefix.
- On master branch: must follow '<task-id>: <conventional commit>' pattern,
e.g. 'DEVX-24: fix: resolve timeout'.
The task ID prefix is configurable via the ``DEVX_TASK_PREFIX`` environment
variable (default: ``DEVX``). Projects consuming devx (e.g., GRM) set
their own prefix (e.g., ``GRM``) so the validator enforces the correct
task ID format for each project.
"""
import re
import subprocess # nosec B404
import sys
import click
from devx.config import CONVENTIONAL_RE
from devx.config import CONVENTIONAL_RE, TASK_PREFIX
from devx.i18n import _
MASTER_TASK_ID_RE = re.compile(r"^DEVX-\d+:")
MASTER_TASK_ID_RE = re.compile(rf"^{TASK_PREFIX}-\d+:")
def get_latest_commit_msg() -> str:
"""Get the latest commit message from git."""
result = subprocess.run( # nosec
["git", "log", "-1", "--format=%B"],
capture_output=True,
text=True,
check=True,
)
return result.stdout.strip()
def first_line(text: str) -> str:
@@ -36,11 +53,26 @@ def get_branch() -> str:
@click.command()
@click.argument("commit_msg_file")
@click.argument("commit_msg_file", required=False)
@click.option("--branch", default=None, help="Override branch detection (for CI use).")
def main(commit_msg_file: str, branch: str | None) -> None:
with open(commit_msg_file) as f:
msg = f.read().strip()
@click.option(
"--git",
"from_git",
is_flag=True,
default=False,
help="Read commit message from git log instead of a file.",
)
def main(commit_msg_file: str | None, branch: str | None, from_git: bool) -> None:
if from_git:
msg = get_latest_commit_msg()
elif commit_msg_file:
if commit_msg_file == "-":
msg = sys.stdin.read().strip()
else:
with open(commit_msg_file, encoding="utf-8") as f:
msg = f.read().strip()
else:
raise click.ClickException(_("Provide a commit message file or use --git."))
if branch is None:
branch = get_branch()
@@ -51,8 +83,9 @@ def main(commit_msg_file: str, branch: str | None) -> None:
raise click.ClickException(
_(
"Oops! Master branch commits must start with a task ID.\n"
" Expected: DEVX-N: <conventional commit message>\n"
" Expected: {prefix}-N: <conventional commit message>\n"
" Got: {subject}",
prefix=TASK_PREFIX,
subject=subject,
)
)
@@ -61,8 +94,9 @@ def main(commit_msg_file: str, branch: str | None) -> None:
raise click.ClickException(
_(
"Oops! Master branch commit must follow conventional format after task ID.\n"
" Expected: DEVX-N: <type>: <description>\n"
" Expected: {prefix}-N: <type>: <description>\n"
" Got: {subject}",
prefix=TASK_PREFIX,
subject=subject,
)
)
@@ -71,8 +105,9 @@ def main(commit_msg_file: str, branch: str | None) -> None:
if MASTER_TASK_ID_RE.match(subject):
raise click.ClickException(
_(
"Oops! Do not include task ID (DEVX-N) in feature branch commits.\n"
" The task ID will be added automatically on merge via CI."
"Oops! Do not include task ID ({prefix}-N) in feature branch commits.\n"
" The task ID will be added automatically on merge via CI.",
prefix=TASK_PREFIX,
)
)
+28
View File
@@ -95,6 +95,13 @@ def ci_doc_coverage(args: tuple[str, ...]) -> None:
_run_module("devx.ci.doc_coverage", list(args))
@ci.command("lint-docs")
@click.argument("args", nargs=-1)
def ci_lint_docs(args: tuple[str, ...]) -> None:
"""Lint documentation files for structure, links, and quality."""
_run_module("devx.ci.lint_docs", list(args))
@ci.command("notify-failure")
@click.argument("args", nargs=-1)
def ci_notify_failure(args: tuple[str, ...]) -> None:
@@ -151,6 +158,20 @@ def ci_validate_commit_msg(args: tuple[str, ...]) -> None:
_run_module("devx.ci.validate_commit_msg", list(args))
@ci.command("distribute-files")
@click.argument("args", nargs=-1)
def ci_distribute_files(args: tuple[str, ...]) -> None:
"""Distribute files across parallel runners (round-robin)."""
_run_module("devx.ci.distribute_files", list(args))
@ci.command("integration-guard")
@click.argument("args", nargs=-1)
def ci_integration_guard(args: tuple[str, ...]) -> None:
"""Run pytest with cross-runner failure detection."""
_run_module("devx.ci.integration_guard", list(args))
@cli.group()
def tools() -> None:
"""Development tool commands."""
@@ -177,6 +198,13 @@ def tools_generate_badges(args: tuple[str, ...]) -> None:
_run_module("devx.tools.generate_badges", list(args))
@tools.command("generate-cliff-config")
@click.argument("args", nargs=-1)
def tools_generate_cliff_config(args: tuple[str, ...]) -> None:
"""Generate a cliff.toml configuration file for the project."""
_run_module("devx.tools.generate_cliff_config", list(args))
@tools.command("install-checkmake")
@click.argument("args", nargs=-1)
def tools_install_checkmake(args: tuple[str, ...]) -> None:
+68 -8
View File
@@ -1,27 +1,87 @@
"""Shared configuration constants for devx scripts and API clients.
All defaults can be overridden via environment variables with the ``DEVX_``
prefix. Projects consuming devx can set these in their ``.env`` files.
Configuration is read from two sources, in priority order:
1. **Environment variables** (``DEVX_`` prefix) — highest priority, used for
CI secrets and per-run overrides.
2. **``[tool.devx]`` section in ``pyproject.toml``** — project defaults,
read from the current working directory.
If neither source provides a value, built-in defaults are used.
"""
from __future__ import annotations
import os
import re
import tomllib
from pathlib import Path
def _load_pyproject_devx() -> dict[str, object]:
"""Load the ``[tool.devx]`` section from pyproject.toml in the CWD.
Returns an empty dict if the file or section is missing.
"""
path = Path("pyproject.toml")
if not path.exists():
return {}
try:
with open(path, "rb") as f: # noqa: PTH123
data: dict[str, object] = tomllib.load(f)
except (tomllib.TOMLDecodeError, OSError):
return {}
tool_raw: object = data.get("tool", {})
if not isinstance(tool_raw, dict):
return {}
tool: dict[str, object] = tool_raw # type: ignore[assignment]
devx_raw: object = tool.get("devx", {})
if not isinstance(devx_raw, dict):
return {}
devx: dict[str, object] = devx_raw # type: ignore[assignment]
return devx
_PYPROJECT = _load_pyproject_devx()
def _get(key: str, env_var: str, default: str) -> str:
"""Get a config value: env var > pyproject.toml > default."""
env_val = os.getenv(env_var)
if env_val is not None:
return env_val
pyproject_val = _PYPROJECT.get(key)
if isinstance(pyproject_val, str):
return pyproject_val
return default
def _get_int(key: str, env_var: str, default: int) -> int:
"""Get an int config value: env var > pyproject.toml > default."""
env_val = os.getenv(env_var)
if env_val is not None:
return int(env_val)
pyproject_val = _PYPROJECT.get(key)
if isinstance(pyproject_val, int):
return pyproject_val
return default
# API endpoints — override via env vars for different Gitea/Vikunja instances
GITEA_API_URL = os.getenv("DEVX_GITEA_API_URL", "https://git.oblachno.oblachno.fyi/api/v1")
VIKUNJA_API_URL = os.getenv("DEVX_VIKUNJA_API_URL", "https://work.oblachno.oblachno.fyi/api/v1")
GITEA_API_URL = _get("gitea_api_url", "DEVX_GITEA_API_URL", "https://git.oblachno.oblachno.fyi/api/v1")
VIKUNJA_API_URL = _get("vikunja_api_url", "DEVX_VIKUNJA_API_URL", "https://work.oblachno.oblachno.fyi/api/v1")
# Organization defaults
REPO_OWNER = os.getenv("DEVX_REPO_OWNER", "oblachno-oss")
# Organization defaults — each project MUST set DEVX_REPO_OWNER explicitly.
# No default: prevents silent 404s when the wrong owner is used.
REPO_OWNER = _get("repo_owner", "DEVX_REPO_OWNER", "")
REPO_NAME = _get("repo_name", "DEVX_REPO_NAME", "")
# Task prefix for Vikunja task IDs — each project sets its own (GRM, DEVX, INFRA, etc.)
TASK_PREFIX = os.getenv("DEVX_TASK_PREFIX", "DEVX")
TASK_PREFIX = _get("task_prefix", "DEVX_TASK_PREFIX", "DEVX")
TASK_ID_RE = re.compile(rf"{TASK_PREFIX}-\d+")
# Vikunja project ID — each project uses a different Vikunja project
VIKUNJA_PROJECT_ID = int(os.getenv("DEVX_VIKUNJA_PROJECT_ID", "6"))
VIKUNJA_PROJECT_ID = _get_int("vikunja_project_id", "DEVX_VIKUNJA_PROJECT_ID", 6)
# HTTP client defaults
DEFAULT_TIMEOUT = 30
+61 -6
View File
@@ -40,15 +40,67 @@ Usage::
from __future__ import annotations
import json
import os
import shutil
import subprocess # nosec B404
from typing import Any
import click
from devx.config import GITEA_API_URL
from devx.i18n import _
class TeaCLIError(Exception):
"""Raised when a tea CLI command fails."""
def configure_tea_login(login_name: str = "devx") -> None:
"""Configure tea CLI login from CI_GITEA_TOKEN and DEVX_GITEA_API_URL.
Idempotent: if a login with the same name already exists, it is not re-added.
Skips silently if tea is not installed or CI_GITEA_TOKEN is not set.
Used by CI scripts (publish, notify_failure) that need tea login but
run in containerized environments where ``make setup`` was not called.
"""
tea_bin = shutil.which("tea")
if tea_bin is None:
click.echo(_("tea not installed — skipping login configuration."))
return
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
click.echo(_("CI_GITEA_TOKEN not set — skipping login configuration."))
return
gitea_url = GITEA_API_URL.replace("/api/v1", "")
result = subprocess.run( # nosec B603
[tea_bin, "login", "list", "--output", "simple"],
capture_output=True,
text=True,
check=False,
)
if result.returncode == 0 and login_name in result.stdout:
click.echo(_("tea login '{name}' already configured.", name=login_name))
return
click.echo(_("Configuring tea login '{name}' for {url}...", name=login_name, url=gitea_url))
subprocess.run( # nosec B603
[tea_bin, "login", "add", "--name", login_name, "--url", gitea_url, "--token", token],
capture_output=True,
text=True,
check=False,
)
subprocess.run( # nosec B603
[tea_bin, "login", "default", login_name],
capture_output=True,
text=True,
check=False,
)
class TeaCLI:
"""Wrapper around the ``tea`` Gitea CLI tool.
@@ -82,12 +134,15 @@ class TeaCLI:
cmd = [self._tea, *args]
if json_output:
cmd.extend(["--output", "json"])
result = subprocess.run( # nosec B603
cmd,
capture_output=True,
text=True,
check=False,
)
try:
result = subprocess.run( # nosec B603
cmd,
capture_output=True,
text=True,
check=False,
)
except FileNotFoundError as e:
raise TeaCLIError(f"tea binary not found ('{self._tea}'). Install tea or add it to PATH.") from e
if result.returncode != 0:
raise TeaCLIError(
f"tea command failed (rc={result.returncode}): {' '.join(args)}\nstderr: {result.stderr.strip()}"
+2 -2
View File
@@ -1,7 +1,7 @@
"""Simple i18n for devx scripts and tools.
Set DEVX_LANG environment variable to override the default English.
Supported: en, bg, de, ru, zh.
Supported: en, bg, de, ru, zh, pl.
Projects can extend translations by setting DEVX_TRANSLATIONS_PATH to a
JSON file with additional keys. Keys from the project's file are merged
@@ -45,7 +45,7 @@ def _(key: str, **kwargs: object) -> str:
If unset, English is always returned regardless of system locale.
"""
lang = os.getenv("DEVX_LANG", "en")
if lang not in ("en", "bg", "de", "ru", "zh"):
if lang not in ("en", "bg", "de", "ru", "zh", "pl"):
lang = "en"
template = TRANSLATIONS.get(key, {}).get(lang, key)
return template.format(**kwargs)
+373
View File
@@ -0,0 +1,373 @@
# devx.mak — Shared Makefile fragment for devx-integrated projects.
#
# This fragment provides common targets for:
# - Vikunja task management and PR creation
# - Workflow validation (actionlint, act_runner)
# - Linting (ruff, pyright, bandit, pip-audit)
# - CI failure notification
# - Environment setup (venv, .env, hooks)
# - Test execution and quality checks
#
# Project config (task prefix, Vikunja project ID, repo owner, repo name)
# is read from [tool.devx] in pyproject.toml by devx.config — no
# Makefile variables needed.
#
# Usage in your Makefile:
#
# # Set DEVX_PYTHON to your venv's Python
# DEVX_PYTHON := $(BIN)/python
#
# # Include the devx fragment (silent if devx not installed yet)
# DEVX_MAK := $(shell $(DEVX_PYTHON) -c \
# "from pathlib import Path; import devx; print(Path(devx.__file__).parent / 'make' / 'devx.mak')" \
# 2>/dev/null)
# -include $(DEVX_MAK)
#
# If devx is not installed, the -include silently skips and the targets
# are simply unavailable (run 'make setup' first).
#
# Variables (set BEFORE including this fragment):
# DEVX_PYTHON — Python executable (default: python3)
# DEVX_PR_BASE — PR base branch (default: master)
# DEVX_VENV — venv directory name (default: .venv)
# DEVX_BIN — venv bin directory (default: $(DEVX_VENV)/bin)
# DEVX_LINT_PATHS — paths for ruff/bandit (default: src/ tests/)
# DEVX_TYPECHECK_PATHS — paths for pyright (default: empty — uses pyright config)
# DEVX_COV_PKG — coverage package name (default: src/devx)
# DEVX_TEST_PATHS — pytest paths (default: tests/)
# DEVX_GITEA_PYPI_HOST — Gitea PyPI host (default: git.oblachno.oblachno.fyi)
# DEVX_GITEA_PYPI_ORG — Gitea PyPI org (default: oblachno-oss)
# DEVX_ACTIONLINT_CFG — actionlint config file (default: .gitea/actionlint.yaml)
# DEVX_WORKFLOW_DIR — workflow directory (default: .gitea/workflows)
DEVX_PYTHON ?= python3
DEVX_PR_BASE ?= master
DEVX_VENV ?= .venv
DEVX_BIN ?= $(DEVX_VENV)/bin
DEVX_LINT_PATHS ?= src/ tests/
DEVX_COV_PKG ?= src/devx
DEVX_TEST_PATHS ?= tests/
DEVX_GITEA_PYPI_HOST ?= git.oblachno.oblachno.fyi
DEVX_GITEA_PYPI_ORG ?= oblachno-oss
DEVX_ACTIONLINT_CFG ?= .gitea/actionlint.yaml
DEVX_WORKFLOW_DIR ?= .gitea/workflows
DEVX_DOCKERFILE_PATHS ?= docker
# PIP_INSTALL — helper to run pip with Gitea private PyPI registry configured.
# Usage: $(DEVX_PIP_INSTALL) install -e '.[ci,lint]'
# CI_GITEA_USERNAME can be set in .env, as an env var, or as a Make variable.
DEVX_PIP_INSTALL := if [ -z "$$CI_GITEA_TOKEN" ]; then . ./.env 2>/dev/null; fi; \
CI_GITEA_TOKEN="$$CI_GITEA_TOKEN"; \
_PYPI_USER="$${CI_GITEA_USERNAME:-emil}"; \
if [ -n "$$CI_GITEA_TOKEN" ] && [ -n "$$_PYPI_USER" ]; then export PIP_EXTRA_INDEX_URL="https://$$_PYPI_USER:$$CI_GITEA_TOKEN@$(DEVX_GITEA_PYPI_HOST)/api/packages/$(DEVX_GITEA_PYPI_ORG)/pypi/simple/"; fi; \
$(DEVX_BIN)/pip
.PHONY: devx-create-task devx-create-pr devx-push devx-push-with-pr devx-check-config
.PHONY: devx-pr-status devx-pr-logs devx-pr-label devx-pr-review
.PHONY: devx-configure-gitea-pypi devx-install-tools devx-install-checkmake devx-checkmake
.PHONY: devx-workflow-lint devx-workflow-dryrun devx-workflow-dryrun-safe devx-workflow-check
.PHONY: devx-notify-failure devx-install-hooks devx-activate-scripts
.PHONY: devx-lint-ruff devx-lint-format devx-typecheck devx-lint-bandit devx-lint-deps devx-lint
.PHONY: devx-clean devx-pre-push
.PHONY: devx-check-mutable-globals devx-check-dep-docs devx-check-test-coverage devx-check-docs devx-check-test-speed
.PHONY: devx-test-unit devx-pytest-cov
.PHONY: devx-setup-image devx-lint-dockerfiles
# ── Vikunja task and PR management ────────────────────────────────────────────
# Create a Vikunja task (project ID read from [tool.devx] in pyproject.toml)
devx-create-task:
@$(DEVX_PYTHON) -m devx.tools.create_task
# Create a PR with title auto-derived from the Vikunja task
# (owner/repo read from [tool.devx] in pyproject.toml)
devx-create-pr:
@$(DEVX_PYTHON) -m devx.tools.create_pr --base $(DEVX_PR_BASE)
# Push current branch to origin
devx-push:
@git push -u origin HEAD
# Validate devx configuration in pyproject.toml
devx-check-config:
@$(DEVX_PYTHON) -m devx.tools.check_config
# Push and create PR in one step
devx-push-with-pr: devx-push devx-create-pr
# Check CI status for a PR (auto-detects current branch's PR)
# Usage: make devx-pr-status
# make devx-pr-status PR=42
# make devx-pr-status PR=42 WAIT=1 TIMEOUT=600
devx-pr-status:
@$(DEVX_PYTHON) -m devx.tools.pr_status \
$(if $(PR),--pr $(PR)) \
$(if $(WAIT),--wait) \
$(if $(TIMEOUT),--timeout $(TIMEOUT))
# Fetch logs for failed CI jobs on a PR
# Usage: make devx-pr-logs
# make devx-pr-logs PR=42
# make devx-pr-logs PR=42 JOB=quality TAIL=50
devx-pr-logs:
@$(DEVX_PYTHON) -m devx.tools.pr_logs \
$(if $(PR),--pr $(PR)) \
$(if $(JOB),--job $(JOB)) \
$(if $(TAIL),--tail $(TAIL))
# Add a label to a PR (default: ready-to-merge)
# Usage: make devx-pr-label
# make devx-pr-label PR=42
# make devx-pr-label PR=42 LABEL=ready-to-merge
devx-pr-label:
@$(DEVX_PYTHON) -m devx.tools.pr_label \
$(if $(PR),--pr $(PR)) \
--label $(or $(LABEL),ready-to-merge)
# Usage: make devx-pr-review PR=42 EVENT=APPROVE BODY="..." CHECKLIST=1,2,3,4,5,6,7,8,9,10,11,12,13
# make devx-pr-review PR=42 EVENT=REQUEST_CHANGES BODY="..."
# make devx-pr-review PR=42 (auto review)
devx-pr-review:
@$(DEVX_PYTHON) -m devx.ci.pr_review \
$(PR) $(DEVX_REPO_OWNER)/$(DEVX_REPO_NAME) \
$(if $(EVENT),--event $(EVENT)) \
$(if $(BODY),--body "$(BODY)") \
$(if $(CHECKLIST),--checklist-confirmed --checklist-categories $(CHECKLIST))
# ── Environment setup ─────────────────────────────────────────────────────────
# Configure Gitea private PyPI registry so pip can find devx and other
# private packages. In CI, CI_GITEA_TOKEN is set as a secret. Locally, it's in .env.
devx-configure-gitea-pypi:
@if [ -z "$$CI_GITEA_TOKEN" ]; then . ./.env 2>/dev/null; fi; \
CI_GITEA_TOKEN="$$CI_GITEA_TOKEN"; \
if [ -z "$$CI_GITEA_TOKEN" ]; then echo "[configure-gitea-pypi] CI_GITEA_TOKEN not set — skipping (devx must be on public PyPI)"; exit 0; fi; \
echo "[configure-gitea-pypi] Gitea PyPI registry configured (CI_GITEA_TOKEN present)."
# Create .env from .env.example if it doesn't exist
devx-env:
@if [ ! -f .env ]; then \
cp .env.example .env; \
echo "Created .env from .env.example — please edit it with your credentials."; \
fi
# Create Python venv with version check
devx-venv:
@python3 -c "import sys; v=sys.version_info; assert v >= (3, 12), f'Python 3.12+ required, found {v.major}.{v.minor}'; print(f'Python {v.major}.{v.minor}.{v.micro} OK')"
$(DEVX_PYTHON) -m venv $(DEVX_VENV)
$(DEVX_BIN)/pip install --upgrade pip setuptools wheel
# Create activate scripts for shell/fish/zsh
devx-activate-scripts:
@test -f activate.sh || (echo '#!/usr/bin/env bash' > activate.sh && echo 'source "$$(cd "$$(dirname "$${BASH_SOURCE[0]}")" && pwd)/.venv/bin/activate"' >> activate.sh && chmod +x activate.sh)
@test -f activate.fish || (echo '#!/usr/bin/env fish' > activate.fish && echo 'set -l script_dir (dirname (status --current-filename))' >> activate.fish && echo 'source "$$script_dir/.venv/bin/activate.fish"' >> activate.fish && chmod +x activate.fish)
@test -f activate.zsh || (echo '#!/usr/bin/env zsh' > activate.zsh && echo '0="$${ZERO:-$${0:#$$ZSH_ARGZERO}}"' >> activate.zsh && echo '0="$${$${(M)0:#/*}:-$$PWD/$$0}"' >> activate.zsh && echo 'source "$${0:A:h}/.venv/bin/activate"' >> activate.zsh && chmod +x activate.zsh)
# Set git hooks path to hooks/
devx-install-hooks:
@git config core.hooksPath hooks
@chmod +x hooks/pre-commit hooks/pre-push 2>/dev/null || true
@echo "core.hooksPath set to hooks/ — tracked hooks are now live."
# ── Tool installation ─────────────────────────────────────────────────────────
# Install CI/CD tools (actionlint, git-cliff, act_runner, tea) to ~/.local/bin
devx-install-tools:
@$(DEVX_PYTHON) -m devx.tools.install_tools
# Install checkmake (Makefile linter)
devx-install-checkmake:
@$(DEVX_PYTHON) -m devx.tools.install_checkmake
# Lint Makefiles with checkmake
devx-checkmake:
@CHECKMAKE_EXE="$$(command -v checkmake 2>/dev/null || echo $(HOME)/.local/bin/checkmake)"; \
if ! command -v "$$CHECKMAKE_EXE" >/dev/null 2>&1 && ! [ -x "$$CHECKMAKE_EXE" ]; then \
echo "[checkmake] checkmake not found. Run: make devx-install-checkmake"; exit 1; \
fi; \
"$$CHECKMAKE_EXE" $(CURDIR)/Makefile
# ── Workflow validation ───────────────────────────────────────────────────────
# Static lint of Gitea Actions workflow YAML files
devx-workflow-lint:
@command -v actionlint >/dev/null 2>&1 || { \
echo "actionlint not found. Install: bash <(curl https://raw.githubusercontent.com/rhysd/actionlint/main/scripts/download-actionlint.bash)"; \
exit 1; \
}
actionlint -config-file $(DEVX_ACTIONLINT_CFG) $(DEVX_WORKFLOW_DIR)/*.yml
# Dry-run all workflows (requires act_runner)
devx-workflow-dryrun:
@command -v act_runner >/dev/null 2>&1 || { echo "act_runner not found. Install: https://gitea.com/gitea/act_runner/releases"; exit 1; }
@echo "Dry-running all workflows (no Docker containers started)..."
act_runner exec --dryrun -W $(DEVX_WORKFLOW_DIR)/ 2>&1 | grep -E 'DRYRUN|ERROR|FAIL|Job'
# Best-effort dry-run (skips if act_runner is not installed)
devx-workflow-dryrun-safe:
@command -v act_runner >/dev/null 2>&1 && { echo "Dry-running workflows..."; act_runner exec --dryrun -W $(DEVX_WORKFLOW_DIR)/ 2>&1 | grep -E 'DRYRUN|ERROR|FAIL|Job'; } || echo "act_runner not found — skipping workflow dry-run (static lint still passed)"
# Static lint + dry-run
devx-workflow-check: devx-workflow-lint devx-workflow-dryrun
@echo "Workflow checks passed (static lint + dry-run)."
# ── CI failure notification ───────────────────────────────────────────────────
# Notify on CI failure — creates a Gitea issue via devx.ci.notify_failure.
# Usage: make devx-notify-failure WORKFLOW=post-merge/release
# Requires: CI_GITEA_TOKEN, GITHUB_REPOSITORY, GITHUB_RUN_ID, GITHUB_SHA
devx-notify-failure:
@. $(DEVX_VENV)/bin/activate 2>/dev/null || true; \
export PATH="$(HOME)/.local/bin:$$PATH"; \
$(DEVX_PYTHON) -m devx.tools.install_tools --tool tea 2>/dev/null || true; \
$(DEVX_PYTHON) -m devx.ci.notify_failure --auto-login \
--repo "$${GITHUB_REPOSITORY}" \
--run-id "$${GITHUB_RUN_ID}" \
--workflow "$(WORKFLOW)" \
--commit "$${GITHUB_SHA}"
# ── Linting ───────────────────────────────────────────────────────────────────
devx-lint-ruff:
@$(DEVX_BIN)/ruff check $(DEVX_LINT_PATHS)
devx-lint-format:
@$(DEVX_BIN)/ruff format --check $(DEVX_LINT_PATHS)
devx-typecheck:
@$(DEVX_BIN)/pyright
devx-lint-bandit:
@$(DEVX_BIN)/bandit -r src/
devx-lint-deps:
@echo "Checking dependencies for known vulnerabilities..."
@$(DEVX_BIN)/python -m ensurepip 2>/dev/null || true
@PIPAPI_PYTHON_LOCATION=$$(pwd)/$(DEVX_VENV)/bin/python \
$(DEVX_BIN)/pip-audit --desc --skip-editable 2>&1 || true
devx-lint: devx-lint-ruff devx-lint-format devx-typecheck devx-lint-bandit
@echo "[devx-lint] Linting checks passed."
# ── Testing ───────────────────────────────────────────────────────────────────
devx-test-unit:
@$(DEVX_BIN)/pytest $(DEVX_TEST_PATHS) -v --no-cov
devx-pytest-cov:
@$(DEVX_BIN)/pytest $(DEVX_TEST_PATHS) -v --cov=$(DEVX_COV_PKG) --cov-report=term-missing --cov-fail-under=100
# ── Quality checks ────────────────────────────────────────────────────────────
# Scan for module-level mutable globals that cause test isolation bugs
devx-check-mutable-globals:
@$(DEVX_PYTHON) -m devx.tools.check_mutable_globals
# Validate that every dependency in pyproject.toml has a documented purpose
devx-check-dep-docs:
@$(DEVX_PYTHON) -m devx.tools.check_pyproject_deps
# Check that changed files have corresponding tests
devx-check-test-coverage:
@$(DEVX_PYTHON) -m devx.tools.check_test_coverage
# Validate agent and user docs for stale file references
devx-check-docs:
@$(DEVX_PYTHON) -m devx.tools.check_agent_docs
# Verify test suite timing
devx-check-test-speed:
@$(DEVX_PYTHON) -m devx.tools.check_test_speed
# ── Pre-push validation ───────────────────────────────────────────────────────
# Run lint + tests before push (projects can override with project-specific targets)
devx-pre-push: devx-lint devx-pytest-cov
@echo "[devx-pre-push] All checks passed. Proceeding with push."
# ── Cleanup ───────────────────────────────────────────────────────────────────
devx-clean:
@find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
@find . -type f -name "*.pyc" -delete 2>/dev/null || true
@rm -rf .coverage htmlcov/ dist/ build/ *.egg-info/ .molecule/ 2>/dev/null || true
# ── Dockerfile linting ────────────────────────────────────────────────────────
#
# Lint Dockerfiles with hadolint. Fails fast if hadolint is not installed
# (no silent skip). Set DEVX_DOCKERFILE_PATHS to the directory containing
# your Dockerfiles (default: docker).
#
# Usage:
# make devx-lint-dockerfiles (lints docker/ directory)
# make devx-lint-dockerfiles DEVX_DOCKERFILE_PATHS=ansible (lints ansible/)
devx-lint-dockerfiles:
@echo "[devx-lint-dockerfiles] Linting Dockerfiles with hadolint..."
@if ! command -v hadolint >/dev/null 2>&1; then \
echo "[devx-lint-dockerfiles] ERROR: hadolint not found. Install from https://github.com/hadolint/hadolint/releases" >&2; \
exit 1; \
fi
@find $(DEVX_DOCKERFILE_PATHS) -name 'Dockerfile*' -exec hadolint {} +
@echo "[devx-lint-dockerfiles] All Dockerfiles passed."
# ── Pre-built image setup ─────────────────────────────────────────────────────
#
# When running inside a pre-built Docker runner image (ci-base, ci-quality,
# ci-full), all deps are already installed in /opt/venv. This target links
# the venv and installs the project itself (with optional extras).
#
# Usage:
# make devx-setup-image (runtime deps only)
# make devx-setup-image EXTRAS=lint (runtime + lint deps)
# make devx-setup-image EXTRAS=ci,lint (runtime + ci + lint deps)
#
# Falls back to setup-ci if /opt/venv is not present (local dev).
# Note: the fallback target name is project-specific (setup-ci, not
# devx-setup-ci) — each project defines its own setup-ci target.
devx-setup-image:
@if [ -d /opt/venv ]; then ln -sf /opt/venv $(DEVX_VENV); . $(DEVX_BIN)/activate; \
_U="$${CI_GITEA_USERNAME:-emil}"; \
if [ -n "$$CI_GITEA_TOKEN" ]; then export PIP_EXTRA_INDEX_URL="https://$$_U:$$CI_GITEA_TOKEN@$(DEVX_GITEA_PYPI_HOST)/api/packages/$(DEVX_GITEA_PYPI_ORG)/pypi/simple/"; fi; \
pip install --no-cache-dir -e .$(if $(EXTRAS),[$(EXTRAS)],); \
echo "[devx-setup-image] Linked /opt/venv$(if $(EXTRAS), with [$(EXTRAS)],)."; \
else echo "[devx-setup-image] /opt/venv not found — falling back to setup-ci"; $(MAKE) setup-ci; fi
# ── Docker image build / push / cleanup ───────────────────────────────────────
#
# Variables:
# DEVX_GITEA_REGISTRY — registry URL (default: git.oblachno.oblachno.fyi)
# DEVX_IMAGE_MANIFEST — path to JSON manifest (default: docker/images.json)
# DEVX_IMAGE_OWNER — package owner for cleanup (default: oblachno-oss)
DEVX_GITEA_REGISTRY ?= git.oblachno.oblachno.fyi
DEVX_IMAGE_MANIFEST ?= docker/images.json
DEVX_IMAGE_OWNER ?= oblachno-oss
# Build all images from manifest (no push)
devx-build-images:
@$(DEVX_PYTHON) -m devx.tools.build_image --manifest $(DEVX_IMAGE_MANIFEST) --pull
# Build and push all images to the Gitea registry
devx-push-images:
@$(DEVX_PYTHON) -m devx.tools.build_image \
--manifest $(DEVX_IMAGE_MANIFEST) \
--registry $(DEVX_GITEA_REGISTRY) \
--push --pull
# Dry-run: show what would be built/pushed
devx-build-images-dry-run:
@$(DEVX_PYTHON) -m devx.tools.build_image \
--manifest $(DEVX_IMAGE_MANIFEST) \
--registry $(DEVX_GITEA_REGISTRY) \
--push --dry-run
# Clean up old image versions (keep last 2 + latest)
devx-clean-images:
@$(DEVX_PYTHON) -m devx.tools.clean_images \
--owner $(DEVX_IMAGE_OWNER) \
--name oblachno-oss/runner-images/ci-base \
--name oblachno-oss/runner-images/ci-quality \
--name oblachno-oss/runner-images/ci-full \
--keep 2
+1
View File
@@ -0,0 +1 @@
"""Molecule testing helpers for Ansible projects."""
+5 -5
View File
@@ -29,7 +29,7 @@ import os
import click
import requests
from devx.config import GITEA_API_URL
from devx.config import GITEA_API_URL, REPO_NAME, REPO_OWNER
DEFAULT_MAX_RUNNERS = 3
@@ -142,12 +142,12 @@ def main(
output_indices: bool,
github_output: bool,
) -> None:
token = os.environ.get("REPO_TOKEN", "")
token = os.environ.get("CI_GITEA_TOKEN", "")
if owner is None:
owner = os.environ.get("DEVX_REPO_OWNER", "oblachno-oss")
owner = os.environ.get("DEVX_REPO_OWNER", "") or REPO_OWNER
if repo is None:
repo = os.environ.get("DEVX_REPO_NAME", "devx")
repo = os.environ.get("DEVX_REPO_NAME", "") or REPO_NAME
count = get_runner_count(GITEA_API_URL, token, owner, repo)
indices = generate_indices(count)
@@ -156,7 +156,7 @@ def main(
gh_output = os.environ.get("GITHUB_OUTPUT")
if not gh_output:
raise click.ClickException("GITHUB_OUTPUT environment variable is not set")
with open(gh_output, "a") as f: # noqa: PTH123
with open(gh_output, "a", encoding="utf-8") as f: # noqa: PTH123
f.write(f"runner-count={count}\n")
f.write(f"runner-indices={json.dumps(indices)}\n")
click.echo(f"Runner count: {count}")
+256 -19
View File
@@ -19,16 +19,19 @@ Usage:
from __future__ import annotations
import tomllib
from dataclasses import dataclass
from pathlib import Path
import click
from devx.ci._shared import lpt_distribute, write_github_env
from devx.i18n import _
from devx.molecule.platforms import PLATFORMS
from devx.molecule.platforms import PLATFORMS, load_platforms
DEFAULT_MAX_RUNNERS = 3
MOLECULE_ROOT = Path("ansible/roles/gitea-runner/molecule")
DEFAULT_ROLES_ROOT = Path("ansible/roles")
@dataclass(frozen=True)
@@ -40,7 +43,8 @@ class TestPair:
def encode(self) -> str:
"""Serialize to a pipe-delimited string for CI consumption."""
return f"{self.scenario}|{self.platform['name']}|{self.platform['image']}|{self.platform['command']}"
cmd = self.platform["command"].replace(" ", "__SPACE__")
return f"{self.scenario}|{self.platform['name']}|{self.platform['image']}|{cmd}"
@staticmethod
def decode(encoded: str) -> TestPair:
@@ -48,7 +52,31 @@ class TestPair:
parts = encoded.split("|")
return TestPair(
scenario=parts[0],
platform={"name": parts[1], "image": parts[2], "command": parts[3]},
platform={"name": parts[1], "image": parts[2], "command": parts[3].replace("__SPACE__", " ")},
)
@dataclass(frozen=True)
class MultiRoleTestPair:
"""A (role, scenario, platform) combination for multi-role projects."""
role: str
scenario: str
platform: dict[str, str]
def encode(self) -> str:
"""Serialize to a pipe-delimited string: ``role|scenario|platform_name|image|command``."""
cmd = self.platform["command"].replace(" ", "__SPACE__")
return f"{self.role}|{self.scenario}|{self.platform['name']}|{self.platform['image']}|{cmd}"
@staticmethod
def decode(encoded: str) -> MultiRoleTestPair:
"""Deserialize from a pipe-delimited string."""
parts = encoded.split("|")
return MultiRoleTestPair(
role=parts[0],
scenario=parts[1],
platform={"name": parts[2], "image": parts[3], "command": parts[4].replace("__SPACE__", " ")},
)
@@ -62,6 +90,33 @@ def discover_scenarios(root: Path | None = None) -> list[str]:
return sorted(scenarios)
def discover_multi_role_scenarios(roles_root: Path | None = None) -> list[tuple[str, str]]:
"""Discover (role, scenario) pairs across all roles under *roles_root*.
Scans ``roles_root/*/molecule/*/`` for scenario directories, skipping
``common`` and directories starting with ``_``. Returns a sorted list of
``(role_name, scenario_name)`` tuples.
"""
if roles_root is None:
roles_root = DEFAULT_ROLES_ROOT
if not roles_root.is_dir():
raise click.ClickException(_("Roles directory not found: {path}", path=str(roles_root)))
pairs: list[tuple[str, str]] = []
for role_dir in sorted(roles_root.iterdir()):
if not role_dir.is_dir():
continue
mol_dir = role_dir / "molecule"
if not mol_dir.is_dir():
continue
for scenario_dir in mol_dir.iterdir():
if not scenario_dir.is_dir():
continue
if scenario_dir.name.startswith("_") or scenario_dir.name == "common":
continue
pairs.append((role_dir.name, scenario_dir.name))
return pairs
def build_pairs(scenarios: list[str], platforms: list[dict[str, str]] | None = None) -> list[TestPair]:
"""Build the full cross-product of scenarios and platforms."""
if platforms is None:
@@ -69,12 +124,135 @@ def build_pairs(scenarios: list[str], platforms: list[dict[str, str]] | None = N
return [TestPair(s, p) for s in scenarios for p in platforms]
def build_multi_role_pairs(
role_scenarios: list[tuple[str, str]],
platforms: list[dict[str, str]] | None = None,
) -> list[MultiRoleTestPair]:
"""Build the full cross-product of (role, scenario) pairs and platforms."""
if platforms is None:
platforms = PLATFORMS
return [MultiRoleTestPair(r, s, p) for r, s in role_scenarios for p in platforms]
# --- Molecule weight configuration ---
#
# Weights are loaded from ``[tool.devx.molecule.weights]`` in
# ``pyproject.toml``. Each project (infra, grm, …) contributes its own
# weights calibrated from actual CI execution times.
#
# Two key formats are supported:
# - ``"scenario" = weight`` — applies to any role with that scenario name
# - ``"role/scenario" = weight`` — role-specific (takes priority)
#
# Example pyproject.toml::
#
# [tool.devx.molecule.weights]
# "nextcloud" = 15
# "app_container/customer-apps" = 11
# "restore/default" = 11
# "default" = 3
#
# If no configuration is found, a generic default weight is used for all
# scenarios (producing a round-robin distribution).
_DEFAULT_SCENARIO_WEIGHT = 3
def _load_molecule_weights(pyproject_path: str = "pyproject.toml") -> tuple[dict[str, int], dict[tuple[str, str], int]]:
"""Load molecule weights from ``[tool.devx.molecule.weights]`` in pyproject.toml.
Returns a tuple of ``(scenario_weights, role_scenario_weights)``:
- ``scenario_weights``: maps scenario name → weight (applies to any role)
- ``role_scenario_weights``: maps (role, scenario) → weight (role-specific)
"""
path = Path(pyproject_path)
if not path.exists():
return {}, {}
try:
with open(path, "rb") as f: # noqa: PTH123
data = tomllib.load(f)
except (tomllib.TOMLDecodeError, OSError):
return {}, {}
weights_raw = data.get("tool", {}).get("devx", {}).get("molecule", {}).get("weights", {})
if not isinstance(weights_raw, dict):
return {}, {}
scenario_weights: dict[str, int] = {}
role_scenario_weights: dict[tuple[str, str], int] = {}
for key, value in weights_raw.items():
if not isinstance(value, int):
continue
if "/" in key:
role, scenario = key.split("/", 1)
role_scenario_weights[(role.lower(), scenario.lower())] = value
else:
scenario_weights[key.lower()] = value
return scenario_weights, role_scenario_weights
# Load weights once at import time (like devx.config and classify_changes)
_SCENARIO_WEIGHTS, _ROLE_SCENARIO_WEIGHTS = _load_molecule_weights()
def _scenario_weight(scenario: str, role: str | None = None) -> int:
"""Estimate a weight for a scenario based on its name and optionally its role.
Role-specific weights (``"role/scenario"``) take priority over
scenario-name-only weights (``"scenario"``). Falls back to the
default weight if no configuration matches.
"""
s = scenario.lower()
if role is not None:
r = role.lower()
key = (r, s)
if key in _ROLE_SCENARIO_WEIGHTS:
return _ROLE_SCENARIO_WEIGHTS[key]
for key, weight in _SCENARIO_WEIGHTS.items():
if key in s:
return weight
return _DEFAULT_SCENARIO_WEIGHT
def _lpt_distribute[T](items: list[T], weights: list[int], max_runners: int) -> list[list[T]]:
"""Distribute *items* across *max_runners* using LPT (delegates to shared utility)."""
return lpt_distribute(items, weights, max_runners)
def distribute_multi_role(pairs: list[MultiRoleTestPair], max_runners: int) -> list[list[MultiRoleTestPair]]:
"""Split *pairs* into *max_runners* balanced groups using LPT scheduling.
Each pair is weighted by role+scenario heuristics (e.g. ``nextcloud`` is
heavier than ``simple-app``). Pairs are sorted by weight descending and
assigned to the runner with the least total weight.
"""
weights = [_scenario_weight(p.scenario, p.role) for p in pairs]
return _lpt_distribute(pairs, weights, max_runners)
def multi_role_pairs_for_runner(
pairs: list[MultiRoleTestPair], runner_index: int, max_runners: int
) -> list[MultiRoleTestPair]:
"""Return the subset of multi-role pairs assigned to *runner_index* (0-based)."""
groups = distribute_multi_role(pairs, max_runners)
if runner_index < 0 or runner_index >= len(groups):
raise click.ClickException(
_("Runner index {index} out of range (0..{max})", index=runner_index, max=max_runners - 1)
)
return groups[runner_index]
def distribute(pairs: list[TestPair], max_runners: int) -> list[list[TestPair]]:
"""Split *pairs* into *max_runners* balanced groups (round-robin)."""
groups: list[list[TestPair]] = [[] for _ in range(max_runners)]
for i, pair in enumerate(pairs):
groups[i % max_runners].append(pair)
return groups
"""Split *pairs* into *max_runners* balanced groups using LPT scheduling.
Each pair is weighted by scenario name heuristics (e.g. ``nextcloud`` is
heavier than ``binary``). Pairs are sorted by weight descending and
assigned to the runner with the least total weight.
"""
weights = [_scenario_weight(p.scenario) for p in pairs]
return _lpt_distribute(pairs, weights, max_runners)
def pairs_for_runner(pairs: list[TestPair], runner_index: int, max_runners: int) -> list[TestPair]:
@@ -92,14 +270,8 @@ def pairs_for_runner(pairs: list[TestPair], runner_index: int, max_runners: int)
def _write_github_env(key: str, value: str) -> None:
"""Append a key=value line to the $GITHUB_ENV file."""
import os
gh_env = os.environ.get("GITHUB_ENV")
if not gh_env:
raise click.ClickException("GITHUB_ENV environment variable is not set")
with open(gh_env, "a") as f: # noqa: PTH123
f.write(f"{key}={value}\n")
"""Append a key=value line to the $GITHUB_ENV file (delegates to shared utility)."""
write_github_env(key, value)
@click.command()
@@ -142,6 +314,26 @@ def _write_github_env(key: str, value: str) -> None:
default=False,
help="With --github-env: write SKIP=true when runner-index exceeds max-runners.",
)
@click.option(
"--molecule-root",
type=click.Path(exists=True, file_okay=False, path_type=Path),
default=None,
help="Custom molecule directory (single-role mode). Default: ansible/roles/gitea-runner/molecule.",
)
@click.option(
"--roles-root",
type=click.Path(exists=True, file_okay=False, path_type=Path),
default=None,
help="Roles directory for multi-role discovery (scans */molecule/*/). "
"Use this for projects with multiple Ansible roles. Default: disabled (single-role mode).",
)
@click.option(
"--platforms-file",
type=click.Path(exists=True, file_okay=True, path_type=Path),
default=None,
help="JSON file with custom platform list (each entry: name, image, command). "
"Overrides the default platform matrix. Useful for projects with custom test images.",
)
def cli(
runner_index: int | None,
max_runners: int,
@@ -149,17 +341,58 @@ def cli(
list_platforms: bool,
github_env: bool,
skip_if_excess: bool,
molecule_root: Path | None,
roles_root: Path | None,
platforms_file: Path | None,
) -> None:
scenarios = discover_scenarios()
platforms = load_platforms(platforms_file)
# Multi-role mode: discover (role, scenario) pairs across all roles
if roles_root is not None:
role_scenarios = discover_multi_role_scenarios(roles_root)
if list_all:
for role, scenario in role_scenarios:
click.echo(f"{role}|{scenario}")
return
if list_platforms:
for p in platforms:
click.echo(f"{p['name']}|{p['image']}|{p['command']}")
return
pairs_mr = build_multi_role_pairs(role_scenarios, platforms)
if runner_index is None:
groups = distribute_multi_role(pairs_mr, max_runners)
for i, group in enumerate(groups):
labels = " ".join(p.encode() for p in group) if group else "(none)"
click.echo(f"Runner {i}: {labels}")
return
if skip_if_excess and github_env and runner_index > max_runners:
click.echo(f"Skipping — runner index {runner_index} > max runners {max_runners}")
_write_github_env("TEST_PAIRS", "")
_write_github_env("SKIP", "true")
return
if runner_index < 1:
raise click.ClickException(f"Runner index {runner_index} is out of range (must be >= 1)")
zero_based = runner_index - 1
assigned = multi_role_pairs_for_runner(pairs_mr, zero_based, max_runners)
encoded = " ".join(p.encode() for p in assigned)
if github_env:
_write_github_env("TEST_PAIRS", encoded)
_write_github_env("SKIP", "false")
click.echo(f"Assigned pairs: {encoded}")
return
click.echo(encoded)
return
# Single-role mode (default or --molecule-root)
scenarios = discover_scenarios(molecule_root)
if list_all:
for s in scenarios:
click.echo(s)
return
if list_platforms:
for p in PLATFORMS:
for p in platforms:
click.echo(f"{p['name']}|{p['image']}|{p['command']}")
return
pairs = build_pairs(scenarios)
pairs = build_pairs(scenarios, platforms)
if runner_index is None:
groups = distribute(pairs, max_runners)
for i, group in enumerate(groups):
@@ -174,6 +407,10 @@ def cli(
_write_github_env("SKIP", "true")
return
# Validate runner index is in range
if runner_index < 1:
raise click.ClickException(f"Runner index {runner_index} is out of range (must be >= 1)")
# Convert 1-based CLI index to 0-based internal index
zero_based = runner_index - 1
assigned = pairs_for_runner(pairs, zero_based, max_runners)
+80 -16
View File
@@ -1,7 +1,11 @@
#!/usr/bin/env python3
"""Run molecule tests sequentially while polling Gitea for other runner failures.
Each pair is encoded as ``scenario|platform_name|platform_image|platform_command``.
Each pair is encoded as one of:
- **Single-role (4-part):** ``scenario|platform_name|platform_image|platform_command``
- **Multi-role (5-part):** ``role|scenario|platform_name|platform_image|platform_command``
Pairs are executed one at a time (molecule scenarios share temp directories and
Docker networks, so parallel execution within a single runner is unsafe).
@@ -9,12 +13,16 @@ A background thread polls the Gitea API. If any other molecule matrix runner
reports failure, the current molecule subprocess is killed and this runner
exits early with code 1.
Usage:
python3 -m devx.molecule.molecule_ci_guard <pair1> <pair2> ...
Usage::
# Single-role (grm-style)
python3 -m devx.molecule.molecule_ci_guard pair1 pair2 ...
# Multi-role (infra-style)
python3 -m devx.molecule.molecule_ci_guard --roles-root ansible/roles pair1 pair2 ...
Environment variables:
GITEA_URL Base URL of the Gitea instance.
REPO_TOKEN API token with repo access.
CI_GITEA_TOKEN API token with repo access.
RUN_ID Workflow run ID (GITHUB_RUN_ID).
JOB_NAME Base job name (GITHUB_JOB), e.g. "molecule-tests".
MATRIX_INDEX Current matrix index (runner-index).
@@ -35,6 +43,7 @@ from pathlib import Path
import click
import requests
from devx.config import REPO_NAME, REPO_OWNER
from devx.i18n import _
POLL_INTERVAL = 10
@@ -95,9 +104,25 @@ def build_molecule_cmd(scenario: str) -> list[str]:
return cmd
def parse_pair(pair: str) -> tuple[str, str, str, str, str]:
"""Parse a pair string into (role, scenario, platform_name, platform_image, platform_command).
Supports both 4-part (single-role) and 5-part (multi-role) formats.
For 4-part pairs, role is empty (caller uses default role dir).
Spaces in the command field are encoded as ``__SPACE__`` to survive
shell word-splitting when ``$TEST_PAIRS`` is expanded unquoted.
"""
parts = pair.split("|")
if len(parts) == 4:
return "", parts[0], parts[1], parts[2], parts[3].replace("__SPACE__", " ")
if len(parts) == 5:
return parts[0], parts[1], parts[2], parts[3], parts[4].replace("__SPACE__", " ")
raise click.ClickException(f"Invalid pair format: {pair!r} (expected 4 or 5 pipe-delimited parts)")
def build_env_for_pair(pair: str, base_env: dict[str, str]) -> dict[str, str]:
"""Build environment for a single molecule pair."""
scenario, platform_name, platform_image, platform_command = pair.split("|")
_role, _scenario, platform_name, platform_image, platform_command = parse_pair(pair)
env = base_env.copy()
env["MOLECULE_PLATFORM_NAME"] = platform_name
env["MOLECULE_PLATFORM_IMAGE"] = platform_image
@@ -106,28 +131,55 @@ def build_env_for_pair(pair: str, base_env: dict[str, str]) -> dict[str, str]:
elif "MOLECULE_PLATFORM_COMMAND" in env:
del env["MOLECULE_PLATFORM_COMMAND"]
env["ANSIBLE_ALLOW_BROKEN_CONDITIONALS"] = "true"
# Use a fresh MOLECULE_HOME per pair to avoid stale config cache
# from previous CI runs (causes "Instances missing" errors).
if "MOLECULE_HOME" not in env:
import tempfile
env["MOLECULE_HOME"] = tempfile.mkdtemp(prefix="molecule-ci-")
return env
def resolve_role_dir(role: str, roles_root: Path | None, repo_root: Path) -> Path:
"""Resolve the working directory for a molecule pair.
For multi-role pairs (role non-empty), uses ``roles_root/role``.
For single-role pairs, uses ``repo_root/ansible/roles/gitea-runner``.
"""
if role:
if roles_root is None:
roles_root = repo_root / "ansible" / "roles"
return roles_root / role
return repo_root / "ansible" / "roles" / "gitea-runner"
@click.command()
@click.argument("pairs", nargs=-1, required=True)
def cli(pairs: tuple[str, ...]) -> None:
@click.option(
"--roles-root",
type=click.Path(exists=True, file_okay=False, path_type=Path),
default=None,
help="Root directory for multi-role pairs (e.g. ansible/roles). Required when pairs use 5-part format.",
)
def cli(pairs: tuple[str, ...], roles_root: Path | None) -> None:
"""Run molecule pairs sequentially, stop if another CI runner fails."""
gitea_url = os.environ.get("GITEA_URL", "")
token = os.environ.get("REPO_TOKEN", "")
token = os.environ.get("CI_GITEA_TOKEN", "")
run_id = int(os.environ.get("RUN_ID", "0"))
job_name = os.environ.get("JOB_NAME", "molecule-tests")
current_index = int(os.environ.get("MATRIX_INDEX", "0"))
repository = os.environ.get("GITEA_REPOSITORY", "oblachno-oss/devx")
owner, sep, repo = repository.partition("/")
repository = os.environ.get("GITEA_REPOSITORY", "")
owner, _sep, repo = repository.partition("/")
if not owner or not repo:
owner, repo = "oblachno-oss", "devx"
owner, repo = REPO_OWNER, REPO_NAME
if not all([gitea_url, token, run_id]):
click.echo(_("GITEA_URL/REPO_TOKEN/RUN_ID not set; running without cross-runner cancellation."))
click.echo(_("GITEA_URL/CI_GITEA_TOKEN/RUN_ID not set; running without cross-runner cancellation."))
repo_root = Path(__file__).resolve().parent.parent.parent.parent
role_dir = repo_root / "ansible" / "roles" / "gitea-runner"
# When devx is installed as a pip package, __file__ resolves to the
# site-packages directory, not the repo root. Use GITHUB_WORKSPACE
# (set by Gitea Actions) or cwd as the repo root.
repo_root = Path(os.environ.get("GITHUB_WORKSPACE", os.getcwd())).resolve()
base_env = os.environ.copy()
base_env.setdefault("DOCKER_HOST", f"unix:///run/user/{os.getuid()}/docker.sock")
@@ -159,16 +211,16 @@ def cli(pairs: tuple[str, ...]) -> None:
if failed_event.is_set():
sys.exit(1)
scenario = pair.split("|")[0]
platform_name = pair.split("|")[1]
role, scenario, platform_name, _img, _cmd = parse_pair(pair)
click.echo(_("Running: {scenario} on {platform}", scenario=scenario, platform=platform_name))
cmd = build_molecule_cmd(scenario)
env = build_env_for_pair(pair, base_env)
cwd = resolve_role_dir(role, roles_root, repo_root)
process = subprocess.Popen( # nosec B603
cmd,
cwd=str(role_dir),
cwd=str(cwd),
env=env,
preexec_fn=os.setsid,
)
@@ -193,12 +245,24 @@ def cli(pairs: tuple[str, ...]) -> None:
sys.exit(1)
rc = process.returncode
if rc != 0:
click.echo(_("FAILED: {pair} exited with code {code}", pair=pair, code=rc))
sys.exit(rc)
click.echo(_("PASSED: {pair}", pair=pair))
# Prune Docker data between scenarios to prevent disk exhaustion
# in Docker-in-Docker molecule containers (each scenario pulls
# hundreds of MB of images that accumulate across pairs).
with contextlib.suppress(subprocess.SubprocessError, OSError):
subprocess.run( # nosec B603, B607
["docker", "system", "prune", "-af", "--volumes"],
check=False,
capture_output=True,
timeout=60,
)
click.echo(_("All molecule tests passed."))
finally:
stop_event.set()
+33 -7
View File
@@ -10,13 +10,39 @@ dev tools and CI scripts.
from __future__ import annotations
#: Supported OS platform matrix.
import json
from pathlib import Path
#: Default supported OS platform matrix.
#: Each entry maps a short name to (image, command).
#: The command must be systemd since rootless Docker requires
#: loginctl/systemctl --user.
#: Uses the project's pre-built molecule-test-base image with
#: ``sleep infinity`` (NOT systemd) to avoid cgroup v2 failures.
PLATFORMS: list[dict[str, str]] = [
{"name": "ubuntu-2204", "image": "geerlingguy/docker-ubuntu2204-ansible:latest", "command": "/lib/systemd/systemd"},
{"name": "ubuntu-2404", "image": "geerlingguy/docker-ubuntu2404-ansible:latest", "command": "/lib/systemd/systemd"},
{"name": "debian-12", "image": "geerlingguy/docker-debian12-ansible:latest", "command": "/lib/systemd/systemd"},
{"name": "archlinux", "image": "marcstraube/archlinux-ansible:latest", "command": "/usr/lib/systemd/systemd"},
{
"name": "ubuntu-2604",
"image": "git.oblachno.oblachno.fyi/oblachno/molecule-test-base:latest",
"command": "sleep infinity",
},
]
def load_platforms(platforms_file: str | Path | None = None) -> list[dict[str, str]]:
"""Load platforms from a JSON file, falling back to PLATFORMS.
Args:
platforms_file: Path to a JSON file with a list of platform dicts.
Each dict must have ``name``, ``image``, and ``command`` keys.
Returns:
List of platform dictionaries.
"""
if platforms_file is None:
return PLATFORMS
path = Path(platforms_file)
if not path.is_file():
return PLATFORMS
with path.open(encoding="utf-8") as f:
data = json.load(f)
if not isinstance(data, list) or not data:
return PLATFORMS
return data
+202
View File
@@ -0,0 +1,202 @@
#!/usr/bin/env python3
"""Ensure Docker is available for molecule tests in CI.
CI runners (e.g. ``gitea/runner-images:ubuntu-latest``) may have the host's
Docker socket mounted. This module verifies Docker is accessible and
sets ``DOCKER_HOST`` explicitly so molecule's Python docker library
connects to the same socket as the Docker CLI.
If the host socket is not available, it tries the rootless socket, then
starts a local ``dockerd`` with the vfs storage driver (requires
privileged container).
Usage::
python3 -m devx.molecule.start_docker [--timeout 30]
"""
from __future__ import annotations
import glob
import os
import subprocess # nosec B404
import sys
import tempfile
import time
import click
from devx.i18n import _
DEFAULT_TIMEOUT = 30
DOCKER_SOCK = "/var/run/docker.sock"
# Rootless socket fallback (e.g. /run/user/994/docker.sock)
ROOTLESS_SOCK = f"/run/user/{os.getuid()}/docker.sock"
def is_docker_ready() -> bool:
"""Check if Docker daemon is responding on the configured socket."""
docker_host = os.environ.get("DOCKER_HOST", f"unix://{DOCKER_SOCK}")
result = subprocess.run( # nosec B603 B607
["docker", "info"],
capture_output=True,
check=False,
env={**os.environ, "DOCKER_HOST": docker_host},
)
return result.returncode == 0
def _diagnose_socket() -> None:
"""Print diagnostic info about the Docker socket."""
click.echo(f"DOCKER_HOST = {os.environ.get('DOCKER_HOST', '(not set)')}")
click.echo(f"Socket path: {DOCKER_SOCK}")
click.echo(f"Socket exists: {os.path.exists(DOCKER_SOCK)}")
if os.path.exists(DOCKER_SOCK):
stat = os.stat(DOCKER_SOCK)
click.echo(f"Socket mode: {oct(stat.st_mode)}")
click.echo(f"Socket uid: {stat.st_uid}, gid: {stat.st_gid}")
# Check if it's a mount point
result = subprocess.run( # nosec B603 B607
["mount"],
capture_output=True,
check=False,
text=True,
)
docker_mounts = [line for line in result.stdout.splitlines() if "docker" in line.lower()]
if docker_mounts:
click.echo("Docker-related mounts:")
for line in docker_mounts:
click.echo(f" {line}")
else:
click.echo("No Docker-related mounts found")
# Check docker context
result = subprocess.run( # nosec B603 B607
["docker", "context", "ls"],
capture_output=True,
check=False,
text=True,
)
click.echo(f"Docker contexts:\n{result.stdout}")
# Try docker info without DOCKER_HOST
result = subprocess.run( # nosec B603 B607
["docker", "info"],
capture_output=True,
check=False,
text=True,
)
click.echo(f"docker info (no DOCKER_HOST): rc={result.returncode}")
if result.returncode != 0:
click.echo(f" stderr: {result.stderr[:500]}")
else:
# Print server version and storage driver
for line in result.stdout.splitlines():
if "Server Version" in line or "Storage Driver" in line or "Docker Root Dir" in line:
click.echo(f" {line.strip()}")
def start_docker_daemon(timeout: int = DEFAULT_TIMEOUT) -> bool:
"""Ensure Docker is ready for molecule tests.
First tries the host socket. If that works, sets ``DOCKER_HOST`` and
returns immediately. If not, tries the rootless socket. If neither
works, starts a local ``dockerd`` with vfs storage driver (requires
privileged container).
Returns ``True`` if Docker is ready, ``False`` if it failed to
start within the timeout.
"""
# Point Docker CLI and Python library to the socket explicitly
os.environ["DOCKER_HOST"] = f"unix://{DOCKER_SOCK}"
# Diagnose socket state
click.echo("--- Docker socket diagnostics ---")
_diagnose_socket()
click.echo("--- End diagnostics ---")
# Check if host Docker is already available
if is_docker_ready():
click.echo(_("Docker daemon already running"))
return True
# Try rootless socket (e.g. /run/user/994/docker.sock)
click.echo(f"Trying rootless socket: {ROOTLESS_SOCK}")
os.environ["DOCKER_HOST"] = f"unix://{ROOTLESS_SOCK}"
if os.path.exists(ROOTLESS_SOCK) and is_docker_ready():
click.echo(_("Docker daemon already running"))
return True
# Scan for any rootless sockets at other UIDs
for sock in sorted(glob.glob("/run/user/*/docker.sock")):
if sock == ROOTLESS_SOCK:
continue
click.echo(f"Trying alternative rootless socket: {sock}")
os.environ["DOCKER_HOST"] = f"unix://{sock}"
if is_docker_ready():
click.echo(_("Docker daemon already running"))
return True
click.echo(_("Host Docker not available, starting local dockerd..."))
# Reset DOCKER_HOST to host socket for local dockerd
os.environ["DOCKER_HOST"] = f"unix://{DOCKER_SOCK}"
# Start local dockerd (requires privileged container)
log_file = tempfile.NamedTemporaryFile( # noqa: SIM115
mode="w", suffix="dockerd.log", delete=False
)
click.echo(f"dockerd log: {log_file.name}")
subprocess.Popen( # nosec B603 B607
[
"dockerd",
"--storage-driver",
"vfs",
"-H",
f"unix://{DOCKER_SOCK}",
],
stdout=log_file,
stderr=subprocess.STDOUT,
start_new_session=True,
)
for _i in range(timeout):
if is_docker_ready():
click.echo(_("Docker daemon started"))
return True
time.sleep(1)
# Print dockerd log on failure
click.echo(_("Docker daemon failed to start"))
click.echo("--- dockerd log ---")
try:
with open(log_file.name, encoding="utf-8") as f:
log_content = f.read()
click.echo(log_content[-3000:] if len(log_content) > 3000 else log_content)
except OSError as e:
click.echo(f"Could not read log: {e}")
click.echo("--- End dockerd log ---")
return False
@click.command()
@click.option(
"--timeout",
default=DEFAULT_TIMEOUT,
type=int,
help="Seconds to wait for Docker daemon to start (default: 30).",
)
def main(timeout: int) -> None:
"""Start Docker daemon for CI molecule tests."""
if start_docker_daemon(timeout):
# Export DOCKER_HOST to GITHUB_ENV for subsequent CI steps
github_env = os.environ.get("GITHUB_ENV")
if github_env and os.environ.get("DOCKER_HOST"):
with open(github_env, "a", encoding="utf-8") as f:
f.write(f"DOCKER_HOST={os.environ['DOCKER_HOST']}\n")
click.echo(f"Exported DOCKER_HOST={os.environ['DOCKER_HOST']} to GITHUB_ENV")
sys.exit(0)
sys.exit(1)
if __name__ == "__main__": # pragma: no cover
main()
+113
View File
@@ -0,0 +1,113 @@
#!/usr/bin/env python3
"""OpenTofu output helpers for CI/CD deployment scripts.
Provides reusable functions for extracting values from ``tofu output``
in a structured way. This eliminates duplicated ``subprocess.run``
boilerplate across deployment and smoke-test scripts.
Typical usage::
from devx.opentofu import get_tofu_output, get_tofu_vm_ip
vms = get_tofu_output("customer_vms", cwd="tofu/environments/staging",
env={"HCLOUD_TOKEN": token})
ip = get_tofu_vm_ip("customer_vms", "oblachno", cwd="tofu/environments/staging",
env={"HCLOUD_TOKEN": token})
"""
from __future__ import annotations
import json
import subprocess # nosec B404
from pathlib import Path
from typing import Any
def get_tofu_output(
output_name: str,
cwd: str | Path | None = None,
env: dict[str, str] | None = None,
) -> Any:
"""Run ``tofu output -json <output_name>`` and return parsed JSON.
Args:
output_name: The OpenTofu output name to query (e.g. ``customer_vms``).
cwd: Directory to run the command in (the tofu env directory).
env: Environment variables for the subprocess (e.g. ``{"HCLOUD_TOKEN": ...}``).
If ``None``, inherits the current environment.
Returns:
Parsed JSON value from the tofu output.
Raises:
RuntimeError: If ``tofu output`` exits with a non-zero code.
json.JSONDecodeError: If stdout is not valid JSON.
"""
result = subprocess.run( # nosec B603, B607
["tofu", "output", "-json", output_name],
cwd=str(cwd) if cwd else None,
capture_output=True,
text=True,
check=False,
env=env,
)
if result.returncode != 0:
raise RuntimeError(f"tofu output failed: {result.stderr}")
return json.loads(result.stdout)
def get_tofu_vm_ip(
output_name: str,
vm_key: str,
cwd: str | Path | None = None,
env: dict[str, str] | None = None,
ip_field: str = "ipv4",
) -> str:
"""Extract a VM IPv4 address from a tofu output map.
The output is expected to be a JSON object mapping VM names to objects
containing an IP field (default ``ipv4``)::
{"staging": {"ipv4": "1.2.3.4", ...}, ...}
Args:
output_name: The tofu output name (e.g. ``customer_vms``).
vm_key: The key inside the output map (e.g. ``"staging"``).
cwd: Directory to run the command in.
env: Environment variables for the subprocess.
ip_field: The field name for the IP address (default ``ipv4``).
Returns:
The IP address string, or empty string if not found.
"""
data = get_tofu_output(output_name, cwd=cwd, env=env)
if not isinstance(data, dict):
return ""
return str(data.get(vm_key, {}).get(ip_field, ""))
def get_tofu_vm_field(
output_name: str,
vm_key: str,
field: str,
cwd: str | Path | None = None,
env: dict[str, str] | None = None,
) -> str:
"""Extract an arbitrary field from a VM entry in tofu output.
Like :func:`get_tofu_vm_ip` but for any field (e.g. ``volume_linux_device``).
Args:
output_name: The tofu output name.
vm_key: The key inside the output map.
field: The field name to extract.
cwd: Directory to run the command in.
env: Environment variables for the subprocess.
Returns:
The field value as a string, or empty string if not found.
"""
data = get_tofu_output(output_name, cwd=cwd, env=env)
if not isinstance(data, dict):
return ""
return str(data.get(vm_key, {}).get(field, ""))
+24
View File
@@ -0,0 +1,24 @@
"""Shared utilities for tools modules."""
from __future__ import annotations
import platform
import click
def arch_string() -> str:
"""Return the architecture string used by release assets.
Maps ``platform.machine()`` to the common release asset naming:
``amd64`` for x86_64, ``arm64`` for aarch64.
Raises:
click.ClickException: If the architecture is not supported.
"""
machine = platform.machine().lower()
if machine in {"x86_64", "amd64"}:
return "amd64"
if machine in {"aarch64", "arm64"}:
return "arm64"
raise click.ClickException(f"Unsupported architecture: {machine}")
+329
View File
@@ -0,0 +1,329 @@
#!/usr/bin/env python3
"""Build and push Docker images to a Gitea container registry.
Replaces raw ``docker build`` / ``docker push`` shell commands with a
tested Python tool. Supports:
- Building from any Dockerfile with a configurable context directory
- Tagging with multiple tags (e.g. ``latest`` + version)
- Optional push to a Gitea registry (with login)
- Dry-run mode (prints commands without executing)
Usage::
# Build a single image
python3 -m devx.tools.build_image \\
--dockerfile docker/ci-base/Dockerfile \\
--tag ci-base:latest \\
--tag ci-base:0.19.3
# Build and push to registry
python3 -m devx.tools.build_image \\
--dockerfile docker/ci-base/Dockerfile \\
--tag ci-base:latest \\
--tag ci-base:0.19.3 \\
--registry git.oblachno.oblachno.fyi \\
--push
# Build multiple images (from a manifest file)
python3 -m devx.tools.build_image --manifest docker/images.json --push
The manifest file is a JSON list of dicts, each with:
- ``name``: image name (e.g. ``ci-base``)
- ``dockerfile``: path to Dockerfile (relative to repo root)
- ``context``: build context directory (optional, defaults to repo root)
- ``tags``: list of tags (optional, defaults to ``["latest"]``)
Registry authentication uses ``CI_GITEA_TOKEN`` and ``CI_GITEA_USERNAME``
environment variables, matching the existing CI workflow patterns.
"""
from __future__ import annotations
import json
import os
import subprocess # nosec B404
from dataclasses import dataclass, field
from pathlib import Path
import click
from devx.i18n import _
@dataclass
class ImageSpec:
"""Specification for a single Docker image to build."""
name: str
dockerfile: str
context: str = "."
tags: list[str] = field(default_factory=lambda: ["latest"])
@classmethod
def from_dict(cls, data: dict[str, object]) -> ImageSpec:
"""Create an ImageSpec from a dict (e.g. from a JSON manifest)."""
name = str(data.get("name", ""))
if not name:
raise ValueError(_("Image manifest entry missing 'name'"))
dockerfile = str(data.get("dockerfile", ""))
if not dockerfile:
raise ValueError(_("Image manifest entry missing 'dockerfile'"))
context = str(data.get("context", "."))
tags_raw = data.get("tags", ["latest"])
if not isinstance(tags_raw, list):
raise ValueError(_("Image 'tags' must be a list"))
tags = [str(t) for t in tags_raw] if tags_raw else ["latest"]
return cls(name=name, dockerfile=dockerfile, context=context, tags=tags)
def load_manifest(path: str | Path) -> list[ImageSpec]:
"""Load a JSON manifest file describing images to build.
The file must contain a JSON list of dicts with at least ``name`` and
``dockerfile`` keys. ``context`` and ``tags`` are optional.
Returns a list of :class:`ImageSpec` instances.
"""
p = Path(path)
if not p.is_file():
raise click.ClickException(_("Manifest file not found: {path}", path=p))
with p.open(encoding="utf-8") as f: # noqa: PTH123
data = json.load(f)
if not isinstance(data, list):
raise click.ClickException(_("Manifest must be a JSON list"))
return [ImageSpec.from_dict(entry) for entry in data]
def build_full_tag(registry: str | None, name: str, tag: str) -> str:
"""Build a full image tag, optionally prefixed with a registry.
>>> build_full_tag(None, "ci-base", "latest")
'ci-base:latest'
>>> build_full_tag("git.example.com", "ci-base", "0.1.0")
'git.example.com/ci-base:0.1.0'
"""
if registry:
return f"{registry}/{name}:{tag}"
return f"{name}:{tag}"
def registry_login(
registry: str,
username: str,
token: str,
*,
dry_run: bool = False,
) -> bool:
"""Log in to a Docker registry.
Returns True on success, False on failure.
In dry-run mode, prints the command without executing.
"""
cmd = ["docker", "login", registry, "-u", username, "--password-stdin"]
if dry_run:
click.echo(f"[dry-run] {' '.join(cmd)}")
return True
result = subprocess.run( # nosec B603
cmd,
input=token,
text=True,
capture_output=True,
check=False,
)
if result.returncode != 0:
click.echo(
_("Registry login failed: {error}", error=result.stderr.strip()),
err=True,
)
return False
click.echo(f"Logged in to {registry}")
return True
def build_image(
spec: ImageSpec,
registry: str | None = None,
*,
dry_run: bool = False,
pull: bool = False,
) -> bool:
"""Build a Docker image from a Dockerfile.
Tags the image with all specified tags, optionally prefixed with the
registry. Returns True on success, False on failure.
"""
if not Path(spec.dockerfile).is_file():
click.echo(
_("Dockerfile not found: {path}", path=spec.dockerfile),
err=True,
)
return False
full_tags = [build_full_tag(registry, spec.name, t) for t in spec.tags]
cmd = ["docker", "build"]
if pull:
cmd.append("--pull")
for ft in full_tags:
cmd.extend(["-t", ft])
cmd.extend(["-f", spec.dockerfile, spec.context])
if dry_run:
click.echo(f"[dry-run] {' '.join(cmd)}")
return True
click.echo(f"Building {spec.name} ({len(full_tags)} tag(s))...")
result = subprocess.run( # nosec B603
cmd,
check=False,
)
if result.returncode != 0:
click.echo(_("Build failed for {name}", name=spec.name), err=True)
return False
click.echo(f"Built {spec.name}")
return True
def push_image(
spec: ImageSpec,
registry: str,
*,
dry_run: bool = False,
) -> bool:
"""Push all tags of a Docker image to the registry.
Returns True if all pushes succeed, False if any fail.
"""
full_tags = [build_full_tag(registry, spec.name, t) for t in spec.tags]
all_ok = True
for ft in full_tags:
cmd = ["docker", "push", ft]
if dry_run:
click.echo(f"[dry-run] {' '.join(cmd)}")
continue
click.echo(f"Pushing {ft}...")
result = subprocess.run( # nosec B603
cmd,
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
click.echo(
_("Push failed for {tag}: {error}", tag=ft, error=result.stderr.strip()),
err=True,
)
all_ok = False
else:
click.echo(f"Pushed {ft}")
return all_ok
def _get_registry_creds() -> tuple[str, str]:
"""Get registry credentials from environment variables."""
token = os.environ.get("CI_GITEA_TOKEN", "")
username = os.environ.get("CI_GITEA_USERNAME", "")
return username, token
@click.command()
@click.option(
"--dockerfile",
"dockerfile",
default=None,
help="Path to Dockerfile (for single-image build).",
)
@click.option(
"--context",
"context",
default=".",
help="Build context directory (for single-image build).",
)
@click.option(
"--name",
"name",
default=None,
help="Image name (for single-image build).",
)
@click.option(
"--tag",
"tags",
multiple=True,
help="Tag(s) for the image. Can be repeated. Defaults to 'latest'.",
)
@click.option(
"--manifest",
"manifest",
default=None,
help="Path to JSON manifest file listing images to build.",
)
@click.option(
"--registry",
"registry",
default=None,
help="Registry URL (e.g. git.example.com). If set with --push, images are tagged and pushed there.",
)
@click.option(
"--push",
is_flag=True,
default=False,
help="Push images to the registry after building.",
)
@click.option(
"--dry-run",
is_flag=True,
default=False,
help="Print commands without executing.",
)
@click.option(
"--pull",
is_flag=True,
default=False,
help="Pass --pull to docker build (always fetch latest base image).",
)
def main(
dockerfile: str | None,
context: str,
name: str | None,
tags: tuple[str, ...],
manifest: str | None,
registry: str | None,
push: bool,
dry_run: bool,
pull: bool,
) -> None:
"""Build and optionally push Docker images to a Gitea registry."""
if manifest:
specs = load_manifest(manifest)
elif dockerfile and name:
tag_list = list(tags) if tags else ["latest"]
specs = [ImageSpec(name=name, dockerfile=dockerfile, context=context, tags=tag_list)]
else:
raise click.ClickException(_("Provide --manifest or both --dockerfile and --name"))
if push:
if not registry:
raise click.ClickException(_("--push requires --registry"))
username, token = _get_registry_creds()
if not token or not username:
raise click.ClickException(
_("Registry credentials required: set CI_GITEA_TOKEN and CI_GITEA_USERNAME env vars")
)
if not registry_login(registry, username, token, dry_run=dry_run):
raise click.ClickException(_("Registry login failed"))
failed: list[str] = []
for spec in specs:
if not build_image(spec, registry, dry_run=dry_run, pull=pull):
failed.append(spec.name)
continue
if push and not push_image(spec, registry, dry_run=dry_run): # type: ignore[arg-type]
failed.append(spec.name)
if failed:
raise click.ClickException(_("Failed images: {names}", names=", ".join(failed)))
click.echo(f"\nDone. {len(specs)} image(s) processed.")
if __name__ == "__main__": # pragma: no cover
main() # pragma: no cover
+237
View File
@@ -0,0 +1,237 @@
#!/usr/bin/env python3
"""Validate agent documentation and user docs for stale file references.
Scans documentation files (``.devin/``, ``docs/``, ``README.md``) for:
- References to files that no longer exist
- References to deleted files (configurable blocklist)
- References to deprecated patterns (configurable regex patterns)
Configuration (``[tool.devx.check_agent_docs]`` in pyproject.toml):
``scan_dirs`` — directories to scan for docs (default: ``[".devin", "docs"]``)
``scan_files`` — specific files to scan (default: ``["README.md", "README.rst"]``)
``scan_extensions`` — file extensions to scan (default: ``[".md", ".yml", ".yaml"]``)
``excluded_paths`` — paths to exclude from scanning (default: ``["docs/retrospectives"]``)
``deleted_files`` — list of file paths that should never be referenced
``deprecated_patterns`` — list of regex patterns for deprecated references
``legitimate_indicators`` — substrings that indicate a legitimate reference to a deprecated pattern
``repo_path_prefixes`` — path prefixes that indicate a repo-relative reference
(default: ``["ansible/", "scripts/", "tofu/", ".devin/", "src/"]``)
``min_path_ref_length`` — minimum length for a path reference to be checked (default: 5)
Usage::
python3 -m devx.tools.check_agent_docs
"""
from __future__ import annotations
import contextlib
import re
from pathlib import Path
import click
from devx.config import _load_pyproject_devx
from devx.i18n import _
MIN_PATH_REF_LENGTH_DEFAULT = 5
# Pattern that matches file path references in markdown or code
FILE_REF_RE = re.compile(
r"(?:`|\")?"
r"([\w\-./]+(?:\.[a-zA-Z0-9]+))"
r"(?:`|\))?"
)
DEFAULT_SCAN_DIRS = [".devin", "docs"]
DEFAULT_SCAN_FILES = ["README.md", "README.rst"]
DEFAULT_SCAN_EXTENSIONS = [".md", ".yml", ".yaml"]
DEFAULT_EXCLUDED_PATHS = ["docs/retrospectives"]
DEFAULT_REPO_PATH_PREFIXES = ["ansible/", "scripts/", "tofu/", ".devin/", "src/"]
def _load_config() -> dict[str, object]:
"""Load check_agent_docs configuration from pyproject.toml."""
devx_cfg = _load_pyproject_devx()
cfg_raw = devx_cfg.get("check_agent_docs", {})
if not isinstance(cfg_raw, dict):
return {}
return cfg_raw # type: ignore[return-value]
def _should_skip(path: Path, excluded_paths: list[str], repo_root: Path) -> bool:
"""Check if a path should be excluded from scanning."""
try:
rel = str(path.relative_to(repo_root))
except ValueError:
return False
return any(excluded in rel for excluded in excluded_paths)
def _is_legitimate_ref(line: str, legitimate_indicators: list[str]) -> bool:
"""Check if a line contains a legitimate reference to a deprecated pattern."""
line_lower = line.lower()
return any(legit.lower() in line_lower for legit in legitimate_indicators)
def _collect_doc_files(
repo_root: Path,
scan_dirs: list[str],
scan_files: list[str],
scan_extensions: list[str],
excluded_paths: list[str],
) -> list[Path]:
"""Collect all documentation files to scan."""
files: list[Path] = []
for scan_dir_name in scan_dirs:
scan_dir = repo_root / scan_dir_name
if not scan_dir.exists():
continue
for ext in scan_extensions:
for path in scan_dir.glob(f"**/*{ext}"):
if not _should_skip(path, excluded_paths, repo_root):
files.append(path)
for readme_name in scan_files:
path = repo_root / readme_name
if path.exists() and not _should_skip(path, excluded_paths, repo_root):
files.append(path)
# Deduplicate while preserving order
seen: set[Path] = set()
unique: list[Path] = []
for f in files:
if f not in seen:
seen.add(f)
unique.append(f)
return unique
def _check_file(
path: Path,
repo_root: Path,
deleted_files: set[str],
deprecated_patterns: list[re.Pattern[str]],
legitimate_indicators: list[str],
repo_path_prefixes: list[str],
min_path_ref_length: int,
skip_ref_prefixes: list[str],
) -> list[str]:
"""Check a single file for stale references."""
issues: list[str] = []
rel_path = path.relative_to(repo_root)
try:
content = path.read_text(encoding="utf-8")
except UnicodeDecodeError:
return issues
for lineno, line in enumerate(content.splitlines(), start=1):
# Check for deleted file references
for deleted in deleted_files:
if deleted in line:
issues.append(f"{rel_path}:{lineno}: references deleted file '{deleted}'")
# Check for deprecated pattern references
for pattern in deprecated_patterns:
if pattern.search(line) and not _is_legitimate_ref(line, legitimate_indicators):
issues.append(f"{rel_path}:{lineno}: matches deprecated pattern '{pattern.pattern}'")
# Check for references to files that don't exist
for match in FILE_REF_RE.finditer(line):
ref = match.group(1)
# Skip URLs, bare words, and short strings
if "/" not in ref or len(ref) < min_path_ref_length:
continue
# Only check references that look like repo paths
if not any(ref.startswith(prefix) for prefix in repo_path_prefixes):
continue
# Skip references matching configured skip prefixes (e.g. aspirational test files)
if any(ref.startswith(prefix) for prefix in skip_ref_prefixes):
continue
candidate = repo_root / ref
if not candidate.exists():
issues.append(f"{rel_path}:{lineno}: references non-existent file '{ref}'")
return issues
@click.command()
def cli() -> None:
"""Validate agent documentation and user docs for stale file references."""
repo_root = Path.cwd()
cfg = _load_config()
scan_dirs_raw = cfg.get("scan_dirs")
scan_dirs: list[str] = [str(d) for d in scan_dirs_raw] if isinstance(scan_dirs_raw, list) else DEFAULT_SCAN_DIRS
scan_files_raw = cfg.get("scan_files")
scan_files: list[str] = [str(d) for d in scan_files_raw] if isinstance(scan_files_raw, list) else DEFAULT_SCAN_FILES
scan_ext_raw = cfg.get("scan_extensions")
scan_extensions: list[str] = (
[str(d) for d in scan_ext_raw] if isinstance(scan_ext_raw, list) else DEFAULT_SCAN_EXTENSIONS
)
excluded_raw = cfg.get("excluded_paths")
excluded_paths: list[str] = (
[str(d) for d in excluded_raw] if isinstance(excluded_raw, list) else DEFAULT_EXCLUDED_PATHS
)
prefixes_raw = cfg.get("repo_path_prefixes")
repo_path_prefixes: list[str] = (
[str(d) for d in prefixes_raw] if isinstance(prefixes_raw, list) else DEFAULT_REPO_PATH_PREFIXES
)
min_len_raw = cfg.get("min_path_ref_length")
min_path_ref_length: int = int(min_len_raw) if isinstance(min_len_raw, int) else MIN_PATH_REF_LENGTH_DEFAULT
skip_prefixes_raw = cfg.get("skip_ref_prefixes", [])
skip_ref_prefixes: list[str] = [str(d) for d in skip_prefixes_raw] if isinstance(skip_prefixes_raw, list) else []
deleted_files: set[str] = set()
deleted_raw = cfg.get("deleted_files", [])
if isinstance(deleted_raw, list):
deleted_files = {str(d) for d in deleted_raw}
deprecated_patterns: list[re.Pattern[str]] = []
deprecated_raw = cfg.get("deprecated_patterns", [])
if isinstance(deprecated_raw, list):
for pattern_str in deprecated_raw:
if isinstance(pattern_str, str):
with contextlib.suppress(re.error):
deprecated_patterns.append(re.compile(pattern_str))
legitimate_indicators: list[str] = []
legit_raw = cfg.get("legitimate_indicators", [])
if isinstance(legit_raw, list):
legitimate_indicators = [str(s) for s in legit_raw]
files = _collect_doc_files(repo_root, scan_dirs, scan_files, scan_extensions, excluded_paths)
all_issues: list[str] = []
for path in sorted(files):
issues = _check_file(
path,
repo_root,
deleted_files,
deprecated_patterns,
legitimate_indicators,
repo_path_prefixes,
min_path_ref_length,
skip_ref_prefixes,
)
all_issues.extend(issues)
if all_issues:
click.echo(f"[check_agent_docs] Found {len(all_issues)} issue(s):\n", err=True)
for issue in all_issues:
click.echo(issue, err=True)
click.echo(
f"\n[check_agent_docs] FAILED: {len(all_issues)} stale reference(s)",
err=True,
)
raise click.ClickException(_("Found {count} stale documentation reference(s)", count=len(all_issues)))
click.echo(_("[check_agent_docs] Passed: scanned {count} file(s), no stale references", count=len(files)))
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/env python3
"""Validate devx configuration consistency in pyproject.toml.
Checks:
1. [tool.devx] section exists with required keys (task_prefix, vikunja_project_id, repo_owner, repo_name)
2. devx version is consistent across all extras that mention it
Usage::
python3 -m devx.tools.check_config
"""
from __future__ import annotations
import re
import tomllib
from pathlib import Path
import click
from devx.i18n import _
@click.command()
def cli() -> None:
"""Validate devx configuration in pyproject.toml."""
path = Path("pyproject.toml")
if not path.exists():
raise click.ClickException(_("pyproject.toml not found in current directory."))
with open(path, "rb") as f: # noqa: PTH123
data = tomllib.load(f)
errors: list[str] = []
# Check [tool.devx] section
devx_cfg = data.get("tool", {}).get("devx", {})
required_keys = {"task_prefix", "vikunja_project_id", "repo_owner", "repo_name"}
missing = required_keys - set(devx_cfg.keys())
if missing:
errors.append(
_("[tool.devx] missing required keys: {keys}", keys=", ".join(sorted(missing))),
)
# Check devx version consistency across extras
optional_deps = data.get("project", {}).get("optional-dependencies", {})
devx_versions: dict[str, str] = {}
for extra_name, deps in optional_deps.items():
for dep in deps:
# Match "devx>=X.Y.Z", "devx==X.Y.Z", "devx>X.Y.Z", etc.
m = re.search(r"\bdevx\s*(>=|==|>|<=|<|~=)\s*([\d.]+)", dep)
if m:
devx_versions[extra_name] = m.group(2)
if devx_versions:
unique_versions = set(devx_versions.values())
if len(unique_versions) > 1:
detail = ", ".join(f"{extra}={v}" for extra, v in sorted(devx_versions.items()))
errors.append(
_("devx version mismatch across extras: {detail}", detail=detail),
)
if errors:
for err in errors:
click.echo(f"ERROR: {err}", err=True)
raise click.ClickException(_("Configuration validation failed."))
click.echo(_("Configuration OK: [tool.devx] present, devx versions consistent."))
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+175
View File
@@ -0,0 +1,175 @@
#!/usr/bin/env python3
"""Detect module-level mutable globals that may cause test isolation bugs.
Scans Python files for patterns like::
_SEEN: set[Path] = set()
_CACHE: dict[Path, Any] = {}
PATHS: list[Path] = []
These are hazardous because one test mutates the container and the next
sees stale state. The script reports the file/line and suggests a factory
function or fixture replacement.
Configuration (``[tool.devx.check_mutable_globals]`` in pyproject.toml):
``scan_dirs`` — list of directories to scan (default: ``["scripts", "tests"]``)
``skip_dirs`` — directory names to skip (default: ``__pycache__``, ``.pytest_cache``, ``venv``, ``.venv``)
``known_safe`` — list of ``"path:line:var_name"`` entries to ignore
Usage::
python3 -m devx.tools.check_mutable_globals
python3 -m devx.tools.check_mutable_globals --scan-dir src --scan-dir tests
"""
from __future__ import annotations
import ast
import contextlib
from pathlib import Path
import click
from devx.config import _load_pyproject_devx
from devx.i18n import _
MUTABLE_TYPES = {"set", "dict", "list"}
PATH_HINTS = ("path", "paths", "seen", "cache", "memo", "registry")
DEFAULT_SCAN_DIRS = ["scripts", "tests"]
DEFAULT_SKIP_DIRS = {"__pycache__", ".pytest_cache", "venv", ".venv"}
def _load_config() -> tuple[list[str], set[str], set[tuple[str, int, str]]]:
"""Load configuration from pyproject.toml [tool.devx.check_mutable_globals]."""
devx_cfg = _load_pyproject_devx()
cfg_raw = devx_cfg.get("check_mutable_globals", {})
if not isinstance(cfg_raw, dict):
return DEFAULT_SCAN_DIRS, DEFAULT_SKIP_DIRS, set()
cfg: dict[str, object] = cfg_raw # type: ignore[assignment]
scan_dirs_raw = cfg.get("scan_dirs", DEFAULT_SCAN_DIRS)
scan_dirs: list[str] = [str(d) for d in scan_dirs_raw] if isinstance(scan_dirs_raw, list) else DEFAULT_SCAN_DIRS
skip_dirs_raw = cfg.get("skip_dirs", list(DEFAULT_SKIP_DIRS))
skip_dirs: set[str] = {str(d) for d in skip_dirs_raw} if isinstance(skip_dirs_raw, list) else DEFAULT_SKIP_DIRS
known_safe_raw = cfg.get("known_safe", [])
known_safe: set[tuple[str, int, str]] = set()
if isinstance(known_safe_raw, list):
for entry in known_safe_raw:
if isinstance(entry, str) and entry.count(":") >= 2:
parts = entry.rsplit(":", 2)
with contextlib.suppress(ValueError):
known_safe.add((parts[0], int(parts[1]), parts[2]))
return scan_dirs, skip_dirs, known_safe
def _should_skip(path: Path, skip_dirs: set[str]) -> bool:
return any(part in skip_dirs for part in path.parts)
def find_mutable_globals(
file_path: Path,
repo_root: Path,
known_safe: set[tuple[str, int, str]],
) -> list[str]:
"""Return a list of issue strings for mutable globals in *file_path*."""
issues: list[str] = []
try:
source = file_path.read_text(encoding="utf-8")
tree = ast.parse(source)
except SyntaxError:
return issues
for node in ast.iter_child_nodes(tree):
if not isinstance(node, ast.AnnAssign | ast.Assign):
continue
names: list[str] = []
if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name):
names.append(node.target.id)
if isinstance(node, ast.Assign):
for target in node.targets:
if isinstance(target, ast.Name):
names.append(target.id)
for name in names:
name_lower = name.lower()
value = node.value
if value is None:
continue
is_mutable_literal = False
if isinstance(value, ast.Call):
if isinstance(value.func, ast.Name):
if value.func.id in MUTABLE_TYPES:
is_mutable_literal = True
elif isinstance(value.func, ast.Attribute):
# e.g. collections.defaultdict
pass
elif isinstance(value, (ast.Dict, ast.List, ast.Set)):
is_mutable_literal = True
if not is_mutable_literal:
continue
# Check if the name or type hint suggests Path usage
has_path_hint = any(hint in name_lower for hint in PATH_HINTS)
has_path_type = False
if isinstance(node, ast.AnnAssign) and node.annotation:
ann = ast.unparse(node.annotation)
has_path_type = "Path" in ann
if has_path_hint or has_path_type:
rel = str(file_path.relative_to(repo_root))
if (rel, node.lineno, name) in known_safe:
continue
value_str = ast.unparse(value) if value is not None else "..."
issues.append(
f"{rel}:{node.lineno}: mutable global {name!r} "
f"({value_str}) — use a factory function or pytest fixture"
)
return issues
@click.command()
@click.option(
"--scan-dir",
multiple=True,
help=_("Additional directory to scan (default: scripts, tests). Can be repeated."),
)
def cli(scan_dir: tuple[str, ...]) -> None:
"""Scan for module-level mutable globals that cause test isolation bugs."""
repo_root = Path.cwd()
config_scan_dirs, skip_dirs, known_safe = _load_config()
# CLI --scan-dir overrides config if provided
scan_dirs = list(scan_dir) if scan_dir else config_scan_dirs
all_issues: list[str] = []
for scan_dir_name in scan_dirs:
scan_path = repo_root / scan_dir_name
if not scan_path.exists():
continue
for py_file in scan_path.rglob("*.py"):
if _should_skip(py_file, skip_dirs):
continue
all_issues.extend(find_mutable_globals(py_file, repo_root, known_safe))
if all_issues:
click.echo(f"[check-mutable-globals] FAILED: {len(all_issues)} issue(s)", err=True)
for issue in all_issues:
click.echo(f" {issue}", err=True)
raise click.ClickException(
_("Found {count} mutable global(s) — use factory functions or pytest fixtures.", count=len(all_issues))
)
click.echo(_("[check-mutable-globals] Passed: no mutable path globals found"))
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+98
View File
@@ -0,0 +1,98 @@
#!/usr/bin/env python3
"""Validate that every dependency in pyproject.toml has a documented purpose.
This script does NOT resolve versions or query PyPI. It only ensures that
every dependency listed in ``[project.dependencies]`` or
``[project.optional-dependencies]`` has a corresponding comment nearby
explaining why it is needed.
Failure means a dependency lacks documentation.
Usage::
python3 -m devx.tools.check_pyproject_deps
python3 -m devx.tools.check_pyproject_deps --file path/to/pyproject.toml
"""
from __future__ import annotations
from pathlib import Path
import click
from devx.i18n import _
def check_deps(pyproject_path: Path) -> list[str]:
"""Return a list of issue strings for undocumented dependencies.
An empty list means all dependencies are documented.
"""
if not pyproject_path.exists():
return [str(pyproject_path) + ": file not found"]
content = pyproject_path.read_text(encoding="utf-8")
lines = content.splitlines()
issues: list[str] = []
in_deps_section = False
prev_was_comment = False
for i, raw_line in enumerate(lines, start=1):
stripped = raw_line.strip()
# Detect section headers
if stripped in ("[project.dependencies]", "[project.optional-dependencies]"):
in_deps_section = True
continue
if stripped.startswith("[") and in_deps_section:
in_deps_section = False
continue
if not in_deps_section:
continue
if stripped == "":
continue
# We're inside a dependency list
if stripped.startswith("#"):
prev_was_comment = True
continue
if stripped.startswith("-") or stripped.startswith('"'):
if not prev_was_comment:
issues.append(f"{pyproject_path.name}:{i}: dependency lacks description comment: {stripped}")
prev_was_comment = False
else:
prev_was_comment = False
return issues
@click.command()
@click.option(
"--file",
"pyproject_file",
type=click.Path(path_type=Path),
default=Path("pyproject.toml"),
help=_("Path to pyproject.toml (default: pyproject.toml in CWD)."),
)
def cli(pyproject_file: Path) -> None:
"""Validate that every dependency in pyproject.toml has a documented purpose."""
issues = check_deps(pyproject_file)
if issues:
click.echo(
_("FAILED: {count} undocumented dependency/ies", count=len(issues)),
err=True,
)
for issue in issues:
click.echo(f" {issue}", err=True)
raise click.ClickException(_("Dependencies must have documentation comments."))
click.echo(_("[check-dep-docs] Passed: all dependencies are documented"))
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+239
View File
@@ -0,0 +1,239 @@
#!/usr/bin/env python3
"""Pre-commit / CI check: ensure every changed or new file has corresponding tests.
Configuration (``[tool.devx.check_test_coverage]`` in pyproject.toml):
``rules`` — list of mapping rules, each with:
``source_pattern`` — glob pattern for source files (e.g. ``"scripts/*.py"``)
``test_paths`` — list of test path templates (e.g. ``["scripts/tests/test_{name}", "tests/unit/test_{name}"]``)
``description`` — human-readable description for error messages
``skip_patterns`` — list of file patterns to skip (e.g. ``["__init__.py", "config.py"]``)
``test_file_indicators`` — substrings that identify a file as a test (default: ``["tests/", "/test_", "_test.py"]``)
``skip_extensions`` — file extensions to skip (default: .md, .yml, .yaml, .json, .tf, .sh, .conf, .service)
Built-in defaults cover common Python project layouts (``scripts/*.py``, ``src/**/*.py``).
Project-specific rules are merged with defaults (first match wins).
Usage::
python3 -m devx.tools.check_test_coverage [--staged-only] [--warn-only]
"""
from __future__ import annotations
import fnmatch
import subprocess # nosec B404
from pathlib import Path
import click
from devx.config import _load_pyproject_devx
from devx.i18n import _
DEFAULT_TEST_INDICATORS = ["tests/", "/test_", "_test.py"]
DEFAULT_SKIP_EXTENSIONS = (".md", ".yml", ".yaml", ".json", ".tf", ".sh", ".conf", ".service")
# Built-in rules for common Python project layouts
BUILTIN_RULES: list[dict[str, object]] = [
{
"source_pattern": "scripts/*.py",
"test_paths": ["scripts/tests/test_{name}", "tests/unit/test_{name}"],
"description": "Missing unit test: scripts/tests/test_{name} or tests/unit/test_{name}",
},
{
"source_pattern": "src/**/*.py",
"test_paths": ["tests/unit/test_{name}", "tests/unit/test_{module}_{name}"],
"description": "Missing unit test: tests/unit/test_{name}",
},
]
def _load_rules() -> tuple[list[dict[str, object]], list[str], list[str], tuple[str, ...]]:
"""Load test coverage rules from pyproject.toml."""
devx_cfg = _load_pyproject_devx()
cfg_raw = devx_cfg.get("check_test_coverage", {})
if not isinstance(cfg_raw, dict):
return BUILTIN_RULES, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS
cfg: dict[str, object] = cfg_raw # type: ignore[assignment]
rules_raw = cfg.get("rules", BUILTIN_RULES)
rules: list[dict[str, object]] = [dict(r) for r in rules_raw] if isinstance(rules_raw, list) else BUILTIN_RULES
skip_raw = cfg.get("skip_patterns", [])
skip_patterns: list[str] = [str(s) for s in skip_raw] if isinstance(skip_raw, list) else []
indicators_raw = cfg.get("test_file_indicators", DEFAULT_TEST_INDICATORS)
indicators: list[str] = (
[str(s) for s in indicators_raw] if isinstance(indicators_raw, list) else DEFAULT_TEST_INDICATORS
)
skip_ext_raw = cfg.get("skip_extensions", list(DEFAULT_SKIP_EXTENSIONS))
if isinstance(skip_ext_raw, list):
skip_ext: tuple[str, ...] = tuple(str(s) for s in skip_ext_raw)
else:
skip_ext = DEFAULT_SKIP_EXTENSIONS
return rules, skip_patterns, indicators, skip_ext
def _changed_files(staged_only: bool, repo_root: Path) -> list[str]:
"""Return list of changed file paths relative to repo root."""
if staged_only:
cmd = ["git", "diff", "--cached", "--name-only", "--diff-filter=ACMR"]
else:
# Compare against origin/master for CI usage
cmd = ["git", "diff", "origin/master...HEAD", "--name-only", "--diff-filter=ACMR"]
result = subprocess.run( # nosec B603, B607
cmd, capture_output=True, text=True, check=False, cwd=repo_root
)
if result.returncode != 0:
# fallback: just use staged files
result = subprocess.run( # nosec B603, B607
["git", "diff", "--cached", "--name-only", "--diff-filter=ACMR"],
capture_output=True,
text=True,
check=False,
cwd=repo_root,
)
return [line.strip() for line in result.stdout.splitlines() if line.strip()]
def _is_test_file(filepath: str, indicators: list[str]) -> bool:
"""Check if a file is a test file."""
return any(indicator in filepath for indicator in indicators)
def _should_skip_file(
filepath: str,
skip_patterns: list[str],
skip_extensions: tuple[str, ...],
) -> bool:
"""Check if a file should be skipped."""
if filepath.startswith("."):
return True
if filepath.endswith(skip_extensions):
return True
name = Path(filepath).name
return any(fnmatch.fnmatch(name, pattern) or fnmatch.fnmatch(filepath, pattern) for pattern in skip_patterns)
def _resolve_test_path(template: str, source_path: str, repo_root: Path) -> Path:
"""Resolve a test path template to an actual path.
Templates can use:
- ``{name}`` — the source file's name (without extension)
- ``{module}`` — the source file's parent directory name
- ``{package_prefix}`` — underscore-joined subdirectories (for nested modules)
"""
path = Path(source_path)
name = path.stem
module = path.parent.name
# Build package prefix for nested modules (e.g. scripts/utils/secrets.py -> utils)
parts = path.parts
package_prefix = ""
if len(parts) > 2:
package_prefix = "_".join(parts[1:-1])
resolved = template.format(
name=name,
module=module,
package_prefix=package_prefix,
)
# Normalize hyphens to underscores (Python module naming)
resolved = resolved.replace("-", "_")
return repo_root / resolved
def _find_missing_tests(
files: list[str],
repo_root: Path,
rules: list[dict[str, object]],
skip_patterns: list[str],
test_indicators: list[str],
skip_extensions: tuple[str, ...],
) -> dict[str, str]:
"""Map each untested file to the reason it's untested."""
missing: dict[str, str] = {}
for f in files:
# Skip test files themselves
if _is_test_file(f, test_indicators):
continue
# Skip config, docs, meta files
if _should_skip_file(f, skip_patterns, skip_extensions):
continue
for rule in rules:
pattern = str(rule.get("source_pattern", ""))
if not fnmatch.fnmatch(f, pattern):
continue
test_templates = rule.get("test_paths", [])
if not isinstance(test_templates, list):
continue
description_template = str(rule.get("description", "Missing test for {f}"))
test_paths = [_resolve_test_path(str(t), f, repo_root) for t in test_templates]
# Check if any test path exists (with .py extension)
found = False
for tp in test_paths:
if tp.with_suffix(".py").exists() or tp.exists():
found = True
break
if not found:
# Format description with file info
name = Path(f).stem
missing[f] = description_template.format(
name=name,
f=f,
test_name=f"test_{name}".replace("-", "_"),
)
break
# If no rule matched, the file is not checked (no test requirement)
# This is intentional — only files matching a rule need tests
return missing
@click.command()
@click.option("--staged-only", is_flag=True, help=_("Only check staged files (for pre-commit)"))
@click.option("--warn-only", is_flag=True, help=_("Print warnings but always exit 0"))
def cli(staged_only: bool, warn_only: bool) -> None:
"""Check that changed files have corresponding tests."""
repo_root = Path.cwd()
rules, skip_patterns, test_indicators, skip_extensions = _load_rules()
files = _changed_files(staged_only, repo_root)
if not files:
click.echo(_("[check_test_coverage] No changed files to check."))
return
missing = _find_missing_tests(files, repo_root, rules, skip_patterns, test_indicators, skip_extensions)
if not missing:
click.echo(f"[check_test_coverage] All {len(files)} changed file(s) have tests.")
return
click.echo("[check_test_coverage] FAILED: missing tests for changed files:\n", err=True)
for f, reason in missing.items():
click.echo(f" {f}", err=True)
click.echo(f" -> {reason}", err=True)
click.echo(
_("\n[check_test_coverage] Fix: add the missing test file(s) before committing."),
err=True,
)
if not warn_only:
raise click.ClickException(_("Missing tests for changed files."))
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+95 -11
View File
@@ -1,12 +1,21 @@
#!/usr/bin/env python3
"""Run unit tests and enforce a maximum execution-time budget.
"""Run unit tests and enforce execution-time budgets.
Checks two quality gates:
1. **Total suite time** must not exceed ``--max-seconds``.
2. **Per-test time** — no individual test may exceed ``--max-single-seconds``.
Usage:
python3 -m devx.tools.check_test_speed [--max-seconds N]
python3 -m devx.tools.check_test_speed [--max-seconds N] [--max-single-seconds S]
The module runs ``make test-unit`` with ``PYTEST_ADDOPTS=--durations=0`` so
that pytest emits per-test timing lines alongside the summary. Both the
total wall-clock time and individual test durations are parsed and validated.
"""
from __future__ import annotations
import os
import re
import subprocess # nosec B404
@@ -14,18 +23,32 @@ import click
from devx.i18n import _
DEFAULT_MAX_SECONDS = 2.0
DEFAULT_MAX_SECONDS = 10.0
DEFAULT_MAX_SINGLE_SECONDS = 0.5
TEST_COMMAND = ["make", "test-unit"]
# Matches pytest summary line: "234 passed in 0.70s"
_TIMING_RE = re.compile(r"(\d+) passed.* in ([0-9.]+)s")
# Matches per-test duration lines from --durations=0:
# 0.51s call tests/test_foo.py::test_bar
_DURATION_LINE_RE = re.compile(r"^(\d+\.?\d*)s\s+(?:setup|call|teardown)\s+(.+)$")
def run_tests() -> tuple[str, str]:
"""Execute the unit-test suite and return (stdout, stderr)."""
"""Execute the unit-test suite and return (stdout, stderr).
Sets ``PYTEST_ADDOPTS=--durations=0`` so pytest emits per-test timings.
"""
env = os.environ.copy()
existing = env.get("PYTEST_ADDOPTS", "")
env["PYTEST_ADDOPTS"] = f"--durations=0 {existing}".strip()
result = subprocess.run( # nosec B603
TEST_COMMAND,
capture_output=True,
text=True,
check=False,
env=env,
)
return result.stdout, result.stderr
@@ -43,8 +66,23 @@ def parse_duration(output: str) -> float:
raise click.ClickException(_("Could not parse test execution time from output."))
def parse_per_test_durations(output: str) -> list[tuple[str, float]]:
"""Extract per-test timings from ``--durations=0`` output.
Returns a list of ``(test_name, seconds)`` tuples sorted by duration
(slowest first).
"""
durations: list[tuple[str, float]] = []
for line in output.splitlines():
match = _DURATION_LINE_RE.match(line.strip())
if match:
durations.append((match.group(2).strip(), float(match.group(1))))
durations.sort(key=lambda x: x[1], reverse=True)
return durations
def check_speed(duration: float, max_seconds: float) -> None:
"""Validate duration is within budget; raise on violation."""
"""Validate total duration is within budget; raise on violation."""
if duration > max_seconds:
raise click.ClickException(
_(
@@ -57,19 +95,58 @@ def check_speed(duration: float, max_seconds: float) -> None:
)
def main(max_seconds: float) -> None:
"""Run tests, parse timing, and enforce the budget."""
def check_per_test_speed(
durations: list[tuple[str, float]],
max_single_seconds: float,
) -> list[str]:
"""Return a list of violation messages for tests exceeding the per-test limit.
An empty list means all tests are within budget.
"""
violations: list[str] = []
for name, elapsed in durations:
if elapsed > max_single_seconds:
violations.append(
_(
"Test '{name}' took {elapsed:.2f}s (limit: {limit}s). "
"Optimise: use lighter fixtures, reduce I/O, or mock external calls.",
name=name,
elapsed=elapsed,
limit=max_single_seconds,
)
)
return violations
def main(max_seconds: float, max_single_seconds: float) -> None:
"""Run tests, parse timings, and enforce both budgets."""
stdout, stderr = run_tests()
combined = stdout + "\n" + stderr
click.echo(combined, err=False)
duration = parse_duration(combined)
check_speed(duration, max_seconds)
if max_single_seconds > 0:
per_test = parse_per_test_durations(combined)
violations = check_per_test_speed(per_test, max_single_seconds)
if violations:
msg = _(
"Per-test speed check FAILED: {count} test(s) exceed {limit}s limit.",
count=len(violations),
limit=max_single_seconds,
)
click.echo(f"\n{msg}", err=True)
for v in violations:
click.echo(f" - {v}", err=True)
raise click.ClickException(msg)
click.echo(
_(
"Unit tests passed in {duration:.2f}s (under {max}s limit).",
"Unit tests passed in {duration:.2f}s (under {max}s limit, all tests under {single}s per-test limit).",
duration=duration,
max=max_seconds,
single=max_single_seconds,
)
)
@@ -80,10 +157,17 @@ def main(max_seconds: float) -> None:
type=float,
default=DEFAULT_MAX_SECONDS,
show_default=True,
help="Maximum allowed execution time in seconds.",
help="Maximum allowed total execution time in seconds.",
)
def cli(max_seconds: float) -> None:
main(max_seconds)
@click.option(
"--max-single-seconds",
type=float,
default=DEFAULT_MAX_SINGLE_SECONDS,
show_default=True,
help="Maximum allowed per-test time in seconds (0 to disable).",
)
def cli(max_seconds: float, max_single_seconds: float) -> None:
main(max_seconds, max_single_seconds)
if __name__ == "__main__": # pragma: no cover
+264
View File
@@ -0,0 +1,264 @@
#!/usr/bin/env python3
"""Clean up old Docker images from a Gitea container registry.
Queries the Gitea API for all versions of a package (container type) and
deletes all but the most recent N versions. The ``latest`` tag is always
preserved if present.
Usage::
# Clean up ci-base images, keep last 2 versions
python3 -m devx.tools.clean_images \\
--owner oblachno-oss \\
--name ci-base \\
--keep 2
# Clean up multiple images
python3 -m devx.tools.clean_images \\
--owner oblachno-oss \\
--name ci-base \\
--name ci-quality \\
--name ci-full \\
--keep 2
# Dry run (list what would be deleted)
python3 -m devx.tools.clean_images \\
--owner oblachno-oss \\
--name ci-base \\
--keep 2 \\
--dry-run
Authentication uses ``CI_GITEA_TOKEN`` environment variable.
"""
from __future__ import annotations
import os
import time
from typing import Any
import click
import requests
from devx.config import GITEA_API_URL, REPO_OWNER
from devx.i18n import _
def list_package_versions(
api_url: str,
owner: str,
name: str,
token: str,
*,
timeout: int = 30,
) -> list[dict[str, Any]]:
"""List all versions of a container package from the Gitea API.
Returns a list of version dicts, each containing at least ``version``
and ``created_at`` fields.
"""
url = f"{api_url}/packages/{owner}?type=container&name={name}"
headers = {"Authorization": f"token {token}"}
all_versions: list[dict[str, Any]] = []
page = 1
while True:
resp = requests.get(
f"{url}&page={page}&limit=50",
headers=headers,
timeout=timeout,
)
resp.raise_for_status()
data = resp.json()
if not data:
break
all_versions.extend(data)
if len(data) < 50:
break
page += 1
return all_versions
def delete_package_version(
api_url: str,
owner: str,
name: str,
version: str,
token: str,
*,
timeout: int = 30,
package_type: str = "container",
max_retries: int = 3,
) -> bool:
"""Delete a specific version of a container package.
Uses the Gitea API endpoint ``DELETE /packages/{owner}/{type}/{name}/{version}``.
Retries on transient failures (5xx, timeouts) up to ``max_retries`` times.
Returns True on success, False on failure.
"""
url = f"{api_url}/packages/{owner}/{package_type}/{name}/{version}"
headers = {"Authorization": f"token {token}"}
for attempt in range(max_retries):
try:
resp = requests.delete(url, headers=headers, timeout=timeout)
except requests.RequestException:
if attempt < max_retries - 1:
time.sleep(2**attempt)
continue
return False
if resp.status_code in (204, 200):
return True
# 404 means already deleted — treat as success
if resp.status_code == 404:
return True
# 5xx is transient — retry
if 500 <= resp.status_code < 600 and attempt < max_retries - 1:
time.sleep(2**attempt)
continue
return False
return False
def sort_versions_by_date(
versions: list[dict[str, Any]],
) -> list[dict[str, Any]]:
"""Sort package versions by creation date, newest first.
Falls back to version string comparison if created_at is missing.
"""
def _sort_key(v: dict[str, Any]) -> str:
return str(v.get("created_at", v.get("version", "")))
return sorted(versions, key=_sort_key, reverse=True)
def select_for_deletion(
versions: list[dict[str, Any]],
keep: int,
) -> list[dict[str, Any]]:
"""Select versions to delete, keeping the most recent ``keep`` versions.
Versions named ``latest`` are always preserved.
"""
sorted_versions = sort_versions_by_date(versions)
to_delete = sorted_versions[keep:]
# Always preserve 'latest' tag
to_delete = [v for v in to_delete if v.get("version") != "latest"]
return to_delete
@click.command()
@click.option(
"--owner",
default=None,
help="Package owner (user or org, default: from [tool.devx] repo_owner).",
)
@click.option(
"--name",
"names",
multiple=True,
required=True,
help="Package name(s). Can be repeated.",
)
@click.option(
"--keep",
default=2,
type=int,
show_default=True,
help="Number of recent versions to keep (excluding 'latest').",
)
@click.option(
"--dry-run",
is_flag=True,
default=False,
help="List versions that would be deleted without actually deleting.",
)
@click.option(
"--api-url",
default=None,
help="Gitea API URL (defaults to DEVX_GITEA_API_URL or built-in default).",
)
def main(
owner: str | None,
names: tuple[str, ...],
keep: int,
dry_run: bool,
api_url: str | None,
) -> None:
"""Clean up old Docker image versions from a Gitea registry."""
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
raise click.ClickException(_("CI_GITEA_TOKEN environment variable required"))
if not owner:
owner = REPO_OWNER
if not owner:
raise click.ClickException(_("Package owner not specified. Use --owner or set [tool.devx] repo_owner."))
base_url = api_url or GITEA_API_URL
total_deleted = 0
total_kept = 0
total_failed = 0
for name in names:
click.echo(_("\n{separator}", separator="=" * 60))
click.echo(_("Package: {owner}/{name}", owner=owner, name=name))
click.echo(_("{separator}", separator="=" * 60))
try:
versions = list_package_versions(base_url, owner, name, token)
except requests.RequestException as exc:
click.echo(
_("Failed to list versions for {name}: {error}", name=name, error=exc),
err=True,
)
total_failed += 1
continue
if not versions:
click.echo(_("No versions found."))
continue
click.echo(_("Found {count} version(s):", count=len(versions)))
for v in sort_versions_by_date(versions):
click.echo(
_(" {version} (created: {created})", version=v.get("version", "?"), created=v.get("created_at", "?"))
)
to_delete = select_for_deletion(versions, keep)
kept_count = len(versions) - len(to_delete)
click.echo(_("\nKeeping {kept}, would delete {count}", kept=kept_count, count=len(to_delete)))
if dry_run:
for v in to_delete:
click.echo(_(" [dry-run] Would delete: {version}", version=v.get("version", "?")))
total_kept += kept_count
continue
deleted_count = 0
failed_count = 0
for v in to_delete:
version = str(v.get("version", ""))
if delete_package_version(base_url, owner, name, version, token):
click.echo(_(" Deleted: {version}", version=version))
deleted_count += 1
else:
click.echo(_(" FAILED to delete: {version}", version=version), err=True)
failed_count += 1
total_deleted += deleted_count
total_kept += kept_count
total_failed += failed_count
click.echo(
_(
"\nDone. Deleted {deleted}, kept {kept}, failed {failed}.",
deleted=total_deleted,
kept=total_kept,
failed=total_failed,
)
)
if total_failed > 0:
raise click.ClickException(_("Failed to delete {count} image version(s)", count=total_failed))
if __name__ == "__main__": # pragma: no cover
main() # pragma: no cover
+27 -7
View File
@@ -6,8 +6,8 @@ The ``tea`` CLI is used for label creation if available, with a
fallback to ``GiteaClient`` if tea is not installed.
Usage:
REPO_TOKEN=<token> python3 -m devx.tools.configure_repo --repo my-repo
REPO_TOKEN=<token> python3 -m devx.tools.configure_repo --repo my-repo --owner my-org
CI_GITEA_TOKEN=<token> python3 -m devx.tools.configure_repo --repo my-repo
CI_GITEA_TOKEN=<token> python3 -m devx.tools.configure_repo --repo my-repo --owner my-org
"""
from __future__ import annotations
@@ -19,7 +19,7 @@ from typing import Any, cast
import click
from devx.api_clients import GiteaClient
from devx.config import GITEA_API_URL, REPO_OWNER
from devx.config import GITEA_API_URL, REPO_NAME, REPO_OWNER
from devx.exceptions import APIError
from devx.i18n import _
@@ -50,11 +50,17 @@ def _default_branch_protection_config() -> dict[str, Any]:
"push_whitelist_usernames": [],
"enable_status_check": True,
"status_check_contexts": _default_status_checks(),
"required_approvals": 0,
"required_approvals": 1,
"dismiss_stale_approvals": True,
"block_on_outdated_branch": True,
"block_on_rejected_reviews": True,
"block_on_official_review_requests": True,
# Prevent admins from force-merging PRs that don't meet branch
# protection requirements (e.g. missing approvals). Without this,
# an admin token can bypass the approval gate via force_merge=true,
# allowing merges that failed the auto-merge CI job to reach master
# and trigger the post-merge release pipeline.
"block_admin_merge_override": True,
}
@@ -102,7 +108,7 @@ def configure_repo(
api_url: Gitea API base URL. If None, uses ``GITEA_API_URL`` from config.
"""
if not token:
raise click.ClickException(_("ERROR: REPO_TOKEN is not set."))
raise click.ClickException(_("ERROR: CI_GITEA_TOKEN is not set."))
url = api_url or GITEA_API_URL
client = GiteaClient(url, token, owner, repo)
@@ -123,6 +129,7 @@ def configure_repo(
click.echo(_(" - Dismiss stale approvals: yes"))
click.echo(_(" - Block outdated branches: yes"))
click.echo(_(" - Block rejected reviews: yes"))
click.echo(_(" - Block admin merge override: yes"))
checks = ", ".join(cast(list[str], bp_config["status_check_contexts"]))
click.echo(_(" - Required status checks: {checks}", checks=checks))
@@ -148,13 +155,26 @@ def configure_repo(
)
def main(repo: str | None, owner: str | None, branch: str, api_url: str | None) -> None:
"""Configure branch protection and repository settings via the Gitea API."""
token = os.environ.get("REPO_TOKEN", "")
token = os.environ.get("CI_GITEA_TOKEN", "")
if repo is None:
repo = os.environ.get("DEVX_REPO_NAME", "")
repo = os.environ.get("DEVX_REPO_NAME", "") or REPO_NAME
if not repo:
raise click.ClickException(_("ERROR: Repository name not specified. Use --repo or set DEVX_REPO_NAME."))
# If DEVX_REPO_NAME contains a slash (e.g. "oblachno/infra"), split into owner/repo.
# This prevents 404s when workflows set DEVX_REPO_NAME to the full path.
if "/" in repo and owner is None:
parts = repo.split("/", 1)
owner, repo = parts[0], parts[1]
click.echo(
_(
"Parsed owner={owner}, repo={repo} from DEVX_REPO_NAME",
owner=owner,
repo=repo,
)
)
if owner is None:
owner = REPO_OWNER
+186
View File
@@ -0,0 +1,186 @@
#!/usr/bin/env python3
"""Create a pull request with the correct title from the Vikunja task.
This tool is run **after** pushing a feature branch. It:
1. Extracts the task ID from the branch name (e.g. ``DEVX-31-fix-foo`` → ``DEVX-31``).
2. Fetches the Vikunja task title for that task ID.
3. Creates a PR with title ``{TASK_PREFIX}-N: <vikunja task title>``.
This eliminates manual PR title entry and ensures the title always
matches the Vikunja task — which is what the auto-merge workflow
validates.
If a PR already exists for the branch, the tool prints its URL and
exits successfully (idempotent).
Usage::
python -m devx.tools.create_pr --branch DEVX-31-fix-foo
The repository is auto-detected from ``DEVX_REPO_OWNER`` /
``DEVX_REPO_NAME`` or ``GITHUB_REPOSITORY`` environment variables.
"""
from __future__ import annotations
import os
import subprocess # nosec B404
import click
from dotenv import load_dotenv
from devx.api_clients import GiteaClient, VikunjaClient
from devx.config import (
DEFAULT_PER_PAGE,
GITEA_API_URL,
REPO_NAME,
REPO_OWNER,
TASK_ID_RE,
TASK_PREFIX,
VIKUNJA_API_URL,
VIKUNJA_PROJECT_ID,
)
from devx.i18n import _
load_dotenv()
def get_repo_name() -> str:
"""Auto-detect repository name from env vars, pyproject.toml, or git remote."""
name = os.environ.get("DEVX_REPO_NAME", "")
if name:
return name
github_repo = os.environ.get("GITHUB_REPOSITORY", "")
if github_repo and "/" in github_repo:
return github_repo.split("/", 1)[1]
if REPO_NAME:
return REPO_NAME
raise click.ClickException(
_("Repository name not set. Use DEVX_REPO_NAME, [tool.devx] repo_name, or GITHUB_REPOSITORY env var."),
)
def extract_task_id(branch: str) -> str:
"""Extract the task ID (e.g. ``DEVX-31``) from a branch name."""
match = TASK_ID_RE.search(branch)
return match.group(0) if match else ""
def get_vikunja_task_title(task_id: str) -> str:
"""Fetch the Vikunja task title for the given task identifier.
Raises ClickException if VIKUNJA_TOKEN is not set or the task is not found.
"""
token = os.environ.get("VIKUNJA_TOKEN", "")
if not token:
raise click.ClickException(_("VIKUNJA_TOKEN is not set. Required to derive PR title."))
client = VikunjaClient(VIKUNJA_API_URL, token)
task = client.find_task_by_identifier(VIKUNJA_PROJECT_ID, task_id, per_page=DEFAULT_PER_PAGE)
if not task:
raise click.ClickException(
_(
"Could not find Vikunja task {task_id} in project {project_id}.",
task_id=task_id,
project_id=VIKUNJA_PROJECT_ID,
),
)
return str(task.get("title", ""))
def find_existing_pr(client: GiteaClient, branch: str) -> dict | None:
"""Return an existing open PR for the branch, or None."""
prs = client.list_prs(state="open")
for pr in prs:
if pr.get("head", {}).get("ref") == branch:
return pr
return None
def create_pr(
branch: str,
base: str,
body: str,
repo_owner: str,
repo_name: str,
) -> dict:
"""Create a PR with the title derived from the Vikunja task.
Returns the PR dict from the Gitea API.
"""
task_id = extract_task_id(branch)
if not task_id:
raise click.ClickException(
_(
"Branch '{branch}' does not contain a task ID.\n Expected format: {prefix}-N-short-description",
branch=branch,
prefix=TASK_PREFIX,
),
)
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
raise click.ClickException(_("CI_GITEA_TOKEN is not set. Required to create a PR."))
vikunja_title = get_vikunja_task_title(task_id)
pr_title = f"{task_id}: {vikunja_title}"
client = GiteaClient(GITEA_API_URL, token, repo_owner, repo_name)
existing = find_existing_pr(client, branch)
if existing:
click.echo(
_(
"PR already exists: #{index} — {url}",
index=existing.get("number", "?"),
url=existing.get("html_url", ""),
),
)
return existing
pr = client.create_pr(title=pr_title, head=branch, base=base, body=body)
click.echo(
_(
"Created PR #{index}: {title}\n {url}",
index=pr.get("number", "?"),
title=pr_title,
url=pr.get("html_url", ""),
),
)
return pr
@click.command()
@click.option("--branch", default=None, help="Head branch (default: auto-detect from git).")
@click.option("--base", default="master", show_default=True, help="Base branch.")
@click.option("--body", default="", help="PR body (markdown). Read from stdin if '-' is passed.")
@click.option("--owner", default=None, help="Repository owner (default: DEVX_REPO_OWNER).")
@click.option("--repo", default=None, help="Repository name (default: DEVX_REPO_NAME or GITHUB_REPOSITORY).")
def cli(branch: str | None, base: str, body: str, owner: str | None, repo: str | None) -> None:
"""Create a PR with the correct title from the Vikunja task."""
if branch is None:
result = subprocess.run( # nosec
["git", "rev-parse", "--abbrev-ref", "HEAD"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
raise click.ClickException(
_("Could not detect current branch: {error}", error=result.stderr.strip()),
)
branch = result.stdout.strip()
if body == "-":
body = click.get_text_stream("stdin").read().strip()
repo_owner = owner or REPO_OWNER
if not repo_owner:
raise click.ClickException(_("Repository owner not set. Use --owner or DEVX_REPO_OWNER env var."))
repo_name = repo or get_repo_name()
create_pr(branch, base, body, repo_owner, repo_name)
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env python3
"""Create a Vikunja task with a detailed HTML description.
This tool is used during the planning phase of the development workflow
to create a well-described task before any code is written. The task
identifier (e.g. ``DEVX-N``, ``GRM-N``, ``OBL-INFRA-N``) is then used
to name the feature branch and the pull request.
Usage::
python -m devx.tools.create_task --title "Add release automation" \\
--description "<h2>Overview</h2><p>Implement automated...</p>"
The project ID and task prefix are read from ``DEVX_VIKUNJA_PROJECT_ID``
and ``DEVX_TASK_PREFIX`` environment variables (or ``.env``).
"""
from __future__ import annotations
import os
import click
from dotenv import load_dotenv
from devx.api_clients import VikunjaClient
from devx.config import TASK_PREFIX, VIKUNJA_API_URL, VIKUNJA_PROJECT_ID
from devx.i18n import _
load_dotenv()
@click.command()
@click.option("--title", required=True, help="Task title (becomes the Vikunja task title).")
@click.option(
"--description",
default="",
help="Task description (HTML supported). Read from stdin if '-' is passed.",
)
@click.option("--project-id", type=int, default=None, help="Vikunja project ID (default: DEVX_VIKUNJA_PROJECT_ID).")
def cli(title: str, description: str, project_id: int | None) -> None:
"""Create a Vikunja task and print its identifier."""
token = os.environ.get("VIKUNJA_TOKEN", "")
if not token:
raise click.ClickException(_("VIKUNJA_TOKEN is not set. Set it in .env or environment."))
pid = project_id if project_id is not None else VIKUNJA_PROJECT_ID
if description == "-":
description = click.get_text_stream("stdin").read().strip()
client = VikunjaClient(VIKUNJA_API_URL, token)
task = client.create_task(pid, title, description)
identifier = task.get("identifier", "")
task_id = task.get("id", "")
click.echo(
_(
"Created Vikunja task: {identifier} (id={task_id})",
identifier=identifier,
task_id=task_id,
)
)
if identifier:
click.echo(
_(
"Next steps:\n"
" 1. git checkout master && git pull\n"
" 2. git checkout -b {prefix}-{num}-short-description\n"
" 3. Implement changes, commit with conventional commit format\n"
" 4. git push -u origin HEAD\n"
" 5. make create-pr (creates PR with title: {identifier}: {title})",
prefix=TASK_PREFIX,
num=identifier.split("-")[-1] if "-" in identifier else "N",
identifier=identifier,
title=title,
)
)
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+246 -63
View File
@@ -5,12 +5,21 @@ Runs pytest-cov, doc-coverage, lint checks, and version extraction,
then writes SVG badge files that can be served as static files from
the Gitea raw file API.
The repo root is resolved from ``GITHUB_WORKSPACE`` or ``os.getcwd()``,
so this module works correctly both when run from a source checkout
and when devx is installed as a pip package in CI.
The package name and coverage target are auto-detected from the
``src/`` directory structure, making this module reusable across
all oblachno-oss repos without per-repo configuration.
Usage:
python3 -m devx.tools.generate_badges --output-dir .badges/
"""
from __future__ import annotations
import os
import re
import subprocess # nosec B404
import sys
@@ -18,8 +27,9 @@ from pathlib import Path
import click
REPO_ROOT = Path(__file__).resolve().parents[4]
from devx.i18n import _
# Coverage regex matches "TOTAL ... NN%" or "TOTAL ... NN.NN%"
_COVERAGE_RE = re.compile(r"TOTAL.*?(\d+(?:\.\d+)?)%")
_PASSED_RE = re.compile(r"(\d+) passed")
_DOC_COVERAGE_RE = re.compile(r"Doc coverage:\s+\d+/\d+\s+\((\d+)%")
@@ -37,18 +47,59 @@ COLOR_HEX: dict[str, str] = {
}
def _find_package_init() -> Path | None:
"""Find the first package __init__.py under src/ that defines __version__."""
src_dir = REPO_ROOT / "src"
if not src_dir.exists():
def resolve_repo_root() -> Path:
"""Resolve the repository root directory.
Uses ``GITHUB_WORKSPACE`` env var (set by Gitea Actions) or
falls back to ``os.getcwd()``. This ensures the correct repo
root is used even when devx is installed as a pip package.
"""
workspace = os.environ.get("GITHUB_WORKSPACE")
if workspace:
path = Path(workspace)
if path.is_dir():
return path
return Path.cwd()
def detect_package_name(repo_root: Path) -> str | None:
"""Auto-detect the Python package name from ``src/`` directory.
Looks for the first subdirectory under ``src/`` that contains
an ``__init__.py`` file with ``__version__``.
Returns the package directory name (e.g., ``devx``,
``gitea_runner_manager``) or ``None`` if no package is found.
"""
src_dir = repo_root / "src"
if not src_dir.is_dir():
return None
for init_file in src_dir.rglob("__init__.py"):
try:
content = init_file.read_text()
except OSError:
for entry in sorted(src_dir.iterdir()):
if not entry.is_dir():
continue
if "__version__" in content:
return init_file
init_file = entry / "__init__.py"
if init_file.exists():
return entry.name
return None
def detect_coverage_target(repo_root: Path) -> str | None:
"""Auto-detect the pytest-cov target from pyproject.toml.
Parses ``addopts`` in ``[tool.pytest.ini_options]`` for
``--cov=src/<package>``. Falls back to ``src/<package>`` if
the package is detected but no explicit cov target is found.
"""
pyproject = repo_root / "pyproject.toml"
if pyproject.exists():
content = pyproject.read_text()
match = re.search(r"--cov=(\S+)", content)
if match:
return match.group(1)
# Fallback: derive from package name
pkg = detect_package_name(repo_root)
if pkg:
return f"src/{pkg}"
return None
@@ -57,14 +108,15 @@ def _xml_escape(text: str) -> str:
return text.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;").replace('"', "&quot;")
def run_command(cmd: list[str]) -> tuple[int, str, str]:
def run_command(cmd: list[str], cwd: Path | None = None) -> tuple[int, str, str]:
"""Run a command and return (returncode, stdout, stderr)."""
root = str(cwd or resolve_repo_root())
result = subprocess.run( # nosec B603
cmd,
capture_output=True,
text=True,
check=False,
cwd=str(REPO_ROOT),
cwd=root,
)
return result.returncode, result.stdout, result.stderr
@@ -139,15 +191,27 @@ def extract_doc_coverage(output: str) -> int | None:
return None
def read_version() -> str:
"""Read __version__ from the package __init__.py."""
init_file = _find_package_init()
if init_file is None:
def read_version(repo_root: Path) -> str:
"""Read __version__ from the package __init__.py under src/.
Auto-detects the package directory and reads ``__version__``
from its ``__init__.py``.
"""
pkg = detect_package_name(repo_root)
if pkg is None:
click.echo(_(" WARNING: No Python package found under src/ — version badge will show 'unknown'"))
return "unknown"
init_file = repo_root / "src" / pkg / "__init__.py"
if not init_file.exists():
click.echo(_(" WARNING: {init_file} not found — version badge will show 'unknown'", init_file=init_file))
return "unknown"
content = init_file.read_text()
match = re.search(r'__version__\s*=\s*["\']([^"\']+)["\']', content)
if match:
return match.group(1)
click.echo(
_(" WARNING: No __version__ found in {init_file} — version badge will show 'unknown'", init_file=init_file)
)
return "unknown"
@@ -179,65 +243,170 @@ def doc_coverage_color(pct: int) -> str:
return "orange"
def generate_badges(output_dir: Path) -> dict[str, dict[str, str | int]]:
"""Generate all badge SVG files and return badge data as a dict."""
badges: dict[str, dict[str, str | int]] = {}
def detect_testpaths(repo_root: Path) -> list[str]:
"""Detect test paths from pyproject.toml or filesystem.
# 1. Code coverage + test count (single pytest-cov run)
rc, stdout, stderr = run_command(
[
sys.executable,
"-m",
"pytest",
"tests/",
"-v",
"--cov=src/devx",
"--cov-report=term-missing",
"--cov-fail-under=0",
]
)
Parses ``testpaths`` in ``[tool.pytest.ini_options]`` from
pyproject.toml. Falls back to ``["tests"]`` if the tests/
directory exists. Returns an empty list if no test paths
are found (pytest will use its own defaults).
"""
pyproject = repo_root / "pyproject.toml"
if pyproject.exists():
content = pyproject.read_text()
# Match: testpaths = ["dir1", "dir2"]
match = re.search(r"testpaths\s*=\s*\[([^\]]+)\]", content)
if match:
paths = re.findall(r'["\']([^"\']+)["\']', match.group(1))
resolved = []
for p in paths:
p = p.strip()
if (repo_root / p).exists():
resolved.append(p)
if resolved:
return resolved
# Fallback: tests/ directory
tests_dir = repo_root / "tests"
if tests_dir.is_dir():
return ["tests"]
return []
def collect_coverage_and_tests(repo_root: Path) -> tuple[dict[str, str | int], dict[str, str | int]]:
"""Run pytest-cov and collect coverage + test count badges.
Returns (coverage_badge, tests_badge). If pytest is not
available or no tests are found, returns 'unknown' badges
with a clear warning explaining the failure.
"""
cov_target = detect_coverage_target(repo_root)
if cov_target is None:
click.echo(_(" WARNING: No coverage target detected (no src/ package, no --cov in pyproject.toml)"))
return make_badge("coverage", "unknown", "lightgrey"), make_badge("tests", "unknown", "lightgrey")
testpaths = detect_testpaths(repo_root)
click.echo(_(" Test paths: {testpaths}", testpaths=testpaths or "(pytest defaults)"))
cmd = [
sys.executable,
"-m",
"pytest",
*testpaths,
"--cov",
cov_target,
"--cov-report=term-missing",
"--cov-fail-under=0",
"-q",
]
rc, stdout, stderr = run_command(cmd, cwd=repo_root)
combined = stdout + "\n" + stderr
coverage = extract_coverage(combined)
if coverage is not None:
badges["coverage"] = make_badge("coverage", f"{coverage:.0f}%", coverage_color(coverage))
cov_badge = make_badge("coverage", f"{coverage:.0f}%", coverage_color(coverage))
else:
badges["coverage"] = make_badge("coverage", "unknown", "red")
click.echo(_(" WARNING: Could not extract coverage from pytest output (rc={rc})", rc=rc))
click.echo(_(" pytest stdout (last 300 chars): {stdout}", stdout=stdout.strip()[-300:]))
click.echo(_(" pytest stderr (last 300 chars): {stderr}", stderr=stderr.strip()[-300:]))
cov_badge = make_badge("coverage", "unknown", "red")
test_count = extract_test_count(combined)
if test_count is not None:
badges["tests"] = make_badge("tests", f"{test_count} passing", "brightgreen" if rc == 0 else "red")
tests_badge = make_badge("tests", f"{test_count} passing", "brightgreen" if rc == 0 else "red")
else:
badges["tests"] = make_badge("tests", "unknown", "red")
click.echo(_(" WARNING: Could not extract test count from pytest output (rc={rc})", rc=rc))
click.echo(_(" pytest stdout (last 300 chars): {stdout}", stdout=stdout.strip()[-300:]))
click.echo(_(" pytest stderr (last 300 chars): {stderr}", stderr=stderr.strip()[-300:]))
tests_badge = make_badge("tests", "unknown", "red")
# 2. Documentation coverage
rc, stdout, _ = run_command(
[
sys.executable,
"-m",
"devx.ci.doc_coverage",
]
return cov_badge, tests_badge
def collect_doc_coverage(repo_root: Path) -> dict[str, str | int]:
"""Run doc_coverage and collect the docs badge."""
rc, stdout, stderr = run_command(
[sys.executable, "-m", "devx.ci.doc_coverage"],
cwd=repo_root,
)
doc_pct = extract_doc_coverage(stdout)
if doc_pct is not None:
badges["docs"] = make_badge("docs", f"{doc_pct}%", doc_coverage_color(doc_pct))
else:
badges["docs"] = make_badge("docs", "unknown", "red")
return make_badge("docs", f"{doc_pct}%", doc_coverage_color(doc_pct))
click.echo(_(" WARNING: Could not extract doc coverage (rc={rc})", rc=rc))
click.echo(_(" stderr: {stderr}", stderr=stderr.strip()[:200]))
return make_badge("docs", "unknown", "red")
# 3. Code quality (ruff + pyright + bandit all pass)
lint_rc, _, _ = run_command([sys.executable, "-m", "ruff", "check", "src/", "tests/"])
format_rc, _, _ = run_command([sys.executable, "-m", "ruff", "format", "--check", "src/", "tests/"])
type_rc, _, _ = run_command([sys.executable, "-m", "pyright"])
bandit_rc, _, _ = run_command([sys.executable, "-m", "bandit", "-r", "src/"])
all_pass = all(rc == 0 for rc in [lint_rc, format_rc, type_rc, bandit_rc])
badges["quality"] = make_badge("code quality", "A" if all_pass else "F", "brightgreen" if all_pass else "red")
def collect_quality(repo_root: Path) -> dict[str, str | int]:
"""Run lint checks and collect the quality badge.
Runs ruff check, ruff format --check, pyright, and bandit.
If any tool is not installed, it is skipped with a warning.
"""
results: list[bool] = []
tool_names: list[str] = []
for cmd, name in [
([sys.executable, "-m", "ruff", "check", "src/", "tests/"], "ruff check"),
([sys.executable, "-m", "ruff", "format", "--check", "src/", "tests/"], "ruff format"),
([sys.executable, "-m", "pyright"], "pyright"),
([sys.executable, "-m", "bandit", "-r", "src/"], "bandit"),
]:
rc, _stdout, stderr = run_command(cmd, cwd=repo_root)
if rc == 0:
results.append(True)
tool_names.append(f"{name}: pass")
else:
results.append(False)
# Distinguish "tool not installed" from "tool found issues"
if "No module named" in stderr or "not found" in stderr.lower():
click.echo(_(" WARNING: {name} not installed — skipping (counted as pass)", name=name))
results[-1] = True
tool_names.append(f"{name}: not installed (skipped)")
else:
tool_names.append(f"{name}: FAIL")
click.echo(_(" WARNING: {name} failed (rc={rc})", name=name, rc=rc))
click.echo(_(" stderr: {stderr}", stderr=stderr.strip()[:200]))
all_pass = all(results)
click.echo(_(" Quality checks: {checks}", checks=", ".join(tool_names)))
return make_badge("code quality", "A" if all_pass else "F", "brightgreen" if all_pass else "red")
def generate_badges(output_dir: Path, repo_root: Path | None = None) -> dict[str, dict[str, str | int]]:
"""Generate all badge SVG files and return badge data as a dict.
Args:
output_dir: Directory to write SVG files.
repo_root: Repository root (auto-detected if None).
"""
root = repo_root or resolve_repo_root()
click.echo(_(" Repo root: {root}", root=root))
pkg = detect_package_name(root)
click.echo(_(" Package: {pkg}", pkg=pkg or "none"))
badges: dict[str, dict[str, str | int]] = {}
# 1. Code coverage + test count (single pytest-cov run)
click.echo(_(" Collecting coverage and tests..."))
cov_badge, tests_badge = collect_coverage_and_tests(root)
badges["coverage"] = cov_badge
badges["tests"] = tests_badge
# 2. Documentation coverage
click.echo(_(" Collecting doc coverage..."))
badges["docs"] = collect_doc_coverage(root)
# 3. Code quality (ruff + pyright + bandit)
click.echo(_(" Collecting code quality..."))
badges["quality"] = collect_quality(root)
# 4. Version
version = read_version()
click.echo(_(" Collecting version..."))
version = read_version(root)
badges["version"] = make_badge("version", f"v{version}", "blue")
# 5. Python version (static but nice)
# 5. Python version (static)
badges["python"] = make_badge("python", "3.12", "blue")
# Write SVG files
@@ -246,7 +415,7 @@ def generate_badges(output_dir: Path) -> dict[str, dict[str, str | int]]:
svg = render_svg(str(badge["label"]), str(badge["message"]), str(badge["color"]))
path = output_dir / f"{name}.svg"
path.write_text(svg)
click.echo(f" Generated: {path}")
click.echo(_(" Generated: {path}", path=path))
return badges
@@ -254,17 +423,31 @@ def generate_badges(output_dir: Path) -> dict[str, dict[str, str | int]]:
@click.command()
@click.option(
"--output-dir",
default=str(REPO_ROOT / ".badges"),
default=".badges",
help="Directory to write badge SVG files.",
)
def cli(output_dir: str) -> None:
@click.option(
"--repo-root",
default=None,
help="Repository root (auto-detected if not specified).",
)
def cli(output_dir: str, repo_root: str | None) -> None:
"""Generate self-contained SVG badge files from project metrics."""
out = Path(output_dir)
click.echo(f"Generating badges in {out}...")
badges = generate_badges(out)
click.echo(f"\nGenerated {len(badges)} badges:")
root = Path(repo_root) if repo_root else None
click.echo(_("Generating badges in {out}...", out=out))
badges = generate_badges(out, repo_root=root)
click.echo(_("\nGenerated {count} badges:", count=len(badges)))
for name, badge in badges.items():
click.echo(f" {name}: {badge['label']}={badge['message']} ({badge['color']})")
click.echo(
_(
" {name}: {label}={message} ({color})",
name=name,
label=badge["label"],
message=badge["message"],
color=badge["color"],
)
)
if __name__ == "__main__": # pragma: no cover
+150
View File
@@ -0,0 +1,150 @@
#!/usr/bin/env python3
"""Generate a cliff.toml configuration file for a project.
Produces a git-cliff configuration with the correct task ID prefix
preprocessor, matching the format used by devx itself. Downstream
repos can use this to avoid duplicating the entire cliff.toml by hand.
Usage::
python -m devx.tools.generate_cliff_config --prefix GRM
python -m devx.tools.generate_cliff_config --prefix GRM --output cliff.toml
python -m devx.tools.generate_cliff_config --prefix GRM --force
"""
from __future__ import annotations
from pathlib import Path
import click
from devx.config import TASK_PREFIX
from devx.i18n import _
# Template uses __PREFIX__ and __PREFIX_REGEX__ as placeholders to avoid
# conflicts with Jinja2's {{ }} and {% %} syntax in the cliff.toml body.
CLIFF_TEMPLATE = """\
# git-cliff configuration for __PREFIX__
# https://git-cliff.org/docs/configuration
# Generated by: python -m devx.tools.generate_cliff_config --prefix __PREFIX__
[changelog]
header = \"\"\"
# Changelog\\n
All notable changes to this project will be documented in this file.\\n
\"\"\"
body = \"\"\"
{% if version %}\\
## [{{ version | trim_start_matches(pat="v") }}] - {{ timestamp | date(format="%Y-%m-%d") }}
{% else %}\\
## [unreleased]
{% endif %}\\
{% for group, commits in commits | group_by(attribute="group") %}
### {{ group | striptags | trim | upper_first }}
{% for commit in commits %}
- {% if commit.scope %}*({{ commit.scope }})* {% endif %}\\
{% if commit.breaking %}[**breaking**] {% endif %}\\
{{ commit.message | upper_first }}\\
{% endfor %}
{% endfor %}
\"\"\"
trim = true
render_always = true
[git]
conventional_commits = true
filter_unconventional = true
require_conventional = false
split_commits = false
protect_breaking_commits = false
filter_commits = false
fail_on_unmatched_commit = false
use_branch_tags = false
topo_order = false
topo_order_commits = true
sort_commits = "oldest"
recurse_submodules = false
commit_preprocessors = [
# Strip __PREFIX__-N: task ID prefix from squash-merge commits so git-cliff sees conventional commits
{ pattern = "^__PREFIX_REGEX__-\\\\d+:\\\\s+", replace = "" },
]
commit_parsers = [
{ message = "^feat", group = "<!-- 0 -->Features" },
{ message = "^fix", group = "<!-- 1 -->Bug Fixes" },
{ message = "^perf", group = "<!-- 4 -->Performance" },
{ message = "^refactor", group = "<!-- 2 -->Refactor" },
# Skip infrastructure-only commits — they don't affect users
{ message = "^doc", skip = true },
{ message = "^test", skip = true },
{ message = "^style", skip = true },
{ message = "^chore", skip = true },
{ message = "^ci", skip = true },
# Skip release commits — they are release artifacts, not features
{ message = "^release:", skip = true },
{ body = ".*security", group = "<!-- 8 -->Security" },
{ message = "^revert", group = "<!-- 9 -->Revert" },
# Skip anything that doesn't match above — safe default
{ message = ".*", skip = true },
]
[bump]
features_always_bump_minor = true
breaking_always_bump_major = false
initial_tag = "0.1.0"
# Refactor commits bump patch — structural changes to src/ or pyproject.toml
# affect users even though no new feature was added.
refactor_always_bump_patch = true
"""
def _generate(prefix: str) -> str:
"""Generate cliff.toml content for the given prefix."""
prefix_regex = prefix.replace("\\", "\\\\")
return CLIFF_TEMPLATE.replace("__PREFIX__", prefix).replace("__PREFIX_REGEX__", prefix_regex)
@click.command()
@click.option(
"--prefix",
default=TASK_PREFIX,
help="Task ID prefix for commit preprocessor (default: from [tool.devx] task_prefix in pyproject.toml).",
)
@click.option(
"--output",
"-o",
default="cliff.toml",
type=click.Path(),
help="Output file path (default: cliff.toml).",
)
@click.option(
"--force",
is_flag=True,
help="Overwrite existing file without prompting.",
)
def main(prefix: str, output: str, force: bool) -> None:
"""Generate a cliff.toml configuration file."""
output_path = Path(output)
if output_path.exists() and not force:
raise click.ClickException(
_(
"{file} already exists. Use --force to overwrite.",
file=str(output_path),
)
)
content = _generate(prefix)
output_path.write_text(content)
click.echo(
_(
"Generated {file} with prefix '{prefix}'.",
file=str(output_path),
prefix=prefix,
)
)
if __name__ == "__main__": # pragma: no cover
main() # pragma: no cover
+10 -15
View File
@@ -7,7 +7,6 @@ pre-built Linux binary from the official GitHub releases.
from __future__ import annotations
import platform
import shutil
import subprocess # nosec B404
import urllib.request
@@ -15,6 +14,8 @@ from pathlib import Path
import click
from devx.tools._shared import arch_string
CHECKMAKE_VERSION = "0.3.2"
RELEASE_URL_TEMPLATE = (
"https://github.com/checkmake/checkmake/releases/download/"
@@ -23,18 +24,11 @@ RELEASE_URL_TEMPLATE = (
TARGET_PATH = Path("/usr/local/bin/checkmake")
def _arch() -> str:
"""Return the architecture string used by checkmake releases."""
machine = platform.machine().lower()
if machine in {"x86_64", "amd64"}:
return "amd64"
if machine in {"aarch64", "arm64"}:
return "arm64"
raise click.ClickException(f"Unsupported architecture: {machine}")
def _install_with_go() -> bool:
"""Install checkmake using go install if Go is available."""
"""Install checkmake using go install if Go is available.
Returns True if the installation succeeded, False if Go is not installed.
"""
go_bin = shutil.which("go")
if go_bin is None:
return False
@@ -51,12 +45,13 @@ def _install_with_go() -> bool:
def _download_binary() -> None:
"""Download the prebuilt checkmake binary for the current architecture."""
url = RELEASE_URL_TEMPLATE.format(arch=_arch())
url = RELEASE_URL_TEMPLATE.format(arch=arch_string())
urllib.request.urlretrieve(url, TARGET_PATH) # nosec B310
TARGET_PATH.chmod(0o755)
def main() -> None:
@click.command()
def cli() -> None:
"""Install checkmake if not already present."""
if shutil.which("checkmake") is not None:
return
@@ -66,4 +61,4 @@ def main() -> None:
if __name__ == "__main__": # pragma: no cover
main() # pragma: no cover
cli() # pragma: no cover
+23 -8
View File
@@ -6,6 +6,7 @@ Handles installation of:
- git-cliff (changelog generator)
- act_runner (Gitea Actions local runner, optional)
- tea (Gitea CLI — official command-line tool for Gitea API operations)
- hadolint (Dockerfile linter)
Each tool is installed to ``~/.local/bin`` if not already on PATH.
Idempotent: skips tools that are already available.
@@ -39,15 +40,14 @@ ACT_RUNNER_VERSION = "0.2.11"
TEA_VERSION = "0.14.1"
HADOLINT_VERSION = "2.12.0"
def _arch() -> str:
"""Return the architecture string used by release assets."""
machine = platform.machine().lower()
if machine in {"x86_64", "amd64"}:
return "amd64"
if machine in {"aarch64", "arm64"}:
return "arm64"
raise click.ClickException(f"Unsupported architecture: {machine}")
"""Return the architecture string used by release assets (delegates to shared utility)."""
from devx.tools._shared import arch_string
return arch_string()
def _ensure_target_dir() -> Path:
@@ -161,7 +161,20 @@ def install_tea() -> bool:
return True
TOOL_NAMES = ["actionlint", "git-cliff", "act_runner", "tea"]
def install_hadolint() -> bool:
"""Install hadolint if not already present. Returns True if installed/skipped."""
if _is_installed("hadolint"):
click.echo("hadolint: already installed")
return True
machine = platform.machine().lower()
arch = "x86_64" if machine in {"x86_64", "amd64"} else "arm64"
url = f"https://github.com/hadolint/hadolint/releases/download/v{HADOLINT_VERSION}/hadolint-Linux-{arch}"
dest = _download_binary(url, "hadolint")
click.echo(f"hadolint: installed to {dest}")
return True
TOOL_NAMES = ["actionlint", "git-cliff", "act_runner", "tea", "hadolint"]
def _install_tool(name: str) -> bool:
@@ -174,6 +187,8 @@ def _install_tool(name: str) -> bool:
return install_act_runner()
if name == "tea":
return install_tea()
if name == "hadolint":
return install_hadolint()
raise click.ClickException(f"Unknown tool: {name}")
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env python3
"""Add a label to a pull request (idempotent).
Commonly used to add the ``ready-to-merge`` label after CI passes and
review is complete. The operation is idempotent — if the label is already
attached, it succeeds without error.
Usage::
# Add ready-to-merge to PR #42
python -m devx.tools.pr_label --pr 42 --label ready-to-merge
# Add label to current branch's PR
python -m devx.tools.pr_label --label ready-to-merge
# Add multiple labels
python -m devx.tools.pr_label --pr 42 --label ready-to-merge --label reviewed
The repository is auto-detected from ``DEVX_REPO_OWNER`` /
``DEVX_REPO_NAME`` or ``GITHUB_REPOSITORY`` environment variables.
"""
from __future__ import annotations
import os
import click
from dotenv import load_dotenv
from devx.api_clients import GiteaClient
from devx.config import GITEA_API_URL, REPO_OWNER
from devx.i18n import _
from devx.tools.create_pr import get_repo_name
from devx.tools.pr_status import _get_current_branch_pr
load_dotenv()
@click.command()
@click.option("--pr", "pr_number", type=int, default=None, help="PR number (default: auto-detect from current branch).")
@click.option("--label", "labels", multiple=True, required=True, help="Label name(s) to add (can be repeated).")
@click.option("--owner", default=None, help="Repository owner (default: DEVX_REPO_OWNER).")
@click.option("--repo", default=None, help="Repository name (default: DEVX_REPO_NAME or GITHUB_REPOSITORY).")
def cli(
pr_number: int | None,
labels: tuple[str, ...],
owner: str | None,
repo: str | None,
) -> None:
"""Add one or more labels to a pull request (idempotent)."""
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
raise click.ClickException(_("CI_GITEA_TOKEN is not set."))
repo_owner = owner or REPO_OWNER
if not repo_owner:
raise click.ClickException(_("Repository owner not set. Use --owner or DEVX_REPO_OWNER env var."))
repo_name = repo or get_repo_name()
client = GiteaClient(GITEA_API_URL, token, repo_owner, repo_name)
if pr_number is None:
pr_number = _get_current_branch_pr(client)
label_list = list(labels)
existing = client.get_pr_label_names(pr_number)
to_add = [lbl for lbl in label_list if lbl not in existing]
already = [lbl for lbl in label_list if lbl in existing]
if already:
for lbl in already:
click.echo(_("Label '{label}' already on PR #{pr}.", label=lbl, pr=pr_number))
if to_add:
client.add_pr_label(pr_number, to_add)
for lbl in to_add:
click.echo(_("Added label '{label}' to PR #{pr}.", label=lbl, pr=pr_number))
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+187
View File
@@ -0,0 +1,187 @@
#!/usr/bin/env python3
"""Fetch logs for failed CI jobs on a pull request.
Lists CI jobs for the latest workflow run of a PR's branch, then fetches
and prints the logs of any failed jobs. Useful for diagnosing CI failures
without navigating the web UI.
Usage::
# Show failed job logs for PR #42
python -m devx.tools.pr_logs --pr 42
# Show failed job logs for current branch's PR
python -m devx.tools.pr_logs
# Show logs for a specific job (by name)
python -m devx.tools.pr_logs --pr 42 --job quality
# Show last N lines of each failed job's logs
python -m devx.tools.pr_logs --pr 42 --tail 50
The repository is auto-detected from ``DEVX_REPO_OWNER`` /
``DEVX_REPO_NAME`` or ``GITHUB_REPOSITORY`` environment variables.
"""
from __future__ import annotations
import os
import click
from dotenv import load_dotenv
from devx.api_clients import APIError, GiteaClient
from devx.config import GITEA_API_URL, REPO_OWNER
from devx.i18n import _
from devx.tools.create_pr import get_repo_name
from devx.tools.pr_status import _get_current_branch_pr
load_dotenv()
def _get_pr_sha(client: GiteaClient, pr_number: int) -> str:
"""Fetch the head SHA of a PR."""
pr = client.get_pr(pr_number)
return pr.get("head", {}).get("sha", "")
def _find_latest_run_by_sha(client: GiteaClient, sha: str) -> dict | None:
"""Find the latest workflow run for a commit SHA.
Gitea Actions API doesn't set head_branch for pull_request events,
so we filter by head_sha instead.
"""
data = client.list_action_runs(limit=50)
for run in data.get("workflow_runs", []):
if run.get("head_sha", "").startswith(sha):
return run
return None
def _find_failed_jobs(jobs: list[dict]) -> list[dict]:
"""Return jobs with conclusion 'failure'."""
return [j for j in jobs if j.get("conclusion") == "failure"]
def _find_job_by_name(jobs: list[dict], name: str) -> dict | None:
"""Find a job by name (case-insensitive partial match)."""
name_lower = name.lower()
for j in jobs:
if name_lower in j.get("name", "").lower():
return j
return None
def _print_job_summary(jobs: list[dict]) -> None:
"""Print a summary table of all jobs and their status."""
for j in jobs:
name = j.get("name", "?")
conclusion = j.get("conclusion", "pending")
status = j.get("status", "?")
symbol = "[FAIL]" if conclusion == "failure" else "[OK]" if conclusion == "success" else f"[{conclusion}]"
click.echo(f" {symbol} {name} (status: {status}, conclusion: {conclusion})")
def _print_failed_steps(job: dict) -> list[int]:
"""Print failed steps for a job. Returns list of failed step numbers."""
failed_steps = []
for step in job.get("steps", []):
if step.get("conclusion") == "failure":
name = step.get("name", "?")
num = step.get("number", "?")
click.echo(f" FAILED step #{num}: {name}")
failed_steps.append(num)
return failed_steps
def _print_logs(client: GiteaClient, job_id: int, tail: int = 0) -> None:
"""Fetch and print logs for a job. If tail > 0, print only last N lines."""
try:
logs = client.get_action_job_logs(job_id)
except APIError as e:
click.echo(_(" Could not fetch logs: {error}", error=str(e)))
return
if tail > 0:
lines = logs.strip().split("\n")
if len(lines) > tail:
click.echo(f" ... (showing last {tail} of {len(lines)} lines)")
logs = "\n".join(lines[-tail:])
for line in logs.split("\n"):
click.echo(f" {line}")
@click.command()
@click.option("--pr", "pr_number", type=int, default=None, help="PR number (default: auto-detect from current branch).")
@click.option("--job", default=None, help="Job name to show logs for (partial match, case-insensitive).")
@click.option("--tail", type=int, default=80, show_default=True, help="Show last N lines of logs (0 = all).")
@click.option("--owner", default=None, help="Repository owner (default: DEVX_REPO_OWNER).")
@click.option("--repo", default=None, help="Repository name (default: DEVX_REPO_NAME or GITHUB_REPOSITORY).")
def cli(
pr_number: int | None,
job: str | None,
tail: int,
owner: str | None,
repo: str | None,
) -> None:
"""Fetch logs for failed CI jobs on a pull request."""
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
raise click.ClickException(_("CI_GITEA_TOKEN is not set."))
repo_owner = owner or REPO_OWNER
if not repo_owner:
raise click.ClickException(_("Repository owner not set. Use --owner or DEVX_REPO_OWNER env var."))
repo_name = repo or get_repo_name()
client = GiteaClient(GITEA_API_URL, token, repo_owner, repo_name)
if pr_number is None:
pr_number = _get_current_branch_pr(client)
click.echo(_("Fetching logs for PR #{pr_number}...", pr_number=pr_number))
sha = _get_pr_sha(client, pr_number)
if not sha:
raise click.ClickException(_("Could not determine head SHA for PR #{pr_number}.", pr_number=pr_number))
run = _find_latest_run_by_sha(client, sha)
if not run:
raise click.ClickException(_("No workflow runs found for SHA {sha}.", sha=sha[:8]))
run_id = run.get("id", 0)
run_status = run.get("status", "?")
click.echo(_("Latest run: #{run_id} (status: {status})", run_id=run_id, status=run_status))
click.echo("")
jobs = client.get_action_run_jobs(run_id)
if not jobs:
click.echo(_("No jobs found for run #{run_id}.", run_id=run_id))
return
_print_job_summary(jobs)
click.echo("")
if job:
target = _find_job_by_name(jobs, job)
if not target:
raise click.ClickException(_("No job matching '{job}' found.", job=job))
click.echo(f"Logs for job '{target.get('name', '?')}' (id={target.get('id')}):")
_print_failed_steps(target)
click.echo("")
_print_logs(client, target["id"], tail)
else:
failed = _find_failed_jobs(jobs)
if not failed:
click.echo(_("No failed jobs."))
return
for fj in failed:
click.echo(f"Logs for failed job '{fj.get('name', '?')}' (id={fj.get('id')}):")
_print_failed_steps(fj)
click.echo("")
_print_logs(client, fj["id"], tail)
click.echo("")
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+174
View File
@@ -0,0 +1,174 @@
#!/usr/bin/env python3
"""Check CI status for a pull request or commit.
Displays the status of all CI checks for a PR (or a specific commit SHA).
Optionally polls until all checks complete (``--wait``).
Usage::
# Check status of PR #42
python -m devx.tools.pr_status --pr 42
# Check status of current branch's PR
python -m devx.tools.pr_status
# Wait for all checks to complete (timeout 600s)
python -m devx.tools.pr_status --pr 42 --wait --timeout 600
# Check a specific commit SHA
python -m devx.tools.pr_status --sha abc1234
The repository is auto-detected from ``DEVX_REPO_OWNER`` /
``DEVX_REPO_NAME`` or ``GITHUB_REPOSITORY`` environment variables.
"""
from __future__ import annotations
import os
import subprocess # nosec B404
import time
import click
from dotenv import load_dotenv
from devx.api_clients import GiteaClient
from devx.config import GITEA_API_URL, REPO_OWNER
from devx.i18n import _
from devx.tools.create_pr import get_repo_name
load_dotenv()
# Status symbols for terminal output
_STATUS_SYMBOLS = {
"success": "[OK]",
"failure": "[FAIL]",
"error": "[FAIL]",
"pending": "[..]",
"skipped": "[SKIP]",
"none": "[--]",
}
def _get_symbol(status: str) -> str:
return _STATUS_SYMBOLS.get(status, f"[{status}]")
def _get_pr_sha(client: GiteaClient, pr_number: int) -> str:
"""Fetch the head SHA of a PR."""
pr = client.get_pr(pr_number)
return pr.get("head", {}).get("sha", "")
def _get_current_branch_pr(client: GiteaClient) -> int:
"""Find the open PR for the current git branch."""
result = subprocess.run( # nosec
["git", "rev-parse", "--abbrev-ref", "HEAD"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
raise click.ClickException(_("Could not detect current branch: {error}", error=result.stderr.strip()))
branch = result.stdout.strip()
prs = client.list_prs(state="open")
for pr in prs:
if pr.get("head", {}).get("ref") == branch:
return int(pr["number"])
raise click.ClickException(_("No open PR found for branch '{branch}'.", branch=branch))
def print_status(client: GiteaClient, sha: str) -> str:
"""Print CI check statuses for a commit SHA. Returns the overall state."""
statuses = client.get_commit_status(sha)
if not statuses:
click.echo(_("No CI checks found for commit {sha}.", sha=sha[:8]))
return "none"
overall = "success"
for s in statuses:
context = s.get("context", "?")
status = s.get("status", "pending")
symbol = _get_symbol(status)
click.echo(f" {symbol} {context}")
if status in ("failure", "error"):
overall = "failure"
elif status == "pending" and overall != "failure":
overall = "pending"
elif status == "skipped" and overall == "success":
overall = "success"
click.echo(f"\n Overall: {_get_symbol(overall)} {overall}")
return overall
def wait_for_completion(
client: GiteaClient,
sha: str,
timeout: int = 600,
interval: int = 30,
) -> str:
"""Poll CI status until all checks complete or timeout. Returns final state."""
click.echo(_("Waiting for CI checks to complete (timeout: {timeout}s)...", timeout=timeout))
deadline = time.time() + timeout
while time.time() < deadline:
state = print_status(client, sha)
if state in ("success", "failure", "error", "none"):
return state
click.echo(f" ...still pending, retrying in {interval}s\n")
time.sleep(interval)
click.echo(_("Timeout reached after {timeout}s.", timeout=timeout))
return "pending"
@click.command()
@click.option("--pr", "pr_number", type=int, default=None, help="PR number (default: auto-detect from current branch).")
@click.option("--sha", default=None, help="Commit SHA to check (alternative to --pr).")
@click.option("--wait", "do_wait", is_flag=True, help="Poll until all checks complete.")
@click.option("--timeout", type=int, default=600, show_default=True, help="Wait timeout in seconds.")
@click.option("--interval", type=int, default=30, show_default=True, help="Poll interval in seconds.")
@click.option("--owner", default=None, help="Repository owner (default: DEVX_REPO_OWNER).")
@click.option("--repo", default=None, help="Repository name (default: DEVX_REPO_NAME or GITHUB_REPOSITORY).")
def cli(
pr_number: int | None,
sha: str | None,
do_wait: bool,
timeout: int,
interval: int,
owner: str | None,
repo: str | None,
) -> None:
"""Check CI status for a pull request or commit."""
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
raise click.ClickException(_("CI_GITEA_TOKEN is not set."))
repo_owner = owner or REPO_OWNER
if not repo_owner:
raise click.ClickException(_("Repository owner not set. Use --owner or DEVX_REPO_OWNER env var."))
repo_name = repo or get_repo_name()
client = GiteaClient(GITEA_API_URL, token, repo_owner, repo_name)
if sha is None:
if pr_number is None:
pr_number = _get_current_branch_pr(client)
click.echo(_("Checking status for PR #{pr_number}...", pr_number=pr_number))
pr = client.get_pr(pr_number)
sha = pr.get("head", {}).get("sha", "")
if not sha:
raise click.ClickException(_("Could not determine head SHA for PR #{pr_number}.", pr_number=pr_number))
click.echo(_("Commit: {sha}", sha=sha[:12]))
click.echo("")
state = wait_for_completion(client, sha, timeout, interval) if do_wait else print_status(client, sha)
if state in ("failure", "error"):
raise click.ClickException(_("CI checks failed."))
if state == "pending" and do_wait:
raise click.ClickException(_("CI checks did not complete within timeout."))
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+123
View File
@@ -0,0 +1,123 @@
#!/usr/bin/env python3
"""Pre-push validation: ensure a Vikunja task exists for the branch.
This tool is designed to run as a git pre-push hook. It extracts the
task ID from the branch name (e.g. ``DEVX-31-fix-foo`` → ``DEVX-31``)
and verifies that a corresponding Vikunja task exists.
If the task does not exist, the hook **fails with guidance** — it does
not auto-create the task. This prevents accidental pushes of branches
without a planning task.
Usage::
python -m devx.tools.pre_push_check --branch DEVX-31-fix-foo
Exit codes:
0 — all checks passed, safe to push
1 — validation failed (missing task, missing token, etc.)
"""
from __future__ import annotations
import os
import subprocess # nosec B404
import click
from dotenv import load_dotenv
from devx.api_clients import VikunjaClient
from devx.config import DEFAULT_PER_PAGE, TASK_ID_RE, TASK_PREFIX, VIKUNJA_API_URL, VIKUNJA_PROJECT_ID
from devx.i18n import _
load_dotenv()
def get_current_branch() -> str:
"""Return the current git branch name, or empty string on error."""
result = subprocess.run( # nosec
["git", "rev-parse", "--abbrev-ref", "HEAD"],
capture_output=True,
text=True,
check=False,
)
return result.stdout.strip()
def extract_task_id(branch: str) -> str:
"""Extract the task ID (e.g. ``DEVX-31``) from a branch name."""
match = TASK_ID_RE.search(branch)
return match.group(0) if match else ""
def task_exists(task_id: str) -> bool:
"""Check if a Vikunja task with the given identifier exists.
Returns ``False`` if VIKUNJA_TOKEN is not set (soft-fail in local mode).
"""
token = os.environ.get("VIKUNJA_TOKEN", "")
if not token:
return False
client = VikunjaClient(VIKUNJA_API_URL, token)
return client.find_task_by_identifier(VIKUNJA_PROJECT_ID, task_id, per_page=DEFAULT_PER_PAGE) is not None
def validate(branch: str) -> None:
"""Run all pre-push validations for the given branch.
Raises ``click.ClickException`` on failure.
"""
if not branch or branch in ("master", "main"):
return
task_id = extract_task_id(branch)
if not task_id:
raise click.ClickException(
_(
"Branch '{branch}' does not contain a task ID.\n"
" Expected format: {prefix}-N-short-description\n"
" Example: {prefix}-42-add-feature\n"
" Fix: rename the branch or create a Vikunja task first:\n"
' python -m devx.tools.create_task --title "Task title"',
branch=branch,
prefix=TASK_PREFIX,
)
)
token = os.environ.get("VIKUNJA_TOKEN", "")
if not token:
click.echo(
_(
"WARNING: VIKUNJA_TOKEN not set — skipping task existence check. "
"Set it in .env to enable full validation.",
),
err=True,
)
return
if not task_exists(task_id):
raise click.ClickException(
_(
"Vikunja task {task_id} not found in project {project_id}.\n"
" Create it first:\n"
' python -m devx.tools.create_task --title "Task title"\n'
" Or check that the task ID in the branch name is correct.",
task_id=task_id,
project_id=VIKUNJA_PROJECT_ID,
)
)
click.echo(_("Pre-push check passed: task {task_id} exists.", task_id=task_id))
@click.command()
@click.option("--branch", default=None, help="Branch name (default: auto-detect from git).")
def cli(branch: str | None) -> None:
"""Validate pre-push preconditions for the current branch."""
if branch is None:
branch = get_current_branch()
validate(branch)
if __name__ == "__main__": # pragma: no cover
cli() # pragma: no cover
+55 -9
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Project setup: install Python deps, pre-commit hooks, and tea CLI login.
"""Project setup: install Python deps, Ansible collections, pre-commit hooks, and tea CLI login.
Usage::
@@ -26,9 +26,24 @@ def _run(cmd: list[str]) -> None:
def _install_python_deps(bin_dir: str, extras: str = "dev") -> None:
"""Install the project with the specified extras in editable mode."""
"""Install the project with the specified extras in editable mode.
In CI (system Python with PIP_BREAK_SYSTEM_PACKAGES=1), a first attempt
uses --break-system-packages. If that fails (e.g. debian-installed
packages without RECORD files), retry with --ignore-installed to skip
uninstalling system packages entirely.
"""
pip = str(Path(bin_dir) / "pip")
_run([pip, "install", "-e", f".[{extras}]"])
cmd = [pip, "install", "-e", f".[{extras}]"]
if os.environ.get("PIP_BREAK_SYSTEM_PACKAGES") == "1":
cmd.append("--break-system-packages")
result = subprocess.run(cmd, check=False) # nosec B603
if result.returncode != 0 and os.environ.get("PIP_BREAK_SYSTEM_PACKAGES") == "1":
click.echo(" Retrying with --ignore-installed to bypass system packages...")
cmd.append("--ignore-installed")
_run(cmd)
elif result.returncode != 0:
raise subprocess.CalledProcessError(result.returncode, cmd)
def _install_pre_commit_hooks(bin_dir: str) -> None:
@@ -38,20 +53,30 @@ def _install_pre_commit_hooks(bin_dir: str) -> None:
_run([pre_commit, "install", "--hook-type", hook_type])
def _install_ansible_collections(bin_dir: str) -> None:
"""Install required Ansible Galaxy collections if requirements exist."""
galaxy = shutil.which("ansible-galaxy") or str(Path(bin_dir) / "ansible-galaxy")
requirements = Path("ansible/requirements.yml")
if not requirements.exists():
click.echo(" ansible/requirements.yml not found — skipping collections.")
return
_run([galaxy, "collection", "install", "-r", str(requirements)])
def _configure_tea_login() -> None:
"""Configure tea CLI login from .env if REPO_TOKEN is set.
"""Configure tea CLI login from .env if CI_GITEA_TOKEN is set.
Idempotent: if a login with the same name already exists, it is not re-added.
Skips if tea is not installed or REPO_TOKEN is not set.
Skips if tea is not installed or CI_GITEA_TOKEN is not set.
"""
tea_bin = shutil.which("tea")
if tea_bin is None:
click.echo("tea: not installed — run 'make install-tools' to install it.")
return
token = os.environ.get("REPO_TOKEN", "")
token = os.environ.get("CI_GITEA_TOKEN", "")
if not token:
click.echo("tea: REPO_TOKEN not set — skipping login configuration.")
click.echo("tea: CI_GITEA_TOKEN not set — skipping login configuration.")
return
api_url = os.environ.get("DEVX_GITEA_API_URL", "https://git.oblachno.oblachno.fyi/api/v1")
@@ -119,18 +144,39 @@ def _verify(bin_dir: str) -> None:
default=False,
help="Skip tea CLI login configuration.",
)
@click.option(
"--no-ansible-collections",
is_flag=True,
default=False,
help="Skip Ansible Galaxy collection installation.",
)
@click.option(
"--skip-install",
is_flag=True,
default=False,
help="Skip pip install (use when deps already installed, e.g. devx came via ci extra).",
)
def main(
bin_dir: str,
extras: str,
no_pre_commit: bool,
no_tea_login: bool,
no_ansible_collections: bool,
skip_install: bool,
) -> None:
"""Install Python deps, pre-commit hooks, and configure tea CLI."""
if not Path(bin_dir).exists():
raise click.ClickException(f"Bin directory not found: {bin_dir}. Run 'python3 -m venv .venv' first.")
click.echo(f"Installing Python dependencies (extras: {extras})...")
_install_python_deps(bin_dir, extras)
if not skip_install:
click.echo(f"Installing Python dependencies (extras: {extras})...")
_install_python_deps(bin_dir, extras)
else:
click.echo("Skipping pip install (--skip-install).")
if not no_ansible_collections:
click.echo("Installing Ansible Galaxy collections...")
_install_ansible_collections(bin_dir)
if not no_pre_commit:
click.echo("Installing pre-commit hooks...")
+2522 -251
View File
File diff suppressed because it is too large Load Diff
+273 -25
View File
@@ -6,7 +6,7 @@ from unittest.mock import MagicMock, patch
import pytest
import requests
from devx.api_clients import GiteaClient, VikunjaClient, _is_retryable, _parse_error
from devx.api_clients import GiteaClient, VikunjaClient, _parse_error
from devx.config import (
DEFAULT_PER_PAGE,
DEFAULT_TIMEOUT,
@@ -137,6 +137,19 @@ class TestGiteaClient:
assert result is None
client.create_label.assert_not_called()
def test_ensure_label_creates_when_others_exist(self) -> None:
"""When labels exist but none match the target name, create a new one."""
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client.list_labels = MagicMock(
return_value=[{"name": "bug", "color": "ff0000"}, {"name": "docs", "color": "007ec6"}]
)
client.create_label = MagicMock(return_value={"name": "ready-to-merge", "color": "2ecc71"})
result = client.ensure_label("ready-to-merge", "2ecc71", "desc")
assert result is not None
assert result["name"] == "ready-to-merge"
client.create_label.assert_called_once_with("ready-to-merge", "2ecc71", "desc")
def test_list_branch_protections(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(
@@ -196,6 +209,18 @@ class TestGiteaClient:
expected_update = {k: v for k, v in TEST_BP_CONFIG.items() if k != "branch_name"}
client.update_branch_protection.assert_called_once_with("master", expected_update)
def test_ensure_branch_protection_creates_when_none_match(self) -> None:
"""When existing protections exist but none match the target branch, create a new one."""
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client.list_branch_protections = MagicMock(
return_value=[{"branch_name": "develop"}, {"branch_name": "staging"}]
)
client.create_branch_protection = MagicMock(return_value={"id": 5, "branch_name": "master"})
result = client.ensure_branch_protection("master", TEST_BP_CONFIG)
assert result["id"] == 5
client.create_branch_protection.assert_called_once_with(TEST_BP_CONFIG)
def test_merge_pr(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(return_value=_mock_response())
@@ -248,6 +273,37 @@ class TestGiteaClient:
timeout=DEFAULT_TIMEOUT,
)
def test_create_pr(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(
return_value=_mock_response({"number": 15, "html_url": "https://git.example.com/pr/15"})
)
result = client.create_pr(title="DEVX-42: Add feature", head="DEVX-42-fix", body="desc")
assert result["number"] == 15
client._session.request.assert_called_once_with(
"POST",
"https://git.example.com/repos/owner/repo/pulls",
timeout=DEFAULT_TIMEOUT,
json={"title": "DEVX-42: Add feature", "head": "DEVX-42-fix", "base": "master", "body": "desc"},
)
def test_create_pr_no_body(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(
return_value=_mock_response({"number": 16, "html_url": "https://git.example.com/pr/16"})
)
result = client.create_pr(title="DEVX-43: Fix bug", head="DEVX-43-fix")
assert result["number"] == 16
call_kwargs = client._session.request.call_args.kwargs
assert "body" not in call_kwargs["json"]
def test_create_pr_custom_base(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(return_value=_mock_response({"number": 17}))
client.create_pr(title="Test", head="branch", base="develop")
call_kwargs = client._session.request.call_args.kwargs
assert call_kwargs["json"]["base"] == "develop"
def test_get_pr_files(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(
@@ -278,6 +334,35 @@ class TestGiteaClient:
timeout=DEFAULT_TIMEOUT,
)
def test_list_prs(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(
return_value=_mock_response([{"number": 1, "title": "feat: add"}, {"number": 2, "title": "fix: bug"}])
)
result = client.list_prs()
assert len(result) == 2
assert result[0]["number"] == 1
client._session.request.assert_called_once_with(
"GET",
"https://git.example.com/repos/owner/repo/pulls",
params={"state": "all"},
timeout=DEFAULT_TIMEOUT,
)
def test_list_prs_with_params(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(return_value=_mock_response([{"number": 3, "title": "docs: update"}]))
result = client.list_prs(state="closed", q="docs")
assert len(result) == 1
client._session.request.assert_called_once_with(
"GET",
"https://git.example.com/repos/owner/repo/pulls",
params={"state": "closed", "q": "docs"},
timeout=DEFAULT_TIMEOUT,
)
def test_get_pr_reviews(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(return_value=_mock_response([{"id": 1, "state": "APPROVED"}]))
@@ -422,7 +507,7 @@ class TestGiteaClient:
assert result["id"] == 1
assert client._session.request.call_count == 2
@patch("devx.api_clients.time.sleep")
@patch("time.sleep")
def test_request_retries_on_429(self, mock_sleep: MagicMock) -> None:
"""Should retry on 429 rate limit with exponential backoff."""
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
@@ -435,7 +520,7 @@ class TestGiteaClient:
assert client._session.request.call_count == 3
assert mock_sleep.call_count == 2
@patch("devx.api_clients.time.sleep")
@patch("time.sleep")
def test_request_retries_on_503(self, mock_sleep: MagicMock) -> None:
"""Should retry on 503 service unavailable."""
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
@@ -447,7 +532,7 @@ class TestGiteaClient:
assert result.json() == {"ok": True}
assert client._session.request.call_count == 2
@patch("devx.api_clients.time.sleep")
@patch("time.sleep")
def test_request_no_retry_on_404(self, mock_sleep: MagicMock) -> None:
"""Should NOT retry on 404 — it's not a transient error."""
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
@@ -460,7 +545,7 @@ class TestGiteaClient:
assert client._session.request.call_count == 1
mock_sleep.assert_not_called()
@patch("devx.api_clients.time.sleep")
@patch("time.sleep")
def test_request_retries_on_connection_error(self, mock_sleep: MagicMock) -> None:
"""Should retry on connection errors."""
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
@@ -470,7 +555,7 @@ class TestGiteaClient:
assert result.json() == {"ok": True}
assert client._session.request.call_count == 2
@patch("devx.api_clients.time.sleep")
@patch("time.sleep")
def test_request_max_retries_exhausted(self, mock_sleep: MagicMock) -> None:
"""Should raise APIError after max retries on persistent 503."""
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
@@ -482,7 +567,7 @@ class TestGiteaClient:
assert exc_info.value.status == 503
assert client._session.request.call_count == 3 # MAX_RETRIES
@patch("devx.api_clients.time.sleep")
@patch("time.sleep")
def test_request_connection_error_exhausted(self, mock_sleep: MagicMock) -> None:
"""Should raise APIError after max retries on persistent connection errors."""
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
@@ -566,7 +651,48 @@ class TestVikunjaClient:
json={"done": True},
)
def test_http_error_raises_api_error(self) -> None:
def test_list_comments(self) -> None:
client = VikunjaClient("https://work.example.com", "tok")
client._session.request = MagicMock(
return_value=_mock_response([{"id": 1, "comment": "first"}, {"id": 2, "comment": "second"}])
)
result = client.list_comments(42)
assert len(result) == 2
assert result[0]["comment"] == "first"
client._session.request.assert_called_once_with(
"GET",
"https://work.example.com/tasks/42/comments",
timeout=DEFAULT_TIMEOUT,
)
def test_update_task_safe(self) -> None:
client = VikunjaClient("https://work.example.com", "tok")
client._session.request = MagicMock(
side_effect=[
_mock_response({"id": 42, "title": "My task", "done": False}),
_mock_response({"id": 42, "title": "My task", "done": True}),
]
)
result = client.update_task_safe(42, done=True)
assert result["done"] is True
assert result["title"] == "My task"
assert client._session.request.call_count == 2
client._session.request.assert_any_call(
"GET",
"https://work.example.com/tasks/42",
timeout=DEFAULT_TIMEOUT,
)
client._session.request.assert_any_call(
"POST",
"https://work.example.com/tasks/42",
timeout=DEFAULT_TIMEOUT,
json={"id": 42, "title": "My task", "done": True},
)
@patch("time.sleep")
def test_http_error_raises_api_error(self, mock_sleep: MagicMock) -> None:
client = VikunjaClient("https://work.example.com", "tok")
mock_resp = MagicMock()
mock_resp.raise_for_status.side_effect = _mock_http_error(http.HTTPStatus.INTERNAL_SERVER_ERROR, "server error")
@@ -587,7 +713,7 @@ class TestVikunjaClient:
client.list_tasks()
assert "connection failed" in str(exc_info.value)
@patch("devx.api_clients.time.sleep")
@patch("time.sleep")
def test_vikunja_retries_on_503(self, mock_sleep: MagicMock) -> None:
"""VikunjaClient should also retry on 503."""
client = VikunjaClient("https://work.example.com", "tok")
@@ -599,7 +725,7 @@ class TestVikunjaClient:
assert len(result) == 1
assert client._session.request.call_count == 2
@patch("devx.api_clients.time.sleep")
@patch("time.sleep")
def test_vikunja_retries_on_connection_error(self, mock_sleep: MagicMock) -> None:
"""VikunjaClient should retry on connection errors."""
client = VikunjaClient("https://work.example.com", "tok")
@@ -609,7 +735,7 @@ class TestVikunjaClient:
assert len(result) == 1
assert client._session.request.call_count == 2
@patch("devx.api_clients.time.sleep")
@patch("time.sleep")
def test_vikunja_max_retries_exhausted(self, mock_sleep: MagicMock) -> None:
"""VikunjaClient should raise APIError after max retries on persistent 503."""
client = VikunjaClient("https://work.example.com", "tok")
@@ -621,7 +747,7 @@ class TestVikunjaClient:
assert exc_info.value.status == 503
assert client._session.request.call_count == 3 # MAX_RETRIES
@patch("devx.api_clients.time.sleep")
@patch("time.sleep")
def test_vikunja_connection_error_exhausted(self, mock_sleep: MagicMock) -> None:
"""VikunjaClient should raise APIError after max retries on persistent connection errors."""
client = VikunjaClient("https://work.example.com", "tok")
@@ -631,21 +757,143 @@ class TestVikunjaClient:
assert exc_info.value.status == 0
assert client._session.request.call_count == 3 # MAX_RETRIES
def test_vikunja_create_task(self) -> None:
client = VikunjaClient("https://work.example.com", "tok")
client._session.request = MagicMock(
return_value=_mock_response({"id": 1, "identifier": "DEVX-1", "title": "Test"})
)
result = client.create_task(6, "Test", "<p>desc</p>")
assert result["identifier"] == "DEVX-1"
client._session.request.assert_called_once_with(
"PUT",
"https://work.example.com/projects/6/tasks",
timeout=DEFAULT_TIMEOUT,
json={"title": "Test", "description": "<p>desc</p>"},
)
class TestIsRetryable:
def test_connection_error_is_retryable(self) -> None:
assert _is_retryable(requests.ConnectionError("refused")) is True
def test_vikunja_create_task_no_description(self) -> None:
client = VikunjaClient("https://work.example.com", "tok")
client._session.request = MagicMock(
return_value=_mock_response({"id": 2, "identifier": "DEVX-2", "title": "No desc"})
)
result = client.create_task(6, "No desc")
assert result["id"] == 2
call_kwargs = client._session.request.call_args.kwargs
assert call_kwargs["json"]["description"] == ""
def test_timeout_is_retryable(self) -> None:
assert _is_retryable(requests.Timeout("timed out")) is True
def test_find_task_by_identifier_found(self) -> None:
client = VikunjaClient("https://work.example.com", "tok")
client._session.request = MagicMock(
return_value=_mock_response([{"identifier": "DEVX-1"}, {"identifier": "DEVX-42", "title": "Found"}])
)
result = client.find_task_by_identifier(6, "DEVX-42", per_page=50)
assert result is not None
assert result["title"] == "Found"
def test_429_is_retryable(self) -> None:
err = _mock_http_error(429, "rate limited")
assert _is_retryable(err) is True
def test_find_task_by_identifier_not_found(self) -> None:
client = VikunjaClient("https://work.example.com", "tok")
client._session.request = MagicMock(
return_value=_mock_response([{"identifier": "DEVX-1"}, {"identifier": "DEVX-2"}])
)
result = client.find_task_by_identifier(6, "DEVX-99", per_page=50)
assert result is None
def test_404_is_not_retryable(self) -> None:
err = _mock_http_error(404, "not found")
assert _is_retryable(err) is False
def test_find_task_by_identifier_empty_project(self) -> None:
client = VikunjaClient("https://work.example.com", "tok")
client._session.request = MagicMock(return_value=_mock_response([]))
result = client.find_task_by_identifier(6, "DEVX-1", per_page=50)
assert result is None
def test_generic_exception_is_not_retryable(self) -> None:
assert _is_retryable(ValueError("oops")) is False
def test_find_task_by_identifier_paginates(self) -> None:
client = VikunjaClient("https://work.example.com", "tok")
full_page = [{"identifier": f"DEVX-{i}"} for i in range(50)]
client._session.request = MagicMock(
side_effect=[
_mock_response(full_page),
_mock_response([{"identifier": "DEVX-50", "title": "Found on page 2"}]),
]
)
result = client.find_task_by_identifier(6, "DEVX-50", per_page=50)
assert result is not None
assert result["title"] == "Found on page 2"
class TestGiteaClientPrLabels:
def test_add_pr_label(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(return_value=_mock_response({}))
client.add_pr_label(42, ["ready-to-merge"])
client._session.request.assert_called_once_with(
"POST",
"https://git.example.com/repos/owner/repo/issues/42/labels",
timeout=DEFAULT_TIMEOUT,
json={"labels": ["ready-to-merge"]},
)
def test_add_pr_label_multiple(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(return_value=_mock_response({}))
client.add_pr_label(42, ["ready-to-merge", "reviewed"])
call_kwargs = client._session.request.call_args.kwargs
assert call_kwargs["json"]["labels"] == ["ready-to-merge", "reviewed"]
def test_get_pr_label_names(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(return_value=_mock_response([{"name": "bug"}, {"name": "ready-to-merge"}]))
result = client.get_pr_label_names(42)
assert result == ["bug", "ready-to-merge"]
client._session.request.assert_called_once_with(
"GET",
"https://git.example.com/repos/owner/repo/issues/42/labels",
timeout=DEFAULT_TIMEOUT,
)
class TestGiteaClientActions:
def test_list_action_runs(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(
return_value=_mock_response({"workflow_runs": [{"id": 1, "status": "completed"}], "total_count": 1})
)
result = client.list_action_runs(branch="feature-branch", limit=1)
assert result["total_count"] == 1
client._session.request.assert_called_once_with(
"GET",
"https://git.example.com/repos/owner/repo/actions/runs",
timeout=DEFAULT_TIMEOUT,
params={"branch": "feature-branch", "limit": 1},
)
def test_get_action_run_jobs(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(
return_value=_mock_response({"jobs": [{"id": 100, "name": "quality", "conclusion": "failure"}]})
)
result = client.get_action_run_jobs(1410)
assert len(result) == 1
assert result[0]["name"] == "quality"
client._session.request.assert_called_once_with(
"GET",
"https://git.example.com/repos/owner/repo/actions/runs/1410/jobs",
timeout=DEFAULT_TIMEOUT,
)
def test_get_action_run_jobs_empty(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
client._session.request = MagicMock(return_value=_mock_response({}))
result = client.get_action_run_jobs(1410)
assert result == []
def test_get_action_job_logs(self) -> None:
client = GiteaClient("https://git.example.com", "tok", "owner", "repo")
mock_resp = MagicMock()
mock_resp.text = "log line 1\nlog line 2"
mock_resp.raise_for_status = MagicMock()
client._session.request = MagicMock(return_value=mock_resp)
result = client.get_action_job_logs(10026)
assert "log line 1" in result
client._session.request.assert_called_once_with(
"GET",
"https://git.example.com/repos/owner/repo/actions/jobs/10026/logs",
timeout=DEFAULT_TIMEOUT,
)
+131 -53
View File
@@ -6,12 +6,12 @@ import click
import pytest
from click.testing import CliRunner
from devx.ci._shared import run_cmd
from devx.ci.auto_merge import (
extract_conventional_msg,
extract_task_id,
main,
read_taskid,
run_cmd,
validate_pr_title,
validate_pr_title_matches_vikunja,
)
@@ -21,23 +21,47 @@ from devx.exceptions import APIError
class TestReadTaskid:
def test_reads_from_file(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-60\n")
assert read_taskid("some-branch") == "DEVX-60"
def test_falls_back_to_branch_name(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
def test_extracts_from_branch_name(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
assert read_taskid("DEVX-19-fix-bug") == "DEVX-19"
def test_returns_empty_when_no_file_no_match(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
def test_returns_empty_when_no_match(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
assert read_taskid("feature-branch") == ""
def test_empty_file_falls_back_to_branch(self, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
def test_warns_on_stale_taskid_file(self, tmp_path, monkeypatch, capsys) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-60\n")
# Branch name takes priority, stale .taskid should produce deprecation warning
assert read_taskid("DEVX-19-fix-bug") == "DEVX-19"
captured = capsys.readouterr()
combined = captured.out + captured.err
assert "WARNING" in combined
assert "deprecated" in combined
assert "DEVX-60" in combined
assert "DEVX-19" in combined
def test_no_warning_when_taskid_file_absent(self, tmp_path, monkeypatch, capsys) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
assert read_taskid("DEVX-42-test") == "DEVX-42"
captured = capsys.readouterr()
assert "WARNING" not in captured.out
def test_no_warning_when_taskid_file_matches_branch(self, tmp_path, monkeypatch, capsys) -> None: # type: ignore[no-untyped-def]
"""No warning when .taskid file content matches the branch task ID."""
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-19\n")
assert read_taskid("DEVX-19-fix-bug") == "DEVX-19"
captured = capsys.readouterr()
assert "WARNING" not in captured.out
def test_no_warning_when_taskid_file_empty(self, tmp_path, monkeypatch, capsys) -> None: # type: ignore[no-untyped-def]
"""No warning when .taskid file exists but is empty."""
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("\n")
assert read_taskid("DEVX-42-test") == "DEVX-42"
assert read_taskid("DEVX-19-fix-bug") == "DEVX-19"
captured = capsys.readouterr()
assert "WARNING" not in captured.out
# -- extract_task_id (legacy fallback) --
@@ -77,9 +101,10 @@ class TestValidatePrTitle:
class TestValidatePrTitleMatchesVikunja:
@patch.dict("os.environ", {}, clear=True)
def test_skips_when_no_token(self) -> None:
# Should not raise — just warn
validate_pr_title_matches_vikunja("DEVX-19: test", "DEVX-19")
def test_raises_when_no_token(self) -> None:
"""Should raise ClickException when VIKUNJA_TOKEN is not set."""
with pytest.raises(click.ClickException, match="VIKUNJA_TOKEN is not set"):
validate_pr_title_matches_vikunja("DEVX-19: test", "DEVX-19")
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"}, clear=True)
@patch("devx.ci.auto_merge.VikunjaClient")
@@ -170,6 +195,29 @@ class TestExtractConventionalMsg:
]
assert extract_conventional_msg(commits) == "feat: add feature"
def test_prefers_feat_over_refactor(self) -> None:
"""When both feat and refactor commits exist, feat wins."""
commits = [
{"commit": {"message": "refactor: add find_task_by_identifier"}},
{"commit": {"message": "fix: remove hardcoded fallbacks"}},
{"commit": {"message": "feat: add manual review support"}},
]
assert extract_conventional_msg(commits) == "feat: add manual review support"
def test_prefers_fix_over_docs(self) -> None:
commits = [
{"commit": {"message": "docs: update README"}},
{"commit": {"message": "fix: resolve bug"}},
]
assert extract_conventional_msg(commits) == "fix: resolve bug"
def test_scope_in_prefix(self) -> None:
commits = [
{"commit": {"message": "refactor(ci): cleanup code"}},
{"commit": {"message": "feat(api): add endpoint"}},
]
assert extract_conventional_msg(commits) == "feat(api): add endpoint"
# -- run_cmd --
@@ -192,11 +240,13 @@ class TestRunCmd:
class TestMain:
@patch.dict("os.environ", {"REPO_TOKEN": "tok", "VIKUNJA_TOKEN": ""}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok", "VIKUNJA_TOKEN": "tok"}, clear=True)
@patch("devx.ci.auto_merge.validate_pr_title_matches_vikunja")
@patch("devx.ci.auto_merge.GiteaClient")
def test_full_merge_flow(self, mock_client_cls: MagicMock, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
def test_full_merge_flow(
self, mock_client_cls: MagicMock, _mock_validate: MagicMock, tmp_path, monkeypatch
) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-19\n")
mock_client = MagicMock()
mock_client.get_pr_commits.return_value = [
@@ -210,68 +260,74 @@ class TestMain:
["DEVX-19-fix-bug", "DEVX-19: Fix timeout", "owner/repo", "7"],
)
assert result.exit_code == 0, result.output
mock_client.merge_pr.assert_called_once_with("7", "DEVX-19: fix: resolve timeout")
mock_client.merge_pr.assert_called_once_with(7, "DEVX-19: fix: resolve timeout")
@patch.dict("os.environ", {"REPO_TOKEN": ""}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": ""}, clear=True)
def test_no_token_raises(self) -> None:
runner = CliRunner()
result = runner.invoke(main, ["DEVX-19-fix", "DEVX-19: test", "owner/repo", "7"])
assert result.exit_code != 0
assert "REPO_TOKEN" in result.output
assert "CI_GITEA_TOKEN" in result.output
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
@patch("devx.ci.auto_merge.GiteaClient")
def test_no_task_id_raises(self, mock_client_cls: MagicMock, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
# No .taskid file, no DEVX-N in branch name
# No DEVX-N in branch name
runner = CliRunner()
result = runner.invoke(main, ["feature-branch", "DEVX-19: test", "owner/repo", "7"])
assert result.exit_code != 0
assert "No task ID" in result.output
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
@patch("devx.ci.auto_merge.GiteaClient")
def test_invalid_pr_title_raises(self, mock_client_cls: MagicMock, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-19\n")
runner = CliRunner()
result = runner.invoke(main, ["DEVX-19-fix", "Bad title", "owner/repo", "7"])
assert result.exit_code != 0
assert "format" in result.output.lower()
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok", "VIKUNJA_TOKEN": "tok"}, clear=True)
@patch("devx.ci.auto_merge.validate_pr_title_matches_vikunja")
@patch("devx.ci.auto_merge.GiteaClient")
def test_merge_behind_master_rebases(self, mock_client_cls: MagicMock, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
def test_merge_behind_master_raises_no_rebase(
self, mock_client_cls: MagicMock, _mock_validate: MagicMock, tmp_path, monkeypatch
) -> None: # type: ignore[no-untyped-def]
"""When branch is behind master, auto-merge should NOT rebase.
Auto-rebasing creates a feedback loop: the force-push triggers a new
pull_request synchronize event, which starts a new CI run, which runs
auto-merge again, which rebases again, etc.
"""
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-19\n")
mock_client = MagicMock()
mock_client.get_pr_commits.return_value = [
{"commit": {"message": "fix: resolve timeout"}},
]
mock_client.merge_pr.side_effect = [
APIError(405, "HEAD branch is behind master"),
None, # Second call succeeds
]
mock_client.merge_pr.side_effect = APIError(405, "HEAD branch is behind master")
mock_client_cls.return_value = mock_client
with patch("devx.ci.auto_merge.run_cmd") as mock_run:
runner = CliRunner()
result = runner.invoke(
main,
["DEVX-19-fix-bug", "DEVX-19: Fix timeout", "owner/repo", "7"],
)
assert result.exit_code == 0, result.output
assert mock_client.merge_pr.call_count == 2
# Should have fetched, rebased, and pushed
assert mock_run.call_count == 5 # config name, config email, fetch, rebase, push
runner = CliRunner()
result = runner.invoke(
main,
["DEVX-19-fix-bug", "DEVX-19: Fix timeout", "owner/repo", "7"],
)
assert result.exit_code != 0
assert "behind master" in result.output.lower()
assert "rebase manually" in result.output.lower()
# Must NOT have called merge_pr twice (no retry after rebase)
assert mock_client.merge_pr.call_count == 1
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok", "VIKUNJA_TOKEN": "tok"}, clear=True)
@patch("devx.ci.auto_merge.validate_pr_title_matches_vikunja")
@patch("devx.ci.auto_merge.GiteaClient")
def test_merge_failure_raises(self, mock_client_cls: MagicMock, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
def test_merge_failure_raises(
self, mock_client_cls: MagicMock, _mock_validate: MagicMock, tmp_path, monkeypatch
) -> None: # type: ignore[no-untyped-def]
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-19\n")
mock_client = MagicMock()
mock_client.get_pr_commits.return_value = [
@@ -288,12 +344,14 @@ class TestMain:
assert result.exit_code != 0
assert "Merge failed" in result.output
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok", "VIKUNJA_TOKEN": "tok"}, clear=True)
@patch("devx.ci.auto_merge.validate_pr_title_matches_vikunja")
@patch("devx.ci.auto_merge.GiteaClient")
def test_no_conventional_msg_raises(self, mock_client_cls: MagicMock, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
def test_no_conventional_msg_raises(
self, mock_client_cls: MagicMock, _mock_validate: MagicMock, tmp_path, monkeypatch
) -> None: # type: ignore[no-untyped-def]
"""When no conventional commit message is found in PR commits, raises."""
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-19\n")
mock_client = MagicMock()
mock_client.get_pr_commits.return_value = []
@@ -307,12 +365,32 @@ class TestMain:
assert result.exit_code != 0
assert "conventional commit" in result.output.lower()
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
def test_invalid_pr_number_raises(self, tmp_path, monkeypatch) -> None:
"""Non-integer PR number should raise."""
monkeypatch.chdir(tmp_path)
runner = CliRunner()
result = runner.invoke(main, ["DEVX-19-fix", "DEVX-19: Test", "owner/repo", "not-a-number"])
assert result.exit_code != 0
assert "PR number must be an integer" in result.output
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
def test_invalid_repo_format_raises(self, tmp_path, monkeypatch) -> None:
"""Repo without owner/name should raise."""
monkeypatch.chdir(tmp_path)
runner = CliRunner()
result = runner.invoke(main, ["DEVX-19-fix", "DEVX-19: Test", "invalidrepo", "7"])
assert result.exit_code != 0
assert "owner/name" in result.output
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok", "VIKUNJA_TOKEN": "tok"}, clear=True)
@patch("devx.ci.auto_merge.validate_pr_title_matches_vikunja")
@patch("devx.ci.auto_merge.GiteaClient")
def test_rebase_retry_failure_raises(self, mock_client_cls: MagicMock, tmp_path, monkeypatch) -> None: # type: ignore[no-untyped-def]
"""When rebase retry also fails, raises with helpful message."""
def test_merge_behind_master_does_not_force_push(
self, mock_client_cls: MagicMock, _mock_validate: MagicMock, tmp_path, monkeypatch
) -> None: # type: ignore[no-untyped-def]
"""Verify no git commands are run when branch is behind master."""
monkeypatch.chdir(tmp_path)
(tmp_path / ".taskid").write_text("DEVX-19\n")
mock_client = MagicMock()
mock_client.get_pr_commits.return_value = [
@@ -321,15 +399,15 @@ class TestMain:
mock_client.merge_pr.side_effect = APIError(405, "HEAD branch is behind master")
mock_client_cls.return_value = mock_client
with patch("devx.ci.auto_merge.run_cmd") as mock_run:
mock_run.side_effect = click.ClickException("git rebase failed")
with patch("devx.ci._shared.run_cmd") as mock_run:
runner = CliRunner()
result = runner.invoke(
main,
["DEVX-19-fix-bug", "DEVX-19: Fix timeout", "owner/repo", "7"],
)
assert result.exit_code != 0
assert "rebase" in result.output.lower()
# No git commands should be run (no rebase, no push)
mock_run.assert_not_called()
def test_main_module_block() -> None:
+737
View File
@@ -0,0 +1,737 @@
"""Unit tests for devx.tools.build_image."""
from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from click import ClickException
from click.testing import CliRunner
import devx.tools.build_image as build_image
from devx.tools.build_image import (
ImageSpec,
build_full_tag,
load_manifest,
push_image,
registry_login,
)
from devx.tools.build_image import (
build_image as do_build,
)
from devx.tools.clean_images import select_for_deletion, sort_versions_by_date
class TestImageSpec:
def test_from_dict_minimal(self) -> None:
spec = ImageSpec.from_dict({"name": "ci-base", "dockerfile": "docker/ci-base/Dockerfile"})
assert spec.name == "ci-base"
assert spec.dockerfile == "docker/ci-base/Dockerfile"
assert spec.context == "."
assert spec.tags == ["latest"]
def test_from_dict_full(self) -> None:
spec = ImageSpec.from_dict(
{
"name": "ci-quality",
"dockerfile": "docker/ci-quality/Dockerfile",
"context": ".",
"tags": ["latest", "0.19.3"],
}
)
assert spec.name == "ci-quality"
assert spec.dockerfile == "docker/ci-quality/Dockerfile"
assert spec.context == "."
assert spec.tags == ["latest", "0.19.3"]
def test_from_dict_missing_name(self) -> None:
with pytest.raises(ValueError, match="missing 'name'"):
ImageSpec.from_dict({"dockerfile": "Dockerfile"})
def test_from_dict_missing_dockerfile(self) -> None:
with pytest.raises(ValueError, match="missing 'dockerfile'"):
ImageSpec.from_dict({"name": "ci-base"})
def test_from_dict_tags_not_list(self) -> None:
with pytest.raises(ValueError, match="tags.*must be a list"):
ImageSpec.from_dict(
{
"name": "ci-base",
"dockerfile": "Dockerfile",
"tags": "latest",
}
)
def test_from_dict_empty_tags_defaults_to_latest(self) -> None:
spec = ImageSpec.from_dict(
{
"name": "ci-base",
"dockerfile": "Dockerfile",
"tags": [],
}
)
assert spec.tags == ["latest"]
class TestBuildFullTag:
def test_no_registry(self) -> None:
assert build_full_tag(None, "ci-base", "latest") == "ci-base:latest"
def test_with_registry(self) -> None:
assert build_full_tag("git.example.com", "ci-base", "0.1.0") == "git.example.com/ci-base:0.1.0"
def test_with_registry_and_path(self) -> None:
assert (
build_full_tag("git.example.com", "oblachno/ci-base", "latest") == "git.example.com/oblachno/ci-base:latest"
)
class TestLoadManifest:
def test_load_valid_manifest(self, tmp_path: Path) -> None:
manifest = tmp_path / "images.json"
manifest.write_text(
json.dumps(
[
{"name": "ci-base", "dockerfile": "docker/ci-base/Dockerfile"},
{"name": "ci-quality", "dockerfile": "docker/ci-quality/Dockerfile", "tags": ["latest", "1.0"]},
]
)
)
specs = load_manifest(manifest)
assert len(specs) == 2
assert specs[0].name == "ci-base"
assert specs[1].tags == ["latest", "1.0"]
def test_load_missing_file(self, tmp_path: Path) -> None:
with pytest.raises(ClickException, match="not found"):
load_manifest(tmp_path / "nonexistent.json")
def test_load_not_a_list(self, tmp_path: Path) -> None:
manifest = tmp_path / "images.json"
manifest.write_text(json.dumps({"name": "ci-base"}))
with pytest.raises(ClickException, match="must be a JSON list"):
load_manifest(manifest)
class TestRegistryLogin:
def test_success(self) -> None:
mock_result = MagicMock(returncode=0, stderr="", stdout="")
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result) as mock_run:
assert registry_login("git.example.com", "user", "token") is True
assert mock_run.call_args.args[0] == [
"docker",
"login",
"git.example.com",
"-u",
"user",
"--password-stdin",
]
assert mock_run.call_args.kwargs["input"] == "token"
def test_failure(self) -> None:
mock_result = MagicMock(returncode=1, stderr="auth failed", stdout="")
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result):
assert registry_login("git.example.com", "user", "bad") is False
def test_dry_run(self) -> None:
with patch("devx.tools.build_image.subprocess.run") as mock_run:
assert registry_login("git.example.com", "user", "token", dry_run=True) is True
mock_run.assert_not_called()
class TestBuildImage:
def test_success(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
spec = ImageSpec(name="ci-base", dockerfile=str(dockerfile), context=".", tags=["latest"])
mock_result = MagicMock(returncode=0)
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result):
assert do_build(spec) is True
def test_dockerfile_not_found(self) -> None:
spec = ImageSpec(name="ci-base", dockerfile="nonexistent/Dockerfile")
assert do_build(spec) is False
def test_build_failure(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
spec = ImageSpec(name="ci-base", dockerfile=str(dockerfile))
mock_result = MagicMock(returncode=1)
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result):
assert do_build(spec) is False
def test_dry_run(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
spec = ImageSpec(name="ci-base", dockerfile=str(dockerfile), tags=["latest", "1.0"])
with patch("devx.tools.build_image.subprocess.run") as mock_run:
assert do_build(spec, dry_run=True) is True
mock_run.assert_not_called()
def test_with_registry(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
spec = ImageSpec(name="ci-base", dockerfile=str(dockerfile), tags=["latest"])
mock_result = MagicMock(returncode=0)
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result) as mock_run:
assert do_build(spec, registry="git.example.com") is True
cmd = mock_run.call_args.args[0]
assert "-t" in cmd
idx = cmd.index("-t")
assert cmd[idx + 1] == "git.example.com/ci-base:latest"
def test_pull_flag(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
spec = ImageSpec(name="ci-base", dockerfile=str(dockerfile))
mock_result = MagicMock(returncode=0)
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result) as mock_run:
assert do_build(spec, pull=True) is True
cmd = mock_run.call_args.args[0]
assert "--pull" in cmd
class TestPushImage:
def test_success(self) -> None:
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest", "1.0"])
mock_result = MagicMock(returncode=0, stderr="", stdout="")
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result) as mock_run:
assert push_image(spec, "git.example.com") is True
assert mock_run.call_count == 2
def test_partial_failure(self) -> None:
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest", "1.0"])
results = [
MagicMock(returncode=0, stderr="", stdout=""),
MagicMock(returncode=1, stderr="push failed", stdout=""),
]
with patch("devx.tools.build_image.subprocess.run", side_effect=results):
assert push_image(spec, "git.example.com") is False
def test_dry_run(self) -> None:
spec = ImageSpec(name="ci-base", dockerfile="Dockerfile", tags=["latest"])
with patch("devx.tools.build_image.subprocess.run") as mock_run:
assert push_image(spec, "git.example.com", dry_run=True) is True
mock_run.assert_not_called()
class TestSortVersions:
def test_sort_by_created_at_desc(self) -> None:
versions = [
{"version": "0.1.0", "created_at": "2025-01-01T00:00:00Z"},
{"version": "0.3.0", "created_at": "2025-03-01T00:00:00Z"},
{"version": "0.2.0", "created_at": "2025-02-01T00:00:00Z"},
]
result = sort_versions_by_date(versions)
assert [v["version"] for v in result] == ["0.3.0", "0.2.0", "0.1.0"]
def test_sort_fallback_to_version(self) -> None:
versions = [
{"version": "0.1.0"},
{"version": "0.3.0"},
{"version": "0.2.0"},
]
result = sort_versions_by_date(versions)
assert [v["version"] for v in result] == ["0.3.0", "0.2.0", "0.1.0"]
class TestSelectForDeletion:
def test_keep_2(self) -> None:
versions = [
{"version": "0.1.0", "created_at": "2025-01-01"},
{"version": "0.2.0", "created_at": "2025-02-01"},
{"version": "0.3.0", "created_at": "2025-03-01"},
{"version": "0.4.0", "created_at": "2025-04-01"},
]
to_delete = select_for_deletion(versions, keep=2)
assert len(to_delete) == 2
assert {v["version"] for v in to_delete} == {"0.1.0", "0.2.0"}
def test_preserve_latest_tag(self) -> None:
versions = [
{"version": "latest", "created_at": "2025-01-01"},
{"version": "0.2.0", "created_at": "2025-02-01"},
{"version": "0.3.0", "created_at": "2025-03-01"},
{"version": "0.4.0", "created_at": "2025-04-01"},
]
to_delete = select_for_deletion(versions, keep=2)
deleted_versions = {v["version"] for v in to_delete}
assert "latest" not in deleted_versions
# latest is oldest by date but still preserved
assert "0.2.0" in deleted_versions
def test_keep_all(self) -> None:
versions = [
{"version": "0.1.0", "created_at": "2025-01-01"},
{"version": "0.2.0", "created_at": "2025-02-01"},
]
to_delete = select_for_deletion(versions, keep=2)
assert len(to_delete) == 0
def test_keep_more_than_available(self) -> None:
versions = [
{"version": "0.1.0", "created_at": "2025-01-01"},
]
to_delete = select_for_deletion(versions, keep=5)
assert len(to_delete) == 0
class TestCleanImagesAPI:
"""Tests for the clean_images module's API functions."""
def test_list_package_versions(self) -> None:
from devx.tools.clean_images import list_package_versions
mock_resp = MagicMock()
mock_resp.json.return_value = [{"version": "0.1.0"}]
mock_resp.raise_for_status = MagicMock()
with patch("devx.tools.clean_images.requests.get", return_value=mock_resp) as mock_get:
versions = list_package_versions(
"https://git.example.com/api/v1",
"oblachno-oss",
"ci-base",
"token",
)
assert versions == [{"version": "0.1.0"}]
assert "page=1" in mock_get.call_args.args[0]
def test_list_package_versions_pagination(self) -> None:
from devx.tools.clean_images import list_package_versions
# First page: 50 items, second page: 3 items, third page: empty
page1 = [{"version": f"0.{i}.0"} for i in range(50)]
page2 = [{"version": f"1.{i}.0"} for i in range(3)]
responses = [
MagicMock(json=MagicMock(return_value=page1), raise_for_status=MagicMock()),
MagicMock(json=MagicMock(return_value=page2), raise_for_status=MagicMock()),
MagicMock(json=MagicMock(return_value=[]), raise_for_status=MagicMock()),
]
with patch("devx.tools.clean_images.requests.get", side_effect=responses):
versions = list_package_versions(
"https://git.example.com/api/v1",
"oblachno-oss",
"ci-base",
"token",
)
assert len(versions) == 53
def test_delete_package_version_success(self) -> None:
from devx.tools.clean_images import delete_package_version
mock_resp = MagicMock(status_code=204)
with patch("devx.tools.clean_images.requests.delete", return_value=mock_resp) as mock_del:
assert (
delete_package_version(
"https://git.example.com/api/v1",
"oblachno-oss",
"ci-base",
"0.1.0",
"token",
)
is True
)
# Verify URL includes container type
url = mock_del.call_args.args[0]
assert "/container/" in url
def test_delete_package_version_404_treated_as_success(self) -> None:
from devx.tools.clean_images import delete_package_version
mock_resp = MagicMock(status_code=404)
with patch("devx.tools.clean_images.requests.delete", return_value=mock_resp):
assert (
delete_package_version(
"https://git.example.com/api/v1",
"oblachno-oss",
"ci-base",
"0.1.0",
"token",
)
is True
)
def test_delete_package_version_failure(self) -> None:
from devx.tools.clean_images import delete_package_version
mock_resp = MagicMock(status_code=403)
with patch("devx.tools.clean_images.requests.delete", return_value=mock_resp):
assert (
delete_package_version(
"https://git.example.com/api/v1",
"oblachno-oss",
"ci-base",
"0.1.0",
"token",
)
is False
)
def test_delete_package_version_retries_on_5xx(self) -> None:
from devx.tools.clean_images import delete_package_version
responses = [
MagicMock(status_code=500),
MagicMock(status_code=502),
MagicMock(status_code=204),
]
with patch("devx.tools.clean_images.requests.delete", side_effect=responses):
with patch("devx.tools.clean_images.time.sleep"):
assert (
delete_package_version(
"https://git.example.com/api/v1",
"oblachno-oss",
"ci-base",
"0.1.0",
"token",
max_retries=3,
)
is True
)
def test_delete_package_version_retries_on_exception(self) -> None:
import requests as req
from devx.tools.clean_images import delete_package_version
responses = [
req.ConnectionError("network down"),
MagicMock(status_code=204),
]
with patch("devx.tools.clean_images.requests.delete", side_effect=responses):
with patch("devx.tools.clean_images.time.sleep"):
assert (
delete_package_version(
"https://git.example.com/api/v1",
"oblachno-oss",
"ci-base",
"0.1.0",
"token",
max_retries=3,
)
is True
)
def test_delete_package_version_exhausts_retries_on_exception(self) -> None:
import requests as req
from devx.tools.clean_images import delete_package_version
with patch(
"devx.tools.clean_images.requests.delete",
side_effect=req.ConnectionError("network down"),
):
with patch("devx.tools.clean_images.time.sleep"):
assert (
delete_package_version(
"https://git.example.com/api/v1",
"oblachno-oss",
"ci-base",
"0.1.0",
"token",
max_retries=2,
)
is False
)
def test_delete_package_version_exhausts_retries_on_5xx(self) -> None:
from devx.tools.clean_images import delete_package_version
with patch(
"devx.tools.clean_images.requests.delete",
return_value=MagicMock(status_code=500),
):
with patch("devx.tools.clean_images.time.sleep"):
assert (
delete_package_version(
"https://git.example.com/api/v1",
"oblachno-oss",
"ci-base",
"0.1.0",
"token",
max_retries=2,
)
is False
)
def test_delete_package_version_zero_retries(self) -> None:
from devx.tools.clean_images import delete_package_version
with patch("devx.tools.clean_images.requests.delete") as mock_del:
assert (
delete_package_version(
"https://git.example.com/api/v1",
"oblachno-oss",
"ci-base",
"0.1.0",
"token",
max_retries=0,
)
is False
)
mock_del.assert_not_called()
class TestCLIBuildImage:
def test_single_image_build(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
runner = CliRunner()
mock_result = MagicMock(returncode=0)
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result):
result = runner.invoke(
build_image.main,
["--dockerfile", str(dockerfile), "--name", "ci-base", "--tag", "latest"],
)
assert result.exit_code == 0
def test_manifest_build(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
manifest = tmp_path / "images.json"
manifest.write_text(
json.dumps(
[
{"name": "ci-base", "dockerfile": str(dockerfile)},
]
)
)
runner = CliRunner()
mock_result = MagicMock(returncode=0)
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result):
result = runner.invoke(
build_image.main,
["--manifest", str(manifest)],
)
assert result.exit_code == 0
def test_missing_dockerfile_and_manifest(self) -> None:
runner = CliRunner()
result = runner.invoke(build_image.main, [])
assert result.exit_code != 0
assert "manifest" in result.output.lower() or "dockerfile" in result.output.lower()
def test_push_without_registry(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
runner = CliRunner()
result = runner.invoke(
build_image.main,
["--dockerfile", str(dockerfile), "--name", "ci-base", "--push"],
)
assert result.exit_code != 0
assert "registry" in result.output.lower()
def test_push_without_credentials(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
runner = CliRunner()
with patch.dict("os.environ", {}, clear=True):
result = runner.invoke(
build_image.main,
["--dockerfile", str(dockerfile), "--name", "ci-base", "--push", "--registry", "git.example.com"],
)
assert result.exit_code != 0
assert "credential" in result.output.lower() or "token" in result.output.lower()
def test_dry_run(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
runner = CliRunner()
with patch("devx.tools.build_image.subprocess.run") as mock_run:
result = runner.invoke(
build_image.main,
["--dockerfile", str(dockerfile), "--name", "ci-base", "--dry-run"],
)
assert result.exit_code == 0
mock_run.assert_not_called()
assert "dry-run" in result.output
def test_build_failure_exits_with_error(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
runner = CliRunner()
mock_result = MagicMock(returncode=1)
with patch("devx.tools.build_image.subprocess.run", return_value=mock_result):
result = runner.invoke(
build_image.main,
["--dockerfile", str(dockerfile), "--name", "ci-base"],
)
assert result.exit_code != 0
def test_push_login_failure(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
runner = CliRunner()
login_result = MagicMock(returncode=1, stderr="auth failed", stdout="")
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "fake", "CI_GITEA_USERNAME": "user"}):
with patch("devx.tools.build_image.subprocess.run", return_value=login_result):
result = runner.invoke(
build_image.main,
["--dockerfile", str(dockerfile), "--name", "ci-base", "--push", "--registry", "git.example.com"],
)
assert result.exit_code != 0
assert "login" in result.output.lower()
def test_push_image_failure(self, tmp_path: Path) -> None:
dockerfile = tmp_path / "Dockerfile"
dockerfile.touch()
runner = CliRunner()
build_result = MagicMock(returncode=0)
login_result = MagicMock(returncode=0, stderr="", stdout="")
push_result = MagicMock(returncode=1, stderr="push failed", stdout="")
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "fake", "CI_GITEA_USERNAME": "user"}):
with patch(
"devx.tools.build_image.subprocess.run",
side_effect=[login_result, build_result, push_result],
):
result = runner.invoke(
build_image.main,
["--dockerfile", str(dockerfile), "--name", "ci-base", "--push", "--registry", "git.example.com"],
)
assert result.exit_code != 0
class TestCLICleanImages:
def test_dry_run(self) -> None:
from devx.tools.clean_images import main as clean_main
runner = CliRunner()
mock_resp = MagicMock()
mock_resp.json.return_value = [
{"version": "0.1.0", "created_at": "2025-01-01"},
{"version": "0.2.0", "created_at": "2025-02-01"},
{"version": "0.3.0", "created_at": "2025-03-01"},
]
mock_resp.raise_for_status = MagicMock()
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "fake"}):
with patch("devx.tools.clean_images.requests.get", return_value=mock_resp):
result = runner.invoke(
clean_main,
["--owner", "oblachno-oss", "--name", "ci-base", "--keep", "1", "--dry-run"],
)
assert result.exit_code == 0
assert "dry-run" in result.output
assert "0.1.0" in result.output
def test_no_token(self) -> None:
from devx.tools.clean_images import main as clean_main
runner = CliRunner()
with patch.dict("os.environ", {}, clear=True):
result = runner.invoke(
clean_main,
["--owner", "oblachno-oss", "--name", "ci-base"],
)
assert result.exit_code != 0
assert "token" in result.output.lower()
def test_no_versions_found(self) -> None:
from devx.tools.clean_images import main as clean_main
runner = CliRunner()
mock_resp = MagicMock()
mock_resp.json.return_value = []
mock_resp.raise_for_status = MagicMock()
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "fake"}):
with patch("devx.tools.clean_images.requests.get", return_value=mock_resp):
result = runner.invoke(
clean_main,
["--owner", "oblachno-oss", "--name", "ci-base", "--dry-run"],
)
assert result.exit_code == 0
assert "No versions" in result.output
def test_actual_delete(self) -> None:
from devx.tools.clean_images import main as clean_main
runner = CliRunner()
list_resp = MagicMock()
list_resp.json.return_value = [
{"version": "0.1.0", "created_at": "2025-01-01"},
{"version": "0.2.0", "created_at": "2025-02-01"},
{"version": "0.3.0", "created_at": "2025-03-01"},
]
list_resp.raise_for_status = MagicMock()
delete_resp = MagicMock(status_code=204)
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "fake"}):
with patch("devx.tools.clean_images.requests.get", return_value=list_resp):
with patch("devx.tools.clean_images.requests.delete", return_value=delete_resp):
result = runner.invoke(
clean_main,
["--owner", "oblachno-oss", "--name", "ci-base", "--keep", "2"],
)
assert result.exit_code == 0
assert "Deleted" in result.output
def test_list_request_exception(self) -> None:
import requests as req
from devx.tools.clean_images import main as clean_main
runner = CliRunner()
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "fake"}):
with patch(
"devx.tools.clean_images.requests.get",
side_effect=req.ConnectionError("network down"),
):
result = runner.invoke(
clean_main,
["--owner", "oblachno-oss", "--name", "ci-base", "--dry-run"],
)
assert result.exit_code != 0
assert "Failed to list" in result.output
def test_delete_failure_in_cli(self) -> None:
from devx.tools.clean_images import main as clean_main
runner = CliRunner()
list_resp = MagicMock()
list_resp.json.return_value = [
{"version": "0.1.0", "created_at": "2025-01-01"},
{"version": "0.2.0", "created_at": "2025-02-01"},
{"version": "0.3.0", "created_at": "2025-03-01"},
]
list_resp.raise_for_status = MagicMock()
delete_resp = MagicMock(status_code=403)
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "fake"}):
with patch("devx.tools.clean_images.requests.get", return_value=list_resp):
with patch("devx.tools.clean_images.requests.delete", return_value=delete_resp):
with patch("devx.tools.clean_images.time.sleep"):
result = runner.invoke(
clean_main,
["--owner", "oblachno-oss", "--name", "ci-base", "--keep", "2"],
)
assert result.exit_code != 0
assert "FAILED" in result.output
@patch("devx.tools.clean_images.REPO_OWNER", "oblachno-oss")
def test_owner_from_config(self) -> None:
from devx.tools.clean_images import main as clean_main
runner = CliRunner()
mock_resp = MagicMock()
mock_resp.json.return_value = []
mock_resp.raise_for_status = MagicMock()
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "fake"}):
with patch("devx.tools.clean_images.requests.get", return_value=mock_resp):
result = runner.invoke(
clean_main,
["--name", "ci-base", "--dry-run"],
)
assert result.exit_code == 0
assert "oblachno-oss/ci-base" in result.output
@patch("devx.tools.clean_images.REPO_OWNER", "")
def test_no_owner_raises(self) -> None:
from devx.tools.clean_images import main as clean_main
runner = CliRunner()
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "fake"}):
result = runner.invoke(
clean_main,
["--name", "ci-base"],
)
assert result.exit_code != 0
assert "owner" in result.output.lower()
+231
View File
@@ -0,0 +1,231 @@
"""Unit tests for devx.tools.check_agent_docs."""
import re
from pathlib import Path
from unittest.mock import patch
from click.testing import CliRunner
from devx.tools.check_agent_docs import (
DEFAULT_REPO_PATH_PREFIXES,
DEFAULT_SCAN_DIRS,
DEFAULT_SCAN_EXTENSIONS,
DEFAULT_SCAN_FILES,
MIN_PATH_REF_LENGTH_DEFAULT,
_check_file,
_collect_doc_files,
_is_legitimate_ref,
_should_skip,
cli,
)
class TestShouldSkip:
def test_skips_excluded_path(self, tmp_path: Path) -> None:
f = tmp_path / "docs" / "retrospectives" / "r.md"
f.parent.mkdir(parents=True)
f.write_text("")
assert _should_skip(f, ["docs/retrospectives"], tmp_path) is True
def test_does_not_skip_normal(self, tmp_path: Path) -> None:
f = tmp_path / "docs" / "guide.md"
f.parent.mkdir(parents=True)
f.write_text("")
assert _should_skip(f, ["docs/retrospectives"], tmp_path) is False
def test_returns_false_for_path_outside_repo(self, tmp_path: Path) -> None:
f = Path("/tmp/some_other_path/guide.md")
assert _should_skip(f, [], tmp_path) is False
class TestIsLegitimateRef:
def test_legitimate_legacy(self) -> None:
assert _is_legitimate_ref("This is legacy code", ["legacy"]) is True
def test_not_legitimate(self) -> None:
assert _is_legitimate_ref("Use this file", ["legacy"]) is False
def test_case_insensitive(self) -> None:
assert _is_legitimate_ref("This is LEGACY", ["legacy"]) is True
class TestCollectDocFiles:
def test_collects_devin_and_docs(self, tmp_path: Path) -> None:
(tmp_path / ".devin").mkdir()
(tmp_path / ".devin" / "guide.md").write_text("")
(tmp_path / "docs").mkdir()
(tmp_path / "docs" / "api.md").write_text("")
(tmp_path / "README.md").write_text("")
files = _collect_doc_files(tmp_path, DEFAULT_SCAN_DIRS, DEFAULT_SCAN_FILES, DEFAULT_SCAN_EXTENSIONS, [])
names = {f.name for f in files}
assert "guide.md" in names
assert "api.md" in names
assert "README.md" in names
def test_excludes_paths(self, tmp_path: Path) -> None:
(tmp_path / "docs" / "retrospectives").mkdir(parents=True)
(tmp_path / "docs" / "retrospectives" / "r.md").write_text("")
(tmp_path / "docs" / "guide.md").write_text("")
files = _collect_doc_files(
tmp_path, DEFAULT_SCAN_DIRS, DEFAULT_SCAN_FILES, DEFAULT_SCAN_EXTENSIONS, ["docs/retrospectives"]
)
names = {f.name for f in files}
assert "guide.md" in names
assert "r.md" not in names
def test_deduplicates(self, tmp_path: Path) -> None:
(tmp_path / "docs").mkdir()
(tmp_path / "docs" / "api.md").write_text("")
files = _collect_doc_files(tmp_path, ["docs", "docs"], DEFAULT_SCAN_FILES, DEFAULT_SCAN_EXTENSIONS, [])
assert len(files) == 1
class TestCheckFile:
def test_detects_deleted_file_ref(self, tmp_path: Path) -> None:
doc = tmp_path / "docs" / "guide.md"
doc.parent.mkdir(parents=True)
doc.write_text("See scripts/old.py for details.\n")
issues = _check_file(
doc, tmp_path, {"scripts/old.py"}, [], [], DEFAULT_REPO_PATH_PREFIXES, MIN_PATH_REF_LENGTH_DEFAULT, []
)
assert any("deleted file" in i for i in issues)
def test_detects_nonexistent_file_ref(self, tmp_path: Path) -> None:
doc = tmp_path / "docs" / "guide.md"
doc.parent.mkdir(parents=True)
doc.write_text("See scripts/nonexistent.py for details.\n")
issues = _check_file(doc, tmp_path, set(), [], [], DEFAULT_REPO_PATH_PREFIXES, MIN_PATH_REF_LENGTH_DEFAULT, [])
assert any("non-existent file" in i for i in issues)
def test_does_not_flag_existing_file(self, tmp_path: Path) -> None:
(tmp_path / "scripts").mkdir()
(tmp_path / "scripts" / "exists.py").write_text("")
doc = tmp_path / "docs" / "guide.md"
doc.parent.mkdir(parents=True)
doc.write_text("See scripts/exists.py for details.\n")
issues = _check_file(doc, tmp_path, set(), [], [], DEFAULT_REPO_PATH_PREFIXES, MIN_PATH_REF_LENGTH_DEFAULT, [])
assert issues == []
def test_detects_deprecated_pattern(self, tmp_path: Path) -> None:
doc = tmp_path / "docs" / "guide.md"
doc.parent.mkdir(parents=True)
doc.write_text("Use ansible/envs/prod/secrets.yml for config.\n")
patterns = [re.compile(r"ansible/envs/[^/]+/secrets\.yml")]
issues = _check_file(
doc, tmp_path, set(), patterns, [], DEFAULT_REPO_PATH_PREFIXES, MIN_PATH_REF_LENGTH_DEFAULT, []
)
assert any("deprecated pattern" in i for i in issues)
def test_legitimate_ref_skips_deprecated(self, tmp_path: Path) -> None:
# Create the referenced file so the non-existent check doesn't trigger
secrets = tmp_path / "ansible" / "envs" / "prod" / "secrets.yml"
secrets.parent.mkdir(parents=True)
secrets.write_text("")
doc = tmp_path / "docs" / "guide.md"
doc.parent.mkdir(parents=True)
doc.write_text("The legacy ansible/envs/prod/secrets.yml is deprecated.\n")
patterns = [re.compile(r"ansible/envs/[^/]+/secrets\.yml")]
issues = _check_file(
doc, tmp_path, set(), patterns, ["deprecated"], DEFAULT_REPO_PATH_PREFIXES, MIN_PATH_REF_LENGTH_DEFAULT, []
)
assert issues == []
def test_unicode_error_returns_empty(self, tmp_path: Path) -> None:
doc = tmp_path / "docs" / "guide.md"
doc.parent.mkdir(parents=True)
doc.write_bytes(b"\xff\xfe\x00\x00")
issues = _check_file(doc, tmp_path, set(), [], [], DEFAULT_REPO_PATH_PREFIXES, MIN_PATH_REF_LENGTH_DEFAULT, [])
assert issues == []
def test_skips_short_ref(self, tmp_path: Path) -> None:
doc = tmp_path / "docs" / "guide.md"
doc.parent.mkdir(parents=True)
doc.write_text("See a.py for details.\n")
issues = _check_file(doc, tmp_path, set(), [], [], DEFAULT_REPO_PATH_PREFIXES, 5, [])
# "a.py" is only 4 chars, below min_path_ref_length
assert issues == []
def test_skips_ref_without_repo_prefix(self, tmp_path: Path) -> None:
doc = tmp_path / "docs" / "guide.md"
doc.parent.mkdir(parents=True)
doc.write_text("See vendor/some/long/path.py for details.\n")
issues = _check_file(doc, tmp_path, set(), [], [], DEFAULT_REPO_PATH_PREFIXES, MIN_PATH_REF_LENGTH_DEFAULT, [])
# "vendor/" is not in repo_path_prefixes
assert issues == []
def test_skip_ref_prefixes_skips_nonexistent(self, tmp_path: Path) -> None:
doc = tmp_path / "docs" / "guide.md"
doc.parent.mkdir(parents=True)
doc.write_text("See scripts/test_foo.py for details.\n")
issues = _check_file(
doc, tmp_path, set(), [], [], DEFAULT_REPO_PATH_PREFIXES, MIN_PATH_REF_LENGTH_DEFAULT, ["scripts/test_"]
)
assert issues == []
class TestCli:
def test_passes_when_no_issues(self, tmp_path: Path) -> None:
(tmp_path / "docs").mkdir()
(tmp_path / "docs" / "guide.md").write_text("All good.\n")
runner = CliRunner()
with (
patch("devx.tools.check_agent_docs._load_config", return_value={}),
patch("devx.tools.check_agent_docs.Path.cwd", return_value=tmp_path),
):
result = runner.invoke(cli, [])
assert result.exit_code == 0
assert "Passed" in result.output
def test_fails_when_stale_ref(self, tmp_path: Path) -> None:
(tmp_path / "docs").mkdir()
(tmp_path / "docs" / "guide.md").write_text("See scripts/deleted.py\n")
cfg = {"deleted_files": ["scripts/deleted.py"]}
runner = CliRunner()
with (
patch("devx.tools.check_agent_docs._load_config", return_value=cfg),
patch("devx.tools.check_agent_docs.Path.cwd", return_value=tmp_path),
):
result = runner.invoke(cli, [])
assert result.exit_code != 0
assert "FAILED" in result.output
def test_load_config_returns_empty_when_not_dict(self) -> None:
from devx.tools.check_agent_docs import _load_config
with patch("devx.tools.check_agent_docs._load_pyproject_devx", return_value={"check_agent_docs": "not a dict"}):
assert _load_config() == {}
def test_load_config_returns_dict_when_valid(self) -> None:
from devx.tools.check_agent_docs import _load_config
cfg = {"scan_dirs": ["custom"]}
with patch("devx.tools.check_agent_docs._load_pyproject_devx", return_value={"check_agent_docs": cfg}):
assert _load_config() == cfg
def test_invalid_regex_pattern_skipped(self, tmp_path: Path) -> None:
(tmp_path / "docs").mkdir()
(tmp_path / "docs" / "guide.md").write_text("All good.\n")
cfg = {"deprecated_patterns": ["[invalid"]}
runner = CliRunner()
with (
patch("devx.tools.check_agent_docs._load_config", return_value=cfg),
patch("devx.tools.check_agent_docs.Path.cwd", return_value=tmp_path),
):
result = runner.invoke(cli, [])
assert result.exit_code == 0
def test_custom_scan_dirs(self, tmp_path: Path) -> None:
custom = tmp_path / "custom_docs"
custom.mkdir()
(custom / "guide.md").write_text("See scripts/deleted.py\n")
cfg = {"scan_dirs": ["custom_docs"], "deleted_files": ["scripts/deleted.py"]}
runner = CliRunner()
with (
patch("devx.tools.check_agent_docs._load_config", return_value=cfg),
patch("devx.tools.check_agent_docs.Path.cwd", return_value=tmp_path),
):
result = runner.invoke(cli, [])
assert result.exit_code != 0
+294
View File
@@ -0,0 +1,294 @@
"""Unit tests for devx.ci.check_auto_merge_ready."""
from unittest.mock import MagicMock, patch
from click.testing import CliRunner
from devx.ci.check_auto_merge_ready import (
cli,
get_pr_title_from_gitea,
get_vikunja_title_optional,
is_branch_behind_master,
)
class TestIsBranchBehindMaster:
@patch("devx.ci.check_auto_merge_ready.subprocess.run")
def test_returns_false_when_ahead(self, mock_run: MagicMock) -> None:
# First: fetch (ok), second: ahead count (ok), third: behind count = 0
mock_run.side_effect = [
MagicMock(returncode=0, stdout="", stderr=""),
MagicMock(returncode=0, stdout="3\n", stderr=""),
MagicMock(returncode=0, stdout="0\n", stderr=""),
]
assert is_branch_behind_master("feature") is False
@patch("devx.ci.check_auto_merge_ready.subprocess.run")
def test_returns_true_when_behind(self, mock_run: MagicMock) -> None:
mock_run.side_effect = [
MagicMock(returncode=0, stdout="", stderr=""),
MagicMock(returncode=0, stdout="0\n", stderr=""),
MagicMock(returncode=0, stdout="5\n", stderr=""),
]
assert is_branch_behind_master("feature") is True
@patch("devx.ci.check_auto_merge_ready.subprocess.run")
def test_returns_false_on_git_error(self, mock_run: MagicMock) -> None:
mock_run.side_effect = [
MagicMock(returncode=0, stdout="", stderr=""),
MagicMock(returncode=1, stdout="", stderr="error"),
]
assert is_branch_behind_master("feature") is False
@patch("devx.ci.check_auto_merge_ready.subprocess.run")
def test_returns_false_on_timeout(self, mock_run: MagicMock) -> None:
import subprocess
mock_run.side_effect = subprocess.TimeoutExpired(cmd="git", timeout=30)
assert is_branch_behind_master("feature") is False
@patch("devx.ci.check_auto_merge_ready.subprocess.run")
def test_returns_false_on_value_error(self, mock_run: MagicMock) -> None:
mock_run.side_effect = [
MagicMock(returncode=0, stdout="", stderr=""),
MagicMock(returncode=0, stdout="3\n", stderr=""),
MagicMock(returncode=0, stdout="not_a_number\n", stderr=""),
]
assert is_branch_behind_master("feature") is False
@patch("devx.ci.check_auto_merge_ready.subprocess.run")
def test_returns_false_on_file_not_found(self, mock_run: MagicMock) -> None:
mock_run.side_effect = FileNotFoundError("git not found")
assert is_branch_behind_master("feature") is False
@patch("devx.ci.check_auto_merge_ready.subprocess.run")
def test_returns_false_when_behind_check_fails(self, mock_run: MagicMock) -> None:
# fetch ok, ahead count ok, behind count command fails
mock_run.side_effect = [
MagicMock(returncode=0, stdout="", stderr=""),
MagicMock(returncode=0, stdout="3\n", stderr=""),
MagicMock(returncode=1, stdout="", stderr="error"),
]
assert is_branch_behind_master("feature") is False
class TestGetPrTitleFromGitea:
def test_returns_none_without_token(self) -> None:
with patch.dict("os.environ", {}, clear=True):
assert get_pr_title_from_gitea("owner/repo", 1) is None
def test_returns_none_with_invalid_repo(self) -> None:
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True):
assert get_pr_title_from_gitea("invalid", 1) is None
@patch("devx.ci.check_auto_merge_ready.GiteaClient")
def test_fetches_title(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.get_pr.return_value = {"title": "DEVX-1: Fix bug"}
mock_client_cls.return_value = mock_client
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True):
result = get_pr_title_from_gitea("owner/repo", 1)
assert result == "DEVX-1: Fix bug"
@patch("devx.ci.check_auto_merge_ready.GiteaClient")
def test_returns_none_on_exception(self, mock_client_cls: MagicMock) -> None:
from devx.exceptions import APIError
mock_client = MagicMock()
mock_client.get_pr.side_effect = APIError(500, "API error")
mock_client_cls.return_value = mock_client
with patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True):
result = get_pr_title_from_gitea("owner/repo", 1)
assert result is None
class TestGetVikunjaTitleOptional:
def test_returns_none_without_token(self) -> None:
with patch.dict("os.environ", {}, clear=True):
assert get_vikunja_title_optional("DEVX-1") is None
@patch("devx.ci.check_auto_merge_ready.VikunjaClient")
def test_returns_title_when_found(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.list_project_tasks.return_value = [{"identifier": "DEVX-1", "title": "Fix bug"}]
mock_client_cls.return_value = mock_client
with patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"}, clear=True):
result = get_vikunja_title_optional("DEVX-1")
assert result == "Fix bug"
@patch("devx.ci.check_auto_merge_ready.VikunjaClient")
def test_returns_none_when_not_found(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.list_project_tasks.return_value = [{"identifier": "DEVX-2", "title": "Other task"}]
mock_client_cls.return_value = mock_client
with patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"}, clear=True):
result = get_vikunja_title_optional("DEVX-1")
assert result is None
@patch("devx.ci.check_auto_merge_ready.VikunjaClient")
def test_paginates_until_found(self, mock_client_cls: MagicMock) -> None:
from devx.config import DEFAULT_PER_PAGE
mock_client = MagicMock()
# First page: full page of non-matching tasks, second page: match
page1 = [{"identifier": f"DEVX-{i}", "title": f"Task {i}"} for i in range(DEFAULT_PER_PAGE)]
page2 = [{"identifier": "DEVX-99", "title": "Found it"}]
mock_client.list_project_tasks.side_effect = [page1, page2]
mock_client_cls.return_value = mock_client
with patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"}, clear=True):
result = get_vikunja_title_optional("DEVX-99")
assert result == "Found it"
@patch("devx.ci.check_auto_merge_ready.VikunjaClient")
def test_returns_none_when_empty_first_page(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.list_project_tasks.return_value = []
mock_client_cls.return_value = mock_client
with patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"}, clear=True):
result = get_vikunja_title_optional("DEVX-1")
assert result is None
class TestCli:
def test_fails_without_task_id(self) -> None:
runner = CliRunner()
with patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX"}, clear=True):
result = runner.invoke(cli, ["--branch", "no-task-id-here"])
assert result.exit_code != 0
def test_local_mode_no_pr_title(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": ""}, clear=True),
patch("devx.ci.check_auto_merge_ready.is_branch_behind_master", return_value=False),
):
result = runner.invoke(cli, ["--branch", "DEVX-1-fix-foo"])
assert result.exit_code == 0
assert "local mode" in result.output
def test_validates_pr_title_format(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": ""}, clear=True),
patch("devx.ci.check_auto_merge_ready.is_branch_behind_master", return_value=False),
):
result = runner.invoke(cli, ["--branch", "DEVX-1-fix-foo", "--pr-title", "Bad title"])
assert result.exit_code != 0
assert "format" in result.output.lower() or "mismatch" in result.output.lower()
def test_passes_with_valid_title(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": ""}, clear=True),
patch("devx.ci.check_auto_merge_ready.is_branch_behind_master", return_value=False),
):
result = runner.invoke(cli, ["--branch", "DEVX-1-fix-foo", "--pr-title", "DEVX-1: Fix foo"])
assert result.exit_code == 0
assert "satisfied" in result.output
def test_skip_behind_check(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": ""}, clear=True),
patch("devx.ci.check_auto_merge_ready.is_branch_behind_master", return_value=True),
):
result = runner.invoke(
cli,
["--branch", "DEVX-1-fix-foo", "--pr-title", "DEVX-1: Fix foo", "--skip-behind-check"],
)
assert result.exit_code == 0
def test_fails_when_behind_master(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": ""}, clear=True),
patch("devx.ci.check_auto_merge_ready.is_branch_behind_master", return_value=True),
):
result = runner.invoke(
cli,
["--branch", "DEVX-1-fix-foo", "--pr-title", "DEVX-1: Fix foo"],
)
assert result.exit_code != 0
assert "behind" in result.output.lower()
def test_skip_vikunja(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": "tok"}, clear=True),
patch("devx.ci.check_auto_merge_ready.is_branch_behind_master", return_value=False),
patch("devx.ci.check_auto_merge_ready.get_vikunja_title_optional", return_value="Different title"),
):
result = runner.invoke(
cli,
["--branch", "DEVX-1-fix-foo", "--pr-title", "DEVX-1: Fix foo", "--skip-vikunja"],
)
assert result.exit_code == 0
def test_fetches_pr_title_from_gitea(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": ""}, clear=True),
patch("devx.ci.check_auto_merge_ready.is_branch_behind_master", return_value=False),
patch("devx.ci.check_auto_merge_ready.get_pr_title_from_gitea", return_value="DEVX-1: Fix foo"),
):
result = runner.invoke(
cli,
["--branch", "DEVX-1-fix-foo", "--repo", "owner/repo", "--pr-number", "1"],
)
assert result.exit_code == 0
assert "from Gitea" in result.output
def test_fails_when_pr_number_but_no_title(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": ""}, clear=True),
patch("devx.ci.check_auto_merge_ready.get_pr_title_from_gitea", return_value=None),
):
result = runner.invoke(
cli,
["--branch", "DEVX-1-fix-foo", "--repo", "owner/repo", "--pr-number", "1"],
)
assert result.exit_code != 0
assert "Could not fetch" in result.output
def test_fails_when_vikunja_token_set_but_task_not_found(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": "tok"}, clear=True),
patch("devx.ci.check_auto_merge_ready.is_branch_behind_master", return_value=False),
patch("devx.ci.check_auto_merge_ready.get_vikunja_title_optional", return_value=None),
):
result = runner.invoke(
cli,
["--branch", "DEVX-1-fix-foo", "--pr-title", "DEVX-1: Fix foo"],
)
assert result.exit_code != 0
assert "Could not find Vikunja task" in result.output
def test_passes_with_vikunja_title_match(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": "tok"}, clear=True),
patch("devx.ci.check_auto_merge_ready.is_branch_behind_master", return_value=False),
patch("devx.ci.check_auto_merge_ready.get_vikunja_title_optional", return_value="Fix foo"),
):
result = runner.invoke(
cli,
["--branch", "DEVX-1-fix-foo", "--pr-title", "DEVX-1: Fix foo"],
)
assert result.exit_code == 0
assert "Vikunja title match OK" in result.output
def test_fails_with_vikunja_title_mismatch(self) -> None:
runner = CliRunner()
with (
patch.dict("os.environ", {"DEVX_TASK_PREFIX": "DEVX", "VIKUNJA_TOKEN": "tok"}, clear=True),
patch("devx.ci.check_auto_merge_ready.is_branch_behind_master", return_value=False),
patch("devx.ci.check_auto_merge_ready.get_vikunja_title_optional", return_value="Different title"),
):
result = runner.invoke(
cli,
["--branch", "DEVX-1-fix-foo", "--pr-title", "DEVX-1: Fix foo"],
)
assert result.exit_code != 0
assert "does not match Vikunja" in result.output
+90
View File
@@ -0,0 +1,90 @@
"""Unit tests for devx.tools.check_config."""
from pathlib import Path
from click.testing import CliRunner
from devx.tools.check_config import cli
class TestCheckConfig:
def test_valid_config(self, tmp_path: Path) -> None:
"""A valid [tool.devx] section with consistent versions passes."""
runner = CliRunner()
with runner.isolated_filesystem(temp_dir=str(tmp_path)) as fs:
Path(fs, "pyproject.toml").write_text(
'[project]\nname = "test"\n'
'[project.optional-dependencies]\nci = ["devx>=0.15.0"]\ndev = ["devx>=0.15.0"]\n'
'[tool.devx]\ntask_prefix = "TEST"\nvikunja_project_id = 1\nrepo_owner = "owner"\nrepo_name = "test"\n'
)
result = runner.invoke(cli)
assert result.exit_code == 0
assert "Configuration OK" in result.output
def test_missing_tool_devx_section(self, tmp_path: Path) -> None:
"""Missing [tool.devx] section fails with error."""
runner = CliRunner()
with runner.isolated_filesystem(temp_dir=str(tmp_path)) as fs:
Path(fs, "pyproject.toml").write_text('[project]\nname = "test"\n')
result = runner.invoke(cli)
assert result.exit_code == 1
assert "missing required keys" in result.output
def test_partial_tool_devx_section(self, tmp_path: Path) -> None:
"""Partial [tool.devx] section fails with missing keys."""
runner = CliRunner()
with runner.isolated_filesystem(temp_dir=str(tmp_path)) as fs:
Path(fs, "pyproject.toml").write_text('[project]\nname = "test"\n[tool.devx]\ntask_prefix = "TEST"\n')
result = runner.invoke(cli)
assert result.exit_code == 1
assert "missing required keys" in result.output
assert "vikunja_project_id" in result.output
assert "repo_owner" in result.output
assert "repo_name" in result.output
def test_version_mismatch(self, tmp_path: Path) -> None:
"""Version mismatch across extras fails."""
runner = CliRunner()
with runner.isolated_filesystem(temp_dir=str(tmp_path)) as fs:
Path(fs, "pyproject.toml").write_text(
'[project]\nname = "test"\n'
"[project.optional-dependencies]\n"
'ci = ["devx>=0.15.0"]\n'
'dev = ["devx>=0.14.2"]\n'
'[tool.devx]\ntask_prefix = "TEST"\nvikunja_project_id = 1\nrepo_owner = "owner"\nrepo_name = "test"\n'
)
result = runner.invoke(cli)
assert result.exit_code == 1
assert "version mismatch" in result.output
def test_no_pyproject_file(self, tmp_path: Path) -> None:
"""Missing pyproject.toml fails."""
runner = CliRunner()
with runner.isolated_filesystem(temp_dir=str(tmp_path)):
result = runner.invoke(cli)
assert result.exit_code == 1
assert "not found" in result.output
def test_no_extras_passes(self, tmp_path: Path) -> None:
"""No optional-dependencies with devx is fine (no versions to compare)."""
runner = CliRunner()
with runner.isolated_filesystem(temp_dir=str(tmp_path)) as fs:
Path(fs, "pyproject.toml").write_text(
'[project]\nname = "test"\n'
'[tool.devx]\ntask_prefix = "TEST"\nvikunja_project_id = 1\nrepo_owner = "owner"\nrepo_name = "test"\n'
)
result = runner.invoke(cli)
assert result.exit_code == 0
assert "Configuration OK" in result.output
def test_single_extra_passes(self, tmp_path: Path) -> None:
"""Single extra with devx version is fine (no mismatch possible)."""
runner = CliRunner()
with runner.isolated_filesystem(temp_dir=str(tmp_path)) as fs:
Path(fs, "pyproject.toml").write_text(
'[project]\nname = "test"\n'
'[project.optional-dependencies]\nci = ["devx>=0.15.0", "pytest"]\n'
'[tool.devx]\ntask_prefix = "TEST"\nvikunja_project_id = 1\nrepo_owner = "owner"\nrepo_name = "test"\n'
)
result = runner.invoke(cli)
assert result.exit_code == 0
+249
View File
@@ -0,0 +1,249 @@
"""Unit tests for devx.tools.check_mutable_globals."""
from pathlib import Path
from unittest.mock import patch
from click.testing import CliRunner
from devx.tools.check_mutable_globals import (
DEFAULT_SCAN_DIRS,
DEFAULT_SKIP_DIRS,
_load_config,
_should_skip,
cli,
find_mutable_globals,
)
class TestFindMutableGlobals:
def test_detects_set_global_with_path_hint(self, tmp_path: Path) -> None:
source = "_SEEN: set[Path] = set()\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert len(issues) == 1
assert "_SEEN" in issues[0]
assert "set()" in issues[0]
def test_detects_dict_global_with_path_hint(self, tmp_path: Path) -> None:
source = "_CACHE: dict[Path, Any] = {}\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert len(issues) == 1
assert "_CACHE" in issues[0]
def test_detects_list_global_with_path_hint(self, tmp_path: Path) -> None:
source = "PATHS: list[Path] = []\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert len(issues) == 1
assert "PATHS" in issues[0]
def test_skips_non_mutable_globals(self, tmp_path: Path) -> None:
source = "_MAX: int = 10\n_SEEN: set[Path] = set()\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert len(issues) == 1
assert "_SEEN" in issues[0]
def test_skips_globals_without_path_hint(self, tmp_path: Path) -> None:
source = "_DATA: dict[str, int] = {}\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert len(issues) == 0
def test_detects_path_type_annotation(self, tmp_path: Path) -> None:
source = "_FILES: set[Path] = set()\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert len(issues) == 1
def test_known_safe_exception(self, tmp_path: Path) -> None:
source = "_SEEN: set[Path] = set()\n"
f = tmp_path / "mod.py"
f.write_text(source)
known_safe = {("mod.py", 1, "_SEEN")}
issues = find_mutable_globals(f, tmp_path, known_safe)
assert len(issues) == 0
def test_syntax_error_returns_empty(self, tmp_path: Path) -> None:
f = tmp_path / "mod.py"
f.write_text("def broken(:\n")
issues = find_mutable_globals(f, tmp_path, set())
assert issues == []
def test_detects_mutable_literal_dict(self, tmp_path: Path) -> None:
source = "_CACHE: dict[Path, Any] = {}\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert len(issues) == 1
def test_detects_mutable_literal_list(self, tmp_path: Path) -> None:
source = "SEEN_PATHS: list[Path] = []\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert len(issues) == 1
def test_detects_mutable_literal_set(self, tmp_path: Path) -> None:
source = "REGISTRY: set[Path] = set()\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert len(issues) == 1
def test_skips_function_definitions(self, tmp_path: Path) -> None:
source = "def foo():\n pass\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert issues == []
def test_handles_assign_with_name_target(self, tmp_path: Path) -> None:
source = "SEEN_PATHS = set()\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert len(issues) == 1
assert "SEEN_PATHS" in issues[0]
def test_skips_annotation_without_value(self, tmp_path: Path) -> None:
source = "_CACHE: dict[Path, Any]\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
assert issues == []
def test_skips_attribute_call(self, tmp_path: Path) -> None:
# collections.defaultdict is an Attribute call, not a Name call
source = "_CACHE: dict[Path, Any] = collections.defaultdict(list)\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
# Attribute calls are skipped (pass), so not flagged as mutable literal
assert issues == []
def test_multiple_assign_targets(self, tmp_path: Path) -> None:
source = "SEEN = CACHE = set()\n"
f = tmp_path / "mod.py"
f.write_text(source)
issues = find_mutable_globals(f, tmp_path, set())
# Both SEEN and CACHE should be flagged
assert len(issues) == 2
class TestShouldSkip:
def test_skips_pycache(self) -> None:
assert _should_skip(Path("/a/__pycache__/b.py"), DEFAULT_SKIP_DIRS) is True
def test_skips_venv(self) -> None:
assert _should_skip(Path("/a/.venv/b.py"), DEFAULT_SKIP_DIRS) is True
def test_does_not_skip_normal(self) -> None:
assert _should_skip(Path("/a/src/b.py"), DEFAULT_SKIP_DIRS) is False
class TestLoadConfig:
def test_defaults_when_no_pyproject(self, tmp_path: Path) -> None:
with patch("devx.tools.check_mutable_globals._load_pyproject_devx", return_value={}):
scan_dirs, skip_dirs, known_safe = _load_config()
assert scan_dirs == DEFAULT_SCAN_DIRS
assert skip_dirs == DEFAULT_SKIP_DIRS
assert known_safe == set()
def test_reads_config_from_pyproject(self) -> None:
cfg = {
"check_mutable_globals": {
"scan_dirs": ["src", "tests"],
"skip_dirs": ["__pycache__", ".tox"],
"known_safe": ["src/mod.py:10:_CACHE"],
}
}
with patch("devx.tools.check_mutable_globals._load_pyproject_devx", return_value=cfg):
scan_dirs, skip_dirs, known_safe = _load_config()
assert scan_dirs == ["src", "tests"]
assert ".tox" in skip_dirs
assert ("src/mod.py", 10, "_CACHE") in known_safe
def test_returns_defaults_when_cfg_not_dict(self) -> None:
with patch(
"devx.tools.check_mutable_globals._load_pyproject_devx",
return_value={"check_mutable_globals": "not a dict"},
):
scan_dirs, skip_dirs, known_safe = _load_config()
assert scan_dirs == DEFAULT_SCAN_DIRS
assert skip_dirs == DEFAULT_SKIP_DIRS
assert known_safe == set()
def test_known_safe_with_invalid_line_number(self) -> None:
cfg = {"check_mutable_globals": {"known_safe": ["mod.py:abc:_CACHE"]}}
with patch("devx.tools.check_mutable_globals._load_pyproject_devx", return_value=cfg):
_, _, known_safe = _load_config()
assert known_safe == set()
def test_scan_dirs_not_list_returns_default(self) -> None:
cfg = {"check_mutable_globals": {"scan_dirs": "not a list"}}
with patch("devx.tools.check_mutable_globals._load_pyproject_devx", return_value=cfg):
scan_dirs, _, _ = _load_config()
assert scan_dirs == DEFAULT_SCAN_DIRS
class TestCli:
def test_passes_when_no_issues(self, tmp_path: Path) -> None:
runner = CliRunner()
with (
patch("devx.tools.check_mutable_globals._load_config", return_value=(["empty_dir"], set(), set())),
patch("devx.tools.check_mutable_globals.Path.cwd", return_value=tmp_path),
):
result = runner.invoke(cli, [])
assert result.exit_code == 0
assert "Passed" in result.output
def test_fails_when_issues_found(self, tmp_path: Path) -> None:
scan_dir = tmp_path / "src"
scan_dir.mkdir()
(scan_dir / "mod.py").write_text("_SEEN: set[Path] = set()\n")
runner = CliRunner()
with (
patch("devx.tools.check_mutable_globals._load_config", return_value=(["src"], set(), set())),
patch("devx.tools.check_mutable_globals.Path.cwd", return_value=tmp_path),
):
result = runner.invoke(cli, [])
assert result.exit_code != 0
assert "FAILED" in result.output
def test_scan_dir_option_overrides_config(self, tmp_path: Path) -> None:
scan_dir = tmp_path / "custom"
scan_dir.mkdir()
(scan_dir / "mod.py").write_text("_SEEN: set[Path] = set()\n")
runner = CliRunner()
with (
patch("devx.tools.check_mutable_globals._load_config", return_value=(["other"], set(), set())),
patch("devx.tools.check_mutable_globals.Path.cwd", return_value=tmp_path),
):
result = runner.invoke(cli, ["--scan-dir", "custom"])
assert result.exit_code != 0
assert "FAILED" in result.output
def test_skips_files_in_skip_dirs(self, tmp_path: Path) -> None:
scan_dir = tmp_path / "src"
pycache = scan_dir / "__pycache__"
pycache.mkdir(parents=True)
(pycache / "mod.py").write_text("_SEEN: set[Path] = set()\n")
runner = CliRunner()
with (
patch("devx.tools.check_mutable_globals._load_config", return_value=(["src"], {"__pycache__"}, set())),
patch("devx.tools.check_mutable_globals.Path.cwd", return_value=tmp_path),
):
result = runner.invoke(cli, [])
assert result.exit_code == 0
assert "Passed" in result.output
+208
View File
@@ -0,0 +1,208 @@
"""Unit tests for devx.tools.check_pyproject_deps."""
from pathlib import Path
from click.testing import CliRunner
from devx.tools.check_pyproject_deps import check_deps, cli
class TestCheckDeps:
def test_no_issues_when_all_documented(self, tmp_path: Path) -> None:
content = """\
[project.dependencies]
# HTTP client
"requests>=2.0"
# CLI framework
"click>=8.0"
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
assert issues == []
def test_finds_undocumented_dependency(self, tmp_path: Path) -> None:
content = """\
[project.dependencies]
# HTTP client
"requests>=2.0"
"click>=8.0"
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
assert len(issues) == 1
assert "click" in issues[0]
def test_finds_multiple_undocumented(self, tmp_path: Path) -> None:
content = """\
[project.dependencies]
"requests>=2.0"
"click>=8.0"
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
assert len(issues) == 2
def test_handles_optional_dependencies(self, tmp_path: Path) -> None:
content = """\
[project.optional-dependencies]
ci = [
# Test runner
"pytest>=8",
"pytest-cov>=4",
]
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
assert len(issues) == 1
assert "pytest-cov" in issues[0]
def test_returns_file_not_found_for_missing_file(self, tmp_path: Path) -> None:
issues = check_deps(tmp_path / "nonexistent.toml")
assert len(issues) == 1
assert "not found" in issues[0]
def test_empty_deps_section_no_issues(self, tmp_path: Path) -> None:
content = """\
[project.dependencies]
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
assert issues == []
def test_skips_non_deps_sections(self, tmp_path: Path) -> None:
content = """\
[project]
name = "test"
version = "0.1.0"
[project.dependencies]
# HTTP
"requests>=2.0"
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
assert issues == []
def test_handles_dash_prefixed_deps(self, tmp_path: Path) -> None:
content = """\
[project.dependencies]
# HTTP client
-requests>=2.0
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
assert issues == []
def test_empty_lines_in_deps_section(self, tmp_path: Path) -> None:
content = """\
[project.dependencies]
# HTTP client
"requests>=2.0"
# CLI
"click>=8.0"
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
assert issues == []
def test_non_dep_non_comment_line_resets_prev(self, tmp_path: Path) -> None:
# A line that's not a comment, not a dep, not empty — resets prev_was_comment
content = """\
[project.dependencies]
# Comment
ci = [
"requests>=2.0",
]
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
# "requests" is preceded by a comment, but the `ci = [` line resets prev_was_comment
# Actually `ci = [` doesn't start with - or ", so it hits the else branch
assert len(issues) == 1
def test_section_transition_exits_deps(self, tmp_path: Path) -> None:
content = """\
[project.dependencies]
# HTTP
"requests>=2.0"
[project.optional-dependencies]
# Test runner
"pytest>=8"
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
# Both deps are documented
assert issues == []
def test_deps_after_other_section_not_checked(self, tmp_path: Path) -> None:
content = """\
[project]
name = "test"
[project.dependencies]
# Documented
"requests>=2.0"
[tool.ruff]
line-length = 120
"undocumented-dep>=1.0"
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
issues = check_deps(f)
# The "undocumented-dep" is in [tool.ruff], not a deps section
assert issues == []
class TestCli:
def test_passes_when_all_documented(self, tmp_path: Path) -> None:
content = """\
[project.dependencies]
# HTTP client
"requests>=2.0"
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
runner = CliRunner()
with __import__("contextlib").chdir(tmp_path):
result = runner.invoke(cli, [])
assert result.exit_code == 0
assert "Passed" in result.output
def test_fails_when_undocumented(self, tmp_path: Path) -> None:
content = """\
[project.dependencies]
"requests>=2.0"
"""
f = tmp_path / "pyproject.toml"
f.write_text(content)
runner = CliRunner()
with __import__("contextlib").chdir(tmp_path):
result = runner.invoke(cli, [])
assert result.exit_code != 0
assert "FAILED" in result.output
def test_custom_file_option(self, tmp_path: Path) -> None:
content = """\
[project.dependencies]
# Documented
"requests>=2.0"
"""
f = tmp_path / "custom.toml"
f.write_text(content)
runner = CliRunner()
result = runner.invoke(cli, ["--file", str(f)])
assert result.exit_code == 0
+251
View File
@@ -0,0 +1,251 @@
"""Unit tests for devx.tools.check_test_coverage."""
from pathlib import Path
from unittest.mock import MagicMock, patch
from click.testing import CliRunner
from devx.tools.check_test_coverage import (
BUILTIN_RULES,
DEFAULT_SKIP_EXTENSIONS,
DEFAULT_TEST_INDICATORS,
_changed_files,
_find_missing_tests,
_is_test_file,
_load_rules,
_resolve_test_path,
_should_skip_file,
cli,
)
class TestIsTestFile:
def test_tests_dir(self) -> None:
assert _is_test_file("tests/unit/test_foo.py", DEFAULT_TEST_INDICATORS) is True
def test_test_prefix(self) -> None:
assert _is_test_file("src/test_foo.py", DEFAULT_TEST_INDICATORS) is True
def test_test_suffix(self) -> None:
assert _is_test_file("src/foo_test.py", DEFAULT_TEST_INDICATORS) is True
def test_non_test_file(self) -> None:
assert _is_test_file("src/foo.py", DEFAULT_TEST_INDICATORS) is False
class TestShouldSkipFile:
def test_skips_dotfiles(self) -> None:
assert _should_skip_file(".gitignore", [], DEFAULT_SKIP_EXTENSIONS) is True
def test_skips_markdown(self) -> None:
assert _should_skip_file("README.md", [], DEFAULT_SKIP_EXTENSIONS) is True
def test_skips_yaml(self) -> None:
assert _should_skip_file("config.yml", [], DEFAULT_SKIP_EXTENSIONS) is True
def test_does_not_skip_python(self) -> None:
assert _should_skip_file("src/foo.py", [], DEFAULT_SKIP_EXTENSIONS) is False
def test_skips_by_pattern(self) -> None:
assert _should_skip_file("src/__init__.py", ["__init__.py"], DEFAULT_SKIP_EXTENSIONS) is True
def test_skips_by_glob_pattern(self) -> None:
assert _should_skip_file("src/config.py", ["config.py"], DEFAULT_SKIP_EXTENSIONS) is True
class TestResolveTestPath:
def test_resolves_name(self, tmp_path: Path) -> None:
result = _resolve_test_path("tests/unit/test_{name}", "src/foo.py", tmp_path)
assert result == tmp_path / "tests" / "unit" / "test_foo"
def test_resolves_module(self, tmp_path: Path) -> None:
result = _resolve_test_path("tests/unit/test_{module}_{name}", "src/pkg/foo.py", tmp_path)
assert result == tmp_path / "tests" / "unit" / "test_pkg_foo"
def test_resolves_package_prefix(self, tmp_path: Path) -> None:
result = _resolve_test_path(
"tests/unit/test_{package_prefix}_{name}",
"scripts/utils/secrets.py",
tmp_path,
)
assert result == tmp_path / "tests" / "unit" / "test_utils_secrets"
def test_normalizes_hyphens(self, tmp_path: Path) -> None:
result = _resolve_test_path("tests/test_{name}", "scripts/my-script.py", tmp_path)
assert result == tmp_path / "tests" / "test_my_script"
class TestFindMissingTests:
def test_finds_missing_test(self, tmp_path: Path) -> None:
files = ["scripts/foo.py"]
rules = BUILTIN_RULES
missing = _find_missing_tests(files, tmp_path, rules, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS)
assert "scripts/foo.py" in missing
def test_no_missing_when_test_exists(self, tmp_path: Path) -> None:
(tmp_path / "scripts" / "tests").mkdir(parents=True)
(tmp_path / "scripts" / "tests" / "test_foo.py").write_text("")
files = ["scripts/foo.py"]
rules = BUILTIN_RULES
missing = _find_missing_tests(files, tmp_path, rules, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS)
assert missing == {}
def test_skips_test_files(self, tmp_path: Path) -> None:
files = ["tests/unit/test_foo.py"]
rules = BUILTIN_RULES
missing = _find_missing_tests(files, tmp_path, rules, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS)
assert missing == {}
def test_skips_non_python_files(self, tmp_path: Path) -> None:
files = ["README.md", "config.yml"]
rules = BUILTIN_RULES
missing = _find_missing_tests(files, tmp_path, rules, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS)
assert missing == {}
def test_no_rule_no_requirement(self, tmp_path: Path) -> None:
files = ["unknown_type.xyz"]
rules = BUILTIN_RULES
missing = _find_missing_tests(files, tmp_path, rules, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS)
assert missing == {}
class TestChangedFiles:
@patch("devx.tools.check_test_coverage.subprocess.run")
def test_staged_only(self, mock_run: MagicMock, tmp_path: Path) -> None:
mock_run.return_value = MagicMock(stdout="file1.py\nfile2.py\n", returncode=0)
files = _changed_files(staged_only=True, repo_root=tmp_path)
assert files == ["file1.py", "file2.py"]
cmd = mock_run.call_args.args[0]
assert "--cached" in cmd
@patch("devx.tools.check_test_coverage.subprocess.run")
def test_ci_mode(self, mock_run: MagicMock, tmp_path: Path) -> None:
mock_run.return_value = MagicMock(stdout="file1.py\n", returncode=0)
files = _changed_files(staged_only=False, repo_root=tmp_path)
assert files == ["file1.py"]
cmd = mock_run.call_args.args[0]
assert "origin/master...HEAD" in cmd
@patch("devx.tools.check_test_coverage.subprocess.run")
def test_fallback_to_staged(self, mock_run: MagicMock, tmp_path: Path) -> None:
# First call fails, second succeeds
mock_run.side_effect = [
MagicMock(stdout="", returncode=1),
MagicMock(stdout="file1.py\n", returncode=0),
]
files = _changed_files(staged_only=False, repo_root=tmp_path)
assert files == ["file1.py"]
class TestLoadRules:
def test_defaults_when_no_config(self) -> None:
with patch("devx.tools.check_test_coverage._load_pyproject_devx", return_value={}):
rules, skip, indicators, skip_ext = _load_rules()
assert rules == BUILTIN_RULES
assert skip == []
assert indicators == DEFAULT_TEST_INDICATORS
assert skip_ext == DEFAULT_SKIP_EXTENSIONS
def test_custom_rules(self) -> None:
cfg = {
"check_test_coverage": {
"rules": [
{
"source_pattern": "lib/*.py",
"test_paths": ["tests/test_{name}"],
"description": "Missing: tests/test_{name}",
}
],
"skip_patterns": ["__init__.py"],
}
}
with patch("devx.tools.check_test_coverage._load_pyproject_devx", return_value=cfg):
rules, skip, indicators, skip_ext = _load_rules()
assert len(rules) == 1
assert rules[0]["source_pattern"] == "lib/*.py"
assert "__init__.py" in skip
def test_returns_defaults_when_cfg_not_dict(self) -> None:
with patch(
"devx.tools.check_test_coverage._load_pyproject_devx", return_value={"check_test_coverage": "not a dict"}
):
rules, skip, indicators, skip_ext = _load_rules()
assert rules == BUILTIN_RULES
assert skip == []
def test_skip_extensions_not_list_returns_default(self) -> None:
cfg = {"check_test_coverage": {"skip_extensions": "not a list"}}
with patch("devx.tools.check_test_coverage._load_pyproject_devx", return_value=cfg):
_, _, _, skip_ext = _load_rules()
assert skip_ext == DEFAULT_SKIP_EXTENSIONS
def test_test_paths_not_list_skips_rule(self, tmp_path: Path) -> None:
files = ["scripts/foo.py"]
rules = [
{
"source_pattern": "scripts/*.py",
"test_paths": "not a list",
"description": "Missing test",
}
]
missing = _find_missing_tests(files, tmp_path, rules, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS)
# Rule matches but test_paths is not a list, so it's skipped — no missing
assert missing == {}
class TestMain:
def test_no_changed_files(self, tmp_path: Path) -> None:
with (
patch("devx.tools.check_test_coverage._changed_files", return_value=[]),
patch(
"devx.tools.check_test_coverage._load_rules",
return_value=(BUILTIN_RULES, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS),
),
patch("devx.tools.check_test_coverage.Path.cwd", return_value=tmp_path),
):
runner = CliRunner()
result = runner.invoke(cli, [])
assert result.exit_code == 0
assert "No changed files" in result.output
def test_all_have_tests(self, tmp_path: Path) -> None:
(tmp_path / "scripts" / "tests").mkdir(parents=True)
(tmp_path / "scripts" / "tests" / "test_foo.py").write_text("")
with (
patch("devx.tools.check_test_coverage._changed_files", return_value=["scripts/foo.py"]),
patch(
"devx.tools.check_test_coverage._load_rules",
return_value=(BUILTIN_RULES, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS),
),
patch("devx.tools.check_test_coverage.Path.cwd", return_value=tmp_path),
):
runner = CliRunner()
result = runner.invoke(cli, [])
assert result.exit_code == 0
assert "have tests" in result.output
def test_missing_test_returns_1(self, tmp_path: Path) -> None:
with (
patch("devx.tools.check_test_coverage._changed_files", return_value=["scripts/foo.py"]),
patch(
"devx.tools.check_test_coverage._load_rules",
return_value=(BUILTIN_RULES, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS),
),
patch("devx.tools.check_test_coverage.Path.cwd", return_value=tmp_path),
):
runner = CliRunner()
result = runner.invoke(cli, [])
assert result.exit_code != 0
def test_warn_only_returns_0(self, tmp_path: Path) -> None:
with (
patch("devx.tools.check_test_coverage._changed_files", return_value=["scripts/foo.py"]),
patch(
"devx.tools.check_test_coverage._load_rules",
return_value=(BUILTIN_RULES, [], DEFAULT_TEST_INDICATORS, DEFAULT_SKIP_EXTENSIONS),
),
patch("devx.tools.check_test_coverage.Path.cwd", return_value=tmp_path),
):
runner = CliRunner()
result = runner.invoke(cli, ["--warn-only"])
assert result.exit_code == 0
+164 -10
View File
@@ -1,4 +1,4 @@
"""Unit tests for scripts/check_test_speed.py."""
"""Unit tests for devx.tools.check_test_speed."""
from unittest.mock import MagicMock, patch
@@ -8,10 +8,13 @@ from click.testing import CliRunner
from devx.tools.check_test_speed import (
DEFAULT_MAX_SECONDS,
DEFAULT_MAX_SINGLE_SECONDS,
TEST_COMMAND,
check_per_test_speed,
check_speed,
cli,
parse_duration,
parse_per_test_durations,
run_tests,
)
@@ -23,12 +26,23 @@ class TestRunTests:
stdout, stderr = run_tests()
assert stdout == "out"
assert stderr == "err"
mock_run.assert_called_once_with(
TEST_COMMAND,
capture_output=True,
text=True,
check=False,
)
mock_run.assert_called_once()
call_kwargs = mock_run.call_args
assert call_kwargs.args[0] == TEST_COMMAND
assert call_kwargs.kwargs["capture_output"] is True
assert call_kwargs.kwargs["text"] is True
assert call_kwargs.kwargs["check"] is False
env = call_kwargs.kwargs["env"]
assert "--durations=0" in env["PYTEST_ADDOPTS"]
@patch("devx.tools.check_test_speed.subprocess.run")
def test_run_tests_preserves_existing_pytest_addopts(self, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(stdout="out", stderr="err", returncode=0)
with patch.dict("os.environ", {"PYTEST_ADDOPTS": "-x"}, clear=False):
run_tests()
env = mock_run.call_args.kwargs["env"]
assert "--durations=0" in env["PYTEST_ADDOPTS"]
assert "-x" in env["PYTEST_ADDOPTS"]
class TestParseDuration:
@@ -48,6 +62,38 @@ class TestParseDuration:
assert "Could not parse" in str(exc.value)
class TestParsePerTestDurations:
def test_parses_call_lines(self) -> None:
output = "0.01s call tests/test_foo.py::test_bar\n"
durations = parse_per_test_durations(output)
assert len(durations) == 1
assert durations[0] == ("tests/test_foo.py::test_bar", 0.01)
def test_parses_setup_and_teardown(self) -> None:
output = (
"0.02s setup tests/test_foo.py::test_bar\n"
"0.01s call tests/test_foo.py::test_bar\n"
"0.00s teardown tests/test_foo.py::test_bar\n"
)
durations = parse_per_test_durations(output)
assert len(durations) == 3
names = [d[0] for d in durations]
assert "tests/test_foo.py::test_bar" in names
def test_sorted_slowest_first(self) -> None:
output = "0.01s call tests/test_a.py::test_slow\n0.50s call tests/test_b.py::test_fast\n"
durations = parse_per_test_durations(output)
assert durations[0][1] >= durations[1][1]
assert durations[0][1] == 0.50
def test_empty_output(self) -> None:
assert parse_per_test_durations("") == []
def test_ignores_non_duration_lines(self) -> None:
output = "Some random line\n234 passed in 0.70s\n"
assert parse_per_test_durations(output) == []
class TestCheckSpeed:
def test_under_budget_passes(self) -> None:
check_speed(1.0, 2.0) # should not raise
@@ -64,6 +110,31 @@ class TestCheckSpeed:
assert "max allowed: 2.0s" in msg
class TestCheckPerTestSpeed:
def test_no_violations_when_all_fast(self) -> None:
durations = [("test_a", 0.1), ("test_b", 0.2)]
assert check_per_test_speed(durations, 0.5) == []
def test_violation_when_test_exceeds_limit(self) -> None:
durations = [("test_slow", 0.6), ("test_fast", 0.1)]
violations = check_per_test_speed(durations, 0.5)
assert len(violations) == 1
assert "test_slow" in violations[0]
assert "0.60s" in violations[0]
def test_multiple_violations(self) -> None:
durations = [("test_a", 0.7), ("test_b", 0.6), ("test_c", 0.1)]
violations = check_per_test_speed(durations, 0.5)
assert len(violations) == 2
def test_exact_limit_passes(self) -> None:
durations = [("test_a", 0.5)]
assert check_per_test_speed(durations, 0.5) == []
def test_empty_durations(self) -> None:
assert check_per_test_speed([], 0.5) == []
def test_main_module_block() -> None:
import devx.tools.check_test_speed as cts
@@ -77,39 +148,71 @@ class TestMain:
@patch("devx.tools.check_test_speed.run_tests")
@patch("devx.tools.check_test_speed.parse_duration")
@patch("devx.tools.check_test_speed.check_speed")
@patch("devx.tools.check_test_speed.parse_per_test_durations")
@patch("devx.tools.check_test_speed.check_per_test_speed")
def test_successful_run(
self,
mock_check_per: MagicMock,
mock_parse_per: MagicMock,
mock_check: MagicMock,
mock_parse: MagicMock,
mock_run: MagicMock,
) -> None:
mock_run.return_value = ("stdout\n", "stderr\n")
mock_parse.return_value = 1.5
mock_parse_per.return_value = []
mock_check_per.return_value = []
runner = CliRunner()
result = runner.invoke(cli, [])
assert result.exit_code == 0
assert "1.50s" in result.output
assert "under 2.0s limit" in result.output
assert "under 10.0s limit" in result.output
mock_run.assert_called_once()
mock_parse.assert_called_once_with("stdout\n\nstderr\n")
mock_check.assert_called_once_with(1.5, DEFAULT_MAX_SECONDS)
mock_parse_per.assert_called_once()
mock_check_per.assert_called_once_with([], DEFAULT_MAX_SINGLE_SECONDS)
@patch("devx.tools.check_test_speed.run_tests")
@patch("devx.tools.check_test_speed.parse_duration")
def test_slow_tests_exit(
def test_slow_total_exits(
self,
mock_parse: MagicMock,
mock_run: MagicMock,
) -> None:
mock_run.return_value = ("out\n", "err\n")
mock_parse.return_value = 3.0
mock_parse.return_value = 15.0
runner = CliRunner()
result = runner.invoke(cli, [])
assert result.exit_code == 1
assert "too slow" in result.output.lower()
@patch("devx.tools.check_test_speed.run_tests")
@patch("devx.tools.check_test_speed.parse_duration")
@patch("devx.tools.check_test_speed.check_speed")
@patch("devx.tools.check_test_speed.parse_per_test_durations")
@patch("devx.tools.check_test_speed.check_per_test_speed")
def test_per_test_violation_exits(
self,
mock_check_per: MagicMock,
mock_parse_per: MagicMock,
mock_check: MagicMock,
mock_parse: MagicMock,
mock_run: MagicMock,
) -> None:
mock_run.return_value = ("out\n", "err\n")
mock_parse.return_value = 3.0
mock_parse_per.return_value = [("test_slow", 0.8)]
mock_check_per.return_value = ["Test 'test_slow' took 0.80s (limit: 0.5s)."]
runner = CliRunner()
result = runner.invoke(cli, [])
assert result.exit_code == 1
assert "Per-test speed check FAILED" in result.output
assert "test_slow" in result.output
@patch("devx.tools.check_test_speed.run_tests")
def test_parse_failure_exits(
self,
@@ -125,16 +228,67 @@ class TestMain:
@patch("devx.tools.check_test_speed.run_tests")
@patch("devx.tools.check_test_speed.parse_duration")
@patch("devx.tools.check_test_speed.check_speed")
@patch("devx.tools.check_test_speed.parse_per_test_durations")
@patch("devx.tools.check_test_speed.check_per_test_speed")
def test_custom_max_seconds(
self,
mock_check_per: MagicMock,
mock_parse_per: MagicMock,
mock_check: MagicMock,
mock_parse: MagicMock,
mock_run: MagicMock,
) -> None:
mock_run.return_value = ("out\n", "err\n")
mock_parse.return_value = 0.5
mock_parse_per.return_value = []
mock_check_per.return_value = []
runner = CliRunner()
result = runner.invoke(cli, ["--max-seconds", "1.5"])
assert result.exit_code == 0
mock_check.assert_called_once_with(0.5, 1.5)
@patch("devx.tools.check_test_speed.run_tests")
@patch("devx.tools.check_test_speed.parse_duration")
@patch("devx.tools.check_test_speed.check_speed")
@patch("devx.tools.check_test_speed.parse_per_test_durations")
@patch("devx.tools.check_test_speed.check_per_test_speed")
def test_disable_per_test_check(
self,
mock_check_per: MagicMock,
mock_parse_per: MagicMock,
mock_check: MagicMock,
mock_parse: MagicMock,
mock_run: MagicMock,
) -> None:
mock_run.return_value = ("out\n", "err\n")
mock_parse.return_value = 1.0
runner = CliRunner()
result = runner.invoke(cli, ["--max-single-seconds", "0"])
assert result.exit_code == 0
mock_parse_per.assert_not_called()
mock_check_per.assert_not_called()
@patch("devx.tools.check_test_speed.run_tests")
@patch("devx.tools.check_test_speed.parse_duration")
@patch("devx.tools.check_test_speed.check_speed")
@patch("devx.tools.check_test_speed.parse_per_test_durations")
@patch("devx.tools.check_test_speed.check_per_test_speed")
def test_custom_max_single_seconds(
self,
mock_check_per: MagicMock,
mock_parse_per: MagicMock,
mock_check: MagicMock,
mock_parse: MagicMock,
mock_run: MagicMock,
) -> None:
mock_run.return_value = ("out\n", "err\n")
mock_parse.return_value = 1.0
mock_parse_per.return_value = []
mock_check_per.return_value = []
runner = CliRunner()
result = runner.invoke(cli, ["--max-single-seconds", "1.0"])
assert result.exit_code == 0
mock_check_per.assert_called_once_with([], 1.0)
+56 -35
View File
@@ -41,9 +41,12 @@ class TestCheckTranslationSet:
src_dir.mkdir()
(src_dir / "mod.py").write_text('_("Hello")\n')
trans_file = tmp_path / "translations.json"
trans_file.write_text(
json.dumps({"Hello": {"en": "Hello", "bg": "Здравей", "de": "Hallo", "ru": "Привет", "zh": "你好"}})
)
all_langs = {"en": "Hello", "bg": "Здравей", "de": "Hallo", "ru": "Привет", "zh": "你好", "pl": "Cześć"}
# Include dynamic keys since collect_keys now adds them for all dirs
data = {"Hello": all_langs}
for dk in check_translations.DYNAMIC_KEYS:
data[dk] = all_langs
trans_file.write_text(json.dumps(data))
result = check_translations.check_translation_set("test", src_dir, trans_file)
assert not result.errors
@@ -67,7 +70,7 @@ class TestCheckTranslationSet:
trans_file.write_text(json.dumps({"Used": {"en": "Used"}, "Dead": {"en": "Dead"}}))
result = check_translations.check_translation_set("test", src_dir, trans_file)
assert any("Dead key" in w for w in result.warnings)
assert any("Dead key" in e for e in result.errors)
def test_missing_language(self, tmp_path: Path) -> None:
src_dir = tmp_path / "src"
@@ -77,7 +80,7 @@ class TestCheckTranslationSet:
trans_file.write_text(json.dumps({"Hello": {"en": "Hello"}}))
result = check_translations.check_translation_set("test", src_dir, trans_file)
assert any("Missing languages" in w for w in result.warnings)
assert any("Missing languages" in e for e in result.errors)
def test_missing_translations_file(self, tmp_path: Path) -> None:
src_dir = tmp_path / "src"
@@ -108,23 +111,19 @@ class TestMain:
result = runner.invoke(check_translations.main, [])
assert result.exit_code == 0
def test_strict_fails_on_warnings(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""--strict should fail if there are missing language warnings."""
warn_result = check_translations.TranslationCheckResult(
def test_errors_fail(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Errors should cause exit code 1."""
error_result = check_translations.TranslationCheckResult(
name="devx",
src_dir=Path("/tmp"),
trans_file=Path("/tmp/t.json"),
used_keys={"a"},
defined_keys={"a"},
warnings=["Dead key: 'bar'"],
)
monkeypatch.setattr(
check_translations,
"check_translation_set",
lambda name, src, trans: warn_result,
errors=["Dead key: 'bar'"],
)
monkeypatch.setattr(check_translations, "check_translation_set", lambda name, src, trans: error_result)
runner = CliRunner()
result = runner.invoke(check_translations.main, ["--strict"])
result = runner.invoke(check_translations.main, [])
assert result.exit_code == 1
def test_fails_on_errors(self, monkeypatch: pytest.MonkeyPatch) -> None:
@@ -174,15 +173,25 @@ class TestMain:
def test_translations_flag(self, tmp_path: Path) -> None:
"""--translations flag should check a specific file."""
trans_file = tmp_path / "translations.json"
trans_file.write_text(
json.dumps({"Hello": {"en": "Hello", "bg": "Здравей", "de": "Hallo", "ru": "Привет", "zh": "你好"}})
)
all_langs = {"en": "Hello", "bg": "Здравей", "de": "Hallo", "ru": "Привет", "zh": "你好", "pl": "Cześć"}
data = {"Hello": all_langs}
for dk in check_translations.DYNAMIC_KEYS:
data[dk] = all_langs
trans_file.write_text(json.dumps(data))
(tmp_path / "mod.py").write_text('_("Hello")\n')
runner = CliRunner()
result = runner.invoke(check_translations.main, ["--translations", str(trans_file)])
assert result.exit_code == 0
def test_no_translations_file_skips(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""When no translations file is found, should pass with skip message."""
monkeypatch.chdir(tmp_path)
runner = CliRunner()
result = runner.invoke(check_translations.main, [])
assert result.exit_code == 0
assert "No translations file found" in result.output
class TestPrintResult:
def test_prints_all_good(self, capsys: pytest.CaptureFixture[str]) -> None:
@@ -247,14 +256,27 @@ class TestDevxI18n:
import devx.i18n
importlib.reload(devx.i18n)
# "ERROR: REPO_TOKEN is not set." has a German translation
result = devx.i18n._("ERROR: REPO_TOKEN is not set.")
# "ERROR: CI_GITEA_TOKEN is not set." has a German translation
result = devx.i18n._("ERROR: CI_GITEA_TOKEN is not set.")
assert "FEHLER" in result
# Restore
monkeypatch.delenv("DEVX_LANG", raising=False)
importlib.reload(devx.i18n)
def test_polish_translation(self, monkeypatch: pytest.MonkeyPatch) -> None:
import importlib
monkeypatch.setenv("DEVX_LANG", "pl")
import devx.i18n
importlib.reload(devx.i18n)
result = devx.i18n._("Running tests...")
assert "Uruchamianie testów" in result
monkeypatch.delenv("DEVX_LANG", raising=False)
importlib.reload(devx.i18n)
def test_unsupported_lang_fallback(self, monkeypatch: pytest.MonkeyPatch) -> None:
import importlib
@@ -294,43 +316,42 @@ class TestCollectKeys:
assert "should_appear" in keys
assert "should_not_appear" not in keys
def test_non_default_dir_no_dynamic_keys(self, tmp_path: Path) -> None:
"""Non-default source dirs should not include DYNAMIC_KEYS."""
def test_non_default_dir_includes_dynamic_keys(self, tmp_path: Path) -> None:
"""Non-default source dirs should also include DYNAMIC_KEYS."""
(tmp_path / "mod.py").write_text('_("mykey")\n')
keys = check_translations.collect_keys(tmp_path)
assert "mykey" in keys
# Dynamic keys should NOT be present for non-default dirs
assert "completed" not in keys
assert "pending" not in keys
# Dynamic keys should be present for all dirs
assert "completed" in keys
assert "pending" in keys
def test_default_dir_includes_dynamic_keys(self) -> None:
"""The default source dir should include DYNAMIC_KEYS."""
keys = check_translations.collect_keys(check_translations.DEFAULT_SRC_DIR)
def test_default_dir_includes_dynamic_keys(self, tmp_path: Path) -> None:
"""collect_keys includes DYNAMIC_KEYS even with an empty source dir."""
keys = check_translations.collect_keys(tmp_path)
assert "completed" in keys
assert "pending" in keys
assert "in_progress" in keys
class TestMainWarnings:
def test_passes_with_warnings(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should pass with exit code 0 and 'PASS with warnings' message."""
warn_result = check_translations.TranslationCheckResult(
class TestMainCleanPass:
def test_passes_clean(self, monkeypatch: pytest.MonkeyPatch) -> None:
"""Should pass with exit code 0 and 'PASS:' message when no errors."""
ok_result = check_translations.TranslationCheckResult(
name="devx",
src_dir=Path("/tmp"),
trans_file=Path("/tmp/t.json"),
used_keys={"a"},
defined_keys={"a"},
warnings=["Dead key: 'bar'"],
)
monkeypatch.setattr(
check_translations,
"check_translation_set",
lambda name, src, trans: warn_result,
lambda name, src, trans: ok_result,
)
runner = CliRunner()
result = runner.invoke(check_translations.main, [])
assert result.exit_code == 0
assert "PASS with warnings" in result.output
assert "PASS:" in result.output
class TestI18nProjectTranslations:
+612 -105
View File
@@ -1,4 +1,16 @@
"""Unit tests for scripts/ci/classify_changes.py."""
"""Unit tests for devx.ci.classify_changes.
Tests cover:
- Glob matching (``_glob_to_regex``, ``_matches_glob``)
- Classifier config loading from pyproject.toml
- ChangeClassifier with layered rules (overrides, patterns, default)
- Tag system (orthogonal categories)
- Backward-compatible API (is_user_facing, is_workflow_only, classify_changes)
- Git helpers (get_changed_files, get_latest_tag, run_git)
- CLI (main with --quiet, --check, --github-output)
"""
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
@@ -9,6 +21,13 @@ from click.testing import CliRunner
import devx.ci.classify_changes as classify_changes_mod
from devx.ci.classify_changes import (
DEFAULT_INFRASTRUCTURE,
ChangeClassifier,
ClassificationResult,
ClassifierConfig,
FileClassification,
_glob_to_regex,
_matches_glob,
classify_changes,
get_changed_files,
get_latest_tag,
@@ -19,98 +38,401 @@ from devx.ci.classify_changes import (
run_git,
)
# ---------------------------------------------------------------------------
# Glob matching tests
# ---------------------------------------------------------------------------
class TestIsUserFacing:
def test_src_is_user_facing(self) -> None:
assert is_user_facing("src/devx/cli.py") is True
def test_ansible_is_user_facing(self) -> None:
assert is_user_facing("ansible/roles/gitea-runner/tasks/main.yml") is True
class TestGlobToRegex:
def test_double_star_matches_anything(self) -> None:
regex = _glob_to_regex(".gitea/**")
assert regex.match(".gitea/workflows/ci.yml")
assert regex.match(".gitea/actionlint.yaml")
assert regex.match(".gitea/a/b/c/d.yml")
assert not regex.match("tests/test_foo.py")
def test_pyproject_is_user_facing(self) -> None:
assert is_user_facing("pyproject.toml") is True
def test_double_star_in_middle(self) -> None:
"""** in the middle of a pattern matches any number of segments."""
regex = _glob_to_regex("src/**/test_*.py")
assert regex.match("src/test_foo.py")
assert regex.match("src/devx/test_cli.py")
assert regex.match("src/a/b/c/test_bar.py")
assert not regex.match("src/cli.py")
def test_workflow_is_not_user_facing(self) -> None:
assert is_user_facing(".gitea/workflows/ci.yml") is False
def test_single_star_matches_within_segment(self) -> None:
regex = _glob_to_regex("src/*/cli.py")
assert regex.match("src/devx/cli.py")
assert regex.match("src/pkg/cli.py")
assert not regex.match("src/devx/sub/cli.py")
def test_ci_scripts_are_not_user_facing(self) -> None:
assert is_user_facing("scripts/ci/release.py") is False
def test_question_mark_matches_single_char(self) -> None:
regex = _glob_to_regex("file?.py")
assert regex.match("file1.py")
assert regex.match("fileA.py")
assert not regex.match("file12.py")
def test_dev_scripts_are_not_user_facing(self) -> None:
"""All scripts under scripts/ are infrastructure (CI/CD, dev tools).
User-facing code lives in src/devx/."""
assert is_user_facing("scripts/check_test_speed.py") is False
assert is_user_facing("scripts/configure_repo.py") is False
assert is_user_facing("scripts/install_checkmake.py") is False
def test_literal_match(self) -> None:
regex = _glob_to_regex("Makefile")
assert regex.match("Makefile")
assert not regex.match("makefile")
def test_shell_scripts_are_not_user_facing(self) -> None:
assert is_user_facing("scripts/setup.sh") is False
assert is_user_facing("scripts/molecule_all.sh") is False
def test_special_chars_escaped(self) -> None:
regex = _glob_to_regex("file.test.py")
assert regex.match("file.test.py")
assert not regex.match("fileXtest.py")
def test_scripts_init_is_not_user_facing(self) -> None:
assert is_user_facing("scripts/__init__.py") is False
def test_version_file_is_not_user_facing(self) -> None:
"""__init__.py only contains __version__ — a release artifact,
not user-facing code. Version bumps alone should not trigger releases."""
assert is_user_facing("src/devx/__init__.py") is False
class TestMatchesGlob:
def test_double_star(self) -> None:
assert _matches_glob(".gitea/workflows/ci.yml", ".gitea/**")
assert _matches_glob("tests/unit/test_cli.py", "tests/**")
assert not _matches_glob("src/devx/cli.py", "tests/**")
def test_api_clients_is_not_user_facing(self) -> None:
"""api_clients.py is used only by CI/CD scripts, not by the GRM CLI."""
assert is_user_facing("src/devx/api_clients.py") is False
def test_exact_match(self) -> None:
assert _matches_glob("Makefile", "Makefile")
assert _matches_glob("src/devx/__init__.py", "src/devx/__init__.py")
assert not _matches_glob("src/devx/cli.py", "src/devx/__init__.py")
def test_docs_are_not_user_facing(self) -> None:
assert is_user_facing("docs/user/getting-started.md") is False
def test_prefix_matching(self) -> None:
assert _matches_glob(".gitea/workflows/ci.yml", ".gitea/")
assert _matches_glob("scripts/ci/release.py", "scripts/")
assert not _matches_glob("tests/test_foo.py", "scripts/")
def test_tests_are_not_user_facing(self) -> None:
assert is_user_facing("tests/unit/test_cli.py") is False
def test_single_star(self) -> None:
assert _matches_glob("src/devx/cli.py", "src/devx/*.py")
assert not _matches_glob("src/devx/sub/cli.py", "src/devx/*.py")
def test_agents_md_is_not_user_facing(self) -> None:
assert is_user_facing("AGENTS.md") is False
def test_makefile_is_not_user_facing(self) -> None:
assert is_user_facing("Makefile") is False
# ---------------------------------------------------------------------------
# ClassifierConfig tests
# ---------------------------------------------------------------------------
class TestClassifierConfig:
def test_from_pyproject_merges_with_defaults(self, tmp_path: Path) -> None:
pyproject = tmp_path / "pyproject.toml"
pyproject.write_text(
"[tool.devx.classify]\n"
'infrastructure = ["scripts/**"]\n'
'infrastructure_overrides = ["src/pkg/__init__.py"]\n'
'user_facing_overrides = ["docs/important.py"]\n'
"\n"
"[tool.devx.classify.tags]\n"
'ansible = ["ansible/**"]\n'
)
config = ClassifierConfig.from_pyproject(str(pyproject))
# Project-specific path is merged with defaults
assert "scripts/**" in config.infrastructure
assert ".gitea/**" in config.infrastructure # from DEFAULT_INFRASTRUCTURE
assert "tests/**" in config.infrastructure # from DEFAULT_INFRASTRUCTURE
assert config.use_defaults is True
assert config.infrastructure_overrides == ["src/pkg/__init__.py"]
assert config.user_facing_overrides == ["docs/important.py"]
assert config.tags == {"ansible": ["ansible/**"]}
def test_from_pyproject_use_defaults_false(self, tmp_path: Path) -> None:
pyproject = tmp_path / "pyproject.toml"
pyproject.write_text('[tool.devx.classify]\nuse_defaults = false\ninfrastructure = [".gitea/**"]\n')
config = ClassifierConfig.from_pyproject(str(pyproject))
assert config.infrastructure == [".gitea/**"]
assert "tests/**" not in config.infrastructure # no defaults
assert config.use_defaults is False
def test_from_pyproject_missing_file_returns_defaults(self) -> None:
config = ClassifierConfig.from_pyproject("/nonexistent/pyproject.toml")
assert config.infrastructure == list(DEFAULT_INFRASTRUCTURE)
assert config.infrastructure_overrides == []
assert config.user_facing_overrides == []
assert config.tags == {}
assert config.use_defaults is True
def test_from_pyproject_missing_section_returns_defaults(self, tmp_path: Path) -> None:
pyproject = tmp_path / "pyproject.toml"
pyproject.write_text('[project]\nname = "test"\n')
config = ClassifierConfig.from_pyproject(str(pyproject))
assert config.infrastructure == list(DEFAULT_INFRASTRUCTURE)
def test_from_pyproject_partial_config(self, tmp_path: Path) -> None:
pyproject = tmp_path / "pyproject.toml"
pyproject.write_text('[tool.devx.classify]\ninfrastructure = ["scripts/**"]\n')
config = ClassifierConfig.from_pyproject(str(pyproject))
assert "scripts/**" in config.infrastructure
assert ".gitea/**" in config.infrastructure # merged with defaults
assert config.infrastructure_overrides == []
assert config.user_facing_overrides == []
assert config.tags == {}
def test_from_pyproject_dedupes_existing_default(self, tmp_path: Path) -> None:
"""Project infrastructure patterns already in defaults are not duplicated."""
pyproject = tmp_path / "pyproject.toml"
pyproject.write_text('[tool.devx.classify]\ninfrastructure = [".gitea/**", "scripts/**"]\n')
config = ClassifierConfig.from_pyproject(str(pyproject))
# .gitea/** should appear only once (deduplicated with defaults)
assert config.infrastructure.count(".gitea/**") == 1
assert "scripts/**" in config.infrastructure
def test_defaults_are_empty_for_bare_constructor(self) -> None:
"""ClassifierConfig() without from_pyproject has empty lists."""
config = ClassifierConfig()
assert config.infrastructure == []
assert config.infrastructure_overrides == []
assert config.user_facing_overrides == []
assert config.tags == {}
assert config.use_defaults is True
def test_default_infrastructure_is_non_empty(self) -> None:
"""The framework ships with a curated default infrastructure list."""
assert len(DEFAULT_INFRASTRUCTURE) > 0
assert ".gitea/**" in DEFAULT_INFRASTRUCTURE
assert "tests/**" in DEFAULT_INFRASTRUCTURE
assert "docs/**" in DEFAULT_INFRASTRUCTURE
def test_default_infrastructure_covers_common_project_files(self) -> None:
"""DEFAULT_INFRASTRUCTURE must cover common project-level files
that are not part of the installed package.
This test prevents regression of the root cause of GRM-64
misclassification: 28 files (scripts/**, REVIEW_CHECKLIST.md)
were classified as user-facing because these patterns were
missing from the defaults.
"""
# Project documentation
assert "AGENTS.md" in DEFAULT_INFRASTRUCTURE
assert "README.md" in DEFAULT_INFRASTRUCTURE
assert "CHANGELOG.md" in DEFAULT_INFRASTRUCTURE
assert "TROUBLESHOOTING.md" in DEFAULT_INFRASTRUCTURE
assert "CONTRIBUTING.md" in DEFAULT_INFRASTRUCTURE
assert "CODE_OF_CONDUCT.md" in DEFAULT_INFRASTRUCTURE
assert "REVIEW_CHECKLIST.md" in DEFAULT_INFRASTRUCTURE
# Build tooling
assert "Makefile" in DEFAULT_INFRASTRUCTURE
assert "cliff.toml" in DEFAULT_INFRASTRUCTURE
assert "uv.lock" in DEFAULT_INFRASTRUCTURE
# Lint config
assert ".pre-commit-config.yaml" in DEFAULT_INFRASTRUCTURE
assert ".ruff.toml" in DEFAULT_INFRASTRUCTURE
assert ".ansible-lint" in DEFAULT_INFRASTRUCTURE
assert ".checkmake.ini" in DEFAULT_INFRASTRUCTURE
assert ".editorconfig" in DEFAULT_INFRASTRUCTURE
# Git config
assert ".gitignore" in DEFAULT_INFRASTRUCTURE
assert ".gitattributes" in DEFAULT_INFRASTRUCTURE
# Agent config
assert ".devin/**" in DEFAULT_INFRASTRUCTURE
# ---------------------------------------------------------------------------
# ChangeClassifier tests
# ---------------------------------------------------------------------------
class TestChangeClassifier:
def _make_classifier(self, **kwargs: object) -> ChangeClassifier:
"""Create a classifier with explicit config (no pyproject.toml needed)."""
config = ClassifierConfig(**kwargs) # type: ignore[arg-type]
return ChangeClassifier(config)
def test_infrastructure_pattern_matches(self) -> None:
classifier = self._make_classifier(infrastructure=[".gitea/**", "tests/**"])
fc = classifier.classify_file(".gitea/workflows/ci.yml")
assert not fc.is_user_facing
assert "infrastructure" in fc.matched_rule
def test_unknown_file_defaults_to_user_facing(self) -> None:
"""Safe default: unknown files are user-facing (require release)."""
assert is_user_facing("some/new/file.type") is True
assert is_user_facing("new_root_file.txt") is True
classifier = self._make_classifier(infrastructure=[".gitea/**"])
fc = classifier.classify_file("src/devx/cli.py")
assert fc.is_user_facing
assert fc.matched_rule is None
assert "default" in fc.reason.lower()
def test_is_workflow_only_inverse(self) -> None:
assert is_workflow_only(".gitea/workflows/ci.yml") is True
assert is_workflow_only("src/devx/cli.py") is False
assert is_workflow_only("pyproject.toml") is False
def test_infrastructure_override(self) -> None:
classifier = self._make_classifier(
infrastructure=[".gitea/**"],
infrastructure_overrides=["src/devx/__init__.py"],
)
fc = classifier.classify_file("src/devx/__init__.py")
assert not fc.is_user_facing
assert fc.matched_rule == "infrastructure_overrides"
def test_user_facing_override_beats_infrastructure(self) -> None:
"""User-facing overrides have highest priority (safety)."""
classifier = self._make_classifier(
infrastructure=["tests/**"],
user_facing_overrides=["tests/test_public_api.py"],
)
fc = classifier.classify_file("tests/test_public_api.py")
assert fc.is_user_facing
assert fc.matched_rule == "user_facing_overrides"
class TestClassifyChanges:
def test_all_user_facing(self) -> None:
files = ["src/devx/cli.py", "ansible/roles/gitea-runner/tasks/main.yml"]
result = classify_changes(files)
assert result["user_facing"] == files
assert result["workflow_only"] == []
def test_user_facing_override_glob_matches_nested(self) -> None:
"""User-facing overrides support glob patterns like infrastructure."""
classifier = self._make_classifier(
infrastructure=[".gitea/**"],
user_facing_overrides=[".gitea/**"],
)
fc = classifier.classify_file(".gitea/workflows/ci.yml")
assert fc.is_user_facing
assert fc.matched_rule == "user_facing_overrides"
def test_all_workflow_only(self) -> None:
files = [".gitea/workflows/ci.yml", "docs/index.md", "AGENTS.md"]
result = classify_changes(files)
assert result["user_facing"] == []
assert result["workflow_only"] == files
def test_user_facing_override_beats_infrastructure_override(self) -> None:
"""User-facing overrides beat infrastructure overrides (safety first)."""
classifier = self._make_classifier(
infrastructure=[".gitea/**"],
infrastructure_overrides=["src/devx/__init__.py"],
user_facing_overrides=["src/devx/__init__.py"],
)
fc = classifier.classify_file("src/devx/__init__.py")
assert fc.is_user_facing
def test_mixed(self) -> None:
def test_tags_are_computed(self) -> None:
classifier = self._make_classifier(
infrastructure=[".gitea/**"],
tags={"ansible": ["ansible/**", ".ansible-lint"], "docs": ["docs/**"]},
)
fc = classifier.classify_file("ansible/tasks/main.yml")
assert "ansible" in fc.tags
assert "docs" not in fc.tags
def test_tags_orthogonal_to_classification(self) -> None:
"""A file can be infrastructure AND tagged."""
classifier = self._make_classifier(
infrastructure=[".gitea/**", "docs/**"],
tags={"docs": ["docs/**"]},
)
fc = classifier.classify_file("docs/index.md")
assert not fc.is_user_facing # infrastructure
assert "docs" in fc.tags # also tagged
def test_classify_multiple_files(self) -> None:
classifier = self._make_classifier(
infrastructure=[".gitea/**", "tests/**"],
infrastructure_overrides=["src/devx/__init__.py"],
tags={"ansible": ["ansible/**"]},
)
files = [
"src/devx/cli.py",
".gitea/workflows/ci.yml",
"pyproject.toml",
"docs/index.md",
"src/devx/__init__.py",
"ansible/tasks/main.yml",
"tests/test_foo.py",
]
result = classify_changes(files)
assert "src/devx/cli.py" in result["user_facing"]
assert "pyproject.toml" in result["user_facing"]
assert ".gitea/workflows/ci.yml" in result["workflow_only"]
assert "docs/index.md" in result["workflow_only"]
result = classifier.classify(files)
assert "src/devx/cli.py" in result.user_facing
assert "ansible/tasks/main.yml" in result.user_facing
assert ".gitea/workflows/ci.yml" in result.infrastructure
assert "src/devx/__init__.py" in result.infrastructure
assert "tests/test_foo.py" in result.infrastructure
assert result.has_user_facing
assert result.has_tag("ansible")
assert "ansible/tasks/main.yml" in result.tags["ansible"]
def test_empty(self) -> None:
result = classify_changes([])
assert result == {"user_facing": [], "workflow_only": []}
def test_classify_empty(self) -> None:
classifier = self._make_classifier(infrastructure=[".gitea/**"])
result = classifier.classify([])
assert not result.has_user_facing
assert result.user_facing == []
assert result.infrastructure == []
def test_reason_is_human_readable(self) -> None:
classifier = self._make_classifier(infrastructure=[".gitea/**"])
fc = classifier.classify_file(".gitea/workflows/ci.yml")
assert ".gitea/**" in fc.reason
fc2 = classifier.classify_file("src/devx/cli.py")
assert "default" in fc2.reason.lower() or "user-facing" in fc2.reason.lower()
class TestClassificationResult:
def test_has_user_facing(self) -> None:
result = ClassificationResult(user_facing=["src/cli.py"])
assert result.has_user_facing
def test_has_user_facing_empty(self) -> None:
result = ClassificationResult()
assert not result.has_user_facing
def test_has_tag(self) -> None:
result = ClassificationResult(tags={"ansible": ["ansible/tasks/main.yml"]})
assert result.has_tag("ansible")
assert not result.has_tag("docs")
# ---------------------------------------------------------------------------
# Backward-compatible API tests
# ---------------------------------------------------------------------------
class TestBackwardCompatibleAPI:
def test_is_workflow_only_with_config(self) -> None:
"""is_workflow_only uses the config-driven classifier by default."""
with patch.object(classify_changes_mod, "_get_classifier") as mock:
classifier = MagicMock()
classifier.classify_file.return_value = FileClassification(
path=".gitea/workflows/ci.yml",
is_user_facing=False,
reason="test",
matched_rule="infrastructure: .gitea/**",
)
mock.return_value = classifier
assert is_workflow_only(".gitea/workflows/ci.yml") is True
def test_is_user_facing_with_config(self) -> None:
with patch.object(classify_changes_mod, "_get_classifier") as mock:
classifier = MagicMock()
classifier.classify_file.return_value = FileClassification(
path="src/devx/cli.py",
is_user_facing=True,
reason="test",
matched_rule=None,
)
mock.return_value = classifier
assert is_user_facing("src/devx/cli.py") is True
def test_legacy_patterns_mode(self) -> None:
"""is_workflow_only with explicit patterns uses legacy prefix matching."""
patterns = frozenset([".gitea/", "tests/"])
assert is_workflow_only(".gitea/workflows/ci.yml", patterns) is True
assert is_workflow_only("tests/test_foo.py", patterns) is True
assert is_workflow_only("src/devx/cli.py", patterns) is False
def test_classify_changes_with_config(self) -> None:
with patch.object(classify_changes_mod, "_get_classifier") as mock:
classifier = MagicMock()
classifier.classify.return_value = ClassificationResult(
user_facing=["src/devx/cli.py"],
infrastructure=[".gitea/workflows/ci.yml"],
)
mock.return_value = classifier
result = classify_changes(["src/devx/cli.py", ".gitea/workflows/ci.yml"])
assert "src/devx/cli.py" in result["user_facing"]
assert ".gitea/workflows/ci.yml" in result["workflow_only"]
def test_classify_changes_legacy_mode(self) -> None:
patterns = frozenset([".gitea/", "tests/"])
result = classify_changes([".gitea/ci.yml", "src/cli.py"], patterns)
assert ".gitea/ci.yml" in result["workflow_only"]
assert "src/cli.py" in result["user_facing"]
def test_has_user_facing_changes_with_config(self) -> None:
with (
patch.object(classify_changes_mod, "get_changed_files", return_value=["src/devx/cli.py"]),
patch.object(classify_changes_mod, "_get_classifier") as mock,
):
classifier = MagicMock()
classifier.classify.return_value = ClassificationResult(
user_facing=["src/devx/cli.py"],
)
mock.return_value = classifier
assert has_user_facing_changes("v0.1.0", "HEAD") is True
def test_has_user_facing_changes_legacy(self) -> None:
with patch.object(classify_changes_mod, "get_changed_files", return_value=[".gitea/ci.yml"]):
patterns = frozenset([".gitea/"])
assert has_user_facing_changes("v0.1.0", "HEAD", patterns) is False
# ---------------------------------------------------------------------------
# Git helper tests
# ---------------------------------------------------------------------------
class TestGetChangedFiles:
@@ -127,23 +449,6 @@ class TestGetChangedFiles:
assert result == []
class TestHasUserFacingChanges:
@patch("devx.ci.classify_changes.get_changed_files")
def test_true_when_user_facing(self, mock_get: MagicMock) -> None:
mock_get.return_value = ["src/devx/cli.py", "docs/index.md"]
assert has_user_facing_changes("v0.1.0", "HEAD") is True
@patch("devx.ci.classify_changes.get_changed_files")
def test_false_when_workflow_only(self, mock_get: MagicMock) -> None:
mock_get.return_value = [".gitea/workflows/ci.yml", "docs/index.md"]
assert has_user_facing_changes("v0.1.0", "HEAD") is False
@patch("devx.ci.classify_changes.get_changed_files")
def test_false_when_no_changes(self, mock_get: MagicMock) -> None:
mock_get.return_value = []
assert has_user_facing_changes("v0.1.0", "HEAD") is False
class TestGetLatestTag:
@patch("subprocess.run")
def test_returns_tag(self, mock_run: MagicMock) -> None:
@@ -170,6 +475,11 @@ class TestRunGit:
run_git(["git", "bad-command"])
# ---------------------------------------------------------------------------
# CLI tests
# ---------------------------------------------------------------------------
class TestMain:
@patch("devx.ci.classify_changes.get_latest_tag", return_value="")
def test_no_tags_outputs_true(self, mock_tag: MagicMock) -> None:
@@ -189,7 +499,7 @@ class TestMain:
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_workflow_only_exits_2(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
mock_changes.return_value = [".gitea/workflows/ci.yml", "docs/index.md"]
mock_changes.return_value = ["docs/index.md", "README.md"]
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 2
@@ -204,9 +514,49 @@ class TestMain:
assert result.exit_code == 0
assert "release needed" in result.output
@patch("devx.ci.classify_changes._get_classifier")
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_default_mode_displays_tags(
self, mock_tag: MagicMock, mock_changes: MagicMock, mock_clf: MagicMock
) -> None:
"""Default mode shows tag files when tags are configured."""
mock_changes.return_value = ["src/devx/cli.py", "ansible/tasks/main.yml"]
mock_clf.return_value = ChangeClassifier(
ClassifierConfig(
infrastructure=[".gitea/**"],
tags={"ansible": ["ansible/**"]},
)
)
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
assert "Ansible files" in result.output
assert "ansible/tasks/main.yml" in result.output
@patch("devx.ci.classify_changes._get_classifier")
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_default_mode_skips_empty_tag(
self, mock_tag: MagicMock, mock_changes: MagicMock, mock_clf: MagicMock
) -> None:
"""Tags with no matching files are skipped in default mode output."""
mock_changes.return_value = ["ansible/tasks/main.yml"]
mock_clf.return_value = ChangeClassifier(
ClassifierConfig(
infrastructure=[".gitea/**"],
tags={"ansible": ["ansible/**"], "docs": ["docs/**"]},
)
)
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
assert "Ansible files" in result.output
# docs tag has no matching files — should not appear
assert "Docs files" not in result.output
@patch("devx.ci.classify_changes.get_latest_tag", return_value="")
def test_no_tags_non_quiet(self, mock_tag: MagicMock) -> None:
"""Non-quiet mode with no tags prints user-facing message."""
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
@@ -215,7 +565,6 @@ class TestMain:
@patch("devx.ci.classify_changes.get_changed_files", return_value=[])
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_no_changes_non_quiet(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
"""Non-quiet mode with no changes prints message."""
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
@@ -224,7 +573,6 @@ class TestMain:
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_quiet_user_facing(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
"""Quiet mode with user-facing changes outputs true."""
mock_changes.return_value = ["src/devx/cli.py"]
runner = CliRunner()
result = runner.invoke(main, ["--quiet"])
@@ -234,8 +582,7 @@ class TestMain:
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_quiet_workflow_only(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
"""Quiet mode with workflow-only changes outputs false."""
mock_changes.return_value = [".gitea/workflows/ci.yml"]
mock_changes.return_value = ["docs/index.md"]
runner = CliRunner()
result = runner.invoke(main, ["--quiet"])
assert result.exit_code == 0
@@ -244,28 +591,39 @@ class TestMain:
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_with_explicit_base(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
"""Explicit --base overrides latest tag."""
mock_changes.return_value = ["src/devx/cli.py"]
runner = CliRunner()
result = runner.invoke(main, ["--base", "v0.2.0", "--head", "HEAD"])
assert result.exit_code == 0
assert "release needed" in result.output
@patch("devx.ci.classify_changes._get_classifier")
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_check_ansible_true(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
"""--check ansible with Ansible changes outputs true."""
def test_check_ansible_true(self, mock_tag: MagicMock, mock_changes: MagicMock, mock_clf: MagicMock) -> None:
mock_changes.return_value = ["ansible/tasks/main.yml", ".gitea/workflows/ci.yml"]
mock_clf.return_value = ChangeClassifier(
ClassifierConfig(
infrastructure=[".gitea/**"],
tags={"ansible": ["ansible/**"]},
)
)
runner = CliRunner()
result = runner.invoke(main, ["--check", "ansible", "--quiet"])
assert result.exit_code == 0
assert "true" in result.output
@patch("devx.ci.classify_changes._get_classifier")
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_check_ansible_false(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
"""--check ansible with no Ansible changes outputs false."""
def test_check_ansible_false(self, mock_tag: MagicMock, mock_changes: MagicMock, mock_clf: MagicMock) -> None:
mock_changes.return_value = ["src/devx/cli.py", ".gitea/workflows/ci.yml"]
mock_clf.return_value = ChangeClassifier(
ClassifierConfig(
infrastructure=[".gitea/**"],
tags={"ansible": ["ansible/**"]},
)
)
runner = CliRunner()
result = runner.invoke(main, ["--check", "ansible", "--quiet"])
assert result.exit_code == 0
@@ -274,7 +632,6 @@ class TestMain:
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_check_user_facing_true(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
"""--check user-facing with user-facing changes outputs true."""
mock_changes.return_value = ["src/devx/cli.py", ".gitea/workflows/ci.yml"]
runner = CliRunner()
result = runner.invoke(main, ["--check", "user-facing", "--quiet"])
@@ -284,27 +641,47 @@ class TestMain:
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_check_user_facing_false(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
"""--check user-facing with only workflow changes outputs false."""
mock_changes.return_value = [".gitea/workflows/ci.yml", "tests/test_foo.py"]
mock_changes.return_value = ["docs/index.md", "tests/test_foo.py"]
runner = CliRunner()
result = runner.invoke(main, ["--check", "user-facing", "--quiet"])
assert result.exit_code == 0
assert "false" in result.output
@patch("devx.ci.classify_changes._get_classifier")
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_check_ansible_non_quiet(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
"""--check ansible in non-quiet mode prints file list."""
def test_check_ansible_non_quiet(self, mock_tag: MagicMock, mock_changes: MagicMock, mock_clf: MagicMock) -> None:
mock_changes.return_value = ["ansible/tasks/main.yml"]
mock_clf.return_value = ChangeClassifier(
ClassifierConfig(
infrastructure=[".gitea/**"],
tags={"ansible": ["ansible/**"]},
)
)
runner = CliRunner()
result = runner.invoke(main, ["--check", "ansible"])
assert result.exit_code == 0
assert "Ansible changes detected" in result.output
@patch("devx.ci.classify_changes._get_classifier")
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_check_unknown_tag_raises(self, mock_tag: MagicMock, mock_changes: MagicMock, mock_clf: MagicMock) -> None:
mock_changes.return_value = ["src/devx/cli.py"]
mock_clf.return_value = ChangeClassifier(
ClassifierConfig(
infrastructure=[".gitea/**"],
tags={"ansible": ["ansible/**"]},
)
)
runner = CliRunner()
result = runner.invoke(main, ["--check", "nonexistent"])
assert result.exit_code != 0
assert "Unknown check category" in result.output
@patch("devx.ci.classify_changes.get_changed_files")
@patch("devx.ci.classify_changes.get_latest_tag", return_value="v0.3.0")
def test_check_user_facing_non_quiet(self, mock_tag: MagicMock, mock_changes: MagicMock) -> None:
"""--check user-facing in non-quiet mode prints file list."""
mock_changes.return_value = ["src/devx/cli.py"]
runner = CliRunner()
result = runner.invoke(main, ["--check", "user-facing"])
@@ -313,7 +690,17 @@ class TestMain:
class TestGithubOutput:
def test_writes_outputs(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
def _make_classifier_with_ansible(self) -> ChangeClassifier:
return ChangeClassifier(
ClassifierConfig(
infrastructure=[".gitea/**", "AGENTS.md"],
tags={"ansible": ["ansible/**"]},
)
)
@patch("devx.ci.classify_changes._get_classifier")
def test_writes_outputs(self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
mock_clf.return_value = self._make_classifier_with_ansible()
gh_file = tmp_path / "output.txt"
monkeypatch.setenv("GITHUB_OUTPUT", str(gh_file))
with patch.object(
@@ -326,7 +713,9 @@ class TestGithubOutput:
assert "ansible-changed=true" in content
assert "user-facing-changed=true" in content
def test_no_changes(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
@patch("devx.ci.classify_changes._get_classifier")
def test_no_changes(self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
mock_clf.return_value = self._make_classifier_with_ansible()
gh_file = tmp_path / "output.txt"
monkeypatch.setenv("GITHUB_OUTPUT", str(gh_file))
with patch.object(classify_changes_mod, "get_changed_files", return_value=[]):
@@ -337,7 +726,9 @@ class TestGithubOutput:
assert "ansible-changed=false" in content
assert "user-facing-changed=false" in content
def test_no_tags(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
@patch("devx.ci.classify_changes._get_classifier")
def test_no_tags(self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
mock_clf.return_value = self._make_classifier_with_ansible()
gh_file = tmp_path / "output.txt"
monkeypatch.setenv("GITHUB_OUTPUT", str(gh_file))
with patch.object(classify_changes_mod, "get_latest_tag", return_value=""):
@@ -355,7 +746,9 @@ class TestGithubOutput:
result = runner.invoke(main, ["--base", "v1.0", "--head", "HEAD", "--github-output"])
assert result.exit_code != 0
def test_workflow_only(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
@patch("devx.ci.classify_changes._get_classifier")
def test_workflow_only(self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
mock_clf.return_value = self._make_classifier_with_ansible()
gh_file = tmp_path / "output.txt"
monkeypatch.setenv("GITHUB_OUTPUT", str(gh_file))
with patch.object(
@@ -367,3 +760,117 @@ class TestGithubOutput:
content = gh_file.read_text()
assert "ansible-changed=false" in content
assert "user-facing-changed=false" in content
@patch("devx.ci.classify_changes._get_classifier")
def test_no_tags_outputs_all_tags_true(
self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""When no tags exist, only user-facing-changed is written."""
mock_clf.return_value = ChangeClassifier(
ClassifierConfig(
infrastructure=[".gitea/**"],
tags={},
)
)
gh_file = tmp_path / "output.txt"
monkeypatch.setenv("GITHUB_OUTPUT", str(gh_file))
with patch.object(classify_changes_mod, "get_latest_tag", return_value=""):
runner = CliRunner()
result = runner.invoke(main, ["--github-output"])
assert result.exit_code == 0
content = gh_file.read_text()
assert "user-facing-changed=true" in content
# No tag outputs since no tags are configured
assert "ansible-changed" not in content
@patch("devx.ci.classify_changes._get_classifier")
def test_force_outputs_true(self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""--force with --github-output writes user-facing-changed=true and all tags true."""
mock_clf.return_value = self._make_classifier_with_ansible()
gh_file = tmp_path / "output.txt"
monkeypatch.setenv("GITHUB_OUTPUT", str(gh_file))
runner = CliRunner()
result = runner.invoke(main, ["--github-output", "--force"])
assert result.exit_code == 0
content = gh_file.read_text()
assert "user-facing-changed=true" in content
assert "ansible-changed=true" in content
assert "Forced user-facing-changed=true" in result.output
@patch("devx.ci.classify_changes._get_classifier")
def test_force_without_github_output_does_nothing(
self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""--force without --github-output falls through to normal classification."""
mock_clf.return_value = self._make_classifier_with_ansible()
monkeypatch.setenv("GITHUB_OUTPUT", str(tmp_path / "output.txt"))
with patch.object(classify_changes_mod, "get_latest_tag", return_value="v1.0"):
with patch.object(classify_changes_mod, "get_changed_files", return_value=[]):
runner = CliRunner()
result = runner.invoke(main, ["--force", "--quiet"])
assert result.exit_code == 0
assert result.output.strip() == "false"
@patch("devx.ci.classify_changes._get_classifier")
def test_force_no_tags(self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""--force with --github-output and no tags writes only user-facing-changed=true."""
mock_clf.return_value = ChangeClassifier(
ClassifierConfig(
infrastructure=[".gitea/**"],
tags={},
)
)
gh_file = tmp_path / "output.txt"
monkeypatch.setenv("GITHUB_OUTPUT", str(gh_file))
runner = CliRunner()
result = runner.invoke(main, ["--github-output", "--force"])
assert result.exit_code == 0
content = gh_file.read_text()
assert "user-facing-changed=true" in content
assert "ansible-changed" not in content
@patch("devx.ci.classify_changes._get_classifier")
def test_force_deploy_env_var(self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""FORCE_DEPLOY=true env var activates force mode without --force flag."""
mock_clf.return_value = self._make_classifier_with_ansible()
gh_file = tmp_path / "output.txt"
monkeypatch.setenv("GITHUB_OUTPUT", str(gh_file))
monkeypatch.setenv("FORCE_DEPLOY", "true")
runner = CliRunner()
result = runner.invoke(main, ["--github-output"])
assert result.exit_code == 0
content = gh_file.read_text()
assert "user-facing-changed=true" in content
assert "ansible-changed=true" in content
@patch("devx.ci.classify_changes._get_classifier")
def test_force_deploy_env_var_false(
self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""FORCE_DEPLOY=false does not activate force mode."""
mock_clf.return_value = self._make_classifier_with_ansible()
gh_file = tmp_path / "output.txt"
monkeypatch.setenv("GITHUB_OUTPUT", str(gh_file))
monkeypatch.setenv("FORCE_DEPLOY", "false")
with patch.object(classify_changes_mod, "get_latest_tag", return_value="v1.0"):
with patch.object(classify_changes_mod, "get_changed_files", return_value=[]):
runner = CliRunner()
result = runner.invoke(main, ["--github-output"])
assert result.exit_code == 0
content = gh_file.read_text()
assert "user-facing-changed=false" in content
@patch("devx.ci.classify_changes._get_classifier")
def test_force_flag_overrides_env_var(
self, mock_clf: MagicMock, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""--force flag works even when FORCE_DEPLOY=false."""
mock_clf.return_value = self._make_classifier_with_ansible()
gh_file = tmp_path / "output.txt"
monkeypatch.setenv("GITHUB_OUTPUT", str(gh_file))
monkeypatch.setenv("FORCE_DEPLOY", "false")
runner = CliRunner()
result = runner.invoke(main, ["--github-output", "--force"])
assert result.exit_code == 0
content = gh_file.read_text()
assert "user-facing-changed=true" in content
+30
View File
@@ -87,6 +87,13 @@ class TestCiCommands:
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.ci.doc_coverage", [])
@patch("devx.cli._run_module")
def test_ci_lint_docs(self, mock_run: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["ci", "lint-docs", "--", "--root", "."])
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.ci.lint_docs", ["--root", "."])
@patch("devx.cli._run_module")
def test_ci_notify_failure(self, mock_run: MagicMock) -> None:
runner = CliRunner()
@@ -166,6 +173,13 @@ class TestToolsCommands:
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.tools.generate_badges", [])
@patch("devx.cli._run_module")
def test_tools_generate_cliff_config(self, mock_run: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["tools", "generate-cliff-config"])
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.tools.generate_cliff_config", [])
@patch("devx.cli._run_module")
def test_tools_install_checkmake(self, mock_run: MagicMock) -> None:
runner = CliRunner()
@@ -218,6 +232,22 @@ class TestMoleculeCommands:
mock_run.assert_called_once_with("devx.molecule.molecule_all", [])
class TestNewCiCommands:
@patch("devx.cli._run_module")
def test_ci_distribute_files(self, mock_run: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["ci", "distribute-files", "--", "--pattern", "*.py"])
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.ci.distribute_files", ["--pattern", "*.py"])
@patch("devx.cli._run_module")
def test_ci_integration_guard(self, mock_run: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["ci", "integration-guard", "--", "-v"])
assert result.exit_code == 0
mock_run.assert_called_once_with("devx.ci.integration_guard", ["-v"])
class TestRunModule:
@patch("importlib.import_module")
def test_run_module_success(self, mock_import: MagicMock) -> None:
+106 -25
View File
@@ -1,12 +1,14 @@
"""Unit tests for config module constants."""
import importlib
from pathlib import Path
from devx.config import (
CONVENTIONAL_RE,
DEFAULT_PER_PAGE,
DEFAULT_TIMEOUT,
GITEA_API_URL,
MAX_RETRIES,
REPO_OWNER,
RETRY_BACKOFF_BASE,
RETRY_STATUS_CODES,
TASK_ID_RE,
@@ -20,25 +22,10 @@ class TestConfigConstants:
assert "api/v1" in GITEA_API_URL
assert "api/v1" in VIKUNJA_API_URL
def test_project_ids(self, monkeypatch: object) -> None:
"""VIKUNJA_PROJECT_ID defaults to 6 when DEVX_VIKUNJA_PROJECT_ID is not set."""
monkeypatch.delenv("DEVX_VIKUNJA_PROJECT_ID", raising=False)
import importlib
import devx.config as cfg
importlib.reload(cfg)
assert cfg.VIKUNJA_PROJECT_ID == 6
# Restore module state
importlib.reload(cfg)
def test_timeouts(self) -> None:
assert DEFAULT_TIMEOUT == 30
assert DEFAULT_PER_PAGE == 50
def test_owner(self) -> None:
assert REPO_OWNER == "oblachno-oss"
def test_task_prefix(self) -> None:
assert TASK_PREFIX == "DEVX"
@@ -64,28 +51,122 @@ class TestConfigConstants:
assert 503 in RETRY_STATUS_CODES
assert 504 in RETRY_STATUS_CODES
def test_env_var_override(self, monkeypatch: object) -> None:
"""Test that env vars override defaults at import time."""
# We can't easily re-import the module, but we can verify
# the constants respect env vars by checking the module source.
class TestPyprojectReading:
"""Test that config.py reads [tool.devx] from pyproject.toml."""
def test_pyproject_provides_values(self) -> None:
"""When pyproject.toml has [tool.devx], values are read from it."""
import devx.config as cfg
assert cfg.GITEA_API_URL # always non-empty
assert cfg.VIKUNJA_API_URL # always non-empty
# devx's own pyproject.toml has task_prefix=DEVX, vikunja_project_id=8
assert cfg.TASK_PREFIX == "DEVX"
assert cfg.VIKUNJA_PROJECT_ID == 8
assert cfg.REPO_OWNER == "oblachno-oss"
def test_env_overrides_pyproject(self, monkeypatch: object) -> None:
"""Env vars take priority over pyproject.toml."""
monkeypatch.setenv("DEVX_TASK_PREFIX", "CUSTOM")
import devx.config as cfg
importlib.reload(cfg)
assert cfg.TASK_PREFIX == "CUSTOM"
assert cfg.TASK_ID_RE.search("CUSTOM-42")
monkeypatch.delenv("DEVX_TASK_PREFIX", raising=False)
importlib.reload(cfg)
def test_no_pyproject_falls_back_to_defaults(self, monkeypatch: object, tmp_path: Path) -> None:
"""When no pyproject.toml exists, defaults are used."""
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("DEVX_TASK_PREFIX", raising=False)
monkeypatch.delenv("DEVX_VIKUNJA_PROJECT_ID", raising=False)
monkeypatch.delenv("DEVX_REPO_OWNER", raising=False)
import devx.config as cfg
importlib.reload(cfg)
assert cfg.TASK_PREFIX == "DEVX"
assert cfg.VIKUNJA_PROJECT_ID == 6
assert cfg.REPO_OWNER == ""
importlib.reload(cfg)
def test_invalid_toml_falls_back_to_defaults(self, monkeypatch: object, tmp_path: Path) -> None:
"""When pyproject.toml is invalid TOML, defaults are used."""
(tmp_path / "pyproject.toml").write_text("invalid toml {{{")
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("DEVX_TASK_PREFIX", raising=False)
import devx.config as cfg
importlib.reload(cfg)
assert cfg.TASK_PREFIX == "DEVX"
importlib.reload(cfg)
def test_no_devx_section_falls_back_to_defaults(self, monkeypatch: object, tmp_path: Path) -> None:
"""When pyproject.toml has no [tool.devx], defaults are used."""
(tmp_path / "pyproject.toml").write_text('[project]\nname = "test"\n')
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("DEVX_TASK_PREFIX", raising=False)
monkeypatch.delenv("DEVX_VIKUNJA_PROJECT_ID", raising=False)
import devx.config as cfg
importlib.reload(cfg)
assert cfg.TASK_PREFIX == "DEVX"
assert cfg.VIKUNJA_PROJECT_ID == 6
importlib.reload(cfg)
def test_pyproject_int_value_used(self, monkeypatch: object, tmp_path: Path) -> None:
"""When pyproject.toml has an int value, it is used (covers _get_int return)."""
(tmp_path / "pyproject.toml").write_text('[project]\nname = "test"\n[tool.devx]\nvikunja_project_id = 42\n')
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("DEVX_VIKUNJA_PROJECT_ID", raising=False)
import devx.config as cfg
importlib.reload(cfg)
assert cfg.VIKUNJA_PROJECT_ID == 42
importlib.reload(cfg)
def test_env_int_override(self, monkeypatch: object, tmp_path: Path) -> None:
"""Env var override for int config takes priority over pyproject.toml."""
(tmp_path / "pyproject.toml").write_text('[project]\nname = "test"\n[tool.devx]\nvikunja_project_id = 42\n')
monkeypatch.chdir(tmp_path)
monkeypatch.setenv("DEVX_VIKUNJA_PROJECT_ID", "99")
import devx.config as cfg
importlib.reload(cfg)
assert cfg.VIKUNJA_PROJECT_ID == 99
importlib.reload(cfg)
def test_tool_not_dict_falls_back_to_defaults(self, monkeypatch: object, tmp_path: Path) -> None:
"""When [tool] is not a dict, defaults are used."""
(tmp_path / "pyproject.toml").write_text('tool = "not a dict"\n')
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("DEVX_TASK_PREFIX", raising=False)
import devx.config as cfg
importlib.reload(cfg)
assert cfg.TASK_PREFIX == "DEVX"
importlib.reload(cfg)
def test_devx_not_dict_falls_back_to_defaults(self, monkeypatch: object, tmp_path: Path) -> None:
"""When [tool.devx] is not a dict, defaults are used."""
(tmp_path / "pyproject.toml").write_text('[tool]\ndevx = "not a dict"\n')
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("DEVX_TASK_PREFIX", raising=False)
import devx.config as cfg
importlib.reload(cfg)
assert cfg.TASK_PREFIX == "DEVX"
importlib.reload(cfg)
class TestTaskPrefixOverride:
def test_task_prefix_from_env(self, monkeypatch: object) -> None:
"""Verify TASK_PREFIX reads from DEVX_TASK_PREFIX env var."""
monkeypatch.setenv("DEVX_TASK_PREFIX", "INFRA")
import importlib
import devx.config as cfg
importlib.reload(cfg)
assert cfg.TASK_PREFIX == "INFRA"
assert cfg.TASK_ID_RE.search("INFRA-42")
assert not cfg.TASK_ID_RE.search("DEVX-42")
# Restore
monkeypatch.delenv("DEVX_TASK_PREFIX", raising=False)
importlib.reload(cfg)
+63 -11
View File
@@ -32,9 +32,10 @@ class TestDefaultConfigs:
assert config["branch_name"] == "master"
assert config["enable_push"] is True
assert config["enable_push_whitelist"] is False
assert config["required_approvals"] == 0
assert config["required_approvals"] == 1
assert isinstance(config["status_check_contexts"], list)
assert "CI / quality (pull_request)" in config["status_check_contexts"]
assert config["block_admin_merge_override"] is True
def test_default_repo_settings_config(self) -> None:
config = _default_repo_settings_config()
@@ -47,7 +48,7 @@ class TestDefaultConfigs:
class TestConfigureRepo:
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
@patch("devx.tools.configure_repo.GiteaClient")
def test_configure_repo_success(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
@@ -58,7 +59,7 @@ class TestConfigureRepo:
mock_client.ensure_branch_protection.assert_called_once()
mock_client.update_repo_settings.assert_called_once()
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
@patch("devx.tools.configure_repo.GiteaClient")
def test_configure_repo_api_error(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
@@ -69,10 +70,10 @@ class TestConfigureRepo:
configure_repo(token="tok", owner="owner", repo="repo")
def test_configure_repo_no_token(self) -> None:
with pytest.raises(click.ClickException, match="REPO_TOKEN"):
with pytest.raises(click.ClickException, match="CI_GITEA_TOKEN"):
configure_repo(token="", owner="owner", repo="repo")
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
@patch("devx.tools.configure_repo.GiteaClient")
def test_configure_repo_custom_configs(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
@@ -102,7 +103,7 @@ class TestConfigureRepo:
class TestMain:
@patch.dict("os.environ", {"REPO_TOKEN": "tok", "DEVX_REPO_NAME": "myrepo"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok", "DEVX_REPO_NAME": "myrepo"}, clear=True)
@patch("devx.tools.configure_repo.GiteaClient")
def test_main_success_with_env_repo(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
@@ -114,7 +115,7 @@ class TestMain:
mock_client.ensure_branch_protection.assert_called_once()
mock_client.update_repo_settings.assert_called_once()
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
@patch("devx.tools.configure_repo.GiteaClient")
def test_main_success_with_cli_repo(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
@@ -125,7 +126,7 @@ class TestMain:
assert result.exit_code == 0
mock_client.ensure_branch_protection.assert_called_once()
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
@patch("devx.tools.configure_repo.GiteaClient")
def test_main_api_error(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
@@ -142,16 +143,17 @@ class TestMain:
runner = CliRunner()
result = runner.invoke(main, ["--repo", "myrepo"])
assert result.exit_code != 0
assert "REPO_TOKEN" in result.output
assert "CI_GITEA_TOKEN" in result.output
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
@patch("devx.tools.configure_repo.REPO_NAME", "")
def test_main_no_repo(self) -> None:
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code != 0
assert "Repository name not specified" in result.output
@patch.dict("os.environ", {"REPO_TOKEN": "tok"}, clear=True)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
@patch("devx.tools.configure_repo.GiteaClient")
def test_main_custom_branch(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
@@ -163,3 +165,53 @@ class TestMain:
mock_client.ensure_branch_protection.assert_called_once()
args = mock_client.ensure_branch_protection.call_args
assert args[0][0] == "develop"
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok", "DEVX_REPO_NAME": "oblachno/infra"}, clear=True)
@patch("devx.tools.configure_repo.GiteaClient")
def test_main_parses_owner_repo_from_env(self, mock_client_cls: MagicMock) -> None:
"""DEVX_REPO_NAME with 'owner/repo' format should be split."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
# Verify GiteaClient was constructed with parsed owner and repo (positional)
call_args = mock_client_cls.call_args
assert call_args[0][2] == "oblachno" # owner is 3rd positional arg
assert call_args[0][3] == "infra" # repo is 4th positional arg
@patch.dict(
"os.environ",
{"CI_GITEA_TOKEN": "tok", "DEVX_REPO_NAME": "infra", "DEVX_REPO_OWNER": "oblachno"},
clear=True,
)
@patch("devx.tools.configure_repo.REPO_OWNER", "oblachno")
@patch("devx.tools.configure_repo.GiteaClient")
def test_main_no_slash_when_owner_set_separately(self, mock_client_cls: MagicMock) -> None:
"""When DEVX_REPO_OWNER is set, DEVX_REPO_NAME should not be split."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
call_args = mock_client_cls.call_args
assert call_args[0][2] == "oblachno" # owner
assert call_args[0][3] == "infra" # repo
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"}, clear=True)
@patch("devx.tools.configure_repo.REPO_NAME", "devx")
@patch("devx.tools.configure_repo.REPO_OWNER", "oblachno-oss")
@patch("devx.tools.configure_repo.GiteaClient")
def test_main_repo_from_pyproject(self, mock_client_cls: MagicMock) -> None:
"""When no env var is set, repo name should come from pyproject.toml."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
runner = CliRunner()
result = runner.invoke(main, [])
assert result.exit_code == 0
call_args = mock_client_cls.call_args
assert call_args[0][2] == "oblachno-oss" # owner
assert call_args[0][3] == "devx" # repo
+192
View File
@@ -0,0 +1,192 @@
"""Unit tests for devx.tools.create_pr."""
from unittest.mock import MagicMock, patch
import click
import pytest
from click.testing import CliRunner
from devx.tools.create_pr import (
cli,
create_pr,
extract_task_id,
find_existing_pr,
get_repo_name,
get_vikunja_task_title,
)
class TestExtractTaskId:
def test_valid(self) -> None:
assert extract_task_id("DEVX-42-fix") == "DEVX-42"
def test_invalid(self) -> None:
assert extract_task_id("feature") == ""
class TestGetRepoName:
@patch.dict("os.environ", {"DEVX_REPO_NAME": "infra"})
def test_from_env(self) -> None:
assert get_repo_name() == "infra"
@patch("devx.tools.create_pr.REPO_NAME", "devx")
@patch.dict("os.environ", {"GITHUB_REPOSITORY": "oblachno/infra"}, clear=True)
def test_env_overrides_pyproject(self) -> None:
assert get_repo_name() == "infra"
@patch("devx.tools.create_pr.REPO_NAME", "devx")
@patch.dict("os.environ", {}, clear=True)
def test_from_pyproject(self) -> None:
assert get_repo_name() == "devx"
@patch("devx.tools.create_pr.REPO_NAME", "")
@patch.dict("os.environ", {"GITHUB_REPOSITORY": "oblachno/infra"}, clear=True)
def test_from_github(self) -> None:
assert get_repo_name() == "infra"
@patch("devx.tools.create_pr.REPO_NAME", "")
@patch.dict("os.environ", {}, clear=True)
def test_missing_raises(self) -> None:
with pytest.raises(click.ClickException, match="Repository name"):
get_repo_name()
class TestGetVikunjaTaskTitle:
@patch("devx.tools.create_pr.VikunjaClient")
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"})
def test_found(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.find_task_by_identifier.return_value = {"identifier": "DEVX-42", "title": "Add feature"}
mock_client_cls.return_value = mock_client
assert get_vikunja_task_title("DEVX-42") == "Add feature"
@patch.dict("os.environ", {}, clear=True)
def test_no_token(self) -> None:
with pytest.raises(click.ClickException, match="VIKUNJA_TOKEN"):
get_vikunja_task_title("DEVX-42")
@patch("devx.tools.create_pr.VikunjaClient")
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"})
def test_not_found(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.find_task_by_identifier.return_value = None
mock_client_cls.return_value = mock_client
with pytest.raises(click.ClickException, match="Could not find"):
get_vikunja_task_title("DEVX-42")
class TestFindExistingPr:
def test_found(self) -> None:
client = MagicMock()
client.list_prs.return_value = [{"head": {"ref": "DEVX-42-fix"}, "number": 10}]
result = find_existing_pr(client, "DEVX-42-fix")
assert result is not None
assert result["number"] == 10
def test_not_found(self) -> None:
client = MagicMock()
client.list_prs.return_value = [{"head": {"ref": "other"}, "number": 10}]
result = find_existing_pr(client, "DEVX-42-fix")
assert result is None
class TestCreatePr:
@patch("devx.tools.create_pr.GiteaClient")
@patch("devx.tools.create_pr.get_vikunja_task_title", return_value="Add feature")
@patch("devx.tools.create_pr.find_existing_pr", return_value=None)
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"})
def test_creates_new_pr(self, mock_find: MagicMock, mock_title: MagicMock, mock_gitea: MagicMock) -> None:
mock_client = MagicMock()
mock_client.create_pr.return_value = {"number": 15, "html_url": "https://git.example.com/pr/15"}
mock_gitea.return_value = mock_client
result = create_pr("DEVX-42-fix", "master", "body", "owner", "repo")
assert result["number"] == 15
mock_client.create_pr.assert_called_once_with(
title="DEVX-42: Add feature",
head="DEVX-42-fix",
base="master",
body="body",
)
@patch("devx.tools.create_pr.GiteaClient")
@patch("devx.tools.create_pr.get_vikunja_task_title", return_value="Add feature")
@patch("devx.tools.create_pr.find_existing_pr")
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"})
def test_existing_pr_idempotent(self, mock_find: MagicMock, mock_title: MagicMock, mock_gitea: MagicMock) -> None:
mock_find.return_value = {"number": 10, "html_url": "https://git.example.com/pr/10"}
mock_client = MagicMock()
mock_gitea.return_value = mock_client
result = create_pr("DEVX-42-fix", "master", "", "owner", "repo")
assert result["number"] == 10
mock_client.create_pr.assert_not_called()
@patch.dict("os.environ", {}, clear=True)
def test_no_repo_token(self) -> None:
with pytest.raises(click.ClickException, match="CI_GITEA_TOKEN"):
create_pr("DEVX-42-fix", "master", "", "owner", "repo")
@patch.dict("os.environ", {"CI_GITEA_TOKEN": "tok"})
def test_no_task_id_in_branch(self) -> None:
with pytest.raises(click.ClickException, match="does not contain a task ID"):
create_pr("feature-branch", "master", "", "owner", "repo")
class TestCli:
@patch("devx.tools.create_pr.create_pr")
@patch("devx.tools.create_pr.subprocess.run")
@patch("devx.tools.create_pr.REPO_OWNER", "owner")
@patch("devx.tools.create_pr.get_repo_name", return_value="repo")
def test_auto_detect_branch(self, mock_repo: MagicMock, mock_run: MagicMock, mock_create: MagicMock) -> None:
mock_run.return_value = MagicMock(stdout="DEVX-42-fix\n", returncode=0)
mock_create.return_value = {"number": 1}
runner = CliRunner()
result = runner.invoke(cli, [])
assert result.exit_code == 0
mock_create.assert_called_once_with("DEVX-42-fix", "master", "", "owner", "repo")
@patch("devx.tools.create_pr.create_pr")
@patch("devx.tools.create_pr.REPO_OWNER", "owner")
@patch("devx.tools.create_pr.get_repo_name", return_value="repo")
def test_explicit_branch(self, mock_repo: MagicMock, mock_create: MagicMock) -> None:
mock_create.return_value = {"number": 1}
runner = CliRunner()
result = runner.invoke(cli, ["--branch", "DEVX-42-fix"])
assert result.exit_code == 0
@patch("devx.tools.create_pr.create_pr")
@patch("devx.tools.create_pr.REPO_OWNER", "owner")
@patch("devx.tools.create_pr.get_repo_name", return_value="repo")
def test_body_from_stdin(self, mock_repo: MagicMock, mock_create: MagicMock) -> None:
mock_create.return_value = {"number": 1}
runner = CliRunner()
result = runner.invoke(cli, ["--branch", "DEVX-42-fix", "--body", "-"], input="PR body text")
assert result.exit_code == 0
mock_create.assert_called_once()
assert mock_create.call_args.args[2] == "PR body text"
@patch("devx.tools.create_pr.REPO_OWNER", "")
@patch("devx.tools.create_pr.get_repo_name", return_value="repo")
def test_missing_owner(self, mock_repo: MagicMock) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["--branch", "DEVX-42-fix"])
assert result.exit_code != 0
assert "owner" in result.output.lower()
@patch("devx.tools.create_pr.create_pr")
@patch("devx.tools.create_pr.get_repo_name", return_value="repo")
def test_explicit_owner(self, mock_repo: MagicMock, mock_create: MagicMock) -> None:
mock_create.return_value = {"number": 1}
runner = CliRunner()
result = runner.invoke(cli, ["--branch", "DEVX-42-fix", "--owner", "custom"])
assert result.exit_code == 0
mock_create.assert_called_once_with("DEVX-42-fix", "master", "", "custom", "repo")
@patch("devx.tools.create_pr.subprocess.run")
@patch("devx.tools.create_pr.REPO_OWNER", "owner")
@patch("devx.tools.create_pr.get_repo_name", return_value="repo")
def test_git_detect_failure(self, mock_repo: MagicMock, mock_run: MagicMock) -> None:
mock_run.return_value = MagicMock(stdout="", stderr="fatal: not a git repository", returncode=128)
runner = CliRunner()
result = runner.invoke(cli, [])
assert result.exit_code != 0
assert "Could not detect" in result.output
+83
View File
@@ -0,0 +1,83 @@
"""Unit tests for devx.tools.create_task."""
from unittest.mock import MagicMock, patch
from click.testing import CliRunner
from devx.tools.create_task import cli
class TestCreateTaskCli:
@patch("devx.tools.create_task.VikunjaClient")
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"})
def test_success(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.create_task.return_value = {"identifier": "DEVX-60", "id": 60}
mock_client_cls.return_value = mock_client
runner = CliRunner()
result = runner.invoke(cli, ["--title", "Add feature X"])
assert result.exit_code == 0
assert "DEVX-60" in result.output
mock_client.create_task.assert_called_once()
@patch.dict("os.environ", {}, clear=True)
def test_missing_token(self) -> None:
runner = CliRunner()
result = runner.invoke(cli, ["--title", "Add feature X"])
assert result.exit_code != 0
assert "VIKUNJA_TOKEN" in result.output
@patch("devx.tools.create_task.VikunjaClient")
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"})
def test_with_description(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.create_task.return_value = {"identifier": "DEVX-61", "id": 61}
mock_client_cls.return_value = mock_client
runner = CliRunner()
result = runner.invoke(
cli,
["--title", "Add feature Y", "--description", "<p>desc</p>"],
)
assert result.exit_code == 0
call_args = mock_client.create_task.call_args
assert call_args.args[1] == "Add feature Y"
assert call_args.args[2] == "<p>desc</p>"
@patch("devx.tools.create_task.VikunjaClient")
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"})
def test_description_from_stdin(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.create_task.return_value = {"identifier": "DEVX-62", "id": 62}
mock_client_cls.return_value = mock_client
runner = CliRunner()
result = runner.invoke(
cli,
["--title", "Add feature Z", "--description", "-"],
input="<p>stdin desc</p>",
)
assert result.exit_code == 0
mock_client.create_task.assert_called_once()
call_args = mock_client.create_task.call_args
assert call_args.args[2] == "<p>stdin desc</p>"
@patch("devx.tools.create_task.VikunjaClient")
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"})
def test_custom_project_id(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.create_task.return_value = {"identifier": "GRM-10", "id": 10}
mock_client_cls.return_value = mock_client
runner = CliRunner()
result = runner.invoke(cli, ["--title", "Task", "--project-id", "3"])
assert result.exit_code == 0
mock_client.create_task.assert_called_once_with(3, "Task", "")
@patch("devx.tools.create_task.VikunjaClient")
@patch.dict("os.environ", {"VIKUNJA_TOKEN": "tok"})
def test_no_identifier_in_response(self, mock_client_cls: MagicMock) -> None:
mock_client = MagicMock()
mock_client.create_task.return_value = {"id": 99}
mock_client_cls.return_value = mock_client
runner = CliRunner()
result = runner.invoke(cli, ["--title", "Task"])
assert result.exit_code == 0
assert "id=99" in result.output

Some files were not shown because too many files have changed in this diff Show More