oblachno-oss/devx
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

feat: extract reusable dev/CI tools from GRM into devx package

Browse Source 
Port core modules (config, exceptions, i18n, api_clients, gitea_cli),
14 CI scripts, 6 dev tools, 5 molecule tools, CLI entry point, workflows,
Makefile, tests (784 tests, 100% coverage), and documentation from GRM.

The devx package is published to the Gitea PyPI registry and consumed
by GRM, infra, and other oblachno-oss projects as a pip dependency.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
emil
2026-06-22 17:01:20 +02:00
co-authored by Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
commit 60fd11419c
86 changed files with 15947 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+306
View File
@@ -0,0 +1,306 @@
# AGENTS.md — Project Conventions for devx
## Build & Test Commands
```bash
make setup # Create venv, install deps, set up hooks, install CI tools
make install-tools # Install actionlint, git-cliff, act_runner to ~/.local/bin
make lint-all # ruff + pyright + bandit + actionlint
make pytest-cov # Unit tests with 100% coverage enforcement
make test-unit # Unit tests without coverage
make workflow-lint # Static lint of .gitea/workflows/*.yml (actionlint)
make workflow-dryrun # Dry-run all workflows in Docker (act_runner exec --dryrun)
make workflow-check # workflow-lint + workflow-dryrun
make clean # Remove caches, build artifacts, coverage data
```
`make setup` automatically installs all development tools:
- **Python deps** via `python -m devx.tools.setup` (pip install -e .[dev], pre-commit hooks)
- **actionlint, git-cliff, act_runner, tea** via `python -m devx.tools.install_tools` (CI/CD tools to ~/.local/bin)
- **tea CLI login** via `python -m devx.tools.setup` (configures `tea login` from `.env` `REPO_TOKEN`)
## Workflow Verification (Before Push)
Workflow YAML files (`.gitea/workflows/*.yml`) are verified with two tools:
1. **actionlint** — Static linter that catches syntax errors, invalid
expressions, unknown keys, type mismatches, and shellcheck issues.
Config: `.gitea/actionlint.yaml` (registers custom `docker` runner label).
Installed automatically by `make setup` via `python -m devx.tools.install_tools`.
2. **act_runner exec --dryrun** — Gitea's own runner in dry-run mode.
Validates job dependencies, step ordering, and Docker image selection
without starting containers. Installed automatically by `make setup`.
Both run via `make workflow-check` and are part of `make lint-all`.
The pre-commit hook runs actionlint automatically when workflow files change.
The CI `quality` job runs `make setup-quality` then `make lint-all`.
CI also runs a best-effort `make workflow-dryrun` step (skipped if act_runner is not installed in the CI Docker image).
## Architecture
devx is a reusable Python package providing development and CI/CD tools for oblachno-oss projects.
### Package Structure
```
src/devx/
├── __init__.py # Version (single source of truth, read by setuptools)
├── cli.py # Click-based CLI entry point (devx command)
├── config.py # Configuration system (DEVX_ env vars, .env loading)
├── api_clients.py # GiteaClient, VikunjaClient — HTTP API wrappers
├── gitea_cli.py # TeaCLI — wrapper around tea CLI with JSON parsing
├── i18n.py # Translation system (gettext-based, translations.json)
├── exceptions.py # Custom exception types
├── translations.json # Translation strings (en, bg)
├── ci/ # CI/CD automation modules (run by workflows)
│ ├── release.py # Automated versioning, tagging, changelog
│ ├── publish.py # Build and publish to Gitea PyPI registry
│ ├── auto_merge.py # Squash-merge PRs with task ID validation
│ ├── classify_changes.py # User-facing vs workflow-only change detection
│ ├── detect_release_commit.py # Detect release commits on master
│ ├── validate_commit_msg.py # Conventional commit validation
│ ├── pr_review.py # Automated PR review
│ ├── post_merge.py # Vikunja task updates after merge
│ ├── sync_wiki.py # Sync documentation to Gitea wiki
│ ├── push_badges.py # Generate and push quality badges
│ ├── notify_failure.py # Create Gitea issues on CI failures
│ ├── check_translations.py # Translation completeness check
│ └── doc_coverage.py # Documentation coverage check
├── tools/ # Developer tooling modules (run locally or by CI)
│ ├── setup.py # Environment setup (venv, deps, hooks)
│ ├── install_tools.py # Install actionlint, git-cliff, act_runner, tea
│ ├── check_test_speed.py # Measure unit test execution time
│ ├── configure_repo.py # Branch protection and label setup
│ └── generate_badges.py # Badge SVG generation
└── molecule/ # Optional molecule testing helpers (for Ansible projects)
```
### Key Design Principles
- **Self-contained package** — `src/devx/` never imports from scripts outside the package
- **Module-based invocation** — All tools invoked via `python -m devx.ci.*` or `python -m devx.tools.*`
- **PYTHONPATH: src** — Workflows set `PYTHONPATH: src` (NOT `.:src` since there are no scripts at repo root)
- **Config via env vars** — `DEVX_*` environment variables with `.env` file fallback
## PR Workflow (Mandatory)
Every change to master goes through this workflow. No exceptions.
### Branch Protection (Required Gitea Settings)
Branch protection and labels are automatically configured by
`python -m devx.tools.configure_repo`, which runs as a `configure-repo` job in
the post-merge workflow on every push to master.
The following rules are enforced for `master`:
- **Require pull request**: No direct pushes to master
- **Require approval review**: At least 1 `APPROVE` review before merge
- **Require status checks**: CI quality must pass
- **Block force pushes**: No history rewriting on master
### 1. Create Vikunja Task
Create a task in Vikunja to get a `DEVX-N` identifier.
### 2. Create Branch
```bash
git checkout master && git pull
git checkout -b DEVX-N-short-description
```
### 3. Implement Changes
- Write code following conventions below
- Write/update tests (100% coverage required)
- Update documentation (CHANGELOG, README, AGENTS.md as needed)
### 4. Commit (Conventional Commits)
Branch commits use conventional commit format (no `DEVX-N:` prefix):
```
feat: add new feature
fix: resolve bug
docs: update README
```
### 5. Push and Create PR
- **PR title format**: `DEVX-N: <vikunja task title>` (must match the Vikunja task title exactly)
- PR body: summary of changes, `Closes DEVX-N`
- Add `ready-to-merge` label **only after review is complete**
### 6. Review the PR
**Automated review (CI `pr-review` job):** Every PR triggers an automated
review via `python -m devx.ci.pr_review`. This job posts a review with
`COMMENT` (no issues) or `REQUEST_CHANGES` (issues found):
- Architecture compliance (no subprocess in CLI, no hardcoded URLs)
- Best practices (no `print()`, no bare `except`, no `TODO`/`FIXME`,
no functions > 50 lines)
- Security (no hardcoded secrets, no `shell=True`, no `eval`/`exec`)
- i18n (no raw strings in `click.echo()` without `_()` wrapper)
- Resource management (no `open()` without `with`, no `Popen()` without cleanup)
- Documentation (source changes must include doc updates)
- Test coverage (source changes must include test updates)
- Commit conventions (conventional commit format on PR commits)
The agent **must** address all `REQUEST_CHANGES` issues before proceeding.
### 7. Address Review Comments
Fix each comment one by one, commit, and push. Re-review until satisfied.
### 8. Approve and Merge
Once all checklist items are verified and comments are addressed, approve
the PR. Then add the `ready-to-merge` label. The auto-merge workflow will:
1. **Validate** PR title format (`DEVX-N: <vikunja task title>`) and match against Vikunja task title
2. **Check** that at least one substantive APPROVE review exists
3. Wait for all CI checks to pass (including the `pr-review` job)
4. Squash-merge with title: `DEVX-N <conventional commit message>` (space-separated, no colon after DEVX-N)
5. The post-merge workflow marks the Vikunja task as done
6. The release workflow automatically versions, tags, and publishes
> **IMPORTANT**: Never manually merge PRs via the API. Always use the auto-merge
> workflow by adding the `ready-to-merge` label.
### Automated Release Pipeline
After a PR is merged to master, the **post-merge workflow**
(`.gitea/workflows/post-merge.yml`) runs automatically:
1. **detect-type** — Checks if the commit is a regular merge or a
release commit (`release: vX.Y.Z`). All subsequent jobs skip for
release commits.
2. **release** — Runs `python -m devx.ci.release` which:
- Checks for user-facing changes via `python -m devx.ci.classify_changes`
- Uses **git-cliff** to calculate the next semver version from conventional commits
- Updates `__version__` in `src/devx/__init__.py` (single source of truth)
- Updates `CHANGELOG.md` with the new version section
- Runs `make lint-ruff` and `make pytest-cov` to verify the release is healthy
- Commits with `release: vX.Y.Z [skip ci]` prefix
- Creates an annotated tag `vX.Y.Z` on the release commit
- Pushes both the commit and tag to master
3. **sync-wiki** — Syncs documentation to the Gitea wiki.
4. **badges** — Generates and pushes quality badge SVGs to the `badges` branch.
5. **vikunja** — Marks the corresponding Vikunja task as done.
The tag push triggers the **publish workflow** (`.gitea/workflows/publish.yml`)
which builds and publishes the package to the Gitea PyPI registry.
### Smart CI: User-Facing vs Workflow-Only Changes
Not all changes require a new release. The project classifies changes
using `python -m devx.ci.classify_changes`:
**Workflow-only paths** (infrastructure → no release needed):
- `.gitea/**` — Gitea Actions workflows
- `tests/**` — Test files
- `AGENTS.md`, `README.md`, `CHANGELOG.md` — Project docs
- `Makefile`, `cliff.toml`, `.pre-commit-config.yaml` — Config
- `.env.example`, `.gitignore` — Config
- `hooks/**` — Git hooks
**User-facing paths** (tool changes → release needed) — everything else:
- `src/devx/**` — Python package source (except `__init__.py`)
- `pyproject.toml` — Package metadata
- Any new file type not in the allowlist
## Script Separation and Import Rules
### Directory Layout
| Directory | Purpose | Release impact |
|-----------|---------|----------------|
| `src/devx/` | User-facing devx package | Changes trigger release |
| `src/devx/ci/` | CI/CD automation (run by workflows) | Part of package |
| `src/devx/tools/` | Developer tooling (run locally or by CI) | Part of package |
| `tests/` | Test files | Workflow-only (no release) |
### Import Rules
1. **`src/devx/` is self-contained** — the package never imports from outside `src/`
2. **CI modules** (`devx.ci.*`) may import from `devx.api_clients`, `devx.config`, `devx.gitea_cli`, `devx.i18n`
3. **Tool modules** (`devx.tools.*`) may import from `devx.api_clients`, `devx.config`, `devx.gitea_cli`
4. **Cross-module imports** within `devx.ci.*` or `devx.tools.*` are allowed but must be documented
### PYTHONPATH Configuration
All workflows use `PYTHONPATH: src` — devx has no scripts at the repo root,
so `.:src` is not needed. The `src` directory is the sole import root.
```yaml
- name: Run script
env:
PYTHONPATH: src
run: python -m devx.ci.example
```
### tea CLI Integration
The `tea` Gitea CLI tool is used for Gitea API interactions. It is installed
by `python -m devx.tools.install_tools` and configured by
`python -m devx.tools.setup` (login profile from `.env` `REPO_TOKEN`).
**`devx.gitea_cli.TeaCLI`** — Python wrapper around `tea` CLI with JSON output parsing:
- `create_issue()` — Create issues with labels
- `list_labels()` / `create_label()` / `add_label()` — Label management
- `create_pr()` / `merge_pr()` / `review_pr()` — Pull request operations
- `create_release()` / `list_releases()` — Release management
### git-cliff Commit Preprocessing
Merge commits on master have the format `DEVX-N <conventional commit>`. The
`cliff.toml` includes a `commit_preprocessors` entry that strips the `DEVX-N `
prefix before parsing. This ensures all merged work appears in the changelog.
### Version Bumping Rules (git-cliff)
| Commit type | Version bump |
|-------------|-------------|
| `feat:` | minor (0.X.0) |
| `fix:` | patch (0.0.X) |
| `feat!:` or `BREAKING CHANGE` | minor (pre-1.0) |
| `chore:`, `ci:`, `docs:` | no bump (excluded by cliff.toml) |
The version source is `__version__` in `src/devx/__init__.py`, read by
setuptools via `dynamic = ["version"]` in `pyproject.toml`.
### Title Format Summary
| What | Format | Example |
|------|--------|---------|
| Branch name | `DEVX-N-short-description` | `DEVX-12-add-release-script` |
| Branch commits | `<conventional commit>` | `feat: add release script` |
| PR title | `DEVX-N: <vikunja task title>` | `DEVX-12: Add release automation` |
| Merge commit | `DEVX-N <conventional commit>` | `DEVX-12 feat: add release script` |
## Config System
devx uses environment variables with `.env` file fallback for configuration.
### DEVX_ Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `DEVX_GITEA_API_URL` | `https://git.oblachno.oblachno.fyi/api/v1` | Gitea API base URL |
| `DEVX_VIKUNJA_API_URL` | `https://work.oblachno.oblachno.fyi/api/v1` | Vikunja API base URL |
| `DEVX_LANG` | `en` | Language for i18n (en, bg) |
| `REPO_TOKEN` | (from .env) | Gitea API token |
| `VIKUNJA_TOKEN` | (from .env) | Vikunja API token |
### Per-Project Overrides
Projects using devx can override the default API URLs and language by setting
`DEVX_*` environment variables or entries in their `.env` file. The config
system loads `.env` automatically via `python-dotenv`.
## Key Conventions
- Python 3.12+ required (ruff/pyright target `py312`)
- 100% test coverage required (`--cov-fail-under=100`)
- Conventional commits on feature branches (no `DEVX-N:` prefix)
- Branch names must include `DEVX-N` task ID
- Line length: 120 chars
- Secrets are passed via environment variables, never on the command line
- All user-facing strings wrapped in `_()` for i18n