Skip to content

⚙️ FEATURE: CI/CD integration — structured exit codes, JSON output, and GitHub Actions step #53

Description

@FernandoCelmer

Problem

pycodeloop run always exits with code 0, even when the agent reports an error, hits max turns, or regressions are detected (#38). This makes it impossible to use pycodeloop reliably in CI pipelines — a failed agent run looks identical to a successful one from the shell's perspective. The "Code as Agent Harness" literature identifies CI/CD pipeline integration as a core harness deployment pattern.

Expected behavior

Exit codes

Code Meaning
0 Agent completed successfully, verification passed
1 Agent error (exception, max turns reached)
2 Regression detected (baseline check failed post-run)
3 Budget exceeded

Machine-readable output

pycodeloop run "fix the auth bug" --ci --output json
{
  "status": "success",
  "turns": 4,
  "tokens": {"input": 12400, "output": 2100},
  "cost_usd": 0.042,
  "regression": false,
  "files_modified": ["src/auth.py", "tests/test_auth.py"]
}

GitHub Actions step

- name: Run pycodeloop agent
  uses: dotflow-io/pycodeloop-action@v1
  with:
    prompt: "Fix failing tests in src/auth.py"
    provider: claude
    model: claude-sonnet-4-6
    check_cmd: "pytest tests/ -q"
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

Suggested implementation

  • Add --ci flag to pycodeloop run CLI command
  • Map agent outcomes to exit codes in cli/commands/run.py
  • Add --output [text|json] flag; JSON mode serializes RunResult dataclass
  • Add files_modified field to RunResult by reading from FileAccessLog
  • Create dotflow-io/pycodeloop-action repository with composite action wrapping pip install pycodeloop && pycodeloop run

References

  • "Code as Agent Harness" — arXiv 2605.18747, Section 5 (Deployment patterns)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestin progressIssue being actively worked on

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions