Post build and test results to pull requests

Moving build, test, and deploy pipelines out of GitHub Actions YAML and into Atmos custom commands removes a lot of glue, but one piece of it always came back. To tell the pull request what happened, the script had to shell out to gh, append to $GITHUB_ENV by hand, and print workflow commands for annotations. None of that works on a laptop, so the reporting half of a pipeline could only be debugged by pushing a commit and waiting for a runner.

Scripts written in the Atmos Automation Language now have a ci module that reports to the CI provider with one call each, and renders the same calls on your terminal when there is no provider.

Reporting was the part you could not run locally

Building, testing, and validating policy are easy to run on your own machine. The last lines of a pipeline step are the problem: the pull request comment, the job summary, the step output, the commit status. They depend on files and tokens that only exist inside a runner, so they stay untested until the pipeline runs for real.

One module, the same script everywhere

From any script (standalone, custom command, workflow, or hook step) you can post a pull request comment, write a job summary, set step outputs, export environment variables and PATH entries for later steps, mask a value in logs, emit inline annotations, create and update a commit status, open a collapsible log group, and upload a SARIF report. The run's context is available through ci.context: provider, event, commit SHA, branch, repository, run URL, and pull request.

With no CI provider detected, every call renders locally instead. A comment becomes a preview, a summary renders as Markdown on the terminal, and outputs print as name=value. You develop and debug the reporting on your laptop, and the CI run executes the same code.

In CI, each call honors the same ci configuration switches as the built-in reporting that Atmos already does for Terraform. Comments and checks are opt-in. A call whose switch is off does not fail silently: it warns, names the switch that is off, and renders locally. When ci.enabled itself is off, that is the switch the warning names, rather than the flag for the individual feature.

The GitHub provider honors GITHUB_API_URL, so the calls work against GitHub Enterprise Server, and ATMOS_CI_GITHUB_API_URL overrides it. A malformed URL fails the command instead of sending your token to an unintended host.

On a CI platform Atmos does not recognize, pass --ci or set ATMOS_CI=true. That forced CI mode makes the generic provider stand in for a real one: the same switches apply, ci.context.local is False, and files named by ATMOS_CI_OUTPUT, ATMOS_CI_SUMMARY, ATMOS_CI_ENV, and ATMOS_CI_PATH receive the outputs, summary, environment exports, and PATH additions. The generic provider reads the run's context from variables such as ATMOS_CI_PR, ATMOS_CI_EVENT, ATMOS_CI_RUN_ID, and ATMOS_CI_RUN_URL.

How to Use It

This custom command runs tests and OPA policy validation, then reports one result to the job summary and to a single pull request comment that later runs update in place:

commands:
  - name: app test
    arguments: [{ name: component, required: true }]
    flags: [{ name: stack, shorthand: s, required: true }]
    steps:
      - name: test
        type: script
        interpreter: starlark
        script: |
          component, stack = ctx.arguments["component"], ctx.flags["stack"]
          tests = ci.group("test", lambda: exec.run(["go", "test", "./..."], check = False, output = "capture"))
          policy = atmos.validate("component", component, flags = {"stack": stack}, check = False, output = "capture")
          ok = tests.exit_code == 0 and policy.exit_code == 0
          body = "## Test: `" + component + "` in `" + stack + "`\n\n" + \
                 "| Check | Result |\n|---|---|\n" + \
                 "| Tests | " + ("✅" if tests.exit_code == 0 else "❌") + " |\n" + \
                 "| OPA policy | " + ("✅" if policy.exit_code == 0 else "❌") + " |\n"
          ci.summary(body)
          ci.comment(body, key = "app-test:" + component + ":" + stack)
          ci.output("tested", "true" if ok else "false")
          if not ok:
              fail("tests or policy failed")

Run it the same way in CI and on a laptop:

atmos app test api --stack=dev

On GitHub Actions, give the job pull-requests: write and set ci.enabled: true and ci.comments.enabled: true. The ci reference covers every call, the gating rules, and the local rendering.

Comments when there is no pull request

The comment above would have nowhere to go on a push to a branch. By default, ci.comment posts to the pull request when the run has one and otherwise leaves a comment on the commit, so the same script reports on both kinds of run. Pass target = "pr" when a missing pull request should be an error, or target = "commit" to always comment on the commit. Commit comments need the contents: write permission.

Templates, image comments, and fork pull requests

Reports that several scripts share do not have to be assembled from strings. Pass template="plan.md" and data={...} to ci.summary or ci.comment and Atmos renders the file from ci.templates.base_path with the same template engine the built-in plan summaries use. The call always names its template: a script has no configured default, so changing ci.summary.template for your Terraform summaries never changes what a script renders. A misspelled key in the data fails the call and names the key, and a template that cannot be found reports the path it looked in.

Container builds and pushes already wrote an image summary to the job. They now go through the same path. With ci.comments.enabled on, each image repository gets a comment on the pull request when one is known, or on the commit otherwise. Later builds update the comment for that image in the same pull request or commit, regardless of tag. Push runs that post commit comments need contents: write permission. Customize the Markdown with ci.templates.container.image.

The hold under pull_request_target and workflow_run applies only when the pull request comes from a fork. For a fork pull request, Atmos holds comments, commit statuses, environment and PATH exports, and SARIF uploads, whether they come from a script or from the native Terraform plan comments. Summaries, outputs, annotations, and masks only touch the current job, so they still run. A pull request from the same repository posts normally, a deleted fork counts as a fork, and a script can read ci.context.pr.fork to decide for itself. Setting ci.allow_unsafe_fork_execution (or ATMOS_ALLOW_UNSAFE_FORK_EXECUTION) releases both this hold and the clone gate of atmos git clone.

Get Involved

Try moving the reporting lines of one pipeline step into a script, and open an issue if a report you need is missing.