Docs: `policy_check` is not a command, but must be allowlisted for it to work

[ribejara-te]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

We are using Atlantis with --gh-team-allowlist='*:plan, *:unlock, sre:apply', and we wanted to enable conftest policy checking, so we configured it and added *:approve_policies to our team allowlist so that policy owners are allowed to run the command (everyone is, but if they're not a policy owner they'll get rejected later down the line).

In our testing, this worked on autoplans, but not on atlantis plan commands. Autoplans would run policy checks, but plan commands wouldn't.

Reproduction Steps

1. Run Atlantis with conftest policy checking enabled and with --gh-team-allowlist='*:plan, *:unlock, *:approve_policies, *:apply'.
2. Raise a PR to trigger an autoplan, it will plan and run the policy checks.
3. Now commant atlantis plan, it will plan but it will NOT run the policy checks.

Solution

After digging through the code, we found this bit over here which makes sure that whoever is running the plan command, is also allowed to run the policy_check command, which is not a command per se in documentation, but is treated as such for the purposes of allowlist evaluation.

And sure thing, we added *:policy_check to our allowlist, and now policy checks always run, as expected.

As far as I could tell, this is not documented anywhere, and given that policy_check is not a command, it's pretty unintuitive that it has to be allowlisted for it to work. This is extra confusing because autoplans do work (autoplans don't have a user associated with it, and thus always pass allowlist evaluation even for the policy_check command).