Kayobe container image build may fail if collections or roles need updating

[markgoddard]

Steps to reproduce

• checkout kayobe-config repo
• add some roles or collections to etc/kayobe/ansible/requirements.yml
• run kayobe control host bootstrap
• modify role or collection version requirements in requirements.yml
• build kayobe container image

Expected

Image builds successfully

Actual

Image fails to build. The command that fails is kayobe control host bootstrap. It fails because the roles or collections in etc/kayobe/ansible/{roles,collections} are of an incompatible version.

Workaround

Remove the roles and collections before building the image.

I also used this change to the Dockerfile which worked, but would flatten any non-galaxy roles/collections:

diff --git a/docker/kayobe/Dockerfile b/docker/kayobe/Dockerfile
index 786a541..e0c83fc 100644
--- a/docker/kayobe/Dockerfile
+++ b/docker/kayobe/Dockerfile
@@ -73,6 +73,7 @@ RUN --mount=type=ssh,uid=1000 --mount=type=tmpfs,target=/tmp/src --mount=type=bi
   sudo chown -Rf stack:stack /tmp/src && \
   # Strip the secrets so that we don't need to pass in a vault-password
   grep -lR "\$ANSIBLE_VAULT" /tmp/src | xargs rm -f && \
+  (rm -rf /tmp/src/etc/kayobe/ansible/roles/* /tmp/src/etc/kayobe/ansible/collections/* || true) && \
   bash /tmp/src/.automation/utils/kayobe-automation-install && \
   (rm -f /stack/.ssh/{id_rsa,id_rsa.pub} || true) && \
   (mkdir /stack/.ansible || true) && \

[jovial]

Interesting issue. I did look at the differences between kayobe control host bootstrap and kayobe control host upgrade and the differences seem to be:

• upgrade uses ansible-galaxy force install
• bootstrap generates the openrc files

Could we potentially add a build ARG to switch the behaviour to use upgrade instead of bootstrap? It is a little concerning that we'd make destructive modifications to a users git checkout by default (if we switched to upgrade by default). It can also be quite useful to not force delete the roles if you are testing some changes. What do you reckon?

[markgoddard]

It's not necessarily just a build time issue - we also run bootstrap when running pipelines.