Reduce incremental Docker build times #27998
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -111,3 +111,7 @@ prime/ | |
|
|
||
| # Manpage | ||
| /man | ||
|
|
||
| Dockerfile | ||
| .dockerignore | ||
| .github/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -78,7 +78,6 @@ jobs: | |
|
|
||
| docker: | ||
| - "Dockerfile" | ||
| - "docker/**" | ||
| - "Makefile" | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,40 @@ | ||
| # Build stage | ||
| FROM docker.io/library/node:20-alpine3.18 AS build-frontend | ||
|
|
||
| ARG GITEA_VERSION | ||
|
|
||
| # Build deps | ||
| RUN apk --no-cache add \ | ||
| build-base \ | ||
| git \ | ||
| && rm -rf /var/cache/apk/* | ||
|
|
||
| # Setup repo | ||
| WORKDIR /usr/src/code.gitea.io/gitea | ||
|
|
||
| COPY Makefile . | ||
|
|
||
| # Download NPM Packages | ||
| COPY package.json . | ||
| COPY package-lock.json . | ||
|
|
||
| RUN make deps-frontend | ||
|
|
||
| # Copy source files | ||
| COPY ./webpack.config.js . | ||
| COPY ./assets ./assets | ||
| COPY ./public ./public | ||
| COPY ./web_src ./web_src | ||
|
|
||
| # Checkout version if set | ||
| COPY ./.git ./.git | ||
| RUN if [ -n "${GITEA_VERSION}" ]; then git checkout "${GITEA_VERSION}"; fi | ||
|
|
||
| # Build frontend | ||
| RUN make clean-all frontend | ||
|
|
||
| # Build stage | ||
| FROM docker.io/library/golang:1.21-alpine3.18 AS build-backend | ||
|
|
||
| ARG GOPROXY | ||
| ENV GOPROXY ${GOPROXY:-direct} | ||
|
|
@@ -13,64 +48,126 @@ ARG CGO_EXTRA_CFLAGS | |
| RUN apk --no-cache add \ | ||
| build-base \ | ||
| git \ | ||
| && rm -rf /var/cache/apk/* | ||
|
|
||
| # Setup repo | ||
| WORKDIR ${GOPATH}/src/code.gitea.io/gitea | ||
|
|
||
| COPY Makefile . | ||
|
|
||
| # Download Golang Modules | ||
| COPY go.mod . | ||
| COPY go.sum . | ||
|
|
||
| RUN make deps-backend | ||
|
|
||
| # Copy source files | ||
| COPY ./build ./build | ||
| COPY ./cmd ./cmd | ||
| COPY ./models ./models | ||
| COPY ./modules ./modules | ||
| COPY ./options ./options | ||
| COPY ./routers ./routers | ||
| COPY ./services ./services | ||
| COPY ./templates ./templates | ||
| COPY ./build.go . | ||
| COPY ./main.go . | ||
|
|
||
| # Checkout version if set | ||
| COPY ./.git ./.git | ||
| RUN if [ -n "${GITEA_VERSION}" ]; then git checkout "${GITEA_VERSION}"; fi | ||
|
|
||
| # Clean directory | ||
| RUN make clean-all | ||
|
|
||
| # Copy frontend build artifacts | ||
| COPY --from=build-frontend /usr/src/code.gitea.io/gitea/public ./public | ||
|
|
||
| # Build backend | ||
| RUN make backend | ||
|
|
||
| # Begin env-to-ini build | ||
| COPY contrib/environment-to-ini/environment-to-ini.go contrib/environment-to-ini/environment-to-ini.go | ||
| COPY ./custom ./custom | ||
|
|
||
| RUN go build contrib/environment-to-ini/environment-to-ini.go | ||
|
|
||
| FROM docker.io/library/alpine:3.18 AS gitea-base | ||
| LABEL maintainer="[email protected]" | ||
|
|
||
| RUN apk --no-cache add \ | ||
| bash \ | ||
| ca-certificates \ | ||
| gettext \ | ||
| git \ | ||
| curl \ | ||
| gnupg \ | ||
| && rm -rf /var/cache/apk/* | ||
|
|
||
| RUN addgroup -S -g 1000 git | ||
|
|
||
| COPY --chmod=644 ./contrib/autocompletion/bash_autocomplete /etc/profile.d/gitea_bash_autocomplete.sh | ||
| COPY --chmod=755 --from=build-backend /go/src/code.gitea.io/gitea/gitea /app/gitea/gitea | ||
| COPY --chmod=755 --from=build-backend /go/src/code.gitea.io/gitea/environment-to-ini /usr/local/bin/environment-to-ini | ||
|
|
||
| FROM gitea-base AS gitea-rootless | ||
|
There was a problem hiding this comment. How does this build work with the two tags in the same file? Would There was a problem hiding this comment. You can see in the docker-dryrun workflow, that the first build-push-action build gitea-base and gitea. Then in the next build-push-action you can see the gitea-base layers have all been cached and only the gitea-rootless layers are run. If not specifying the target when building, docker build will export the final image "as the one to run". |
||
|
|
||
| EXPOSE 2222 3000 | ||
|
|
||
| RUN apk --no-cache add \ | ||
| dumb-init \ | ||
| && rm -rf /var/cache/apk/* | ||
|
|
||
| RUN adduser \ | ||
| -S -H -D \ | ||
| -h /var/lib/gitea/git \ | ||
| -s /bin/bash \ | ||
| -u 1000 \ | ||
| -G git \ | ||
| git | ||
|
|
||
| RUN mkdir -p /var/lib/gitea /etc/gitea | ||
| RUN chown git:git /var/lib/gitea /etc/gitea | ||
|
|
||
| # Copy local files | ||
| COPY --chmod=755 docker/rootless / | ||
|
|
||
| # git:git | ||
| USER 1000:1000 | ||
| ENV GITEA_WORK_DIR /var/lib/gitea | ||
| ENV GITEA_CUSTOM /var/lib/gitea/custom | ||
| ENV GITEA_TEMP /tmp/gitea | ||
| ENV TMPDIR /tmp/gitea | ||
|
|
||
| # TODO add to docs the ability to define the ini to load (useful to test and revert a config) | ||
| ENV GITEA_APP_INI /etc/gitea/app.ini | ||
| ENV HOME "/var/lib/gitea/git" | ||
| VOLUME ["/var/lib/gitea", "/etc/gitea"] | ||
| WORKDIR /var/lib/gitea | ||
|
|
||
| ENTRYPOINT ["/usr/bin/dumb-init", "--", "/usr/local/bin/docker-entrypoint.sh"] | ||
| CMD [] | ||
|
|
||
| FROM gitea-base AS gitea | ||
|
|
||
| EXPOSE 22 3000 | ||
|
|
||
| RUN apk --no-cache add \ | ||
| linux-pam \ | ||
| openssh \ | ||
| s6 \ | ||
| sqlite \ | ||
| su-exec \ | ||
| && rm -rf /var/cache/apk/* | ||
|
|
||
| RUN adduser \ | ||
| -S -H -D \ | ||
| -h /data/git \ | ||
| -s /bin/bash \ | ||
| -u 1000 \ | ||
| -G git \ | ||
| git && \ | ||
| echo "git:*" | chpasswd -e | ||
|
|
||
| ENV USER git | ||
| ENV GITEA_CUSTOM /data/gitea | ||
|
|
@@ -80,7 +177,4 @@ VOLUME ["/data"] | |
| ENTRYPOINT ["/usr/bin/entrypoint"] | ||
| CMD ["/bin/s6-svscan", "/etc/s6"] | ||
|
|
||
| COPY --chmod=755 docker/root / | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thoughts on removing this line, can see it has been untouched in this file for 5 years so guessing it might be a part of a workflow?
Disadavntage of including these lines is that any time a maintainer commits to git, they will be re-running the
COPY ./.git ./.gitstep. Currently this happend transparently.To replicate the functionality you can use this feature of docker build
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is due to the makefile using git calls to determine the version information. This could (and tbh should) be pulled out for the reasons you mentioned that whenever git repo changes it invalidates the cache (even if you are attempting to build the same tag). We do some magic in the makefile to calculate the version, and this logic is used for more than just docker (eg xgo crossarch builds) which means it's kinda load bearing and changes to it need a closer eye when reviewing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at the failed build, the
.gitfolder isn't present during CI runs - I think this is more to do with the dyrun workflow needing to do a full checkout once it has detected a file has changed?In the Dockerfile
GITEA_VERSIONis anARG, from what I can tell these are only accessible within the Dockerfile, and aren't in the build as environment variables, so should be safe to take out?I think in the GitHub release workflows, it is using the
.gitdirectory to determine version information.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think if we were to go down the route of not relying on
.gitfor Docker builds, we may want to go down the route of something like....This way, when developing you will get a nominal "develop" value for your Gitea version, but when compiling for release, it can be specified as a build argument?