Your Container Does Not Need an Operating System
Ship your Go service on FROM scratch. It works, the image lands around 10 MB, and your vulnerability scanner goes quiet. The reason most people try it once and retreat to Alpine is not size and it is not ideology. It is that four or five specific things break, and every single one of them fails with an error message that tells you nothing useful about the actual cause.
My favorite is exec /myapp: no such file or directory. The binary is right there. You can see it in the layer. Docker is not lying to you, it just phrased it terribly. So: here is the full list of what breaks on an empty filesystem, in the order you will hit it. Work through this once and you can ship on scratch forever. If you would rather not, gcr.io/distroless/static-debian12 solves most of it for you at a cost of about two megabytes, and picking that instead is a perfectly respectable engineering decision.
Scratch is a real base image, by the way. It is a reserved name that means “start from nothing.” No files, no directories, not even /. The multi-stage build post covers the mechanics of getting your binary out of a builder stage, so I am assuming you already know what COPY --from= does.
Step One: Prove Your Binary Is Actually Static
Everyone copies CGO_ENABLED=0 off a blog post and assumes the job is done. It usually is. When it is not, you get the no such file or directory lie, because the kernel found your binary, read its ELF header, saw a request for /lib64/ld-linux-x86-64.so.2, and could not find that. The missing file is the dynamic loader, not your program.
Check before you build the image, not after it crashes in staging:
$ file ./myapp./myapp: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped
$ ldd ./myapp not a dynamic executable
$ go version -m ./myapp | head -20./myapp: go1.26.0 build -buildmode=exe build -trimpath=true build CGO_ENABLED=0 build GOOS=linux build GOARCH=amd64 build vcs.revision=8f2c1a...If file says “dynamically linked” you are not going on scratch yet. Things that quietly flip you back to dynamic linking: any SQLite driver that wraps the C library (mattn/go-sqlite3 is the classic), image codecs that bind to libjpeg or libwebp, anything speaking to a system keyring, and a handful of Kerberos or LDAP wrappers. go version -m is the honest one here because it reports what the compiler actually did rather than what you thought you asked for. Wire that ldd check into CI and you will never debug this at 2 AM.
DNS: The netgo Tag and the Files Nobody Copied
Go has two DNS resolvers. With cgo enabled it can call the system getaddrinfo. With cgo off it uses a pure-Go resolver that reads /etc/resolv.conf and /etc/nsswitch.conf itself.
The good news is that /etc/resolv.conf and /etc/hosts are bind-mounted into the container by the runtime at start time, so they exist even on scratch. You do not need to bake them in, and you should not try.
/etc/nsswitch.conf is different. Nothing mounts it, and it is what tells the resolver whether to consult /etc/hosts before DNS or the other way round. Go’s fallback behavior when the file is missing has shifted between releases, which means the failure mode is “hostname lookups work on my laptop and in three of four environments.” Copy the file and delete the variable:
COPY --from=builder /etc/nsswitch.conf /etc/nsswitch.confWhile you are at it, build with -tags netgo,osusergo. Those tags force the pure-Go resolver and the pure-Go os/user implementation regardless of what CGO_ENABLED happens to be at the time. It makes the intent explicit instead of relying on an environment variable surviving every refactor of your build pipeline.
Timezones: Why Every Timestamp Is Suddenly UTC
time.LoadLocation("America/New_York") reads the tzdata database out of /usr/share/zoneinfo. Scratch does not have /usr, let alone the zoneinfo tree, so the call returns an error. If your code ignores that error (be honest, it does), every timestamp silently renders in UTC and your cron-style scheduler fires at the wrong hour. Nothing crashes. It is just wrong, quietly, for a week.
The fix is one line of Go:
import ( "time" _ "time/tzdata" // embeds the IANA tz database into the binary)That blank import compiles the whole tz database into your executable. It costs roughly 450 KB, which on a 10 MB image is noise, and it means the timezone data ships with the code that depends on it. The alternative is copying the database out of the builder and pointing ZONEINFO at wherever you put it (COPY --from=builder /usr/share/zoneinfo /zoneinfo plus ENV ZONEINFO=/zoneinfo), which also works and which I would only bother with if you need to update tz rules without recompiling. For a service, embed it.
There Is No /tmp, and You Cannot mkdir One
Scratch has no directories. Not empty ones, none. So os.CreateTemp fails, and more sneakily, net/http fails: r.FormFile calls ParseMultipartForm with a 32 MB memory budget, and anything over that budget spills to a temp file on disk. Your upload endpoint works fine for small files in testing and dies on the first real one.
You cannot fix this with RUN mkdir /tmp because RUN needs a shell and there is no shell. Two things that do work:
# Option A: make an empty dir in the builder, lift it across# (in the builder stage: RUN mkdir -m 1777 /emptytmp)COPY --from=builder --chown=65532:65532 /emptytmp /tmp
# Option B: WORKDIR creates directories, and needs no shell to do itWORKDIR /appOption A is the one you want for /tmp specifically, since it lets you set ownership for your non-root user. Create a fresh empty directory to copy rather than copying the builder’s own /tmp, which tends to accumulate junk from the toolchain and would ride along into your image. At runtime you can also mount a real tmpfs over it, which is better practice anyway: --mount=type=tmpfs,destination=/tmp for Docker, or an emptyDir volume in Kubernetes. Then your writable scratch space lives in RAM and your image layer stays read-only, which is where a container filesystem should be.
The User Problem Is Smaller Than It Looks
The non-root containers post covers why you care, so I will stay on the scratch-specific part: you do not actually need /etc/passwd.
USER 65532:65532The kernel enforces permissions on numeric UIDs and GIDs. It has no interest in names. That line gives you a genuinely unprivileged process with zero extra files. 65532 is the UID distroless uses for its nonroot user, and borrowing it keeps volume permissions consistent if you ever switch bases.
You need a passwd entry only if your own code (or a dependency) calls user.Current() or user.Lookup(), which happens more often than you would guess inside libraries that try to resolve a home directory for config. When you do need it, generate exactly one line in the builder rather than copying the host’s file:
RUN echo 'app:x:65532:65532:app:/nonexistent:/sbin/nologin' > /passwd.minimalThen COPY --from=builder /passwd.minimal /etc/passwd. One line, no accidental disclosure of every account on your build machine.
HEALTHCHECK Without curl or a Shell
This is the one that makes people give up, and it has a genuinely nice answer.
The usual healthcheck is HEALTHCHECK CMD curl -f http://localhost:8080/health. On scratch that fails twice over: no curl, and the shell-form CMD needs /bin/sh to interpret it. So stop reaching for an external tool and put the check inside the binary you already trust:
func main() { if len(os.Args) > 1 && os.Args[1] == "healthcheck" { resp, err := http.Get("http://127.0.0.1:8080/health") if err != nil || resp.StatusCode != http.StatusOK { os.Exit(1) } os.Exit(0) } // ... normal server startup}HEALTHCHECK --interval=30s --timeout=3s --start-period=5s \ CMD ["/myapp", "healthcheck"]Exec form, no shell, no extra binary, and the check ships and versions in lockstep with the server. I now do this even on images that have a shell.
The 2 AM Problem: Debugging Nothing
docker exec -it mycontainer sh returns exec: "sh": executable file not found. Your options, best first:
docker debug <container> attaches a toolbox shell through a sidecar without modifying the running container. Docker Desktop / Docker Business feature, and the smoothest experience if you have it.
kubectl debug -it mypod --image=busybox:1.37 --target=myapp is the Kubernetes answer. Ephemeral containers join your pod and share the target’s process namespace, so you get busybox tooling pointed at a process running in an image that contains none.
nsenter from the host, when you have host access and nothing else:
$ PID=$(docker inspect -f '{{.State.Pid}}' mycontainer)$ sudo nsenter -t "$PID" -n -p /bin/shNote the missing -m there, and do not add it back. Without it you get the host’s /bin/sh running inside the container’s network and PID namespaces, which is exactly what you want: host tooling (ss, tcpdump, curl) pointed at the container’s sockets and processes. Add -m to enter the mount namespace too and nsenter will try to resolve /bin/sh inside the container’s filesystem, find nothing, and hand you the same “no such file” error you were trying to escape.
A parallel :debug tag built from the same builder stage onto gcr.io/distroless/base-debian12:debug, which includes busybox. Same binary, same layers up to the final stage, shell available. Push both, run the slim one.
The real lesson is upstream of all four: an image with no shell means printf-debugging on a live container is off the table, so your structured logging, your metrics, and a net/http/pprof endpoint stop being nice-to-haves. Build the observability in, because you will not be poking around by hand.
Good News: You Do Not Need tini
Every “Docker PID 1” article tells you to add tini or dumb-init to reap zombies and forward signals. For a Go binary on scratch you can skip it. A typical Go service never forks, so there are no zombies to reap (if you do shell out with os/exec, cmd.Wait() reaps for you), and the signal handling is four lines you should have anyway:
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)defer stop()
<-ctx.Done()shutdownCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)defer cancel()srv.Shutdown(shutdownCtx)If your entrypoint were a shell script, you would need an init to forward SIGTERM past sh. You have no shell, so the problem removes itself. One of the rare cases where taking things away fixes something.
Builds You Can Audit Later
Three flags worth setting, beyond the -ldflags="-w -s" stripping the multi-stage post already covers:
-trimpath removes absolute filesystem paths from the binary. Without it your stack traces in production contain /home/kingpin/go/src/..., which is both a small information leak and a guarantee that two developers building the same commit produce different bytes.
-buildvcs=true stamps the git revision. The default is auto, which silently produces no VCS info when .git is not in your build context, and it usually is not. Setting it to true makes the build fail loudly instead, so you find out at build time rather than when you are trying to work out which commit is running in prod.
Between those two, go version -m on a shipped binary tells you the exact source it came from. Scratch also makes image digests reproducible in a way a package-managed base never will, because there is no apt layer underneath drifting every time upstream rebuilds.
About Those Zero-CVE Screenshots
Scan a scratch image with Trivy or Grype and you will often get zero findings. Part of that is real: there is no OpenSSL, no bash, no glibc, no apt, so an entire category of CVE genuinely does not apply to you. That is the strongest argument for this whole approach.
Part of it is a measurement artifact. Those scanners are very good at reading OS package manifests, and a scratch image has no package manifest to read. Your Go module dependencies still have vulnerabilities and the OS scanner is not the tool that finds them. Use govulncheck ./... in CI, which checks the Go vulnerability database against the functions you actually call, and generate an SBOM with syft so the scanner has something to chew on. A zero from Trivy on a scratch image means “no OS packages to be vulnerable,” which is good but is not the same claim as “no vulnerabilities.”
Putting It Together
# syntax=docker/dockerfile:1FROM golang:1.26 AS builder
WORKDIR /srcCOPY go.mod go.sum ./RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY . .RUN --mount=type=cache,target=/root/.cache/go-build \ CGO_ENABLED=0 GOOS=linux \ go build -trimpath -buildvcs=true \ -tags netgo,osusergo \ -ldflags="-w -s" \ -o /out/myapp ./cmd/myapp
# fail the build here rather than at runtimeRUN ldd /out/myapp 2>&1 | grep -q "not a dynamic executable"RUN echo 'app:x:65532:65532:app:/nonexistent:/sbin/nologin' > /out/passwdRUN mkdir -m 1777 /out/emptytmp
FROM scratch AS production
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/COPY --from=builder /etc/nsswitch.conf /etc/nsswitch.confCOPY --from=builder /out/passwd /etc/passwdCOPY --from=builder --chown=65532:65532 /out/emptytmp /tmpCOPY --from=builder /out/myapp /myapp
USER 65532:65532EXPOSE 8080HEALTHCHECK --interval=30s --timeout=3s --start-period=5s \ CMD ["/myapp", "healthcheck"]ENTRYPOINT ["/myapp"]The CA certificates line is table stakes for anything making outbound HTTPS calls and it is covered elsewhere on the site, but leaving it out is still the single most common scratch bug, so it stays in the template. Note the ldd assertion in the builder: that turns a runtime mystery into a build failure with an obvious cause.
So Which Base Should You Actually Use
Typical numbers for a small Go HTTP service, embedded tzdata included:
| Base | Final image | What you get |
|---|---|---|
scratch | 8 to 12 MB | Nothing. You bring every file yourself. |
gcr.io/distroless/static-debian12 | 10 to 14 MB | ca-certs, tzdata, /tmp, a nonroot user, all pre-solved |
alpine:3.22 | 15 to 18 MB | All of the above plus busybox, a shell, and apk |
golang:1.26 (builder, shipped by accident) | 800 MB+ | A full Go toolchain in production, somehow |
The gap between scratch and distroless/static is a couple of megabytes. Nobody’s deploy pipeline is bottlenecked on 2 MB, so decide on operational grounds instead:
Use distroless/static by default. It is the pragmatic answer for most teams. Same size class, same absence of a shell and a package manager, and the ca-certs / tzdata / /tmp / nonroot problems are already handled by someone who thought about them harder than you will on a Thursday afternoon.
Use scratch when you want to know exactly what is in your image because you put every byte there deliberately, or when you are shipping a single-purpose binary to a lot of places and 2 MB times a lot of pulls is actually money, or when you are building something you want to be able to audit file by file.
Use Alpine when you have cgo dependencies you cannot remove, or when your operations story genuinely depends on getting a shell in the container. That is a legitimate requirement and pretending otherwise is how you end up with an unmaintainable image that only its author can debug.
What you should not do is ship the builder. A golang image in production is a compiler, a package cache, and a git client sitting next to your service for no reason, like keeping the forklift in the living room because it was useful on moving day.
Related Reading
- Multi-Stage Docker Builds: Stop Shipping Your node_modules to Production
- Running Docker Containers as Non-Root (And Why You Should)
- Distroless Images: When Minimal Goes Too Far
- Alpine vs. Distroless: Choosing Your Minimalist Base
- Building CLI Tools in Go: Because Shell Scripts Have a Maximum Complexity