dockersrc/rust
1
0
Fork 0
mirror of https://github.com/dockersrc/rust synced 2026-06-24 14:01:04 -04:00
Code Issues Actions Packages Projects Releases Wiki Activity

⚡ Native cross-compile stage + sccache on by default ⚡

Browse Source 
Eliminates QEMU arm64 build (was ~15 hours) by adding a
`rust-tools` stage that mirrors the Go image's `go-tools` pattern:
runs on the build platform (amd64), cross-compiles or downloads all
~50 Rust tools for the target arch, and activates sccache throughout.
Build time for linux/arm64: ~15 hours → ~30–60 minutes.
How it works:
- `FROM --platform=$BUILDPLATFORM rust:alpine AS rust-tools` runs natively
- `cargo binstall --target $RUST_TARGET` fetches GitHub release prebuilts
(zero compilation for most tools)
- Missing prebuilts fall back to `cargo install --target` using the
`aarch64-linux-musl-gcc` cross-toolchain (no QEMU, native speed)
- Native (x86_64) sccache installed as RUSTC_WRAPPER so source-compiled
tools are cached across Docker rebuilds
- `CARGO_INSTALL_ROOT=/rust-tools` keeps tool binaries separate from
rustup shims; `COPY --from=rust-tools /rust-tools/bin/` drops them
into the build stage before `05-custom.sh` runs
Pure-Rust / no-OpenSSL feature flags where possible (CGO=0 equivalent):
- `sqlx-cli`: `--features rustls` (was native-tls)
- `sea-orm-cli`: `--features runtime-tokio-rustls` (was native-tls default)
- `trunk`: already defaults to rustls — no change needed
- `probe-rs`: prebuilt if available, silently skipped otherwise (needs libusb)
sccache is now active by default in all login shells via profile.d/rust.sh
(`RUSTC_WRAPPER=sccache`); opt out with `-e RUSTC_WRAPPER=`.
- Dockerfile: add 110-line rust-tools stage with musl cross-toolchain,
sccache, cargo-binstall, all tool installs for target arch
- Dockerfile: COPY --from=rust-tools before 05-custom.sh RUN step
- Dockerfile: set RUSTC_WRAPPER in build stage; drop unused cargo-registry
and cargo-git cache mounts (no longer needed there)
- rootfs/root/docker/setup/05-custom.sh: remove cargo-binstall bootstrap
and all cargo binstall/install blocks; add RUSTC_WRAPPER to profile.d
- README.md: document rust-tools stage, native cross-compile architecture,
expected build times, sccache-on-by-default, pure-Rust feature flags,
updated cache mount ID table

Dockerfile
README.md
rootfs/root/docker/setup/05-custom.sh
This commit is contained in:
Jason Hempstead
2026-06-21 17:58:20 -04:00
parent 20cd92f941
commit a6acb207b6
3 changed files with 203 additions and 111 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Dockerfile
+141 -3
View File
@@ -30,6 +30,140 @@ ARG DISTRO_VERSION="${IMAGE_VERSION}"
ARG BUILD_VERSION="${BUILD_DATE}"
FROM tianon/gosu:latest AS gosu
# ─── native cross-compile stage ───────────────────────────────────────────────
# Mirrors the go-tools pattern: runs on the build platform (always native amd64
# in CI). For arm64 targets the musl cross-toolchain compiles natively instead
# of under QEMU — turning a 15-hour emulated build into a 30-60 minute one.
# cargo binstall --target fetches prebuilt binaries from GitHub releases;
# for tools without prebuilts it falls back to native cross-compilation.
FROM --platform=$BUILDPLATFORM rust:alpine AS rust-tools
ARG TARGETARCH
# musl-cross provides aarch64-linux-musl-gcc for arm64 cross-compilation
RUN apk add --no-cache musl-cross curl jq
# Resolve Docker TARGETARCH → Rust target triple
RUN case "${TARGETARCH}" in \
amd64) echo "x86_64-unknown-linux-musl" ;; \
arm64) echo "aarch64-unknown-linux-musl" ;; \
*) echo "unsupported TARGETARCH: ${TARGETARCH}" >&2; exit 1 ;; \
esac > /tmp/rust-target
# Register the cross-compile target with the native (x86_64) Rust toolchain
RUN RUST_TARGET="$(cat /tmp/rust-target)" && rustup target add "${RUST_TARGET}"
# Linker and compiler overrides for aarch64-musl; harmless when TARGETARCH=amd64
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-musl-gcc
ENV CC_aarch64_unknown_linux_musl=aarch64-linux-musl-gcc
ENV CXX_aarch64_unknown_linux_musl=aarch64-linux-musl-g++
ENV AR_aarch64_unknown_linux_musl=aarch64-linux-musl-ar
# Install native (x86_64) sccache to /usr/local/bin for use as RUSTC_WRAPPER
# during this stage; kept separate from the target-arch sccache in /rust-tools/bin
RUN set -e; \
SCCACHE_VER="$(curl -fsSL https://api.github.com/repos/mozilla/sccache/releases/latest | jq -r '.tag_name')"; \
SCCACHE_ASSET="sccache-${SCCACHE_VER}-x86_64-unknown-linux-musl"; \
curl -fsSL "https://github.com/mozilla/sccache/releases/download/${SCCACHE_VER}/${SCCACHE_ASSET}.tar.gz" \
| tar xz -C /tmp; \
install -m 755 "/tmp/${SCCACHE_ASSET}/sccache" /usr/local/bin/sccache; \
rm -rf "/tmp/${SCCACHE_ASSET}"
# Bootstrap native (x86_64) cargo-binstall
RUN curl -fsSL "https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-x86_64-unknown-linux-musl.tgz" \
| tar xz -C /usr/local/cargo/bin cargo-binstall && \
chmod 755 /usr/local/cargo/bin/cargo-binstall
# All tool binaries land in /rust-tools/bin — cleanly separate from rustup shims
RUN mkdir -p /rust-tools/bin
# CARGO_INSTALL_ROOT routes both `cargo install` and `cargo binstall` to /rust-tools/bin
ENV CARGO_INSTALL_ROOT=/rust-tools
# Install all Rust tools for the target arch with the native build cache active.
# Prebuilts are downloaded directly; source fallbacks cross-compile on amd64.
RUN --mount=type=cache,id=cargo-registry-native,sharing=shared,target=/usr/local/cargo/registry \
--mount=type=cache,id=cargo-git-native,sharing=locked,target=/usr/local/cargo/git \
--mount=type=cache,id=sccache-native,sharing=locked,target=/root/.cache/sccache \
set -o pipefail; \
RUST_TARGET="$(cat /tmp/rust-target)"; \
export RUSTC_WRAPPER=/usr/local/bin/sccache; \
export SCCACHE_DIR=/root/.cache/sccache; \
cargo binstall -y --target "${RUST_TARGET}" \
cargo-edit \
cargo-watch \
cargo-update \
cargo-outdated \
cargo-expand \
cargo-info \
bacon \
cargo-llvm-cov \
cargo-tarpaulin \
cargo-audit \
cargo-deny \
cargo-machete \
cargo-semver-checks \
cargo-make \
cargo-deb \
cargo-generate \
cargo-release \
cargo-chef \
cargo-zigbuild \
just \
tokei \
hyperfine \
wasm-pack \
wasm-tools \
wasm-bindgen-cli \
cbindgen \
cargo-binutils \
cargo-bloat \
cargo-asm \
mdbook \
mdbook-toc \
sccache \
typos-cli \
taplo-cli \
cargo-sort \
cargo-hack \
cargo-criterion \
dprint \
cargo-careful \
cargo-public-api \
cargo-spellcheck \
cargo-geiger \
grcov || true; \
cargo binstall -y --target "${RUST_TARGET}" cargo-nextest 2>/dev/null || \
cargo install --locked --target "${RUST_TARGET}" cargo-nextest || true; \
cargo binstall -y --target "${RUST_TARGET}" cargo-dist 2>/dev/null || \
cargo install --target "${RUST_TARGET}" cargo-dist || true; \
cargo binstall -y --target "${RUST_TARGET}" cargo-msrv 2>/dev/null || \
cargo install --target "${RUST_TARGET}" cargo-msrv; \
cargo binstall -y --target "${RUST_TARGET}" cargo-mutants 2>/dev/null || \
cargo install --target "${RUST_TARGET}" cargo-mutants; \
cargo binstall -y --target "${RUST_TARGET}" flip-link 2>/dev/null || \
cargo install --target "${RUST_TARGET}" flip-link; \
cargo binstall -y --target "${RUST_TARGET}" cargo-ndk 2>/dev/null || \
cargo install --target "${RUST_TARGET}" cargo-ndk; \
cargo binstall -y --target "${RUST_TARGET}" trunk 2>/dev/null || \
cargo install --target "${RUST_TARGET}" trunk 2>/dev/null || true; \
cargo binstall -y --target "${RUST_TARGET}" cargo-udeps 2>/dev/null || \
cargo install --target "${RUST_TARGET}" cargo-udeps || true; \
cargo binstall -y --target "${RUST_TARGET}" cargo-fuzz 2>/dev/null || \
cargo install --target "${RUST_TARGET}" cargo-fuzz || true; \
cargo binstall -y --target "${RUST_TARGET}" cargo-minimal-versions 2>/dev/null || \
cargo install --target "${RUST_TARGET}" cargo-minimal-versions || true; \
cargo binstall -y --target "${RUST_TARGET}" cross 2>/dev/null || \
cargo install --target "${RUST_TARGET}" cross 2>/dev/null || true; \
cargo binstall -y --target "${RUST_TARGET}" samply 2>/dev/null || true; \
cargo binstall -y --target "${RUST_TARGET}" flamegraph 2>/dev/null || \
cargo install --target "${RUST_TARGET}" flamegraph || true; \
cargo install --target "${RUST_TARGET}" probe-rs --features cli 2>/dev/null || true; \
cargo install --target "${RUST_TARGET}" sqlx-cli \
--no-default-features --features rustls,postgres,mysql,sqlite 2>/dev/null || true; \
cargo install --target "${RUST_TARGET}" sea-orm-cli \
--no-default-features --features codegen,sqlx-mysql,sqlx-postgres,sqlx-sqlite,runtime-tokio-rustls 2>/dev/null || true
FROM ${PULL_URL}:${DISTRO_VERSION} AS build
ARG TZ
ARG USER
@@ -157,11 +291,15 @@ RUN echo "Custom Applications"; \
$SHELL_OPTS; \
echo ""
RUN --mount=type=cache,id=cargo-registry,sharing=shared,target=/usr/local/share/cargo/registry \
--mount=type=cache,id=cargo-git-${TARGETARCH},sharing=locked,target=/usr/local/share/cargo/git \
--mount=type=cache,id=rustup-downloads-${TARGETARCH},sharing=locked,target=/usr/local/share/rustup/downloads \
# Target-arch tool binaries compiled natively in the rust-tools stage;
# copied here before 05-custom.sh runs so the symlink loop picks them up
COPY --from=rust-tools /rust-tools/bin/ /usr/local/share/cargo/bin/
RUN --mount=type=cache,id=rustup-downloads-${TARGETARCH},sharing=locked,target=/usr/local/share/rustup/downloads \
--mount=type=cache,id=sccache-build-${TARGETARCH},sharing=locked,target=/root/.cache/sccache \
echo "Running custom commands"; \
export RUSTC_WRAPPER=/usr/local/share/cargo/bin/sccache; \
export SCCACHE_DIR=/root/.cache/sccache; \
if [ -f "/root/docker/setup/05-custom.sh" ];then echo "Running the custom script";/root/docker/setup/05-custom.sh||{ echo "Failed to execute /root/docker/setup/05-custom.sh" && exit 10; };echo "Done running the custom script";fi; \
echo ""
README.md
+57 -23
View File
@@ -283,9 +283,9 @@ Run miri with: `cargo +nightly miri test`
| `sqlx-cli` | Compile-time SQL verification and migration runner for sqlx |
| `sea-orm-cli` | Migration generator and entity scaffolder for SeaORM |
These are installed with broadly compatible feature flags. Projects with
unusual feature requirements may need to `cargo install` them again with
project-specific flags.
Both are built with `rustls` instead of `native-tls` (pure-Rust TLS stack,
no OpenSSL dependency) and support postgres, mysql, and sqlite. Projects with
unusual feature requirements may `cargo install` them again with different flags.
---
@@ -320,24 +320,25 @@ docker run --rm -v "$PWD:/app" \
casjaysdev/rust:latest
```
### Enable sccache compilation caching
### sccache compilation caching (on by default)
`sccache` is installed and `SCCACHE_DIR` is pre-configured to
`/root/.cache/sccache`. It is **not** activated by default. Opt in
per run with `-e RUSTC_WRAPPER=sccache`:
`sccache` is installed and **active by default**. `RUSTC_WRAPPER=sccache` is
set in `/etc/profile.d/rust.sh` so every login shell automatically routes
`rustc` invocations through the cache. `SCCACHE_DIR` points to
`/root/.cache/sccache`, which is declared as a Docker volume.
```shell
docker run --rm -v "$PWD:/app" \
-v rust-sccache:/root/.cache/sccache \
-e RUSTC_WRAPPER=sccache \
casjaysdev/rust:latest
casjaysdev/rust:latest cargo build --release
```
With `RUSTC_WRAPPER=sccache`, sccache intercepts every `rustc` invocation and
serves cached object files on cache hits, dramatically speeding up incremental
Cache hits skip recompilation entirely, dramatically speeding up incremental
and repeated builds. `CARGO_INCREMENTAL` is forced to `0` because cargo's own
incremental compilation conflicts with sccache's shared cache.
To opt out: `-e RUSTC_WRAPPER=`
#### Remote sccache backends
Point sccache at S3, Redis, GCS, or Azure by setting the relevant env vars
@@ -356,20 +357,27 @@ docker run --rm -v "$PWD:/app" \
### BuildKit cache mounts (for image builds)
The `Dockerfile` uses `--mount=type=cache` on both the package install and
toolchain install steps. This keeps the apk index, cargo registry, rustup
downloads, and sccache populated between `docker build` runs so rebuilding
the image after a change does not re-download anything:
The `Dockerfile` uses `--mount=type=cache` across all stages. This keeps the
apk index, cargo registry, rustup downloads, and sccache populated between
`docker build` runs so rebuilding after a change does not re-download or
recompile anything:
```shell
# BuildKit is the default since Docker 23; no flags needed
docker build --tag casjaysdev/rust:local .
```
Cache mount IDs: `apk-cache-<arch>`, `cargo-registry`, `cargo-git-<arch>`,
`rustup-downloads-<arch>`, `sccache-build-<arch>` (where `<arch>` is
`amd64` or `arm64` — per-arch IDs prevent cross-arch cache corruption in
multi-platform builds).
| Stage | Cache mount ID | Contents |
|-------|----------------|----------|
| `build` | `apk-cache-<arch>` | Alpine package index and downloaded APKs |
| `build` | `rustup-downloads-<arch>` | rustup toolchain and component tarballs |
| `build` | `sccache-build-<arch>` | sccache compiled-artifact cache for the build stage |
| `rust-tools` | `cargo-registry-native` | Cargo registry index and crate tarballs (native) |
| `rust-tools` | `cargo-git-native` | Cargo git dependencies (native) |
| `rust-tools` | `sccache-native` | sccache cache for source-compiled tools in the native stage |
`<arch>` is `amd64` or `arm64`. Per-arch IDs prevent cross-arch cache
corruption in multi-platform builds.
---
@@ -381,8 +389,8 @@ multi-platform builds).
| `RUSTUP_HOME` | `/usr/local/share/rustup` | Toolchains and components |
| `RUSTUP_TOOLCHAIN` | `stable` | Default channel |
| `SCCACHE_DIR` | `/root/.cache/sccache` | Local sccache storage directory |
| `CARGO_INCREMENTAL` | `0` | Disabled — required when using sccache |
| `RUSTC_WRAPPER` | *(unset)* | Set to `sccache` to activate compilation caching |
| `CARGO_INCREMENTAL` | `0` | Disabled — conflicts with sccache shared cache |
| `RUSTC_WRAPPER` | `sccache` | Active by default; set to empty string to disable |
| `CARGO_WORKDIR` | *(unset)* | Override working directory for `rust-workflow` |
| `CARGO_BUILD_TARGET` | *(unset)* | Cross-compile triple for `rust-workflow` |
| `TZ` | `America/New_York` | Override at run time with `-e TZ=...` |
@@ -523,8 +531,34 @@ docker build --tag casjaysdev/rust:local .
```
BuildKit is required (default in Docker 23+). Cache mounts keep subsequent
builds fast — the cargo registry, rustup downloads, and sccache data persist
in BuildKit's own cache layer storage.
builds fast — cargo registry, rustup downloads, and sccache data persist in
BuildKit's own cache layer storage.
### Build architecture — native cross-compilation
The `Dockerfile` uses the same `--platform=$BUILDPLATFORM` pattern as the
Go image's `go-tools` stage. A dedicated `rust-tools` stage runs natively on
the build host (always amd64 in CI) and cross-compiles or downloads all
~50 Rust tool binaries for the target arch before the main build stage ever
starts. This eliminates QEMU emulation for tool compilation:
| How | What |
|-----|------|
| `cargo binstall --target <triple>` | Fetches prebuilt binaries from GitHub releases — no compilation |
| `cargo install --target <triple>` | Source-compiles natively on amd64 via the `musl-cross` toolchain |
| `sccache` (native x86_64) | Caches all source compilations across rebuilds |
Tools with C dependencies use pure-Rust alternatives wherever possible:
`rustls` instead of `native-tls`/OpenSSL (sqlx-cli, sea-orm-cli, trunk);
bundled SQLite via `rusqlite`'s `bundled` feature. `probe-rs` (needs libusb)
is best-effort: downloaded if a prebuilt exists, silently skipped otherwise.
**Expected multi-arch build times:**
| Arch | Before (QEMU) | After (native cross-compile) |
|------|---------------|------------------------------|
| `linux/amd64` | ~20 min | ~20 min |
| `linux/arm64` | ~15 hours | ~30–60 min |
---
rootfs/root/docker/setup/05-custom.sh
+5 -85
View File
@@ -146,91 +146,10 @@ rustup target add \
aarch64-linux-android
# - - - - - - - - - - - - - - - - - - - - - - - - -
# Bootstrap cargo-binstall — downloads prebuilt binaries instead of
# compiling every tool from source, cutting install time dramatically
BINSTALL_ARCH="$(uname -m)"
BINSTALL_URL="https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-${BINSTALL_ARCH}-unknown-linux-musl.tgz"
curl -sSfL "$BINSTALL_URL" -o /tmp/cargo-binstall.tgz
tar xzf /tmp/cargo-binstall.tgz -C /tmp cargo-binstall
install -m 755 /tmp/cargo-binstall "$CARGO_HOME/bin/cargo-binstall"
rm -f /tmp/cargo-binstall.tgz /tmp/cargo-binstall
# - - - - - - - - - - - - - - - - - - - - - - - - -
# Workflow and development tools — most have musl prebuilt binaries for amd64 and arm64;
# || true lets the build continue if an individual tool has no prebuilt and source
# compilation fails (e.g. missing a C dep); failures are visible in the build log
cargo binstall -y \
cargo-edit \
cargo-watch \
cargo-update \
cargo-outdated \
cargo-expand \
cargo-info \
bacon \
cargo-llvm-cov \
cargo-tarpaulin \
cargo-audit \
cargo-deny \
cargo-machete \
cargo-semver-checks \
cargo-make \
cargo-deb \
cargo-generate \
cargo-release \
cargo-chef \
cargo-zigbuild \
just \
tokei \
hyperfine \
wasm-pack \
wasm-tools \
wasm-bindgen-cli \
cbindgen \
cargo-binutils \
cargo-bloat \
cargo-asm \
mdbook \
mdbook-toc \
sccache \
typos-cli \
taplo-cli \
cargo-sort \
cargo-hack \
cargo-criterion \
dprint \
cargo-careful \
cargo-public-api \
cargo-spellcheck \
cargo-geiger \
grcov || true
# Tools that occasionally lack musl prebuilts — fall back to source compilation
# cargo-nextest requires --locked when building from source (locked-tripwire guard)
cargo binstall -y cargo-nextest 2>/dev/null || cargo install --locked cargo-nextest || true
cargo binstall -y cargo-dist 2>/dev/null || cargo install cargo-dist || true
cargo binstall -y cargo-msrv 2>/dev/null || cargo install cargo-msrv
cargo binstall -y cargo-mutants 2>/dev/null || cargo install cargo-mutants
cargo binstall -y flip-link 2>/dev/null || cargo install flip-link
cargo binstall -y cargo-ndk 2>/dev/null || cargo install cargo-ndk
cargo binstall -y trunk 2>/dev/null || cargo install trunk 2>/dev/null || true
cargo binstall -y cargo-udeps 2>/dev/null || cargo install cargo-udeps || true
cargo binstall -y cargo-fuzz 2>/dev/null || cargo install cargo-fuzz || true
cargo binstall -y cargo-minimal-versions 2>/dev/null || cargo install cargo-minimal-versions || true
# cross (the cross-rs cross-compilation runner)
cargo binstall -y cross 2>/dev/null || cargo install cross 2>/dev/null || true
# probe-rs requires the cli feature flag and is best built from source
cargo install probe-rs --features cli 2>/dev/null || true
# samply and cargo-flamegraph require a system perf or dtrace — best-effort
cargo binstall -y samply 2>/dev/null || true
cargo binstall -y flamegraph 2>/dev/null || cargo install flamegraph || true
# sqlx-cli and sea-orm-cli need project-specific feature flags at runtime;
# install a broadly compatible build here as a convenience
cargo install sqlx-cli --no-default-features --features native-tls,postgres,mysql,sqlite 2>/dev/null || true
cargo install sea-orm-cli 2>/dev/null || true
# All Rust tool binaries (cargo-edit, cargo-deny, sqlx-cli, sea-orm-cli, probe-rs,
# sccache, etc.) are compiled natively in the Dockerfile's rust-tools stage and
# copied to $CARGO_HOME/bin before this script runs — no cargo install here.
# The symlink loop below picks them all up.
# - - - - - - - - - - - - - - - - - - - - - - - - -
# Linker configuration for cross-compilation targets.
@@ -292,6 +211,7 @@ export CARGO_HOME="${CARGO_HOME}"
export RUSTUP_TOOLCHAIN="stable"
export PATH="${CARGO_HOME}/bin:\${PATH}"
export SCCACHE_DIR="\${SCCACHE_DIR:-/root/.cache/sccache}"
export RUSTC_WRAPPER="\${RUSTC_WRAPPER:-sccache}"
export CARGO_INCREMENTAL="\${CARGO_INCREMENTAL:-0}"
PROFILE