feat(build): produce per-arch pushed-artifact tarballs for native add-ons (#3425) #3470

Merged

mfreeman451 merged 1 commit from feat/addon-bundle-tarball into staging

2026-05-31 16:44:57 +00:00

mfreeman451 commented

2026-05-31 16:40:30 +00:00

Owner

What

delivery-models task 1.2 (build half) — produce the per-arch signed-able pushed-artifact gzip tarball at build time.

The agent side already lands: stageAddonArtifact fetches an artifact, verifies sha256 + ed25519, and auto-detects a gzip tarball (binary + manifest/config + systemd units) vs. a bare binary, extracting the tarball flat under current/ (traversal/symlink/hardlink/decompression-bomb guarded). This PR adds the build side that produces that tarball, so a pushed-artifact add-on can ship units + config alongside its binary.

Changes

build/native_addons/assemble_addon_bundle.py

New --tarball os/arch=out_path. write_tarball() emits a deterministic gzip tar: entries sorted by name, mtime=0, uid/gid=0, blank uname/gname, and gzip mtime=0 so the .tar.gz is byte-for-byte reproducible (stable sha256 for signing). Binary is 0755 (named by binary_name), manifest/config/units 0644, all flattened to single-segment names — exactly what the agent's extractAddonTarball expects.
Each arch's tarball_file + tarball_sha256 is recorded in metadata.json next to the existing bare-binary sha256.
Fully backward-compatible: with no --tarball, the zip/sha/metadata output is unchanged (no tarball fields).

build/native_addons/defs.bzl / addon_inventory.bzl

A bundle that sets pushed_artifact_tarball: True emits {name}.{os}.{arch}.tar.gz per platform plus an all_tarballs filegroup.
Optional unit_entries (systemd .service/.timer units) ship in both the zip bundle and the flattened tarball — the mechanism a systemd-supervised add-on (e.g. netprobe) will use.
Both sample bundles opt in, so CI exercises the new genrule + write_tarball path (bare binary + addon.yaml + config.schema.json).

Verification

Ran the assembler end to end locally against a crafted binary + manifest + systemd unit (bazel itself can't run here — CI is the build-side gate):

Layout/modes: tarball contains flat addon.yaml/config.schema.json/<binary>/<unit>; binary 0755, rest 0644; no zip name collision (real archive_path is bin/<os>/<arch>/<binary>, so Path(...).name correctly yields the binary name).
Reproducibility: two runs produce identical sha256 per arch.
Metadata: recorded tarball_sha256 matches the on-disk file for both arches.
Backward-compat: without --tarball, no tarball files and no tarball fields in metadata.
py_compile + buildifier -mode=check clean; openspec validate add-native-addon-delivery-models --strict passes.

Downstream (not in this PR)

Signing the tarball and publishing it as the referenced pushed-artifact (so the agent assignment's artifact_sha256 = tarball_sha256) belong to the build-signing change's sign/publish half — secret-blocked (Cosign + ed25519 upload keys). metadata.json now carries tarball_file/tarball_sha256 so the release/discovery-index step can pick the tarball without re-deriving it.
The os-package add-on template (the other prong of task 1.2) is still pending.

tasks.md

Updated add-native-addon-delivery-models/tasks.md 1.2 to reflect both the agent-side extraction and this build-side production are done; remaining = tarball signing (build-signing, secret-blocked) + os-package template.

🤖 Generated with Claude Code

## What delivery-models **task 1.2 (build half)** — produce the per-arch signed-able `pushed-artifact` gzip tarball at build time. The agent side already lands: `stageAddonArtifact` fetches an artifact, verifies sha256 + ed25519, and auto-detects a **gzip tarball** (binary + manifest/config + systemd units) vs. a bare binary, extracting the tarball flat under `current/` (traversal/symlink/hardlink/decompression-bomb guarded). This PR adds the **build side that produces that tarball**, so a pushed-artifact add-on can ship units + config alongside its binary. ## Changes **`build/native_addons/assemble_addon_bundle.py`** - New `--tarball os/arch=out_path`. `write_tarball()` emits a **deterministic** gzip tar: entries sorted by name, `mtime=0`, `uid/gid=0`, blank uname/gname, and `gzip mtime=0` so the `.tar.gz` is byte-for-byte reproducible (stable sha256 for signing). Binary is `0755` (named by `binary_name`), manifest/config/units `0644`, all flattened to **single-segment** names — exactly what the agent's `extractAddonTarball` expects. - Each arch's `tarball_file` + `tarball_sha256` is recorded in `metadata.json` next to the existing bare-binary `sha256`. - Fully backward-compatible: with no `--tarball`, the zip/sha/metadata output is unchanged (no tarball fields). **`build/native_addons/defs.bzl` / `addon_inventory.bzl`** - A bundle that sets `pushed_artifact_tarball: True` emits `{name}.{os}.{arch}.tar.gz` per platform plus an `all_tarballs` filegroup. - Optional `unit_entries` (systemd `.service`/`.timer` units) ship in both the zip bundle and the flattened tarball — the mechanism a systemd-supervised add-on (e.g. netprobe) will use. - Both sample bundles opt in, so CI exercises the new genrule + `write_tarball` path (bare binary + `addon.yaml` + `config.schema.json`). ## Verification Ran the assembler end to end locally against a crafted binary + manifest + systemd unit (bazel itself can't run here — CI is the build-side gate): - **Layout/modes:** tarball contains flat `addon.yaml`/`config.schema.json`/`<binary>`/`<unit>`; binary `0755`, rest `0644`; no zip name collision (real `archive_path` is `bin/<os>/<arch>/<binary>`, so `Path(...).name` correctly yields the binary name). - **Reproducibility:** two runs produce identical sha256 per arch. - **Metadata:** recorded `tarball_sha256` matches the on-disk file for both arches. - **Backward-compat:** without `--tarball`, no tarball files and no tarball fields in metadata. - `py_compile` + `buildifier -mode=check` clean; `openspec validate add-native-addon-delivery-models --strict` passes. ## Downstream (not in this PR) - **Signing** the tarball and **publishing** it as the referenced pushed-artifact (so the agent assignment's `artifact_sha256` = `tarball_sha256`) belong to the build-signing change's sign/publish half — **secret-blocked** (Cosign + ed25519 upload keys). `metadata.json` now carries `tarball_file`/`tarball_sha256` so the release/discovery-index step can pick the tarball without re-deriving it. - The **`os-package`** add-on template (the other prong of task 1.2) is still pending. ## tasks.md Updated `add-native-addon-delivery-models/tasks.md` 1.2 to reflect both the agent-side extraction and this build-side production are done; remaining = tarball signing (build-signing, secret-blocked) + `os-package` template. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

mfreeman451 added 1 commit

2026-05-31 16:40:30 +00:00

feat(build): produce per-arch pushed-artifact tarballs for native add-ons (#3425 )

Secret Scan / gitleaks (pull_request) Successful in 21s

Details

lint / lint (push) Successful in 1m40s

Details

lint / lint (pull_request) Successful in 54s

Details

Golang Tests / test-go (push) Successful in 2m27s

Details

CI / build (pull_request) Failing after 2m22s

Details

45e27d15dc

delivery-models task 1.2 (build half). The agent already fetches, verifies,
and extracts a gzip-tarball pushed-artifact (binary + manifest/config + systemd
units, flattened under current/); this adds the build side that produces it.

assemble_addon_bundle.py:
- new --tarball os/arch=out_path; write_tarball() emits a deterministic gzip tar
  (entries sorted, mtime=0, uid/gid=0, gzip mtime=0 for a reproducible sha256),
  binary 0755 + manifest/config/units 0644, all flattened to single-segment
  names matching the agent's extractAddonTarball.
- per-arch tarball name + sha256 recorded in metadata.json alongside the bare
  binary sha256; no --tarball -> output is byte-for-byte unchanged.

defs.bzl / addon_inventory.bzl:
- bundles that set pushed_artifact_tarball: True emit {name}.{os}.{arch}.tar.gz
  plus an all_tarballs filegroup; optional unit_entries ship systemd units in
  the bundle/tarball. Both samples opt in so CI exercises the path.

Verified locally end to end: tarball layout/modes, byte-for-byte reproducibility
across runs, metadata sha256 matches the file, and backward-compat without
--tarball. buildifier + py_compile clean; openspec --strict passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

mfreeman451 reviewed

2026-05-31 16:44:52 +00:00

mfreeman451 left a comment

lgtm

mfreeman451 merged commit b262aac497 into staging

2026-05-31 16:44:57 +00:00

mfreeman451 referenced this pull request from a commit

2026-05-31 16:44:59 +00:00

Merge pull request 'feat(build): produce per-arch pushed-artifact tarballs for native add-ons (#3425)' (#3470) from feat/addon-bundle-tarball into staging

mfreeman451 referenced this pull request from a commit

2026-05-31 16:52:57 +00:00

feat(build): add os-package native add-on template (#3425)