Tracker: host-network-visibility + native add-on framework OpenSpec proposals #3456

New issue

Open

opened 2026-05-29 17:19:49 +00:00 by mfreeman451 · 0 comments

mfreeman451 commented

2026-05-29 17:19:49 +00:00

Owner

Purpose

Umbrella tracker for host-network-visibility (netprobe / eBPF / DPI), the native add-on framework, and the netprobe + Bumblebee migrations onto that framework. This issue is an index — canonical work-tracking is each proposal's tasks.md; don't move individual sub-tasks here.

Scoped 2026-06-01 to netprobe / eBPF / DPI / Bumblebee + the add-on framework they ride on. The remote-access / Teleport cluster, per-agent-availability, streamed-config, dependency-catalog, and SRQL proposals were removed from this tracker (tracked elsewhere).

Status snapshot (2026-06-01)

Proposal	Done / Total	Status
`add-host-network-visibility-sidecar`	203/272	netprobe sidecar (p0f/JA4 fingerprinting, DPI, eBPF process attribution). Phases 1–4 mostly done; Phase 5 (remote pcapng) + 6 deferred.
`add-agent-feature-sets`	17/45	Framework spine. Agent-sidecar path done; the rest delivered via the 4 sub-proposals below.
`add-native-addon-delivery-models`	10/11	Per-arch pushed-artifact tarballs, os-package template, supervision models, verified activation + rollback. Remaining: §1.1 rdp-adapter carve (a remote-access binary — out of this tracker's scope).
`add-native-addon-build-signing`	6/13	Build→sign→publish→import landed + live-validated. Remaining: §5.2 verify-before-release rejects tampered/unsigned in CI; §3.1 dead-code/size gate. (Checkbox count lags the merged pipeline.)
`add-native-addon-rust-sdk`	7/7	✅ Complete. Rust handshake/gRPC SDK + `rust-sample` reference add-on (Go↔Rust go-plugin interop).
`add-native-addon-edge-ops`	3/9	Status read model + SRQL entity landed (#3455). Remaining: Edge Ops UI — per-cohort targeting, approval review, assigned/installed/active drift card, onboarding feature-set selection.
`migrate-netprobe-to-native-addon`	5/15	Active (this session). Carve + bundle + signed systemd-service unit + eBPF-object delivery + agent attach/assignment-gated cutover + config-generator contract merged. Remaining: §2.3 status, §2.4 rollback tests, §3.1 publish+seed real AddonPackage, §3.3 Edge Ops, §4.x verification (esp. §4.3 e2e).
`migrate-bumblebee-to-native-addon`	0/15	Drafted, not merged (branch `feat/migrate-bumblebee-native-addon`). Deliver Bumblebee as a native add-on (pushed-artifact / systemd-timer). Framework deps now merged → ready to start; clean parallel workstream.
`add-bumblebee-agent-exposure`	32/33	Developer-endpoint exposure scanning. Phase 6 nearly complete.

Native add-on framework — build → sign → publish → import → assign → activate

The end-to-end delivery loop is now wired in code (all merged this session, #3425):

Build / sign / publish: per-arch tarball assembler + os-package template (#3470, #3471), netprobe carve + bundle (#3472), build fixes (#3473), per-arch ed25519 signer (#3474), OCI publish/sign/verify/index lane (#3475 — live-validated against the registry + OpenBao cosign).
Import → AddonPackage: core verify-then-mirror importer (#3476) + DB test (#3477), object-storage mirror (#3478), web-ng OCI orchestration on a shared ForgejoOciClient (#3479) + FirstPartyImporter dedup & native e2e (#3480).
Assign → activate (netprobe): signed systemd-service unit in the bundle (#3481), sidecar attach mode (#3482), assignment-gated cutover + apply-on-connect (#3483), control-plane config-generator emits the netprobe assignment (#3484), eBPF object delivery so capture actually runs (#3486). Plus build/CI fixes (#3485 rdp-adapter crate ref, #3487 bazel deps).
Validated to a real eBPF object build on a Linux agent host (sr-test-pve04, kernel 6.8): valid BPF ELF carrying the flow→process programs (kprobe tcp_close / udp_recvmsg / udp_sendmsg, kretprobe, tc classifier, maps, BTF).

Outstanding (netprobe / eBPF / DPI / Bumblebee)

netprobe migration (migrate-netprobe-to-native-addon):

§2.3 per-add-on status (installed/active/degraded + capture) through the AddonStatus read model.
§2.4 rollback failure-path tests (machinery exists; untested).
§3.1 publish a real netprobe bundle on Linux CI → import → approve → assign.
§3.3 netprobe selectable in Edge Ops (targeting + drift card).
§4.x verification — esp. §4.3 scratch-Linux-agent e2e (publish → assign → fetch → setcap → enable unit → netprobe loads the eBPF object → capture → ingest). Needs an agent build carrying the §2.2 cutover + a published bundle. sr-test-pve04 is the scratch agent.

build-signing: §5.2 verify-before-release rejects unsigned/tampered in CI; §3.1 whydeadcode/size gate.

edge-ops: the Edge Ops UI (targeting / approval / drift / onboarding) covering netprobe + Bumblebee.

Bumblebee migration: the whole migrate-bumblebee-to-native-addon change (drafted, not started).

Parallelizable workstream (for a second agent)

migrate-bumblebee-to-native-addon is ready to hand to another agent: a complete, independent migration (Bumblebee scanner → native add-on, pushed-artifact / systemd-timer) that consumes the now-merged framework and does not touch the netprobe agent-side code currently in flight. The netprobe migration PRs (#3481–#3486) are the working template; shared surface is minimal (delivery-models systemd-timer supervision, already implemented).

What to do here

Don't copy individual sub-tasks here — they live in each proposal's tasks.md.
Update the table when a proposal moves drafted → active → complete.
Use this issue to coordinate cross-proposal sequencing.

## Purpose Umbrella tracker for **host-network-visibility (netprobe / eBPF / DPI)**, the **native add-on framework**, and the **netprobe + Bumblebee migrations** onto that framework. This issue is an index — canonical work-tracking is each proposal's `tasks.md`; don't move individual sub-tasks here. > **Scoped 2026-06-01** to netprobe / eBPF / DPI / Bumblebee + the add-on framework they ride on. The remote-access / Teleport cluster, per-agent-availability, streamed-config, dependency-catalog, and SRQL proposals were removed from this tracker (tracked elsewhere). ## Status snapshot (2026-06-01) | Proposal | Done / Total | Status | | --- | --- | --- | | `add-host-network-visibility-sidecar` | 203/272 | netprobe sidecar (p0f/JA4 fingerprinting, DPI, eBPF process attribution). Phases 1–4 mostly done; Phase 5 (remote pcapng) + 6 deferred. | | `add-agent-feature-sets` | 17/45 | Framework spine. Agent-sidecar path done; the rest delivered via the 4 sub-proposals below. | | `add-native-addon-delivery-models` | 10/11 | Per-arch pushed-artifact tarballs, os-package template, supervision models, verified activation + rollback. Remaining: §1.1 rdp-adapter carve (a remote-access binary — out of this tracker's scope). | | `add-native-addon-build-signing` | 6/13 | Build→sign→publish→import **landed + live-validated**. Remaining: §5.2 verify-before-release rejects tampered/unsigned in CI; §3.1 dead-code/size gate. (Checkbox count lags the merged pipeline.) | | `add-native-addon-rust-sdk` | 7/7 | ✅ Complete. Rust handshake/gRPC SDK + `rust-sample` reference add-on (Go↔Rust go-plugin interop). | | `add-native-addon-edge-ops` | 3/9 | Status read model + SRQL entity landed (#3455). Remaining: Edge Ops UI — per-cohort targeting, approval review, assigned/installed/active drift card, onboarding feature-set selection. | | `migrate-netprobe-to-native-addon` | 5/15 | **Active (this session).** Carve + bundle + signed systemd-service unit + eBPF-object delivery + agent attach/assignment-gated cutover + config-generator contract merged. Remaining: §2.3 status, §2.4 rollback tests, §3.1 publish+seed real AddonPackage, §3.3 Edge Ops, §4.x verification (esp. §4.3 e2e). | | `migrate-bumblebee-to-native-addon` | 0/15 | **Drafted, not merged** (branch `feat/migrate-bumblebee-native-addon`). Deliver Bumblebee as a native add-on (pushed-artifact / systemd-timer). Framework deps now merged → **ready to start; clean parallel workstream.** | | `add-bumblebee-agent-exposure` | 32/33 | Developer-endpoint exposure scanning. Phase 6 nearly complete. | ## Native add-on framework — build → sign → publish → import → assign → activate The end-to-end delivery loop is now wired **in code** (all merged this session, #3425): - **Build / sign / publish:** per-arch tarball assembler + os-package template (#3470, #3471), netprobe carve + bundle (#3472), build fixes (#3473), per-arch ed25519 signer (#3474), OCI publish/sign/verify/index lane (#3475 — live-validated against the registry + OpenBao cosign). - **Import → AddonPackage:** core verify-then-mirror importer (#3476) + DB test (#3477), object-storage mirror (#3478), web-ng OCI orchestration on a shared `ForgejoOciClient` (#3479) + FirstPartyImporter dedup & native e2e (#3480). - **Assign → activate (netprobe):** signed systemd-service unit in the bundle (#3481), sidecar **attach mode** (#3482), **assignment-gated cutover + apply-on-connect** (#3483), control-plane config-generator emits the netprobe assignment (#3484), **eBPF object delivery so capture actually runs** (#3486). Plus build/CI fixes (#3485 rdp-adapter crate ref, #3487 bazel deps). - **Validated to a real eBPF object build on a Linux agent host** (`sr-test-pve04`, kernel 6.8): valid BPF ELF carrying the flow→process programs (kprobe `tcp_close` / `udp_recvmsg` / `udp_sendmsg`, kretprobe, tc classifier, maps, BTF). ## Outstanding (netprobe / eBPF / DPI / Bumblebee) **netprobe migration** (`migrate-netprobe-to-native-addon`): - §2.3 per-add-on status (installed/active/degraded + capture) through the AddonStatus read model. - §2.4 rollback failure-path tests (machinery exists; untested). - §3.1 publish a real netprobe bundle on **Linux CI** → import → approve → assign. - §3.3 netprobe selectable in Edge Ops (targeting + drift card). - §4.x verification — esp. **§4.3 scratch-Linux-agent e2e** (publish → assign → fetch → setcap → enable unit → netprobe loads the eBPF object → capture → ingest). Needs an agent build carrying the §2.2 cutover + a published bundle. `sr-test-pve04` is the scratch agent. **build-signing:** §5.2 verify-before-release rejects unsigned/tampered in CI; §3.1 whydeadcode/size gate. **edge-ops:** the Edge Ops UI (targeting / approval / drift / onboarding) covering netprobe + Bumblebee. **Bumblebee migration:** the whole `migrate-bumblebee-to-native-addon` change (drafted, not started). ## Parallelizable workstream (for a second agent) `migrate-bumblebee-to-native-addon` is **ready to hand to another agent**: a complete, independent migration (Bumblebee scanner → native add-on, `pushed-artifact` / `systemd-timer`) that consumes the now-merged framework and does **not** touch the netprobe agent-side code currently in flight. The netprobe migration PRs (#3481–#3486) are the working template; shared surface is minimal (delivery-models `systemd-timer` supervision, already implemented). ## What to do here - Don't copy individual sub-tasks here — they live in each proposal's `tasks.md`. - Update the table when a proposal moves drafted → active → complete. - Use this issue to coordinate cross-proposal sequencing.

mfreeman451 self-assigned this

2026-05-29 17:31:24 +00:00

mfreeman451 referenced this issue

2026-06-01 05:39:11 +00:00

feat(addons): migrate Bumblebee scanner to native add-on #3493