#2908 Merge Flowgger and OTEL into unified log-collector #2969

Merged

mfreeman451 merged 1 commit from refs/pull/2969/head into staging

2026-02-26 20:51:32 +00:00

mfreeman451 commented

2026-02-26 20:39:48 +00:00

(Migrated from github.com)

Owner

Imported from GitHub pull request.

Original GitHub pull request: #2920
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/pull/2920
Original created: 2026-02-26T20:39:48Z
Original updated: 2026-02-26T20:51:34Z
Original head: carverauto/serviceradar:2908-refactor-merge-otel-and-flowgger-log-collectors
Original base: staging
Original merged: 2026-02-26T20:51:32Z by @mfreeman451

Summary

Replaces two independent log-collection daemons (serviceradar-flowgger, serviceradar-otel) with a single unified binary (serviceradar-log-collector) that composes both as library dependencies
Config delegation model: unified TOML config points to native sub-collector configs — no config format migration needed
Single tonic-health gRPC endpoint on port 50044 replaces per-daemon health checks for K8s probes
Backward-compatible NATS subjects preserved (logs.syslog, logs.otel, otel.traces, otel.metrics, otel.metrics.raw)
Removed old Dockerfiles, entrypoints, systemd units, Helm templates, and k8s manifests for flowgger/otel
Updated CI workflows, build scripts, Helm values, SPIRE IDs, and cert generation

Key changes

New crate: rust/log-collector/ — thin orchestration layer with syslog and otel feature flags
Library conversion: rust/flowgger/ and rust/otel/ converted from binaries to library-only crates
Bug fix: OTEL nats_output.rs now handles pre-existing JetStream streams with different subjects (fetch-and-update fallback)
Deferred: NATS output unification and unified Prometheus metrics (preserves Flowgger upstream compatibility)

Test plan

Syslog UDP input → NATS logs.syslog (RFC 3164 → GELF JSON, verified via docker-compose)
OTEL gRPC traces → NATS otel.traces + perf metrics → otel.metrics
OTEL gRPC logs → NATS logs.otel
OTEL gRPC metrics → NATS otel.metrics.raw
gRPC health check: all 4 services ("", "log-collector", "flowgger", "otel") return SERVING
mTLS credential loading verified (both pipelines connect to NATS with log-collector.pem certs)
CI rust tests pass
Docker image builds in CI

Closes #2908

🤖 Generated with Claude Code

Imported from GitHub pull request. Original GitHub pull request: #2920 Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/pull/2920 Original created: 2026-02-26T20:39:48Z Original updated: 2026-02-26T20:51:34Z Original head: carverauto/serviceradar:2908-refactor-merge-otel-and-flowgger-log-collectors Original base: staging Original merged: 2026-02-26T20:51:32Z by @mfreeman451 --- ## Summary - Replaces two independent log-collection daemons (`serviceradar-flowgger`, `serviceradar-otel`) with a single unified binary (`serviceradar-log-collector`) that composes both as library dependencies - Config delegation model: unified TOML config points to native sub-collector configs — no config format migration needed - Single tonic-health gRPC endpoint on port 50044 replaces per-daemon health checks for K8s probes - Backward-compatible NATS subjects preserved (`logs.syslog`, `logs.otel`, `otel.traces`, `otel.metrics`, `otel.metrics.raw`) - Removed old Dockerfiles, entrypoints, systemd units, Helm templates, and k8s manifests for flowgger/otel - Updated CI workflows, build scripts, Helm values, SPIRE IDs, and cert generation ## Key changes - **New crate**: `rust/log-collector/` — thin orchestration layer with `syslog` and `otel` feature flags - **Library conversion**: `rust/flowgger/` and `rust/otel/` converted from binaries to library-only crates - **Bug fix**: OTEL `nats_output.rs` now handles pre-existing JetStream streams with different subjects (fetch-and-update fallback) - **Deferred**: NATS output unification and unified Prometheus metrics (preserves Flowgger upstream compatibility) ## Test plan - [x] Syslog UDP input → NATS `logs.syslog` (RFC 3164 → GELF JSON, verified via docker-compose) - [x] OTEL gRPC traces → NATS `otel.traces` + perf metrics → `otel.metrics` - [x] OTEL gRPC logs → NATS `logs.otel` - [x] OTEL gRPC metrics → NATS `otel.metrics.raw` - [x] gRPC health check: all 4 services (`""`, `"log-collector"`, `"flowgger"`, `"otel"`) return SERVING - [x] mTLS credential loading verified (both pipelines connect to NATS with `log-collector.pem` certs) - [ ] CI rust tests pass - [ ] Docker image builds in CI Closes #2908 🤖 Generated with [Claude Code](https://claude.com/claude-code)