2332 feat remove kv for configdynamic config generation #2688
No reviewers
Labels
No labels
1week
2weeks
Failed compliance check
IP cameras
NATS
Possible security concern
Review effort 1/5
Review effort 2/5
Review effort 3/5
Review effort 4/5
Review effort 5/5
UI
aardvark
accessibility
amd64
api
arm64
auth
back-end
bgp
blog
bug
build
checkers
ci-cd
cleanup
cnpg
codex
core
dependencies
device-management
documentation
duplicate
dusk
ebpf
enhancement
eta 1d
eta 1hr
eta 3d
eta 3hr
feature
fieldsurvey
github_actions
go
good first issue
help wanted
invalid
javascript
k8s
log-collector
mapper
mtr
needs-triage
netflow
network-sweep
observability
oracle
otel
plug-in
proton
python
question
reddit
redhat
research
rperf
rperf-checker
rust
sdk
security
serviceradar-agent
serviceradar-agent-gateway
serviceradar-web
serviceradar-web-ng
siem
snmp
sysmon
topology
ubiquiti
wasm
wontfix
zen-engine
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
carverauto/serviceradar!2688
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "refs/pull/2688/head"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Imported from GitHub pull request.
Original GitHub pull request: #2335
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/pull/2335
Original created: 2026-01-18T07:04:53Z
Original updated: 2026-01-18T07:26:07Z
Original head: carverauto/serviceradar:2332-feat-remove-kv-for-configdynamic-config-generation
Original base: staging
Original merged: 2026-01-18T07:26:06Z by @mfreeman451
User description
IMPORTANT: Please sign the Developer Certificate of Origin
Thank you for your contribution to ServiceRadar. Please note, when contributing, the developer must include
a DCO sign-off statement indicating the DCO acceptance in one commit message. Here
is an example DCO Signed-off-by line in a commit message:
Describe your changes
Issue ticket number and link
Code checklist before requesting a review
PR Type
Enhancement
Description
Removed KV store backend support from configuration loading across all services; services now exclusively support file and environment variable configuration sources
Deleted KV-related code including
SetKVStore(),OverlayFromKV(), KV client initialization, bootstrap seeding, and watch/reload functionality from Go and Rust implementationsSimplified configuration architecture by removing KV overlay merging, type coercion helpers, and KV-based template publishing fallbacks
Updated all service configurations (docker-compose, systemd, Helm) to use
CONFIG_SOURCE=fileinstead ofkvRemoved KV dependencies from multiple services: agent, otel, flowgger, trapd, rperf-client, zen consumer, and config-sync tool
Updated documentation and specifications to reflect file-based configuration only, including migration guidance and deprecation notices
Code cleanup including import reorganization, formatting improvements, and removal of unused KV-related error variables
Preserved KV support in config-sync tool by localizing KV client creation function
Diagram Walkthrough
File Walkthrough
13 files
config.go
Remove KV store backend from configuration loadingpkg/config/config.go
environment variable configuration sources
SetKVStore()method andOverlayFromKV()function that handledKV configuration merging
decodeJSONObject,normalizeOverlayTypes,coerceOverlayValue, etc.) for type coercionloadAndValidateWithSource()to return error whenCONFIG_SOURCE=kvis usedserver.go
Remove KV store setup and bootstrap from agent serverpkg/agent/server.go
setupDataStores()function and KV client initialization logicbootstrapKVDefaults()method that seeded default checkerconfigs to KV
resolution functions
NewServer()to skip KV setup and removedconfigStorefieldinitialization
loadCheckerConfig()to load only fromfile
service.go
Remove KV manager and watching from service bootstrappkg/config/bootstrap/service.go
Service()functionnewKVManager(),resolveKeyContext(),applyKVOverlays(),attachWatcherHooks()Resultstruct by removingmanager,cfgLoader, andpinnedPathfieldsStartWatch()andClose()methods no-ops; configuration is nowfile-based only
template.go
Remove KV template publishing fallbackpkg/config/bootstrap/template.go
publishTemplateWithFallback()function that attempted KVpublishing with core fallback
persistTemplateToKV()andretryTemplatePublish()functions forKV template storage
removing KV as a fallback
templatePublishRetryBackoffconstant; onlytemplatePublishTimeoutremainsmain.go
Localize KV client creation in config-sync toolcmd/tools/config-sync/main.go
newKVServiceClientFromEnv()function to replace removedshared utility
modes from environment variables
dependency on shared config package
main.go
Remove KV config source handling from fakercmd/faker/main.go
CONFIG_SOURCE=kvenvironmentvariable
bootstrap path
errors.go
Remove KV-related error definitionspkg/agent/errors.go
errNoSecurityConfigKVanderrFailedToInitializeKVClienterrDataServiceUnavailableerrorlib.rs
Remove KV support from Rust config bootstrap libraryrust/config-bootstrap/src/lib.rs
Bootstrap::new()and
load()methodswatch()method for KV change monitoringoverlay_kv(),sanitize_config(), and relatedsanitization support
BootstrapOptionsstruct by removingkv_key,seed_kv, andwatch_kvfieldsconfiguration
main.rs
Remove KV configuration from OTEL servicecmd/otel/src/main.rs
CONFIG_SOURCE=kvdetection and KV key configurationArcimport andCONFIG_PATHconstantkv_key,seed_kv, andwatch_kvparameters
main.rs
Remove KV support from rperf-clientcmd/checkers/rperf-client/src/main.rs
RestartHandleimport and KV watch/restart spawning codemain.rs
Remove KV configuration from zen consumercmd/consumers/zen/src/main.rs
CONFIG_SOURCE=kvdetection and KV key configurationRestartHandleimport and KV watch/restart spawning code--seed-kvCLI argumentmain.rs
Remove KV support from trapd servicecmd/trapd/src/main.rs
RestartHandleimport and KV watch/restart spawning codemain.rs
Remove KV configuration from flowgger servicecmd/flowgger/src/main.rs
RestartHandleimport and KV watch/restart spawning codeusage
16 files
main.go
Code formatting alignmentcmd/checkers/dusk/main.go
parser.rs
Code formatting improvements in parserrust/srql/src/parser.rs
strip_prefix()andstrip_suffix()chainsagents.rs
Import and query formatting improvementsrust/srql/src/query/agents.rs
services.rs
Import and query formatting improvementsrust/srql/src/query/services.rs
devices.rs
Import and query formatting improvementsrust/srql/src/query/devices.rs
cpu_metrics.rs
Import and query formatting improvementsrust/srql/src/query/cpu_metrics.rs
interfaces.rs
Import statement reorganizationrust/srql/src/query/interfaces.rs
timeseries_metrics.rs
Import statement reorganizationrust/srql/src/query/timeseries_metrics.rs
main.rs
Remove unused importcmd/ebpf/profiler/profiler/src/main.rs
warnimport fromlogcrateprocess_metrics.rs
Import statement reorganizationrust/srql/src/query/process_metrics.rs
device_updates.rs
Import statement reorganizationrust/srql/src/query/device_updates.rs
disk_metrics.rs
Import statement reorganizationrust/srql/src/query/disk_metrics.rs
gateways.rs
Import statement reorganizationrust/srql/src/query/gateways.rs
memory_metrics.rs
Import statement reorganizationrust/srql/src/query/memory_metrics.rs
harness.rs
Code formatting improvementsrust/srql/tests/support/harness.rs
traces.rs
Import statement reorganizationrust/srql/src/query/traces.rs
16 files
docker-compose.yml
Update docker-compose to use file-based configurationdocker-compose.yml
CONFIG_SOURCEfromkvtofilefor otel, flowgger, trapd,db-event-writer, and mapper services
KV_CERT_*, KV_SERVER_NAME) from affected services
values.yaml
Update Helm values for file-based configurationhelm/serviceradar/values.yaml
configSourcefromkvtofileenabled: false) with note thatKV-backed seeding is deprecated
configWatchEnabled: false) with note that KVwatch/reload is no longer supported
_helpers.tpl
Simplify Helm KV environment templatehelm/serviceradar/templates/_helpers.tpl
CONFIG_SOURCE=fileBUILD.bazel
Update Bazel build configuration for KV removalpkg/config/BUILD.bazel
kv_client.go,kv_repair.go,kv_loader.go,kv_watch.go,watchers.go,watchers_snapshot.gotoml_mask.goto build sourcesdb-event-writer.yaml
Update db-event-writer Helm template for file-based confighelm/serviceradar/templates/db-event-writer.yaml
CONFIG_SOURCEto hardcodedfilevalueserviceradar-dusk-checker.service
Update dusk-checker systemd service configurationpackaging/dusk-checker/systemd/serviceradar-dusk-checker.service
CONFIG_SOURCEfromkvtofileserviceradar-rperf-checker.service
Update rperf-checker systemd service configurationpackaging/rperf-checker/systemd/serviceradar-rperf-checker.service
CONFIG_SOURCEfromkvtofileserviceradar-mapper.service
Update mapper systemd service configurationpackaging/mapper/systemd/serviceradar-mapper.service
CONFIG_SOURCEfromkvtofileserviceradar-profiler.service
Update profiler systemd service configurationpackaging/profiler/systemd/serviceradar-profiler.service
CONFIG_SOURCEfromkvtofileserviceradar-sync.service
Update sync systemd service configurationpackaging/sync/systemd/serviceradar-sync.service
CONFIG_SOURCEfromkvtofileserviceradar-otel.service
Update otel systemd service configurationpackaging/otel/systemd/serviceradar-otel.service
CONFIG_SOURCEfromkvtofileserviceradar-event-writer.service
Update event-writer systemd service configurationpackaging/event-writer/systemd/serviceradar-event-writer.service
CONFIG_SOURCEfromkvtofileserviceradar-agent.service
Update agent systemd service configurationpackaging/agent/systemd/serviceradar-agent.service
CONFIG_SOURCEfromkvtofileserviceradar-trapd.service
Update trapd systemd service configurationpackaging/trapd/systemd/serviceradar-trapd.service
CONFIG_SOURCEfromkvtofileserviceradar-flowgger.service
Update flowgger systemd service configurationpackaging/flowgger/systemd/serviceradar-flowgger.service
CONFIG_SOURCEfromkvtofileserviceradar-zen.service
Update zen systemd service configurationpackaging/zen/systemd/serviceradar-zen.service
CONFIG_SOURCEfromkvtofile11 files
agents.md
Update agent documentation for file-based configurationdocs/docs/agents.md
KV store
/etc/serviceradarinstead ofKV keys
management
kv-ci-plan.md
Mark KV CI plan as deprecateddocs/kv-ci-plan.md
been removed
README.md
Update config-bootstrap README for file-based configurationrust/config-bootstrap/README.md
watch support
overlay
spec.md
Add specification for removing KV service configurationopenspec/changes/remove-edge-kv-config/specs/kv-configuration/spec.md
service configuration
configuration
precedence, and KV-based observability
design.md
Add design document for KV configuration removalopenspec/changes/remove-edge-kv-config/design.md
configuration
KV for rules (zen)
snmp.md
Update SNMP documentation for file-based configurationdocs/docs/snmp.md
configuration/profile data
edits or pinned overlay
proposal.md
Add proposal for removing KV service configurationopenspec/changes/remove-edge-kv-config/proposal.md
configuration
merging)
tasks.md
Add task checklist for KV removal projectopenspec/changes/remove-edge-kv-config/tasks.md
pending)
project.md
Update project documentation for KV configuration removalopenspec/project.md
file- or gRPC-based (no KV auto-reload)
kv-configuration.md
Update KV configuration documentationdocs/docs/kv-configuration.md
instead of KV config
configuration
README.md
Improve configuration troubleshooting documentationdocker/README.md
fileorenv/etc/serviceradar22 files
Imported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2335#issuecomment-3764997152
Original created: 2026-01-18T07:06:13Z
PR Compliance Guide 🔍
Below is a summary of compliance checks for this PR:
No security concerns identified
No security vulnerabilities detected by AI analysis. Human verification advised for critical code.🎫 #2332
KV store.
pkg/configby removing KV-related configurationloading/overlay/watch/reload code.
and/or get config over gRPC (agent), instead of KV.
using
CONFIG_SOURCE=kvfor affected services.(and possibly others).
possibly
rust/kvutil), if applicable.Codebase context is not defined
Follow the guide to enable codebase context checks.
Generic: Meaningful Naming and Self-Documenting Code
Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting
Status: Passed
Generic: Robust Error Handling and Edge Case Management
Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation
Status:
Silent error fallback: The new
newKVServiceClientFromEnvreturns(nil, nil, nil)for multiplemisconfiguration/error cases (missing certs, unknown
KV_SEC_MODE), silently disabling KVsync instead of returning an actionable error or warning.
Referred Code
Compliance status legend
🟢 - Fully Compliant🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label
Imported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2335#issuecomment-3764998057
Original created: 2026-01-18T07:07:21Z
PR Code Suggestions ✨
Explore these optional code suggestions:
Avoid nil KV client usage
In
newKVClient, add a check for anilclient returned fromnewKVServiceClientFromEnvand returnnilto prevent creating a brokenkvgrpc.Client.cmd/tools/config-sync/main.go [121-128]
[To ensure code accuracy, apply this suggestion manually]Suggestion importance[1-10]: 8
__
Why: The suggestion correctly identifies a potential nil pointer issue where a
nilclient could be passed tokvgrpc.New, preventing a likely panic and improving robustness.Return error on partial mTLS configuration
Return an explicit error from
newKVServiceClientFromEnvif mTLS is configuredbut required environment variables are missing, instead of silently returning
nil.cmd/tools/config-sync/main.go [142-148]
Suggestion importance[1-10]: 7
__
Why: The suggestion improves error handling by making a silent failure an explicit error, which enhances debuggability for misconfigurations.
Remove unused KV environment variables
Remove unused KV-related environment variables (
KV_ADDRESS,KV_SEC_MODE,KV_CERT_DIR) as the configuration source is now a file.packaging/dusk-checker/systemd/serviceradar-dusk-checker.service [9-12]
Suggestion importance[1-10]: 6
__
Why: The suggestion correctly identifies that with
CONFIG_SOURCE=file, theKV_*environment variables are likely obsolete and should be removed for better code hygiene and clarity.Fix line continuation backslash
In the
curlcommand example, replace the double backslash (\) with a singlebackslash (
</code>) for correct line continuation in the shell.docs/docs/kv-configuration.md [34-35]
Suggestion importance[1-10]: 5
__
Why: The suggestion correctly identifies a typo in a shell command example within the documentation, improving its accuracy and usability for developers.
Simplify error message handling logic
Refactor the error message handling for failed template registrations to be more
concise by inverting the conditional logic.
pkg/config/bootstrap/template.go [116-122]
Suggestion importance[1-10]: 2
__
Why: This is a minor stylistic change that offers a marginal improvement in code conciseness without affecting functionality.
Imported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2335#issuecomment-3765007800
Original created: 2026-01-18T07:21:19Z
CI Feedback 🧐
(Feedback updated until commit
github.com/carverauto/serviceradar@f8fd106366)A test triggered by this PR failed. Here is an AI-generated analysis of the failure:
Action: build
Failed stage: Configure SRQL fixture database for tests [❌]
Failed test name: ""
Failure summary:
The action failed during CI setup because the required secret
SRQL_TEST_DATABASE_CA_CERTwas notconfigured.
- The workflow explicitly checks for
SRQL_TEST_DATABASE_CA_CERTto verify TLS for theSRQL test fixture and aborts when it is empty.
- Log shows:
SRQL_TEST_DATABASE_CA_CERT secret mustbe configured to verify SRQL fixture TLS.followed byProcess completed with exit code 1.(aroundlines 636-637).
Relevant error logs: