fixing zen pipeline #2824
No reviewers
Labels
No labels
1week
2weeks
Failed compliance check
IP cameras
NATS
Possible security concern
Review effort 1/5
Review effort 2/5
Review effort 3/5
Review effort 4/5
Review effort 5/5
UI
aardvark
accessibility
amd64
api
arm64
auth
back-end
bgp
blog
bug
build
checkers
ci-cd
cleanup
cnpg
codex
core
dependencies
device-management
documentation
duplicate
dusk
ebpf
enhancement
eta 1d
eta 1hr
eta 3d
eta 3hr
feature
fieldsurvey
github_actions
go
good first issue
help wanted
invalid
javascript
k8s
log-collector
mapper
mtr
needs-triage
netflow
network-sweep
observability
oracle
otel
plug-in
proton
python
question
reddit
redhat
research
rperf
rperf-checker
rust
sdk
security
serviceradar-agent
serviceradar-agent-gateway
serviceradar-web
serviceradar-web-ng
siem
snmp
sysmon
topology
ubiquiti
wasm
wontfix
zen-engine
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
carverauto/serviceradar!2824
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "refs/pull/2824/head"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Imported from GitHub pull request.
Original GitHub pull request: #2656
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/pull/2656
Original created: 2026-02-01T06:26:11Z
Original updated: 2026-02-01T08:14:41Z
Original head: carverauto/serviceradar:bug/netflow-zen-rules
Original base: staging
Original merged: 2026-02-01T08:14:33Z by @mfreeman451
User description
IMPORTANT: Please sign the Developer Certificate of Origin
Thank you for your contribution to ServiceRadar. Please note, when contributing, the developer must include
a DCO sign-off statement indicating the DCO acceptance in one commit message. Here
is an example DCO Signed-off-by line in a commit message:
Describe your changes
Issue ticket number and link
Code checklist before requesting a review
PR Type
Enhancement, Bug fix
Description
Add NetFlow Zen rule bootstrap with retry logic for deployment tooling
Implement multi-subject stream support in NetFlow collector configuration
Migrate NetFlow stream from dedicated 'flows' to shared 'events' stream
Create netflow_to_ocsf transformation rule for OCSF compliance
Add db-event-writer mapping for processed NetFlow events to database
Diagram Walkthrough
File Walkthrough
8 files
Add stream_subjects configuration fieldImplement multi-subject stream management logicCreate NetFlow to OCSF transformation ruleAdd retry logic for rule installationCreate Helm bootstrap job with retry logicAdd NetFlow to OCSF transformation ruleCreate Kubernetes bootstrap job for rulesAdd NetFlow to OCSF transformation rule18 files
Add NetFlow subject and decision groupPackage NetFlow rule with Zen imageMap processed NetFlow to database tableAdd NetFlow Zen rule and decision groupMap processed NetFlow to database tableUpdate NetFlow collector stream configurationAdd NetFlow subject to Zen consumerInclude bootstrap job in Kubernetes manifestsMap processed NetFlow to database tableUpdate NetFlow collector stream configurationPackage NetFlow rule with Zen artifactsAdd NetFlow rule to package configurationMap processed NetFlow to database tableUpdate NetFlow collector stream configurationExport NetFlow rule file for packagingUpdate NetFlow collector stream configurationUpdate test configuration for new streamUpdate test configuration stream name1 files
Update test script stream name reference7 files
Update testing documentation stream referencesDocument stream_subjects configuration parameterUpdate troubleshooting stream name referencesDocument NetFlow rule bootstrap change proposalDefine Docker Compose bootstrap requirementsDefine rule bootstrap specification requirementsDocument implementation tasks for bootstrap1 files
Imported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2656#issuecomment-3830466637
Original created: 2026-02-01T06:26:59Z
PR Compliance Guide 🔍
Below is a summary of compliance checks for this PR:
Subject capture escalation
Description: The new JetStream management logic updates/creates the stream to include
stream_subjects_resolved()from runtime config, so if an attacker can modify the collectorconfiguration they could add arbitrary subjects and cause the service to “claim” and
retain messages for subjects it should not manage (cross-tenant/cross-service subject
capture), expanding blast radius beyond
flows.raw.netflow.publisher.rs [117-150]
Referred Code
Mutable image tags
Description: The new bootstrap Job pulls container images using mutable
:latesttags (e.g.,ghcr.io/carverauto/serviceradar-tools:latestandghcr.io/carverauto/serviceradar-zen:latest) withimagePullPolicy: Always, which increasessupply-chain risk because a compromised/updated tag can change executed code without a
manifest change.
serviceradar-zen-rules-bootstrap.yaml [23-38]
Referred Code
🎫 No ticket provided
Codebase context is not defined
Follow the guide to enable codebase context checks.
Generic: Meaningful Naming and Self-Documenting Code
Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting
Status: Passed
Generic: Secure Error Handling
Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.
Status: Passed
Generic: Robust Error Handling and Edge Case Management
Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation
Status:
Swallowed error context: The
matchonjs.get_stream(...)usesErr(_)which discards the underlying error and canmask non-"not found" failures, reducing diagnosability and risking incorrect
stream creation behavior.
Referred Code
Generic: Security-First Input Validation and Data Handling
Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities
Status:
Missing subjects validation:
validate()enforces non-emptysubjectbut does not validate that entries in optionalstream_subjectsare non-empty/valid, allowing potentially invalid external config to reachJetStream stream creation/update.
Referred Code
Generic: Comprehensive Audit Trails
Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.
Status:
Unstructured audit logs: The new rule installation retry loop logs actions via plain
echowithout structured fields(e.g., actor/user ID, timestamp) needed to reliably reconstruct deployment-time changes.
Referred Code
Generic: Secure Logging Practices
Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.
Status:
Non-structured logging: The new logging output for rule installation is plain text (not structured) and may be
harder to audit/monitor consistently depending on the logging pipeline requirements.
Referred Code
Compliance status legend
🟢 - Fully Compliant🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label
Imported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2656#issuecomment-3830468003
Original created: 2026-02-01T06:28:06Z
PR Code Suggestions ✨
Explore these optional code suggestions:
Propagate non-not-found errors
Refactor the error handling for
js.get_streaminpublisher.rs. Instead ofcatching all errors with
Err(_), specifically handle "not found" errors andpropagate all other errors to avoid masking critical issues like permission
failures.
rust/netflow-collector/src/publisher.rs [119-150]
Suggestion importance[1-10]: 8
__
Why: This is a critical improvement to error handling. The current code's
Err(_)branch masks important errors like authentication or permission failures, which would be difficult to debug. The suggestion correctly proposes to handle only "not found" errors and propagate all others.Simplify stream creation and update
Simplify the NATS stream creation and update logic in
publisher.rsby replacingthe complex
matchblock with a single, atomicget_or_create_streamcall.rust/netflow-collector/src/publisher.rs [148]
Suggestion importance[1-10]: 7
__
Why: The suggestion correctly points out that the complex logic for creating/updating a NATS stream can be replaced by a single, atomic
get_or_create_streamcall, which simplifies the code and improves maintainability.Improve subject resolution logic efficiency
Refactor
stream_subjects_resolvedinconfig.rsto use aHashSetfor moreefficient and idiomatic deduplication of subjects, instead of sorting and
calling
dedup()on aVec.rust/netflow-collector/src/config.rs [163-174]
Suggestion importance[1-10]: 4
__
Why: The suggestion offers a more idiomatic and potentially more performant way to deduplicate subjects using a
HashSet. While the current implementation is correct, this change improves code quality and clarity.Consolidate duplicated rule installation logic
The shell script logic for installing Zen rules with retries is duplicated in
Docker, Helm, and Kubernetes configurations. This logic should be consolidated
into a shared script or built into the
zen-put-ruletool to improvemaintainability.
Examples:
docker/compose/zen-install-rules.sh [54-81]
helm/serviceradar/templates/zen-rules-bootstrap-job.yaml [71-98]
Solution Walkthrough:
Before:
After:
Suggestion importance[1-10]: 7
__
Why: The suggestion correctly identifies significant code duplication of the retry logic across multiple deployment configuration files, which impacts future maintainability.
Ensure script exits on error
Add
set -etodocker/compose/zen-install-rules.shto ensure the script exitsimmediately if a rule installation fails, preventing silent failures.
docker/compose/zen-install-rules.sh [87-93]
[To ensure code accuracy, apply this suggestion manually]Suggestion importance[1-10]: 7
__
Why: This suggestion correctly identifies that the script would not fail if an
install_rulecommand fails, leading to a silent error in the deployment process. Addingset -eis a crucial fix for script robustness.Exit Job on install failures
Add strict mode (
set -euo pipefail) and explicit exit checks (|| exit 1) to theinstall_rulecommands inzen-rules-bootstrap-job.yamlto ensure the Job failsupon rule installation failure.
helm/serviceradar/templates/zen-rules-bootstrap-job.yaml [103-117]
[To ensure code accuracy, apply this suggestion manually]Suggestion importance[1-10]: 7
__
Why: This suggestion correctly identifies that the bootstrap script within the Kubernetes Job manifest lacks error handling, which could cause the Job to succeed silently even if rule installation fails. This is a significant reliability improvement.
Imported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2656#issuecomment-3830593850
Original created: 2026-02-01T08:14:30Z
CI Feedback 🧐
A test triggered by this PR failed. Here is an AI-generated analysis of the failure:
Action: build
Failed stage: Configure SRQL fixture database for tests [❌]
Failed test name: ""
Failure summary:
The action failed during environment/fixture setup because the required secret
SRQL_TEST_DATABASE_CA_CERTwas not configured.- The log explicitly reports:
SRQL_TEST_DATABASE_CA_CERT secret must be configured to verify SRQL fixture TLS.(line 676)- The
workflow then exited with
exit code 1(line 677).Relevant error logs: