Chore/netflow parser 090 #2852

Merged
mikemiles-dev merged 4 commits from refs/pull/2852/head into staging 2026-02-05 04:20:03 +00:00
mikemiles-dev commented 2026-02-05 01:56:57 +00:00 (Migrated from github.com)
Owner

Imported from GitHub pull request.

Original GitHub pull request: #2702
Original author: @mikemiles-dev
Original URL: https://github.com/carverauto/serviceradar/pull/2702
Original created: 2026-02-05T01:56:57Z
Original updated: 2026-02-05T04:20:03Z
Original head: mikemiles-dev/serviceradar:chore/netflow_parser_090
Original base: staging
Original merged: 2026-02-05T04:20:03Z by @mfreeman451

IMPORTANT: Please sign the Developer Certificate of Origin

Thank you for your contribution to ServiceRadar. Please note, when contributing, the developer must include
a DCO sign-off statement indicating the DCO acceptance in one commit message. Here
is an example DCO Signed-off-by line in a commit message:

Signed-off-by: J. Doe <j.doe@domain.com>

Describe your changes

Code checklist before requesting a review

  • I have signed the DCO?
  • The build completes without errors?
  • All tests are passing when running make test?
Imported from GitHub pull request. Original GitHub pull request: #2702 Original author: @mikemiles-dev Original URL: https://github.com/carverauto/serviceradar/pull/2702 Original created: 2026-02-05T01:56:57Z Original updated: 2026-02-05T04:20:03Z Original head: mikemiles-dev/serviceradar:chore/netflow_parser_090 Original base: staging Original merged: 2026-02-05T04:20:03Z by @mfreeman451 --- ## IMPORTANT: Please sign the Developer Certificate of Origin Thank you for your contribution to ServiceRadar. Please note, when contributing, the developer must include a [DCO sign-off statement]( https://developercertificate.org/) indicating the DCO acceptance in one commit message. Here is an example DCO Signed-off-by line in a commit message: ``` Signed-off-by: J. Doe <j.doe@domain.com> ``` ## Describe your changes ## Issue ticket number and link ## Code checklist before requesting a review - [ ] I have signed the DCO? - [ ] The build completes without errors? - [ ] All tests are passing when running make test?
qodo-code-review[bot] commented 2026-02-05 01:57:18 +00:00 (Migrated from github.com)
Author
Owner

Imported GitHub PR comment.

Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2702#issuecomment-3850645124
Original created: 2026-02-05T01:57:18Z

ⓘ You are approaching your monthly quota for Qodo. Upgrade your plan

Review Summary by Qodo

Add pending flows caching support and upgrade netflow parser to 0.9.0

✨ Enhancement

Grey Divider

Walkthroughs

Description
• Upgrade netflow_parser to 0.9.0 with pending flows caching support
• Add configuration for buffering flows arriving before their template
• Refactor converter to handle NoTemplate flowsets with debug logging
• Update template event callback to differentiate pending vs lost flows
• Add pending flows metrics reporting for V9 and IPFIX protocols
Diagram
flowchart LR
  A["netflow_parser 0.9.0"] -->|PendingFlowsConfig| B["Listener"]
  B -->|pending_flows enabled| C["AutoScopedParser"]
  C -->|NoTemplate flowsets| D["Converter"]
  D -->|debug logging| E["Metrics Reporter"]
  F["Config"] -->|PendingFlowsCacheConfig| B
Grey Divider

File Changes

1. rust/netflow-collector/src/config.rs ⚙️ Configuration changes +37/-0

Add pending flows cache configuration structure

rust/netflow-collector/src/config.rs


2. rust/netflow-collector/src/converter.rs ✨ Enhancement +316/-298

Handle NoTemplate flowsets with debug logging

rust/netflow-collector/src/converter.rs


3. rust/netflow-collector/src/listener.rs ✨ Enhancement +71/-51

Integrate pending flows config into parser builder

rust/netflow-collector/src/listener.rs


View more (9)
4. rust/netflow-collector/src/main.rs ✨ Enhancement +13/-0

Log pending flows cache configuration at startup

rust/netflow-collector/src/main.rs


5. rust/netflow-collector/src/metrics.rs ✨ Enhancement +30/-0

Report pending flows metrics for V9 and IPFIX

rust/netflow-collector/src/metrics.rs


6. rust/netflow-collector/src/publisher.rs 🧪 Tests +1/-1

Update test config to include pending_flows field

rust/netflow-collector/src/publisher.rs


7. rust/netflow-collector/Cargo.toml Dependencies +1/-1

Upgrade netflow_parser dependency to 0.9.0

rust/netflow-collector/Cargo.toml


8. rust/netflow-collector/quick-test.sh ⚙️ Configuration changes +1/-1

Update netflow_generator Docker image to 0.2.7

rust/netflow-collector/quick-test.sh


9. rust/netflow-collector/TESTING.md 📝 Documentation +3/-3

Update netflow_generator Docker image references

rust/netflow-collector/TESTING.md


10. packaging/netflow-collector/config/netflow-collector.json ⚙️ Configuration changes +5/-0

Add pending flows cache configuration example

packaging/netflow-collector/config/netflow-collector.json


11. rust/netflow-collector/netflow-collector-test.json ⚙️ Configuration changes +5/-0

Add pending flows cache configuration to test config

rust/netflow-collector/netflow-collector-test.json


12. rust/netflow-collector/netflow-collector.json ⚙️ Configuration changes +5/-0

Add pending flows cache configuration to default config

rust/netflow-collector/netflow-collector.json


Grey Divider

Qodo Logo

Imported GitHub PR comment. Original author: @qodo-code-review[bot] Original URL: https://github.com/carverauto/serviceradar/pull/2702#issuecomment-3850645124 Original created: 2026-02-05T01:57:18Z --- <pre>ⓘ You are approaching your monthly quota for Qodo. <a href="https://www.qodo.ai/pricing">Upgrade your plan</a></pre> <h3>Review Summary by Qodo</h3> Add pending flows caching support and upgrade netflow parser to 0.9.0 <code>✨ Enhancement</code> <img src="https://www.qodo.ai/wp-content/uploads/2025/11/light-grey-line.svg" height="10%" alt="Grey Divider"> <h3>Walkthroughs</h3> <details open> <summary>Description</summary> <br/> <pre> • Upgrade netflow_parser to 0.9.0 with pending flows caching support • Add configuration for buffering flows arriving before their template • Refactor converter to handle NoTemplate flowsets with debug logging • Update template event callback to differentiate pending vs lost flows • Add pending flows metrics reporting for V9 and IPFIX protocols </pre> </details> <details> <summary>Diagram</summary> <br/> > ```mermaid flowchart LR A["netflow_parser 0.9.0"] -->|PendingFlowsConfig| B["Listener"] B -->|pending_flows enabled| C["AutoScopedParser"] C -->|NoTemplate flowsets| D["Converter"] D -->|debug logging| E["Metrics Reporter"] F["Config"] -->|PendingFlowsCacheConfig| B ``` </details> <img src="https://www.qodo.ai/wp-content/uploads/2025/11/light-grey-line.svg" height="10%" alt="Grey Divider"> <h3>File Changes</h3> <details> <summary>1. rust/netflow-collector/src/config.rs <code>⚙️ Configuration changes</code> <code> +37/-0 </code> </summary> <br/> >Add pending flows cache configuration structure > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-ea8fb7fc6e55836ada19b9c1ca26f2a88575289a3fee3005183916a8454d0fc9'> rust/netflow-collector/src/config.rs </a> <hr/> </details> <details> <summary>2. rust/netflow-collector/src/converter.rs <code>✨ Enhancement</code> <code> +316/-298 </code> </summary> <br/> >Handle NoTemplate flowsets with debug logging > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-873d955c2362720f0772872c1dfaba79c512b09dc5c794009ea466ba73cbaae4'> rust/netflow-collector/src/converter.rs </a> <hr/> </details> <details> <summary>3. rust/netflow-collector/src/listener.rs <code>✨ Enhancement</code> <code> +71/-51 </code> </summary> <br/> >Integrate pending flows config into parser builder > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-3ec4d0175a1f407a9c9246268da3491d1fdfacd2e8c00d536fde36357e993999'> rust/netflow-collector/src/listener.rs </a> <hr/> </details> <details><summary><ins><strong>View more (9)</strong></ins></summary><br/> <details> <summary>4. rust/netflow-collector/src/main.rs <code>✨ Enhancement</code> <code> +13/-0 </code> </summary> <br/> >Log pending flows cache configuration at startup > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-ec91a491c51b432333dc1b7add92effe5fdabfad9a7a807a1af3a05a9602c3bb'> rust/netflow-collector/src/main.rs </a> <hr/> </details> <details> <summary>5. rust/netflow-collector/src/metrics.rs <code>✨ Enhancement</code> <code> +30/-0 </code> </summary> <br/> >Report pending flows metrics for V9 and IPFIX > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-d3fef810bfa2a5b985b71300423ec1d9c6eea321fee261710cd3257b521ad7e2'> rust/netflow-collector/src/metrics.rs </a> <hr/> </details> <details> <summary>6. rust/netflow-collector/src/publisher.rs <code>🧪 Tests</code> <code> +1/-1 </code> </summary> <br/> >Update test config to include pending_flows field > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-10dec5866b4bc416c62590bdf2d3ded6c234b8d423ec4aebc70eac891ff0c07d'> rust/netflow-collector/src/publisher.rs </a> <hr/> </details> <details> <summary>7. rust/netflow-collector/Cargo.toml <code> Dependencies </code> <code> +1/-1 </code> </summary> <br/> >Upgrade netflow_parser dependency to 0.9.0 > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-03c9bd143b73f35c01bd56390e03d9bde76065f53473ed6a70aa49c6dd1d67b3'> rust/netflow-collector/Cargo.toml </a> <hr/> </details> <details> <summary>8. rust/netflow-collector/quick-test.sh <code>⚙️ Configuration changes</code> <code> +1/-1 </code> </summary> <br/> >Update netflow_generator Docker image to 0.2.7 > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-c0bfb16cf98bf7ee9339462c2c54e7518086e0adbf2bc0ef1a30315ae9919f20'> rust/netflow-collector/quick-test.sh </a> <hr/> </details> <details> <summary>9. rust/netflow-collector/TESTING.md <code>📝 Documentation</code> <code> +3/-3 </code> </summary> <br/> >Update netflow_generator Docker image references > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-2fd22117bc767137b2b988b14694a0b9c1be7fe91f2174f77fbea9401399a9bc'> rust/netflow-collector/TESTING.md </a> <hr/> </details> <details> <summary>10. packaging/netflow-collector/config/netflow-collector.json <code>⚙️ Configuration changes</code> <code> +5/-0 </code> </summary> <br/> >Add pending flows cache configuration example > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-f69f967d5a0aa98e0a1167dc137203c6433bf734daeb112956bc4c11aa4dd67e'> packaging/netflow-collector/config/netflow-collector.json </a> <hr/> </details> <details> <summary>11. rust/netflow-collector/netflow-collector-test.json <code>⚙️ Configuration changes</code> <code> +5/-0 </code> </summary> <br/> >Add pending flows cache configuration to test config > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-938af48a34400ecdbd476d7589d28af0cb991e4c297587186917e3d8d931a955'> rust/netflow-collector/netflow-collector-test.json </a> <hr/> </details> <details> <summary>12. rust/netflow-collector/netflow-collector.json <code>⚙️ Configuration changes</code> <code> +5/-0 </code> </summary> <br/> >Add pending flows cache configuration to default config > ><a href='https://github.com/carverauto/serviceradar/pull/2702/files#diff-11279498c772a86f80c6e32c2d357b896337db740a27fe65b2393a00e28d6f17'> rust/netflow-collector/netflow-collector.json </a> <hr/> </details> </details> <img src="https://www.qodo.ai/wp-content/uploads/2025/11/light-grey-line.svg" height="10%" alt="Grey Divider"> <a href="https://www.qodo.ai"><img src="https://www.qodo.ai/wp-content/uploads/2025/03/qodo-logo.svg" width="80" alt="Qodo Logo"></a>
qodo-code-review[bot] commented 2026-02-05 02:02:10 +00:00 (Migrated from github.com)
Author
Owner

Imported GitHub PR comment.

Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2702#issuecomment-3850656948
Original created: 2026-02-05T02:02:10Z

ⓘ You are approaching your monthly quota for Qodo. Upgrade your plan

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (2) 📎 Requirement gaps (0)

Grey Divider
Action required
1. Non-ASCII checkmarks in TESTING.md 📘 Rule violation ✓ Correctness
Description
• The modified Markdown file rust/netflow-collector/TESTING.md contains non-ASCII symbols (e.g.,
  , , ).
• This violates the repository requirement that modified Markdown documentation be ASCII-only, which
  can cause encoding/rendering issues in some tooling.
Code

rust/netflow-collector/TESTING.md[R17-18]

+docker run --rm --network host ghcr.io/mikemiles-dev/netflow_generator:0.2.7 \
  --dest 127.0.0.1:2055 --once
Evidence
PR Compliance ID 11 requires ASCII-only content in any modified Markdown files. TESTING.md was
modified in this PR and contains multiple non-ASCII characters (checkmarks/crosses) elsewhere in the
file.

AGENTS.md
rust/netflow-collector/TESTING.md[307-325]
rust/netflow-collector/TESTING.md[515-525]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`rust/netflow-collector/TESTING.md` contains non-ASCII characters (e.g., `✓`, `✗`, `✅`) but modified Markdown files must be ASCII-only.

## Issue Context
This PR touched `TESTING.md`, so the file must comply with the ASCII-only Markdown rule.

## Fix Focus Areas
- rust/netflow-collector/TESTING.md[307-325]
- rust/netflow-collector/TESTING.md[515-525]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Pending flows lack TTL 🐞 Bug ⛯ Reliability
Suggestion Impact:Introduced a pending flow TTL field in config with a default (300s), added validation bounds for ttl and other pending-flow limits, and updated the listener to set PendingFlowsConfig.ttl to Some(Duration::from_secs(ttl_secs)) rather than None.

code diff:

# File: rust/netflow-collector/src/config.rs
@@ -73,6 +73,8 @@
     pub max_entries_per_template: usize,
     #[serde(default = "default_max_entry_size_bytes")]
     pub max_entry_size_bytes: usize,
+    #[serde(default = "default_pending_ttl_secs")]
+    pub ttl_secs: u64,
 }
 
 #[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Eq)]
@@ -160,6 +162,10 @@
 
 fn default_max_entry_size_bytes() -> usize {
     65535
+}
+
+fn default_pending_ttl_secs() -> u64 {
+    300
 }
 
 impl Config {
@@ -184,14 +190,17 @@
             anyhow::bail!("subject cannot be empty");
         }
         if let Some(pf) = &self.pending_flows {
-            if pf.max_pending_flows == 0 {
-                anyhow::bail!("pending_flows.max_pending_flows must be > 0");
-            }
-            if pf.max_entries_per_template == 0 {
-                anyhow::bail!("pending_flows.max_entries_per_template must be > 0");
-            }
-            if pf.max_entry_size_bytes == 0 {
-                anyhow::bail!("pending_flows.max_entry_size_bytes must be > 0");
+            if pf.max_pending_flows == 0 || pf.max_pending_flows > 10_000 {
+                anyhow::bail!("pending_flows.max_pending_flows must be 1..=10,000");
+            }
+            if pf.max_entries_per_template == 0 || pf.max_entries_per_template > 100_000 {
+                anyhow::bail!("pending_flows.max_entries_per_template must be 1..=100,000");
+            }
+            if pf.max_entry_size_bytes == 0 || pf.max_entry_size_bytes > 1_048_576 {
+                anyhow::bail!("pending_flows.max_entry_size_bytes must be 1..=1,048,576");
+            }
+            if pf.ttl_secs == 0 || pf.ttl_secs > 3600 {
+                anyhow::bail!("pending_flows.ttl_secs must be 1..=3,600");
             }
         }
         Ok(())

# File: rust/netflow-collector/src/config.rs
@@ -73,6 +73,8 @@
     pub max_entries_per_template: usize,
     #[serde(default = "default_max_entry_size_bytes")]
     pub max_entry_size_bytes: usize,
+    #[serde(default = "default_pending_ttl_secs")]
+    pub ttl_secs: u64,
 }
 
 #[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Eq)]
@@ -160,6 +162,10 @@
 
 fn default_max_entry_size_bytes() -> usize {
     65535
+}
+
+fn default_pending_ttl_secs() -> u64 {
+    300
 }
 
 impl Config {
@@ -184,14 +190,17 @@
             anyhow::bail!("subject cannot be empty");
         }
         if let Some(pf) = &self.pending_flows {
-            if pf.max_pending_flows == 0 {
-                anyhow::bail!("pending_flows.max_pending_flows must be > 0");
-            }
-            if pf.max_entries_per_template == 0 {
-                anyhow::bail!("pending_flows.max_entries_per_template must be > 0");
-            }
-            if pf.max_entry_size_bytes == 0 {
-                anyhow::bail!("pending_flows.max_entry_size_bytes must be > 0");
+            if pf.max_pending_flows == 0 || pf.max_pending_flows > 10_000 {
+                anyhow::bail!("pending_flows.max_pending_flows must be 1..=10,000");
+            }
+            if pf.max_entries_per_template == 0 || pf.max_entries_per_template > 100_000 {
+                anyhow::bail!("pending_flows.max_entries_per_template must be 1..=100,000");
+            }
+            if pf.max_entry_size_bytes == 0 || pf.max_entry_size_bytes > 1_048_576 {
+                anyhow::bail!("pending_flows.max_entry_size_bytes must be 1..=1,048,576");
+            }
+            if pf.ttl_secs == 0 || pf.ttl_secs > 3600 {
+                anyhow::bail!("pending_flows.ttl_secs must be 1..=3,600");
             }
         }
         Ok(())

# File: rust/netflow-collector/src/listener.rs
@@ -8,7 +8,7 @@
 use prost::Message;
 use std::net::SocketAddr;
 use std::sync::{Arc, Mutex};
-use std::time::{SystemTime, UNIX_EPOCH};
+use std::time::{Duration, SystemTime, UNIX_EPOCH};
 use tokio::net::UdpSocket;
 use tokio::sync::mpsc;
 
@@ -61,7 +61,7 @@
                 if pending_enabled {
                     debug!(
                         "Missing template - ID: {}, Protocol: {:?}. \
-                         Flow data buffered for replay when template arrives.",
+                         Pending flow cache enabled; data queued if capacity allows.",
                         template_id, protocol
                     );
                 } else {
@@ -98,7 +98,7 @@
                 max_pending_flows: pf.max_pending_flows,
                 max_entries_per_template: pf.max_entries_per_template,
                 max_entry_size_bytes: pf.max_entry_size_bytes,
-                ttl: None,
+                ttl: Some(Duration::from_secs(pf.ttl_secs)),
             });

Description
• Pending flow caching is enabled via config, but the implementation hard-codes ttl: None, meaning
  buffered flows may be replayed arbitrarily late (until size-based eviction).
• This can produce very stale/out-of-order flow delivery when templates arrive late and makes
  memory/latency behavior harder to reason about operationally.
Code

rust/netflow-collector/src/listener.rs[R96-102]

+        if let Some(pf) = &config.pending_flows {
+            builder = builder.with_pending_flows(PendingFlowsConfig {
+                max_pending_flows: pf.max_pending_flows,
+                max_entries_per_template: pf.max_entries_per_template,
+                max_entry_size_bytes: pf.max_entry_size_bytes,
+                ttl: None,
+            });
Evidence
Listener always sets ttl: None when enabling pending flows, and the application config schema
provides no TTL field to allow operators to bound retention/replay age.

rust/netflow-collector/src/listener.rs[91-103]
rust/netflow-collector/src/config.rs[68-76]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Pending flow caching is configured, but the code forces `ttl: None`, so buffered flows may be replayed arbitrarily late (until size-based eviction). This can lead to very stale/out-of-order flow emission and makes resource behavior harder to control.

### Issue Context
`PendingFlowsCacheConfig` currently only exposes size-based limits. `Listener::new` always sets `ttl: None` when pending flow buffering is enabled.

### Fix Focus Areas
- rust/netflow-collector/src/config.rs[68-76]
- rust/netflow-collector/src/config.rs[173-197]
- rust/netflow-collector/src/listener.rs[96-103]

### Suggested fix
1. Extend `PendingFlowsCacheConfig` with an optional TTL field (e.g., `ttl_seconds: Option&lt;u64&gt;` with a sensible default like 300s).
2. Add validation: if provided, TTL must be &gt; 0.
3. Convert TTL seconds to the type expected by `PendingFlowsConfig.ttl` (likely `Some(Duration::from_secs(ttl_seconds))`).
4. Update example JSON configs to include/mention TTL so operators can tune it.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended
3. pending_flows lacks upper bounds 📘 Rule violation ⛨ Security
Suggestion Impact:Updated pending_flows validation to enforce maximum allowed values (not just >0) for max_pending_flows, max_entries_per_template, and max_entry_size_bytes; also introduced and bounded a new ttl_secs pending flow setting.

code diff:

@@ -184,14 +190,17 @@
             anyhow::bail!("subject cannot be empty");
         }
         if let Some(pf) = &self.pending_flows {
-            if pf.max_pending_flows == 0 {
-                anyhow::bail!("pending_flows.max_pending_flows must be > 0");
-            }
-            if pf.max_entries_per_template == 0 {
-                anyhow::bail!("pending_flows.max_entries_per_template must be > 0");
-            }
-            if pf.max_entry_size_bytes == 0 {
-                anyhow::bail!("pending_flows.max_entry_size_bytes must be > 0");
+            if pf.max_pending_flows == 0 || pf.max_pending_flows > 10_000 {
+                anyhow::bail!("pending_flows.max_pending_flows must be 1..=10,000");
+            }
+            if pf.max_entries_per_template == 0 || pf.max_entries_per_template > 100_000 {
+                anyhow::bail!("pending_flows.max_entries_per_template must be 1..=100,000");
+            }
+            if pf.max_entry_size_bytes == 0 || pf.max_entry_size_bytes > 1_048_576 {
+                anyhow::bail!("pending_flows.max_entry_size_bytes must be 1..=1,048,576");
+            }
+            if pf.ttl_secs == 0 || pf.ttl_secs > 3600 {
+                anyhow::bail!("pending_flows.ttl_secs must be 1..=3,600");
             }
         }

Description
• The new pending_flows settings are validated only for > 0, with no maximum limits enforced for
  max_pending_flows, max_entries_per_template, or max_entry_size_bytes.
• Because these values come from external JSON config files, extremely large values could trigger
  excessive memory usage (potential DoS) or unstable runtime behavior.
Code

rust/netflow-collector/src/config.rs[R186-196]

+        if let Some(pf) = &self.pending_flows {
+            if pf.max_pending_flows == 0 {
+                anyhow::bail!("pending_flows.max_pending_flows must be > 0");
+            }
+            if pf.max_entries_per_template == 0 {
+                anyhow::bail!("pending_flows.max_entries_per_template must be > 0");
+            }
+            if pf.max_entry_size_bytes == 0 {
+                anyhow::bail!("pending_flows.max_entry_size_bytes must be > 0");
+            }
+        }
Evidence
PR Compliance ID 6 requires validation/sanitization of external inputs. The new external
configuration fields are accepted as-is after only a non-zero check, leaving potentially dangerous
boundary values unvalidated.

Rule 6: Generic: Security-First Input Validation and Data Handling
rust/netflow-collector/src/config.rs[186-196]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`pending_flows` config values are external inputs and currently only checked for non-zero values. Missing upper bounds can allow extreme values that may cause memory exhaustion or degraded stability.

## Issue Context
These values are read from JSON config files and then passed into `netflow_parser` pending-flow buffering configuration.

## Fix Focus Areas
- rust/netflow-collector/src/config.rs[186-196]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


4. Optimistic MissingTemplate log 🐞 Bug ✧ Quality
Suggestion Impact:Updated the pending-enabled MissingTemplate debug log wording to be non-guaranteeing ("data queued if capacity allows") rather than claiming buffering for replay will occur.

code diff:

@@ -61,7 +61,7 @@
                 if pending_enabled {
                     debug!(
                         "Missing template - ID: {}, Protocol: {:?}. \
-                         Flow data buffered for replay when template arrives.",
+                         Pending flow cache enabled; data queued if capacity allows.",
                         template_id, protocol
                     );
                 } else {

Description
• When pending-flow buffering is enabled, MissingTemplate events are logged at debug and the
  message states flow data is buffered for replay.
• Since buffering success/failure isn’t verified at this call site, the wording can mislead
  operators; consider adjusting the message and/or surfacing drop/replay-failure signals more
  prominently alongside the existing pending-flow metrics logs.
Code

rust/netflow-collector/src/listener.rs[R57-74]

+            MissingTemplate {
+                template_id,
+                protocol,
+            } => {
+                if pending_enabled {
+                    debug!(
+                        "Missing template - ID: {}, Protocol: {:?}. \
+                         Flow data buffered for replay when template arrives.",
+                        template_id, protocol
+                    );
+                } else {
+                    warn!(
+                        "Missing template - ID: {}, Protocol: {:?}. \
+                         Flow data received before template definition - data lost.",
+                        template_id, protocol
+                    );
+                }
+            }
Evidence
The template-event callback unconditionally claims buffering when pending_enabled is true, while
pending-flow failure signals are only available via periodic metrics logging elsewhere.

rust/netflow-collector/src/listener.rs[57-74]
rust/netflow-collector/src/metrics.rs[47-61]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
When pending flows are enabled, the `MissingTemplate` log line says flow data is buffered for replay. This code path does not verify whether buffering actually succeeded; the message can therefore be misleading.

### Issue Context
Pending-flow outcomes are surfaced via periodic metrics logs (`pending_dropped`, `pending_replay_failed`, etc.). The template-event callback currently changes severity (warn-&gt;debug) and uses optimistic wording.

### Fix Focus Areas
- rust/netflow-collector/src/listener.rs[57-74]
- rust/netflow-collector/src/metrics.rs[47-61]

### Suggested fix
1. Change the pending-enabled log message to avoid guarantees (e.g., &quot;pending flow cache enabled; flow data may be buffered for replay&quot;).
2. Consider logging at `info` (or `warn` with rate limiting) if operators need visibility that templates are missing even when buffering is enabled.
3. Optionally add a short hint in the log pointing to the pending-flow counters (cached/replayed/dropped/replay_failed) to drive correct troubleshooting.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider
ⓘ The new review experience is currently in Beta. Learn more
Grey Divider

Qodo Logo

Imported GitHub PR comment. Original author: @qodo-code-review[bot] Original URL: https://github.com/carverauto/serviceradar/pull/2702#issuecomment-3850656948 Original created: 2026-02-05T02:02:10Z --- <pre>ⓘ You are approaching your monthly quota for Qodo. <a href="https://www.qodo.ai/pricing">Upgrade your plan</a></pre> <h3>Code Review by Qodo</h3> <code>🐞 Bugs (2)</code> <code>📘 Rule violations (2)</code> <code>📎 Requirement gaps (0)</code> <img src="https://www.qodo.ai/wp-content/uploads/2025/11/light-grey-line.svg" height="10%" alt="Grey Divider"> <br/> <img src="https://www.qodo.ai/wp-content/uploads/2026/01/action-required.png" height="20" alt="Action required"> <details> <summary> 1. Non-ASCII checkmarks in <b><i>TESTING.md</i></b> <code>📘 Rule violation</code> <code>✓ Correctness</code></summary> <br/> > <details open> ><summary>Description</summary> ><br/> > ><pre> >• The modified Markdown file <b><i>rust/netflow-collector/TESTING.md</i></b> contains non-ASCII symbols (e.g., > <b><i>✓</i></b>, <b><i>✗</i></b>, <b><i>✅</i></b>). >• This violates the repository requirement that modified Markdown documentation be ASCII-only, which > can cause encoding/rendering issues in some tooling. ></pre> ></details> > <details open> ><summary>Code</summary> ><br/> > ><code>[rust/netflow-collector/TESTING.md[R17-18]](https://github.com/carverauto/serviceradar/pull/2702/files#diff-2fd22117bc767137b2b988b14694a0b9c1be7fe91f2174f77fbea9401399a9bcR17-R18)</code> > >```diff >+docker run --rm --network host ghcr.io/mikemiles-dev/netflow_generator:0.2.7 \ > --dest 127.0.0.1:2055 --once >``` ></details> > <details > ><summary>Evidence</summary> ><br/> > ><pre> >PR Compliance ID 11 requires ASCII-only content in any modified Markdown files. <b><i>TESTING.md</i></b> was >modified in this PR and contains multiple non-ASCII characters (checkmarks/crosses) elsewhere in the >file. ></pre> > > <code>AGENTS.md</code> > <code>[rust/netflow-collector/TESTING.md[307-325]](https://github.com/carverauto/serviceradar/blob/4efb8c8500bec5dd9a8d8e35c34a73d94325e7d3/rust/netflow-collector/TESTING.md/#L307-L325)</code> > <code>[rust/netflow-collector/TESTING.md[515-525]](https://github.com/carverauto/serviceradar/blob/4efb8c8500bec5dd9a8d8e35c34a73d94325e7d3/rust/netflow-collector/TESTING.md/#L515-L525)</code> ></details> > <details> ><summary>Agent prompt</summary> ><br/> > >``` >The issue below was found during a code review. Follow the provided context and guidance below and implement a solution > >## Issue description >`rust/netflow-collector/TESTING.md` contains non-ASCII characters (e.g., `✓`, `✗`, `✅`) but modified Markdown files must be ASCII-only. > >## Issue Context >This PR touched `TESTING.md`, so the file must comply with the ASCII-only Markdown rule. > >## Fix Focus Areas >- rust/netflow-collector/TESTING.md[307-325] >- rust/netflow-collector/TESTING.md[515-525] >``` > <code>ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools</code> ></details> <hr/> </details> <details> <summary>✅ <s> 2. Pending flows lack TTL <code>🐞 Bug</code> <code>⛯ Reliability</code></s></summary> ___ <details><summary><b>Suggestion Impact:</b></summary>Introduced a pending flow TTL field in config with a default (300s), added validation bounds for ttl and other pending-flow limits, and updated the listener to set PendingFlowsConfig.ttl to Some(Duration::from_secs(ttl_secs)) rather than None. code diff: ```diff # File: rust/netflow-collector/src/config.rs @@ -73,6 +73,8 @@ pub max_entries_per_template: usize, #[serde(default = "default_max_entry_size_bytes")] pub max_entry_size_bytes: usize, + #[serde(default = "default_pending_ttl_secs")] + pub ttl_secs: u64, } #[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Eq)] @@ -160,6 +162,10 @@ fn default_max_entry_size_bytes() -> usize { 65535 +} + +fn default_pending_ttl_secs() -> u64 { + 300 } impl Config { @@ -184,14 +190,17 @@ anyhow::bail!("subject cannot be empty"); } if let Some(pf) = &self.pending_flows { - if pf.max_pending_flows == 0 { - anyhow::bail!("pending_flows.max_pending_flows must be > 0"); - } - if pf.max_entries_per_template == 0 { - anyhow::bail!("pending_flows.max_entries_per_template must be > 0"); - } - if pf.max_entry_size_bytes == 0 { - anyhow::bail!("pending_flows.max_entry_size_bytes must be > 0"); + if pf.max_pending_flows == 0 || pf.max_pending_flows > 10_000 { + anyhow::bail!("pending_flows.max_pending_flows must be 1..=10,000"); + } + if pf.max_entries_per_template == 0 || pf.max_entries_per_template > 100_000 { + anyhow::bail!("pending_flows.max_entries_per_template must be 1..=100,000"); + } + if pf.max_entry_size_bytes == 0 || pf.max_entry_size_bytes > 1_048_576 { + anyhow::bail!("pending_flows.max_entry_size_bytes must be 1..=1,048,576"); + } + if pf.ttl_secs == 0 || pf.ttl_secs > 3600 { + anyhow::bail!("pending_flows.ttl_secs must be 1..=3,600"); } } Ok(()) # File: rust/netflow-collector/src/config.rs @@ -73,6 +73,8 @@ pub max_entries_per_template: usize, #[serde(default = "default_max_entry_size_bytes")] pub max_entry_size_bytes: usize, + #[serde(default = "default_pending_ttl_secs")] + pub ttl_secs: u64, } #[derive(Debug, Serialize, Deserialize, Clone, PartialEq, Eq)] @@ -160,6 +162,10 @@ fn default_max_entry_size_bytes() -> usize { 65535 +} + +fn default_pending_ttl_secs() -> u64 { + 300 } impl Config { @@ -184,14 +190,17 @@ anyhow::bail!("subject cannot be empty"); } if let Some(pf) = &self.pending_flows { - if pf.max_pending_flows == 0 { - anyhow::bail!("pending_flows.max_pending_flows must be > 0"); - } - if pf.max_entries_per_template == 0 { - anyhow::bail!("pending_flows.max_entries_per_template must be > 0"); - } - if pf.max_entry_size_bytes == 0 { - anyhow::bail!("pending_flows.max_entry_size_bytes must be > 0"); + if pf.max_pending_flows == 0 || pf.max_pending_flows > 10_000 { + anyhow::bail!("pending_flows.max_pending_flows must be 1..=10,000"); + } + if pf.max_entries_per_template == 0 || pf.max_entries_per_template > 100_000 { + anyhow::bail!("pending_flows.max_entries_per_template must be 1..=100,000"); + } + if pf.max_entry_size_bytes == 0 || pf.max_entry_size_bytes > 1_048_576 { + anyhow::bail!("pending_flows.max_entry_size_bytes must be 1..=1,048,576"); + } + if pf.ttl_secs == 0 || pf.ttl_secs > 3600 { + anyhow::bail!("pending_flows.ttl_secs must be 1..=3,600"); } } Ok(()) # File: rust/netflow-collector/src/listener.rs @@ -8,7 +8,7 @@ use prost::Message; use std::net::SocketAddr; use std::sync::{Arc, Mutex}; -use std::time::{SystemTime, UNIX_EPOCH}; +use std::time::{Duration, SystemTime, UNIX_EPOCH}; use tokio::net::UdpSocket; use tokio::sync::mpsc; @@ -61,7 +61,7 @@ if pending_enabled { debug!( "Missing template - ID: {}, Protocol: {:?}. \ - Flow data buffered for replay when template arrives.", + Pending flow cache enabled; data queued if capacity allows.", template_id, protocol ); } else { @@ -98,7 +98,7 @@ max_pending_flows: pf.max_pending_flows, max_entries_per_template: pf.max_entries_per_template, max_entry_size_bytes: pf.max_entry_size_bytes, - ttl: None, + ttl: Some(Duration::from_secs(pf.ttl_secs)), }); ``` </details> <br/> > <details open> ><summary>Description</summary> ><br/> > ><pre> >• Pending flow caching is enabled via config, but the implementation hard-codes <b><i>ttl: None</i></b>, meaning > buffered flows may be replayed arbitrarily late (until size-based eviction). >• This can produce very stale/out-of-order flow delivery when templates arrive late and makes > memory/latency behavior harder to reason about operationally. ></pre> ></details> > <details open> ><summary>Code</summary> ><br/> > ><code>[rust/netflow-collector/src/listener.rs[R96-102]](https://github.com/carverauto/serviceradar/pull/2702/files#diff-3ec4d0175a1f407a9c9246268da3491d1fdfacd2e8c00d536fde36357e993999R96-R102)</code> > >```diff >+ if let Some(pf) = &config.pending_flows { >+ builder = builder.with_pending_flows(PendingFlowsConfig { >+ max_pending_flows: pf.max_pending_flows, >+ max_entries_per_template: pf.max_entries_per_template, >+ max_entry_size_bytes: pf.max_entry_size_bytes, >+ ttl: None, >+ }); >``` ></details> > <details > ><summary>Evidence</summary> ><br/> > ><pre> >Listener always sets <b><i>ttl: None</i></b> when enabling pending flows, and the application config schema >provides no TTL field to allow operators to bound retention/replay age. ></pre> > > <code>[rust/netflow-collector/src/listener.rs[91-103]](https://github.com/carverauto/serviceradar/blob/4efb8c8500bec5dd9a8d8e35c34a73d94325e7d3/rust/netflow-collector/src/listener.rs/#L91-L103)</code> > <code>[rust/netflow-collector/src/config.rs[68-76]](https://github.com/carverauto/serviceradar/blob/4efb8c8500bec5dd9a8d8e35c34a73d94325e7d3/rust/netflow-collector/src/config.rs/#L68-L76)</code> ></details> > <details> ><summary>Agent prompt</summary> ><br/> > >``` >The issue below was found during a code review. Follow the provided context and guidance below and implement a solution > >### Issue description >Pending flow caching is configured, but the code forces `ttl: None`, so buffered flows may be replayed arbitrarily late (until size-based eviction). This can lead to very stale/out-of-order flow emission and makes resource behavior harder to control. > >### Issue Context >`PendingFlowsCacheConfig` currently only exposes size-based limits. `Listener::new` always sets `ttl: None` when pending flow buffering is enabled. > >### Fix Focus Areas >- rust/netflow-collector/src/config.rs[68-76] >- rust/netflow-collector/src/config.rs[173-197] >- rust/netflow-collector/src/listener.rs[96-103] > >### Suggested fix >1. Extend `PendingFlowsCacheConfig` with an optional TTL field (e.g., `ttl_seconds: Option&lt;u64&gt;` with a sensible default like 300s). >2. Add validation: if provided, TTL must be &gt; 0. >3. Convert TTL seconds to the type expected by `PendingFlowsConfig.ttl` (likely `Some(Duration::from_secs(ttl_seconds))`). >4. Update example JSON configs to include/mention TTL so operators can tune it. >``` > <code>ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools</code> ></details> <hr/> </details> <br/> <img src="https://www.qodo.ai/wp-content/uploads/2026/01/review-recommended.png" height="20" alt="Remediation recommended"> <details> <summary>✅ <s> 3. <b><i>pending_flows</i></b> lacks upper bounds <code>📘 Rule violation</code> <code>⛨ Security</code></s></summary> ___ <details><summary><b>Suggestion Impact:</b></summary>Updated pending_flows validation to enforce maximum allowed values (not just >0) for max_pending_flows, max_entries_per_template, and max_entry_size_bytes; also introduced and bounded a new ttl_secs pending flow setting. code diff: ```diff @@ -184,14 +190,17 @@ anyhow::bail!("subject cannot be empty"); } if let Some(pf) = &self.pending_flows { - if pf.max_pending_flows == 0 { - anyhow::bail!("pending_flows.max_pending_flows must be > 0"); - } - if pf.max_entries_per_template == 0 { - anyhow::bail!("pending_flows.max_entries_per_template must be > 0"); - } - if pf.max_entry_size_bytes == 0 { - anyhow::bail!("pending_flows.max_entry_size_bytes must be > 0"); + if pf.max_pending_flows == 0 || pf.max_pending_flows > 10_000 { + anyhow::bail!("pending_flows.max_pending_flows must be 1..=10,000"); + } + if pf.max_entries_per_template == 0 || pf.max_entries_per_template > 100_000 { + anyhow::bail!("pending_flows.max_entries_per_template must be 1..=100,000"); + } + if pf.max_entry_size_bytes == 0 || pf.max_entry_size_bytes > 1_048_576 { + anyhow::bail!("pending_flows.max_entry_size_bytes must be 1..=1,048,576"); + } + if pf.ttl_secs == 0 || pf.ttl_secs > 3600 { + anyhow::bail!("pending_flows.ttl_secs must be 1..=3,600"); } } ``` </details> <br/> > <details open> ><summary>Description</summary> ><br/> > ><pre> >• The new <b><i>pending_flows</i></b> settings are validated only for <b><i>&gt; 0</i></b>, with no maximum limits enforced for > <b><i>max_pending_flows</i></b>, <b><i>max_entries_per_template</i></b>, or <b><i>max_entry_size_bytes</i></b>. >• Because these values come from external JSON config files, extremely large values could trigger > excessive memory usage (potential DoS) or unstable runtime behavior. ></pre> ></details> > <details open> ><summary>Code</summary> ><br/> > ><code>[rust/netflow-collector/src/config.rs[R186-196]](https://github.com/carverauto/serviceradar/pull/2702/files#diff-ea8fb7fc6e55836ada19b9c1ca26f2a88575289a3fee3005183916a8454d0fc9R186-R196)</code> > >```diff >+ if let Some(pf) = &self.pending_flows { >+ if pf.max_pending_flows == 0 { >+ anyhow::bail!("pending_flows.max_pending_flows must be > 0"); >+ } >+ if pf.max_entries_per_template == 0 { >+ anyhow::bail!("pending_flows.max_entries_per_template must be > 0"); >+ } >+ if pf.max_entry_size_bytes == 0 { >+ anyhow::bail!("pending_flows.max_entry_size_bytes must be > 0"); >+ } >+ } >``` ></details> > <details > ><summary>Evidence</summary> ><br/> > ><pre> >PR Compliance ID 6 requires validation/sanitization of external inputs. The new external >configuration fields are accepted as-is after only a non-zero check, leaving potentially dangerous >boundary values unvalidated. ></pre> > > <code>Rule 6: Generic: Security-First Input Validation and Data Handling</code> > <code>[rust/netflow-collector/src/config.rs[186-196]](https://github.com/carverauto/serviceradar/blob/4efb8c8500bec5dd9a8d8e35c34a73d94325e7d3/rust/netflow-collector/src/config.rs/#L186-L196)</code> ></details> > <details> ><summary>Agent prompt</summary> ><br/> > >``` >The issue below was found during a code review. Follow the provided context and guidance below and implement a solution > >## Issue description >`pending_flows` config values are external inputs and currently only checked for non-zero values. Missing upper bounds can allow extreme values that may cause memory exhaustion or degraded stability. > >## Issue Context >These values are read from JSON config files and then passed into `netflow_parser` pending-flow buffering configuration. > >## Fix Focus Areas >- rust/netflow-collector/src/config.rs[186-196] >``` > <code>ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools</code> ></details> <hr/> </details> <details> <summary>✅ <s> 4. Optimistic MissingTemplate log <code>🐞 Bug</code> <code>✧ Quality</code></s></summary> ___ <details><summary><b>Suggestion Impact:</b></summary>Updated the pending-enabled MissingTemplate debug log wording to be non-guaranteeing ("data queued if capacity allows") rather than claiming buffering for replay will occur. code diff: ```diff @@ -61,7 +61,7 @@ if pending_enabled { debug!( "Missing template - ID: {}, Protocol: {:?}. \ - Flow data buffered for replay when template arrives.", + Pending flow cache enabled; data queued if capacity allows.", template_id, protocol ); } else { ``` </details> <br/> > <details open> ><summary>Description</summary> ><br/> > ><pre> >• When pending-flow buffering is enabled, <b><i>MissingTemplate</i></b> events are logged at <b><i>debug</i></b> and the > message states flow data is buffered for replay. >• Since buffering success/failure isn’t verified at this call site, the wording can mislead > operators; consider adjusting the message and/or surfacing drop/replay-failure signals more > prominently alongside the existing pending-flow metrics logs. ></pre> ></details> > <details open> ><summary>Code</summary> ><br/> > ><code>[rust/netflow-collector/src/listener.rs[R57-74]](https://github.com/carverauto/serviceradar/pull/2702/files#diff-3ec4d0175a1f407a9c9246268da3491d1fdfacd2e8c00d536fde36357e993999R57-R74)</code> > >```diff >+ MissingTemplate { >+ template_id, >+ protocol, >+ } => { >+ if pending_enabled { >+ debug!( >+ "Missing template - ID: {}, Protocol: {:?}. \ >+ Flow data buffered for replay when template arrives.", >+ template_id, protocol >+ ); >+ } else { >+ warn!( >+ "Missing template - ID: {}, Protocol: {:?}. \ >+ Flow data received before template definition - data lost.", >+ template_id, protocol >+ ); >+ } >+ } >``` ></details> > <details > ><summary>Evidence</summary> ><br/> > ><pre> >The template-event callback unconditionally claims buffering when <b><i>pending_enabled</i></b> is true, while >pending-flow failure signals are only available via periodic metrics logging elsewhere. ></pre> > > <code>[rust/netflow-collector/src/listener.rs[57-74]](https://github.com/carverauto/serviceradar/blob/4efb8c8500bec5dd9a8d8e35c34a73d94325e7d3/rust/netflow-collector/src/listener.rs/#L57-L74)</code> > <code>[rust/netflow-collector/src/metrics.rs[47-61]](https://github.com/carverauto/serviceradar/blob/4efb8c8500bec5dd9a8d8e35c34a73d94325e7d3/rust/netflow-collector/src/metrics.rs/#L47-L61)</code> ></details> > <details> ><summary>Agent prompt</summary> ><br/> > >``` >The issue below was found during a code review. Follow the provided context and guidance below and implement a solution > >### Issue description >When pending flows are enabled, the `MissingTemplate` log line says flow data is buffered for replay. This code path does not verify whether buffering actually succeeded; the message can therefore be misleading. > >### Issue Context >Pending-flow outcomes are surfaced via periodic metrics logs (`pending_dropped`, `pending_replay_failed`, etc.). The template-event callback currently changes severity (warn-&gt;debug) and uses optimistic wording. > >### Fix Focus Areas >- rust/netflow-collector/src/listener.rs[57-74] >- rust/netflow-collector/src/metrics.rs[47-61] > >### Suggested fix >1. Change the pending-enabled log message to avoid guarantees (e.g., &quot;pending flow cache enabled; flow data may be buffered for replay&quot;). >2. Consider logging at `info` (or `warn` with rate limiting) if operators need visibility that templates are missing even when buffering is enabled. >3. Optionally add a short hint in the log pointing to the pending-flow counters (cached/replayed/dropped/replay_failed) to drive correct troubleshooting. >``` > <code>ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools</code> ></details> <hr/> </details> <img src="https://www.qodo.ai/wp-content/uploads/2025/11/light-grey-line.svg" height="10%" alt="Grey Divider"> <pre>ⓘ The new review experience is currently in Beta. <a href="https://docs.qodo.ai/qodo-documentation/code-review">Learn more</a></pre> <img src="https://www.qodo.ai/wp-content/uploads/2025/11/light-grey-line.svg" height="10%" alt="Grey Divider"> <!-- https://github.com/carverauto/serviceradar/commit/4efb8c8500bec5dd9a8d8e35c34a73d94325e7d3 --> <a href="https://www.qodo.ai"><img src="https://www.qodo.ai/wp-content/uploads/2025/03/qodo-logo.svg" width="80" alt="Qodo Logo"></a>
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
carverauto/serviceradar!2852
No description provided.