feat: Netflow stats #1083

New issue

Closed

opened 2026-03-28 04:31:30 +00:00 by mfreeman451 · 0 comments

mfreeman451 commented

2026-03-28 04:31:30 +00:00

Owner

Imported from GitHub.

Original GitHub issue: #2965
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/2965
Original created: 2026-03-01T06:45:09Z

The "Top N" Dashboards (The Essentials)
These are the immediate, snapshot views every network admin looks for when someone says, "The network is slow."
Top Talkers (Source IPs): Who is generating the most traffic? (Measure by both Bytes and Packets).
Top Listeners (Destination IPs): Who is receiving the most traffic?
Top Conversations (Source IP ↔ Destination IP): Which specific pairs of devices are exchanging the most data?
Top Applications/Ports: Traffic broken down by Layer 4 destination port (e.g., 443/HTTPS, 53/DNS, 22/SSH). Bonus: Allow users to map custom port ranges to internal application names.
Top Protocols: Breakdown by L3/L4 protocol (TCP, UDP, ICMP, GRE, ESP, etc.).
Time-Series & Capacity Planning
Admins need to know how traffic changes over time to justify bandwidth upgrades or find out what happened at 3:00 AM last night.
Bandwidth Over Time: Stacked area charts showing Ingress vs. Egress traffic in Bits Per Second (bps) and Packets Per Second (pps) across specific interfaces.
95th Percentile Utilization: The industry standard for billing and capacity planning. Show the 95th percentile bandwidth usage for a given interface over a month.
Subnet / VLAN Traffic Distribution: Group IPs by CIDR blocks to show which departments, branch offices, or data center racks are consuming the network.
Packet Size Distribution: Are flows mostly small packets (VoIP, interactive sessions, or SYN floods) or large packets (file transfers, backups)?
Security & Troubleshooting (SecOps / NetOps)
NetFlow is invaluable for spotting anomalies, misconfigurations, and active attacks.
TCP Flag Distribution: A massive spike in SYN packets without corresponding ACKs usually indicates a SYN Flood DDoS attack. A spike in RST (Reset) packets indicates dropped connections or firewall blocks.
Top Denied Flows: If you are ingesting firewall logs or specific IPFIX fields that indicate flow drops, plotting these shows active probing or broken ACLs.
Connection / Flow Rate (Flows per Second): A sudden spike in the number of unique flows, even if the overall bandwidth is low, usually indicates a port scanner (like Nmap) or a botnet brute-forcing the network.
Long-Lived vs. Short-Lived Connections: Highlight flows that have been active for days (data exfiltration, persistent tunnels) vs. thousands of 1-packet flows (spoofed traffic).
Routing & Internet Edge (For ISPs and Large Enterprises)
If your users are running BGP or managing a network edge, this is where your platform will shine.
Top Autonomous Systems (ASNs): Map public IPs to their BGP ASNs. This tells admins which ISPs or cloud providers (e.g., AWS, Cloudflare, local ISPs) they are sending the most traffic to. Useful for peering decisions.
Traffic by Geo-Location (GeoIP): Map public IPs to countries/cities. A map visualization showing traffic sources and destinations is a classic NOC dashboard feature.
Next-Hop IP Stats: Where is traffic being routed after it leaves this device?
Quality of Service (QoS)
Traffic by ToS / DSCP (Differentiated Services Code Point): Group traffic by its QoS tag. This allows voice engineers to verify that VoIP traffic is actually being tagged as EF (Expedited Forwarding) and not falling into the default/best-effort queues.
Pro-Tips for the UI & Architecture
If you want your open-source project to stand out against commercial giants, focus on these UX/Architecture features:
Contextual Drill-Downs: If an admin clicks on a "Top Talker" IP, the UI should immediately pivot to a dashboard filtered only for that IP, showing what ports it's using, who it's talking to, and its historical traffic curve.
Enrichment is Everything: Raw IPs are hard to read. Build a pipeline that automatically resolves IPs to DNS hostnames, maps public IPs to GeoIP/ASNs, and maps private subnets to user-defined names (e.g., 10.1.5.0/24 = HR VLAN).
Alerting on Thresholds: Allow users to set baselines (e.g., "Alert me if TCP port 22 traffic exceeds 500 Mbps for more than 5 minutes" or "Alert me if we see traffic from North Korea").
Data Rollups: Raw NetFlow data gets massive quickly. Keep raw data for 7–14 days for forensic troubleshooting, but automatically roll it up into 1-hour and 1-day aggregates so users can look at 1-year historical trends without crashing the database.

Imported from GitHub. Original GitHub issue: #2965 Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/2965 Original created: 2026-03-01T06:45:09Z --- 1. The "Top N" Dashboards (The Essentials) These are the immediate, snapshot views every network admin looks for when someone says, "The network is slow." Top Talkers (Source IPs): Who is generating the most traffic? (Measure by both Bytes and Packets). Top Listeners (Destination IPs): Who is receiving the most traffic? Top Conversations (Source IP ↔ Destination IP): Which specific pairs of devices are exchanging the most data? Top Applications/Ports: Traffic broken down by Layer 4 destination port (e.g., 443/HTTPS, 53/DNS, 22/SSH). Bonus: Allow users to map custom port ranges to internal application names. Top Protocols: Breakdown by L3/L4 protocol (TCP, UDP, ICMP, GRE, ESP, etc.). 2. Time-Series & Capacity Planning Admins need to know how traffic changes over time to justify bandwidth upgrades or find out what happened at 3:00 AM last night. Bandwidth Over Time: Stacked area charts showing Ingress vs. Egress traffic in Bits Per Second (bps) and Packets Per Second (pps) across specific interfaces. 95th Percentile Utilization: The industry standard for billing and capacity planning. Show the 95th percentile bandwidth usage for a given interface over a month. Subnet / VLAN Traffic Distribution: Group IPs by CIDR blocks to show which departments, branch offices, or data center racks are consuming the network. Packet Size Distribution: Are flows mostly small packets (VoIP, interactive sessions, or SYN floods) or large packets (file transfers, backups)? 3. Security & Troubleshooting (SecOps / NetOps) NetFlow is invaluable for spotting anomalies, misconfigurations, and active attacks. TCP Flag Distribution: A massive spike in SYN packets without corresponding ACKs usually indicates a SYN Flood DDoS attack. A spike in RST (Reset) packets indicates dropped connections or firewall blocks. Top Denied Flows: If you are ingesting firewall logs or specific IPFIX fields that indicate flow drops, plotting these shows active probing or broken ACLs. Connection / Flow Rate (Flows per Second): A sudden spike in the number of unique flows, even if the overall bandwidth is low, usually indicates a port scanner (like Nmap) or a botnet brute-forcing the network. Long-Lived vs. Short-Lived Connections: Highlight flows that have been active for days (data exfiltration, persistent tunnels) vs. thousands of 1-packet flows (spoofed traffic). 4. Routing & Internet Edge (For ISPs and Large Enterprises) If your users are running BGP or managing a network edge, this is where your platform will shine. Top Autonomous Systems (ASNs): Map public IPs to their BGP ASNs. This tells admins which ISPs or cloud providers (e.g., AWS, Cloudflare, local ISPs) they are sending the most traffic to. Useful for peering decisions. Traffic by Geo-Location (GeoIP): Map public IPs to countries/cities. A map visualization showing traffic sources and destinations is a classic NOC dashboard feature. Next-Hop IP Stats: Where is traffic being routed after it leaves this device? 5. Quality of Service (QoS) Traffic by ToS / DSCP (Differentiated Services Code Point): Group traffic by its QoS tag. This allows voice engineers to verify that VoIP traffic is actually being tagged as EF (Expedited Forwarding) and not falling into the default/best-effort queues. Pro-Tips for the UI & Architecture If you want your open-source project to stand out against commercial giants, focus on these UX/Architecture features: Contextual Drill-Downs: If an admin clicks on a "Top Talker" IP, the UI should immediately pivot to a dashboard filtered only for that IP, showing what ports it's using, who it's talking to, and its historical traffic curve. Enrichment is Everything: Raw IPs are hard to read. Build a pipeline that automatically resolves IPs to DNS hostnames, maps public IPs to GeoIP/ASNs, and maps private subnets to user-defined names (e.g., 10.1.5.0/24 = HR VLAN). Alerting on Thresholds: Allow users to set baselines (e.g., "Alert me if TCP port 22 traffic exceeds 500 Mbps for more than 5 minutes" or "Alert me if we see traffic from North Korea"). Data Rollups: Raw NetFlow data gets massive quickly. Keep raw data for 7–14 days for forensic troubleshooting, but automatically roll it up into 1-hour and 1-day aggregates so users can look at 1-year historical trends without crashing the database.

Rows
Columns