carverauto/serviceradar

Fork 0

feat: device identification #3423

New issue

Open

opened 2026-05-27 04:44:20 +00:00 by mfreeman451 · 1 comment

mfreeman451 commented

2026-05-27 04:44:20 +00:00

Owner

Is your feature request related to a problem?

Need to build into our discovery engine a device identification/fingerprinting capability. Not exactly sure how this is going to work with devices that aren't discovered but imported through integrations like armis or netbox, need to work that out in this change as well.

This change should occur in serviceradar-agent, there should probably be some admin/settings UI where users create profiles for this based on a SRQL query for the scope like we do for other things (mapper, sysmon, etc).

Describe the solution you'd like

A clear and concise description of what you want to happen.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Additional context

Add any other context or screenshots about the feature request here.

**Is your feature request related to a problem?** Need to build into our discovery engine a device identification/fingerprinting capability. Not exactly sure how this is going to work with devices that aren't discovered but imported through integrations like armis or netbox, need to work that out in this change as well. This change should occur in `serviceradar-agent`, there should probably be some admin/settings UI where users create profiles for this based on a SRQL query for the scope like we do for other things (mapper, sysmon, etc). **Describe the solution you'd like** A clear and concise description of what you want to happen. **Describe alternatives you've considered** A clear and concise description of any alternative solutions or features you've considered. **Additional context** Add any other context or screenshots about the feature request here.

mfreeman451 commented

2026-05-27 04:44:31 +00:00

Author

Owner

Product Requirements Document (PRD): Rust-Based Passive Fingerprinting Sidecar for ServiceRadar Agent

1. Overview

Project Name: Fingerprinting Sidecar Integration (fingerprintd)

Objective:
Add robust, passive OS/device/application fingerprinting capabilities to the ServiceRadar agent without depending on Nmap or heavy Go-native libraries. Use the Rust library huginn-net (multi-protocol passive fingerprinting: TCP/p0f-style, HTTP, TLS/JA4) as a sidecar binary bundled with the Go-based agent.

This follows modern observability patterns (e.g., Datadog Agent's multi-binary architecture) while keeping the core Go agent lightweight.

Key Motivation:

Avoid maintaining a full custom Go fingerprint engine + signature DB.
Leverage mature, high-performance Rust implementation.
Enable passive fingerprinting from live traffic with low overhead.
Support future extensibility (more protocols, better accuracy).

2. Goals & Non-Goals

In Scope:

Bundle a statically compiled Rust binary (fingerprintd) with the ServiceRadar agent installer/package.
Reliable inter-process communication between Go agent and Rust sidecar.
Passive TCP, HTTP, and TLS fingerprinting.
OS, device type, browser/server, and application-level insights.
Lifecycle management (start/stop/restart) from the Go agent.
Configuration passthrough and result caching in the agent.
Cross-platform support (primarily Linux, with Windows/macOS where feasible).

Out of Scope:

Active probing (no Nmap-style behavior).
Full replacement of existing discovery/mapper engine — this augments it.
Real-time deep packet inspection for all traffic (focus on targeted interfaces/flows).
Replacing all service detection (banner grabbing can still live in Go).

3. Architecture

High-Level Design:

Go Agent (main process): Discovery engine, configuration, storage, API.
fingerprintd (Rust sidecar): Stateless or lightly stateful fingerprint engine based on huginn-net.
Communication: Unix Domain Socket (primary) for low-latency, message-based IPC. Fallback to stdio if needed.

Bundling Strategy (inspired by Datadog Agent practices):

Compile fingerprintd as a fully static binary (x86_64-unknown-linux-musl target recommended for portability).
Include the binary in the agent distribution tarball / deb / rpm / etc. (e.g., under /usr/lib/serviceradar/bin/fingerprintd).
On install, the Go installer places it and sets appropriate permissions (capabilities for packet capture if needed: CAP_NET_RAW).
Use build tags and conditional compilation in Go where possible to keep agent size down (per Datadog's dependency pruning and dead-code elimination techniques).

Process Management:

Agent starts fingerprintd as a child process (or systemd service in advanced setups).
Health monitoring + auto-restart on crash.
Graceful shutdown.

4. Requirements

Functional Requirements

Fingerprinting Capabilities
- TCP stack analysis (OS, TTL, window size, options order, etc.).
- HTTP header analysis (client/server signatures).
- TLS/JA4 fingerprinting.
- Return structured results: OS family/version, device type, confidence, protocol-specific details.
Integration Points
- Agent feeds raw packets (or parsed metadata) from gopacket/libpcap captures.
- Sidecar returns enriched fingerprint data to feed into the mapper/discovery engine.
- Support batch and single-packet requests.
Configuration
- Shared config file or CLI flags passed from Go agent.
- Enable/disable specific analyzers (TCP only, full suite, etc.).
- Signature DB update mechanism (if huginn-net supports external DBs).

Non-Functional Requirements

Performance: < 5ms per packet average.
Resource Usage: Low CPU/memory footprint (sidecar should be lightweight).
Reliability: Auto-restart, connection retry, error handling.
Security: Least privilege (drop root after binding if possible), input validation.
Size: Keep impact on total agent bundle minimal (static Rust binary expected ~5-15 MB).

5. Technical Implementation Details

Rust Side (fingerprintd):

Thin CLI wrapper around huginn-net crates.
Unix socket server using tokio.
JSON or Protobuf request/response format for simplicity and type safety.
Example request: { "packet": base64_data, "interface": "eth0", ... }
Example response: structured fingerprint object.

Go Side:

exec.Command + Unix socket dial.
Sidecar manager with lifecycle methods (Start(), Stop(), Restart()).
Client wrapper to send packets and receive results.
Integration hook in discovery pipeline.

Deployment & Packaging:

Include binary in release assets.
Installer scripts handle extraction and path setup.
Optional: Embed as asset and extract on first run (like some Datadog components).

6. Risks & Mitigations

Rust Binary Portability — Use musl target + test on multiple distros.
IPC Overhead — Benchmark Unix socket vs alternatives; optimize message size.
Maintenance — Monitor huginn-net upstream; plan for signature updates.
Agent Size — Apply Datadog-style techniques (build tags, dependency auditing with go-size-analyzer and goda) to minimize impact of new Go wrapper code.
Permissions — Document required capabilities clearly.

7. Success Metrics

Successful passive OS detection accuracy comparable to p0f on test traffic.
< 2% increase in overall agent bundle size.
Zero-downtime fingerprinting during sidecar restarts.
Positive feedback from internal testing on real networks.

8. Timeline & Next Steps

Phase 1: Prototype Rust wrapper + Go manager (1-2 weeks).
Phase 2: Integration with existing discovery engine + testing.
Phase 3: Packaging, docs, and release.

**Product Requirements Document (PRD): Rust-Based Passive Fingerprinting Sidecar for ServiceRadar Agent** ### 1. Overview **Project Name:** Fingerprinting Sidecar Integration (`fingerprintd`) **Objective:** Add robust, passive OS/device/application fingerprinting capabilities to the ServiceRadar agent without depending on Nmap or heavy Go-native libraries. Use the Rust library **huginn-net** (multi-protocol passive fingerprinting: TCP/p0f-style, HTTP, TLS/JA4) as a sidecar binary bundled with the Go-based agent. This follows modern observability patterns (e.g., Datadog Agent's multi-binary architecture) while keeping the core Go agent lightweight. **Key Motivation:** - Avoid maintaining a full custom Go fingerprint engine + signature DB. - Leverage mature, high-performance Rust implementation. - Enable passive fingerprinting from live traffic with low overhead. - Support future extensibility (more protocols, better accuracy). ### 2. Goals & Non-Goals **In Scope:** - Bundle a statically compiled Rust binary (`fingerprintd`) with the ServiceRadar agent installer/package. - Reliable inter-process communication between Go agent and Rust sidecar. - Passive TCP, HTTP, and TLS fingerprinting. - OS, device type, browser/server, and application-level insights. - Lifecycle management (start/stop/restart) from the Go agent. - Configuration passthrough and result caching in the agent. - Cross-platform support (primarily Linux, with Windows/macOS where feasible). **Out of Scope:** - Active probing (no Nmap-style behavior). - Full replacement of existing discovery/mapper engine — this augments it. - Real-time deep packet inspection for all traffic (focus on targeted interfaces/flows). - Replacing all service detection (banner grabbing can still live in Go). ### 3. Architecture **High-Level Design:** - **Go Agent** (main process): Discovery engine, configuration, storage, API. - **fingerprintd** (Rust sidecar): Stateless or lightly stateful fingerprint engine based on huginn-net. - **Communication**: Unix Domain Socket (primary) for low-latency, message-based IPC. Fallback to stdio if needed. **Bundling Strategy** (inspired by Datadog Agent practices): - Compile `fingerprintd` as a fully static binary (`x86_64-unknown-linux-musl` target recommended for portability). - Include the binary in the agent distribution tarball / deb / rpm / etc. (e.g., under `/usr/lib/serviceradar/bin/fingerprintd`). - On install, the Go installer places it and sets appropriate permissions (capabilities for packet capture if needed: `CAP_NET_RAW`). - Use build tags and conditional compilation in Go where possible to keep agent size down (per Datadog's dependency pruning and dead-code elimination techniques). **Process Management:** - Agent starts `fingerprintd` as a child process (or systemd service in advanced setups). - Health monitoring + auto-restart on crash. - Graceful shutdown. ### 4. Requirements #### Functional Requirements 1. **Fingerprinting Capabilities** - TCP stack analysis (OS, TTL, window size, options order, etc.). - HTTP header analysis (client/server signatures). - TLS/JA4 fingerprinting. - Return structured results: OS family/version, device type, confidence, protocol-specific details. 2. **Integration Points** - Agent feeds raw packets (or parsed metadata) from gopacket/libpcap captures. - Sidecar returns enriched fingerprint data to feed into the mapper/discovery engine. - Support batch and single-packet requests. 3. **Configuration** - Shared config file or CLI flags passed from Go agent. - Enable/disable specific analyzers (TCP only, full suite, etc.). - Signature DB update mechanism (if huginn-net supports external DBs). #### Non-Functional Requirements - **Performance**: < 5ms per packet average. - **Resource Usage**: Low CPU/memory footprint (sidecar should be lightweight). - **Reliability**: Auto-restart, connection retry, error handling. - **Security**: Least privilege (drop root after binding if possible), input validation. - **Size**: Keep impact on total agent bundle minimal (static Rust binary expected ~5-15 MB). ### 5. Technical Implementation Details **Rust Side (`fingerprintd`):** - Thin CLI wrapper around huginn-net crates. - Unix socket server using tokio. - JSON or Protobuf request/response format for simplicity and type safety. - Example request: `{ "packet": base64_data, "interface": "eth0", ... }` - Example response: structured fingerprint object. **Go Side:** - `exec.Command` + Unix socket dial. - Sidecar manager with lifecycle methods (`Start()`, `Stop()`, `Restart()`). - Client wrapper to send packets and receive results. - Integration hook in discovery pipeline. **Deployment & Packaging:** - Include binary in release assets. - Installer scripts handle extraction and path setup. - Optional: Embed as asset and extract on first run (like some Datadog components). ### 6. Risks & Mitigations - **Rust Binary Portability** — Use musl target + test on multiple distros. - **IPC Overhead** — Benchmark Unix socket vs alternatives; optimize message size. - **Maintenance** — Monitor huginn-net upstream; plan for signature updates. - **Agent Size** — Apply Datadog-style techniques (build tags, dependency auditing with `go-size-analyzer` and `goda`) to minimize impact of new Go wrapper code. - **Permissions** — Document required capabilities clearly. ### 7. Success Metrics - Successful passive OS detection accuracy comparable to p0f on test traffic. - < 2% increase in overall agent bundle size. - Zero-downtime fingerprinting during sidecar restarts. - Positive feedback from internal testing on real networks. ### 8. Timeline & Next Steps - Phase 1: Prototype Rust wrapper + Go manager (1-2 weeks). - Phase 2: Integration with existing discovery engine + testing. - Phase 3: Packaging, docs, and release.

mfreeman451 referenced this issue from a commit

2026-05-28 06:07:33 +00:00

Adopt license-clean fingerprint stack (drop huginn-net + JA4+)

mfreeman451 referenced this issue from a commit

2026-05-28 06:22:54 +00:00

Add multi-corpus fingerprint ensemble (MuonFP + Recog + Satori)