2339 buglogs lost otel log formatschema #2699
No reviewers
Labels
No labels
1week
2weeks
Failed compliance check
IP cameras
NATS
Possible security concern
Review effort 1/5
Review effort 2/5
Review effort 3/5
Review effort 4/5
Review effort 5/5
UI
aardvark
accessibility
amd64
api
arm64
auth
back-end
bgp
blog
bug
build
checkers
ci-cd
cleanup
cnpg
codex
core
dependencies
device-management
documentation
duplicate
dusk
ebpf
enhancement
eta 1d
eta 1hr
eta 3d
eta 3hr
feature
fieldsurvey
github_actions
go
good first issue
help wanted
invalid
javascript
k8s
log-collector
mapper
mtr
needs-triage
netflow
network-sweep
observability
oracle
otel
plug-in
proton
python
question
reddit
redhat
research
rperf
rperf-checker
rust
sdk
security
serviceradar-agent
serviceradar-agent-gateway
serviceradar-web
serviceradar-web-ng
siem
snmp
sysmon
topology
ubiquiti
wasm
wontfix
zen-engine
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
carverauto/serviceradar!2699
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "refs/pull/2699/head"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Imported from GitHub pull request.
Original GitHub pull request: #2354
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/pull/2354
Original created: 2026-01-18T17:47:30Z
Original updated: 2026-01-18T17:50:24Z
Original head: carverauto/serviceradar:2339-buglogs-lost-otel-log-formatschema
Original base: staging
Original merged: 2026-01-18T17:50:22Z by @mfreeman451
User description
IMPORTANT: Please sign the Developer Certificate of Origin
Thank you for your contribution to ServiceRadar. Please note, when contributing, the developer must include
a DCO sign-off statement indicating the DCO acceptance in one commit message. Here
is an example DCO Signed-off-by line in a commit message:
Describe your changes
Issue ticket number and link
Code checklist before requesting a review
PR Type
Bug fix, Enhancement
Description
Restore OTEL log schema fields (resource_attributes, scope_name, scope_version) across ingestion and UI
Normalize non-OTEL log sources into OTEL record format with proper field mapping
Enhance log detail view to display resource attributes and scope metadata
Refactor log parsing to extract and preserve OTEL-specific fields from multiple source formats
Diagram Walkthrough
File Walkthrough
4 files
Refactor log parsing to extract OTEL schema fieldsDisplay resource attributes in log detail viewRefactor JSON log parsing to preserve OTEL fieldsExtract and normalize OTEL log record fields2 files
Add tests for resource attributes renderingUpdate tests for OTEL field extraction4 files
Fix formatting in test assertionsMinor spacing adjustment in UI textReformat query filter code for readabilityReformat UUID parsing error handling5 files
Update documentation on MCP tools usageAdd change proposal for OTEL log schema restorationAdd design document for OTEL schema changesAdd requirements for OTEL log schema visibilityAdd implementation tasks for OTEL schema restorationImported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2354#issuecomment-3765541377
Original created: 2026-01-18T17:48:11Z
PR Compliance Guide 🔍
Below is a summary of compliance checks for this PR:
No security concerns identified
No security vulnerabilities detected by AI analysis. Human verification advised for critical code.🎫 #2339
metadata, attributes, trace/span ids, etc.) are preserved and visible again in the Logs UI
rather than only showing
Time | Level | Service | Message.information.
sources (e.g., syslog/GELF) into an OTEL record format for consistency.
Codebase context is not defined
Follow the guide to enable codebase context checks.
Generic: Comprehensive Audit Trails
Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.
Status: Passed
Generic: Meaningful Naming and Self-Documenting Code
Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting
Status: Passed
Generic: Secure Error Handling
Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.
Status: Passed
Generic: Robust Error Handling and Edge Case Management
Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation
Status:
Silent marshal failure: The new
encodeAttributeshelper silently returns an empty string whenjson.Marshalfails,losing context without any error propagation or logging.
Referred Code
Generic: Secure Logging Practices
Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.
Status:
Attribute passthrough risk: The new normalization merges arbitrary
entryfields intoattributes/resource_attributeswhich could inadvertently store sensitive values from log payloads, and the diff does not
show any explicit allowlist/redaction controls.
Referred Code
Generic: Security-First Input Validation and Data Handling
Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities
Status:
Unbounded parsing: The new
parseAttributeValue/parseKeyValueMaplogic accepts and parses free-form stringsinto maps without visible size/complexity limits, which may be risky for hostile or
extremely large log inputs.
Referred Code
Compliance status legend
🟢 - Fully Compliant🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label
Imported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2354#issuecomment-3765541543
Original created: 2026-01-18T17:48:22Z
CI Feedback 🧐
A test triggered by this PR failed. Here is an AI-generated analysis of the failure:
Action: build
Failed stage: Configure SRQL fixture database for tests [❌]
Failed test name: ""
Failure summary:
The action failed during environment/secret validation because the required secret
SRQL_TEST_DATABASE_CA_CERTwas not configured (it is empty in the job env at lines 540 and 626).The
workflow explicitly checks for this secret to verify the SRQL fixture TLS and aborts with:
SRQL_TEST_DATABASE_CA_CERT secret must be configured to verify SRQL fixture TLS.(line 636), exitingwith code 1 (line 637).
Relevant error logs:
Imported GitHub PR comment.
Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2354#issuecomment-3765542567
Original created: 2026-01-18T17:49:26Z
PR Code Suggestions ✨
Explore these optional code suggestions:
Consolidate redundant log processing logic
The PR introduces nearly identical log normalization logic in both Go and Elixir
components. This should be consolidated into a single service to avoid
redundancy and reduce maintenance.
Examples:
pkg/consumers/db-event-writer/json_logs.go [99-172]
elixir/serviceradar_core/lib/serviceradar/event_writer/processors/logs.ex [82-117]
Solution Walkthrough:
Before:
After:
Suggestion importance[1-10]: 9
__
Why: The suggestion correctly identifies significant duplication of complex log normalization logic between the Go (
json_logs.go) and Elixir (logs.ex) components, which is a major architectural concern impacting maintainability.Skip all-zero IDs
Update
bytes_to_hexto returnNoneif the input byte slice contains only zeros,preventing invalid all-zero trace or span IDs from being generated.
cmd/consumers/zen/src/otel_logs.rs [146-156]
Suggestion importance[1-10]: 7
__
Why: The suggestion correctly identifies that all-zero trace/span IDs are invalid and should be treated as absent, improving data quality by preventing meaningless IDs from being processed.
Support numeric & bool attrs
Modify
get_attr_stringto convert numeric and boolean attribute values tostrings, ensuring they are not dropped if they are not already string types.
cmd/consumers/zen/src/otel_logs.rs [135-144]
Suggestion importance[1-10]: 7
__
Why: This change makes the
get_attr_stringfunction more robust by handling non-string attribute values, preventing potential data loss when attributes like version numbers are numeric.Correct attribute map merge priority
Refactor
mergeAttributeMapsto be more idiomatic by iterating over theextramapfirst, then the
basemap, which simplifies the logic by removing an existencecheck.
pkg/consumers/db-event-writer/json_logs.go [489-507]
Suggestion importance[1-10]: 2
__
Why: The suggestion correctly identifies the desired merge priority but misinterprets the existing code, which already implements it; the proposed change is a minor refactoring for readability with no functional difference.
Use empty JSON object
In
encodeAttributes, return an empty JSON object ({}) instead of an empty stringwhen the attribute map is empty or marshaling fails to ensure consumers always
receive valid JSON.
pkg/consumers/db-event-writer/json_logs.go [476-487]
Suggestion importance[1-10]: 6
__
Why: Returning an empty JSON object (
{}) instead of an empty string for attribute fields ensures type consistency for downstream consumers, which is a good practice for data integrity.