bug(core): #636

New issue

Closed

opened 2026-03-28 04:26:45 +00:00 by mfreeman451 · 0 comments

mfreeman451 commented

2026-03-28 04:26:45 +00:00

Owner

Imported from GitHub.

Original GitHub issue: #1919
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/1919
Original created: 2025-11-03T06:25:56Z

Describe the bug
Seeing lots of broken connections in the proton logs and these errors in the core logs after our recent update #1918

{"level":"info","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","update_count":1,"total_batches":1,"time":"2025-11-03T06:
24:38Z","message":"Successfully published device updates to device_updates stream"}
{"level":"info","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","poller_id":"k8s-poller","device_id":"serviceradar:poller
:k8s-poller","time":"2025-11-03T06:24:38Z","message":"Successfully registered poller as device"}
{"level":"warn","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","error":"lookup existing devices: error iterating rows: c
ontext deadline exceeded","time":"2025-11-03T06:24:39Z","message":"Failed to annotate _first_seen metadata"}
{"level":"warn","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","error":"lookup tombstoned devices: failed to batch query
 unified devices: context deadline exceeded","time":"2025-11-03T06:24:39Z","message":"Failed to filter updates against tombstones"}
{"level":"warn","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","error":"failed to register service device: failed to pub
lish device updates: failed to prepare device updates batch 1 (records 0-1): context deadline exceeded","poller_id":"k8s-poller","time":"2025-11-03T06:24:39Z","message
":"Failed to register service device for poller"}
{"level":"info","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","total_services":10,"time":"2025-11-03T06:24:39Z","messag
e":"Completed service message assembly"}
{"level":"info","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","poller_id":"k8s-poller","services":10,"sweep_services":0
,"payload_bytes":4786,"time":"2025-11-03T06:24:39Z","message":"StreamStatus summary"}
{"level":"info","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","agent_id":"k8s-agent","poller_id":"k8s-poller","time":"2
025-11-03T06:24:40Z","message":"Auto-registering agent from heartbeat"}

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Imported from GitHub. Original GitHub issue: #1919 Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/1919 Original created: 2025-11-03T06:25:56Z --- **Describe the bug** Seeing lots of broken connections in the proton logs and these errors in the core logs after our recent update #1918 ``` {"level":"info","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","update_count":1,"total_batches":1,"time":"2025-11-03T06: 24:38Z","message":"Successfully published device updates to device_updates stream"} {"level":"info","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","poller_id":"k8s-poller","device_id":"serviceradar:poller :k8s-poller","time":"2025-11-03T06:24:38Z","message":"Successfully registered poller as device"} {"level":"warn","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","error":"lookup existing devices: error iterating rows: c ontext deadline exceeded","time":"2025-11-03T06:24:39Z","message":"Failed to annotate _first_seen metadata"} {"level":"warn","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","error":"lookup tombstoned devices: failed to batch query unified devices: context deadline exceeded","time":"2025-11-03T06:24:39Z","message":"Failed to filter updates against tombstones"} {"level":"warn","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","error":"failed to register service device: failed to pub lish device updates: failed to prepare device updates batch 1 (records 0-1): context deadline exceeded","poller_id":"k8s-poller","time":"2025-11-03T06:24:39Z","message ":"Failed to register service device for poller"} {"level":"info","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","total_services":10,"time":"2025-11-03T06:24:39Z","messag e":"Completed service message assembly"} {"level":"info","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","poller_id":"k8s-poller","services":10,"sweep_services":0 ,"payload_bytes":4786,"time":"2025-11-03T06:24:39Z","message":"StreamStatus summary"} {"level":"info","trace_id":"77385d5688622679af3da34c54c4f2f9","span_id":"eb69cff45a82af19","component":"core","agent_id":"k8s-agent","poller_id":"k8s-poller","time":"2 025-11-03T06:24:40Z","message":"Auto-registering agent from heartbeat"} ``` **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll down to '....' 4. See error **Expected behavior** A clear and concise description of what you expected to happen. **Screenshots** If applicable, add screenshots to help explain your problem. **Desktop (please complete the following information):** - OS: [e.g. iOS] - Browser [e.g. chrome, safari] - Version [e.g. 22] **Smartphone (please complete the following information):** - Device: [e.g. iPhone6] - OS: [e.g. iOS8.1] - Browser [e.g. stock browser, safari] - Version [e.g. 22] **Additional context** Add any other context about the problem here.