bug(core): core crashing from bad poller #339

New issue

Closed

opened 2026-03-28 04:23:37 +00:00 by mfreeman451 · 1 comment

mfreeman451 commented

2026-03-28 04:23:37 +00:00

Owner

Imported from GitHub.

Original GitHub issue: #956
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/956
Original created: 2025-06-18T03:22:43Z

we have a misconfigured poller that cant communicate to the core, the core is crashing:

Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 Parsed 0 CPU metrics for poller demo-staging with timestamp
Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 Service sysmon on poller demo-staging has AgentID: default-agent
Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 Service ping on poller demo-staging has AgentID: default-agent
Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 Successfully updated poller status for demo-staging
Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 Updated API server state for poller: demo-staging
Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 gRPC call: /monitoring.PollerService/ReportStatus Duration: 32.81918ms Error: <nil>
Jun 18 03:21:26 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:26 Successfully flushed 1 timeseries metrics for poller demo-staging
Jun 18 03:21:26 demo-staging serviceradar-core[73687]: panic: runtime error: index out of range [0] with length 0
Jun 18 03:21:26 demo-staging serviceradar-core[73687]: goroutine 8 [running]:
Jun 18 03:21:26 demo-staging serviceradar-core[73687]: github.com/carverauto/serviceradar/pkg/core.(*Server).flushSysmonMetrics(0xc000244280, {0x102a9c8, 0x1f4e900})
Jun 18 03:21:26 demo-staging serviceradar-core[73687]:         /Users/mfreeman/src/serviceradar/pkg/core/server.go:219 +0x2cf
Jun 18 03:21:26 demo-staging serviceradar-core[73687]: github.com/carverauto/serviceradar/pkg/core.(*Server).flushAllBuffers(0xc000244280, {0x102a9c8, 0x1f4e900})
Jun 18 03:21:26 demo-staging serviceradar-core[73687]:         /Users/mfreeman/src/serviceradar/pkg/core/server.go:169 +0xa5
Jun 18 03:21:26 demo-staging serviceradar-core[73687]: github.com/carverauto/serviceradar/pkg/core.(*Server).flushBuffers(0xc000244280, {0x102a9c8, 0x1f4e900})
Jun 18 03:21:26 demo-staging serviceradar-core[73687]:         /Users/mfreeman/src/serviceradar/pkg/core/server.go:157 +0x8e
Jun 18 03:21:26 demo-staging serviceradar-core[73687]: created by github.com/carverauto/serviceradar/pkg/core.NewServer in goroutine 1
Jun 18 03:21:26 demo-staging serviceradar-core[73687]:         /Users/mfreeman/src/serviceradar/pkg/core/server.go:110 +0x707

Imported from GitHub. Original GitHub issue: #956 Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/956 Original created: 2025-06-18T03:22:43Z --- we have a misconfigured poller that cant communicate to the core, the core is crashing: ``` Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 Parsed 0 CPU metrics for poller demo-staging with timestamp Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 Service sysmon on poller demo-staging has AgentID: default-agent Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 Service ping on poller demo-staging has AgentID: default-agent Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 Successfully updated poller status for demo-staging Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 Updated API server state for poller: demo-staging Jun 18 03:21:21 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:21 gRPC call: /monitoring.PollerService/ReportStatus Duration: 32.81918ms Error: <nil> Jun 18 03:21:26 demo-staging serviceradar-core[73687]: 2025/06/18 03:21:26 Successfully flushed 1 timeseries metrics for poller demo-staging Jun 18 03:21:26 demo-staging serviceradar-core[73687]: panic: runtime error: index out of range [0] with length 0 Jun 18 03:21:26 demo-staging serviceradar-core[73687]: goroutine 8 [running]: Jun 18 03:21:26 demo-staging serviceradar-core[73687]: github.com/carverauto/serviceradar/pkg/core.(*Server).flushSysmonMetrics(0xc000244280, {0x102a9c8, 0x1f4e900}) Jun 18 03:21:26 demo-staging serviceradar-core[73687]: /Users/mfreeman/src/serviceradar/pkg/core/server.go:219 +0x2cf Jun 18 03:21:26 demo-staging serviceradar-core[73687]: github.com/carverauto/serviceradar/pkg/core.(*Server).flushAllBuffers(0xc000244280, {0x102a9c8, 0x1f4e900}) Jun 18 03:21:26 demo-staging serviceradar-core[73687]: /Users/mfreeman/src/serviceradar/pkg/core/server.go:169 +0xa5 Jun 18 03:21:26 demo-staging serviceradar-core[73687]: github.com/carverauto/serviceradar/pkg/core.(*Server).flushBuffers(0xc000244280, {0x102a9c8, 0x1f4e900}) Jun 18 03:21:26 demo-staging serviceradar-core[73687]: /Users/mfreeman/src/serviceradar/pkg/core/server.go:157 +0x8e Jun 18 03:21:26 demo-staging serviceradar-core[73687]: created by github.com/carverauto/serviceradar/pkg/core.NewServer in goroutine 1 Jun 18 03:21:26 demo-staging serviceradar-core[73687]: /Users/mfreeman/src/serviceradar/pkg/core/server.go:110 +0x707 ```

mfreeman451 commented

2026-03-28 04:23:37 +00:00

Author

Owner

Imported GitHub comment.

Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/956#issuecomment-2986484714
Original created: 2025-06-19T03:36:10Z

closing, fixed

Imported GitHub comment. Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/956#issuecomment-2986484714 Original created: 2025-06-19T03:36:10Z --- closing, fixed