bug(agent): checker requests #760

New issue

Closed

opened 2026-03-28 04:28:15 +00:00 by mfreeman451 · 1 comment

mfreeman451 commented

2026-03-28 04:28:15 +00:00

Owner

Imported from GitHub.

Original GitHub issue: #2331
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/2331
Original created: 2026-01-18T05:31:27Z

Describe the bug

Not sure why we are seeing these, this is what we would have expected to see if the poller (which is now completely removed/deprecated) were reaching down and out to the agent for status from these external checkers. The new model is the agent pushes everything up to it, and any external checker that the agent used to reach out to, now has to also push up to the agent. We are moving as much as we can into the serviceradar-agent service itself (anything written in golang that gets deployed in the edge) but there still might be some edge cases, either way, these messages in the logs don't make sense and I'm not sure where they are coming from. We also see something reporting statuses for a bunch of services to the core-elx, it presumably also comes from this.

{"level":"warn","component":"agent","request":{"service_name":"snmp","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"}{"level":"info","component":"agent","type":"grpc","name":"snmp","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"}                                                    {"level":"warn","component":"agent","request":{"service_name":"trapd","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"}                                                                                                                                                                                          {"level":"info","component":"agent","type":"grpc","name":"trapd","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"}                                                   {"level":"warn","component":"agent","request":{"service_name":"zen","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"} {"level":"info","component":"agent","type":"grpc","name":"zen","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"}                                                     {"level":"warn","component":"agent","request":{"service_name":"db-event-writer","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"}                                                                                                                                                                                {"level":"info","component":"agent","type":"grpc","name":"db-event-writer","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"}                                         {"level":"warn","component":"agent","request":{"service_name":"flowgger","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"}                                                                                                                                                                                       {"level":"info","component":"agent","type":"grpc","name":"flowgger","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"}                                                {"level":"warn","component":"agent","request":{"service_name":"mapper","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"}                                                                                                                                                                                         {"level":"info","component":"agent","type":"grpc","name":"mapper","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"}                                                  {"level":"warn","component":"agent","request":{"service_name":"rperf-checker","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"}                                                                                                                                                                                  {"level":"info","component":"agent","status_count":8,"time":"2026-01-18T05:29:02Z","message":"Successfully pushed status to gateway"}

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Imported from GitHub. Original GitHub issue: #2331 Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/2331 Original created: 2026-01-18T05:31:27Z --- **Describe the bug** Not sure why we are seeing these, this is what we would have expected to see if the poller (which is now completely removed/deprecated) were reaching down and out to the agent for status from these external checkers. The new model is the agent pushes everything up to it, and any external checker that the agent used to reach out to, now has to also push up to the agent. We are moving as much as we can into the `serviceradar-agent` service itself (anything written in golang that gets deployed in the edge) but there still might be some edge cases, either way, these messages in the logs don't make sense and I'm not sure where they are coming from. We also see something reporting statuses for a bunch of services to the core-elx, it presumably also comes from this. ``` {"level":"warn","component":"agent","request":{"service_name":"snmp","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"}{"level":"info","component":"agent","type":"grpc","name":"snmp","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"} {"level":"warn","component":"agent","request":{"service_name":"trapd","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"} {"level":"info","component":"agent","type":"grpc","name":"trapd","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"} {"level":"warn","component":"agent","request":{"service_name":"zen","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"} {"level":"info","component":"agent","type":"grpc","name":"zen","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"} {"level":"warn","component":"agent","request":{"service_name":"db-event-writer","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"} {"level":"info","component":"agent","type":"grpc","name":"db-event-writer","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"} {"level":"warn","component":"agent","request":{"service_name":"flowgger","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"} {"level":"info","component":"agent","type":"grpc","name":"flowgger","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"} {"level":"warn","component":"agent","request":{"service_name":"mapper","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"} {"level":"info","component":"agent","type":"grpc","name":"mapper","details":"","time":"2026-01-18T05:29:02Z","message":"Checker request"} {"level":"warn","component":"agent","request":{"service_name":"rperf-checker","service_type":"grpc","agent_id":"k8s-agent"},"time":"2026-01-18T05:29:02Z","message":"GatewayId is empty in request"} {"level":"info","component":"agent","status_count":8,"time":"2026-01-18T05:29:02Z","message":"Successfully pushed status to gateway"} ``` **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll down to '....' 4. See error **Expected behavior** A clear and concise description of what you expected to happen. **Screenshots** If applicable, add screenshots to help explain your problem. **Desktop (please complete the following information):** - OS: [e.g. iOS] - Browser [e.g. chrome, safari] - Version [e.g. 22] **Smartphone (please complete the following information):** - Device: [e.g. iPhone6] - OS: [e.g. iOS8.1] - Browser [e.g. stock browser, safari] - Version [e.g. 22] **Additional context** Add any other context about the problem here.

mfreeman451 commented

2026-03-28 04:28:15 +00:00

Author

Owner

Imported GitHub comment.

Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/2331#issuecomment-3765123067
Original created: 2026-01-18T09:59:37Z

closing as completed

Imported GitHub comment. Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/2331#issuecomment-3765123067 Original created: 2026-01-18T09:59:37Z --- closing as completed