bug(zen): SPIFFE Workload API unavailable #800

New issue

Closed

opened 2026-03-28 04:28:40 +00:00 by mfreeman451 · 0 comments

mfreeman451 commented

2026-03-28 04:28:40 +00:00

Owner

Imported from GitHub.

Original GitHub issue: #2401
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/2401
Original created: 2026-01-20T04:21:55Z

Describe the bug

Waiting for NATS service at serviceradar-nats:4222...                                                                                                                                      NATS service is ready!                                                                                                                                                                     Waiting for data service at serviceradar-datasvc:50057...                                                                                                                                  Data service is ready!                                                                                                                                                                     Rules already installed, skipping initial rule installation                                                                                                                                Starting ServiceRadar Zen with config: /etc/serviceradar/zen.json                                                                                                                          [2026-01-20T04:21:01Z INFO  async_nats::connector] connected successfully server=4222 max_payload=1048576                                                                                  [2026-01-20T04:21:01Z INFO  zen::nats] connected to nats at tls://serviceradar-nats:4222                                                                                                   [2026-01-20T04:21:01Z INFO  async_nats] event: connected                                                                                                                                   [2026-01-20T04:21:01Z INFO  zen] using stream events                                                                                                                                       [2026-01-20T04:21:01Z INFO  zen] using consumer zen-consumer                                                                                                                               [2026-01-20T04:21:01Z INFO  zen::engine] initialized decision engine with bucket serviceradar-datasvc                                                                                      [2026-01-20T04:21:01Z INFO  zen] waiting for messages on subjects: ["logs.syslog", "logs.snmp", "logs.otel"]                                                                               [2026-01-20T04:21:01Z INFO  zen::rule_watcher] watching rules under agents/k8s-zen-consumer/events/                                                                                        [2026-01-20T04:21:01Z WARN  zen::spiffe] SPIFFE Workload API unavailable (Grpc(Status { code: PermissionDenied, message: "no identity issued", metadata: MetadataMap { headers: {"content-type": "application/grpc"} }, source: None })); retrying in 2s                                                                                                                              [2026-01-20T04:21:03Z WARN  zen::spiffe] SPIFFE Workload API unavailable (Grpc(Status { code: PermissionDenied, message: "no identity issued", metadata: MetadataMap { headers: {"content-type": "application/grpc"} }, source: None })); retrying in 2s                                                                                                                              [2026-01-20T04:21:05Z WARN  zen::spiffe] SPIFFE Workload API unavailable (Grpc(Status { code: PermissionDenied, message: "no identity issued", metadata: MetadataMap { headers: {"content-type": "application/grpc"} }, source: None })); retrying in 2s

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Imported from GitHub. Original GitHub issue: #2401 Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/2401 Original created: 2026-01-20T04:21:55Z --- **Describe the bug** ``` Waiting for NATS service at serviceradar-nats:4222... NATS service is ready! Waiting for data service at serviceradar-datasvc:50057... Data service is ready! Rules already installed, skipping initial rule installation Starting ServiceRadar Zen with config: /etc/serviceradar/zen.json [2026-01-20T04:21:01Z INFO async_nats::connector] connected successfully server=4222 max_payload=1048576 [2026-01-20T04:21:01Z INFO zen::nats] connected to nats at tls://serviceradar-nats:4222 [2026-01-20T04:21:01Z INFO async_nats] event: connected [2026-01-20T04:21:01Z INFO zen] using stream events [2026-01-20T04:21:01Z INFO zen] using consumer zen-consumer [2026-01-20T04:21:01Z INFO zen::engine] initialized decision engine with bucket serviceradar-datasvc [2026-01-20T04:21:01Z INFO zen] waiting for messages on subjects: ["logs.syslog", "logs.snmp", "logs.otel"] [2026-01-20T04:21:01Z INFO zen::rule_watcher] watching rules under agents/k8s-zen-consumer/events/ [2026-01-20T04:21:01Z WARN zen::spiffe] SPIFFE Workload API unavailable (Grpc(Status { code: PermissionDenied, message: "no identity issued", metadata: MetadataMap { headers: {"content-type": "application/grpc"} }, source: None })); retrying in 2s [2026-01-20T04:21:03Z WARN zen::spiffe] SPIFFE Workload API unavailable (Grpc(Status { code: PermissionDenied, message: "no identity issued", metadata: MetadataMap { headers: {"content-type": "application/grpc"} }, source: None })); retrying in 2s [2026-01-20T04:21:05Z WARN zen::spiffe] SPIFFE Workload API unavailable (Grpc(Status { code: PermissionDenied, message: "no identity issued", metadata: MetadataMap { headers: {"content-type": "application/grpc"} }, source: None })); retrying in 2s ``` **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll down to '....' 4. See error **Expected behavior** A clear and concise description of what you expected to happen. **Screenshots** If applicable, add screenshots to help explain your problem. **Desktop (please complete the following information):** - OS: [e.g. iOS] - Browser [e.g. chrome, safari] - Version [e.g. 22] **Smartphone (please complete the following information):** - Device: [e.g. iPhone6] - OS: [e.g. iOS8.1] - Browser [e.g. stock browser, safari] - Version [e.g. 22] **Additional context** Add any other context about the problem here.