bug(core-elx): Failed to refresh otel_trace_summaries #997

New issue

Closed

opened 2026-03-28 04:30:39 +00:00 by mfreeman451 · 0 comments

mfreeman451 commented

2026-03-28 04:30:39 +00:00

Owner

Imported from GitHub.

Original GitHub issue: #2775
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/2775
Original created: 2026-02-10T16:04:16Z

Describe the bug

│ core-elx 16:02:48.631 [error] Postgrex.Protocol (#PID<0.6256.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.256110.0> (ServiceRadar.Jobs.RefreshTraceSummariesWorker) time ││ d out because it queued and checked out the connection for longer than 15000ms                                                                                                                ││ core-elx                                                                                                                                                                                      ││ core-elx #PID<0.256110.0> (ServiceRadar.Jobs.RefreshTraceSummariesWorker) was at location:                                                                                                    ││ core-elx                                                                                                                                                                                      ││ core-elx     (stdlib 7.1) gen.erl:243: :gen.do_call/4                                                                                                                                         ││ core-elx     (stdlib 7.1) gen_statem.erl:3261: :gen_statem.call/3                                                                                                                             ││ core-elx     (ssl 11.4) ssl_gen_statem.erl:1284: :ssl_gen_statem.call/2                                                                                                                       ││ core-elx     (postgrex 0.21.1) lib/postgrex/protocol.ex:3261: Postgrex.Protocol.msg_recv/4                                                                                                    ││ core-elx     (postgrex 0.21.1) lib/postgrex/protocol.ex:2263: Postgrex.Protocol.recv_bind/3                                                                                                   │
│ core-elx     (postgrex 0.21.1) lib/postgrex/protocol.ex:2118: Postgrex.Protocol.bind_execute_close/4                                                                                          ││ core-elx     (db_connection 2.8.1) lib/db_connection/holder.ex:354: DBConnection.Holder.holder_apply/4                                                                                        │
│ core-elx     (db_connection 2.8.1) lib/db_connection.ex:1555: DBConnection.run_execute/5                                                                                                      ││ core-elx                                                                                                                                                                                      │
│ core-elx 16:02:48.775 [error] Failed to refresh otel_trace_summaries: ssl recv: closed (the connection was closed by the pool, possibly due to a timeout or because the pool has been termina ││ ted)

web-ng 16:03:22.824 [error] Postgrex.Protocol (#PID<0.7108.0> ("db_conn_9")) disconnected: ** (DBConnection.ConnectionError) client #PID<0.601553.0> (ServiceRadar.Jobs.RefreshTraceSummariesWorker) timed out because it queued and checked out the connection for longer than 15000ms                                                                                                           web-ng                                                                                                                                                                                           web-ng #PID<0.601553.0> (ServiceRadar.Jobs.RefreshTraceSummariesWorker) was at location:                                                                                                         web-ng                                                                                                                                                                                           web-ng     (stdlib 7.1) gen.erl:243: :gen.do_call/4                                                                                                                                              web-ng     (stdlib 7.1) gen_statem.erl:3261: :gen_statem.call/3                                                                                                                                  web-ng     (ssl 11.4) ssl_gen_statem.erl:1284: :ssl_gen_statem.call/2                                                                                                                            web-ng     (postgrex 0.22.0) lib/postgrex/protocol.ex:3272: Postgrex.Protocol.msg_recv/4                                                                                                         web-ng     (postgrex 0.22.0) lib/postgrex/protocol.ex:2274: Postgrex.Protocol.recv_bind/3                                                                                                        web-ng     (postgrex 0.22.0) lib/postgrex/protocol.ex:2129: Postgrex.Protocol.bind_execute_close/4                                                                                               web-ng     (db_connection 2.9.0) lib/db_connection/holder.ex:356: DBConnection.Holder.holder_apply/4                                                                                             web-ng     (db_connection 2.9.0) lib/db_connection.ex:1539: DBConnection.run_execute/5                                                                                                           web-ng                                                                                                                                                                                           web-ng 16:03:23.033 [error] Failed to refresh otel_trace_summaries: ssl recv: closed (the connection was closed by the pool, possibly due to a timeout or because the pool has been terminated)

Not sure why we're trying to run this job on both nodes either, should be one or the other.

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Imported from GitHub. Original GitHub issue: #2775 Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/2775 Original created: 2026-02-10T16:04:16Z --- **Describe the bug** ``` │ core-elx 16:02:48.631 [error] Postgrex.Protocol (#PID<0.6256.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.256110.0> (ServiceRadar.Jobs.RefreshTraceSummariesWorker) time ││ d out because it queued and checked out the connection for longer than 15000ms ││ core-elx ││ core-elx #PID<0.256110.0> (ServiceRadar.Jobs.RefreshTraceSummariesWorker) was at location: ││ core-elx ││ core-elx (stdlib 7.1) gen.erl:243: :gen.do_call/4 ││ core-elx (stdlib 7.1) gen_statem.erl:3261: :gen_statem.call/3 ││ core-elx (ssl 11.4) ssl_gen_statem.erl:1284: :ssl_gen_statem.call/2 ││ core-elx (postgrex 0.21.1) lib/postgrex/protocol.ex:3261: Postgrex.Protocol.msg_recv/4 ││ core-elx (postgrex 0.21.1) lib/postgrex/protocol.ex:2263: Postgrex.Protocol.recv_bind/3 │ │ core-elx (postgrex 0.21.1) lib/postgrex/protocol.ex:2118: Postgrex.Protocol.bind_execute_close/4 ││ core-elx (db_connection 2.8.1) lib/db_connection/holder.ex:354: DBConnection.Holder.holder_apply/4 │ │ core-elx (db_connection 2.8.1) lib/db_connection.ex:1555: DBConnection.run_execute/5 ││ core-elx │ │ core-elx 16:02:48.775 [error] Failed to refresh otel_trace_summaries: ssl recv: closed (the connection was closed by the pool, possibly due to a timeout or because the pool has been termina ││ ted) ``` ``` web-ng 16:03:22.824 [error] Postgrex.Protocol (#PID<0.7108.0> ("db_conn_9")) disconnected: ** (DBConnection.ConnectionError) client #PID<0.601553.0> (ServiceRadar.Jobs.RefreshTraceSummariesWorker) timed out because it queued and checked out the connection for longer than 15000ms web-ng web-ng #PID<0.601553.0> (ServiceRadar.Jobs.RefreshTraceSummariesWorker) was at location: web-ng web-ng (stdlib 7.1) gen.erl:243: :gen.do_call/4 web-ng (stdlib 7.1) gen_statem.erl:3261: :gen_statem.call/3 web-ng (ssl 11.4) ssl_gen_statem.erl:1284: :ssl_gen_statem.call/2 web-ng (postgrex 0.22.0) lib/postgrex/protocol.ex:3272: Postgrex.Protocol.msg_recv/4 web-ng (postgrex 0.22.0) lib/postgrex/protocol.ex:2274: Postgrex.Protocol.recv_bind/3 web-ng (postgrex 0.22.0) lib/postgrex/protocol.ex:2129: Postgrex.Protocol.bind_execute_close/4 web-ng (db_connection 2.9.0) lib/db_connection/holder.ex:356: DBConnection.Holder.holder_apply/4 web-ng (db_connection 2.9.0) lib/db_connection.ex:1539: DBConnection.run_execute/5 web-ng web-ng 16:03:23.033 [error] Failed to refresh otel_trace_summaries: ssl recv: closed (the connection was closed by the pool, possibly due to a timeout or because the pool has been terminated) ``` Not sure why we're trying to run this job on both nodes either, should be one or the other. **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll down to '....' 4. See error **Expected behavior** A clear and concise description of what you expected to happen. **Screenshots** If applicable, add screenshots to help explain your problem. **Desktop (please complete the following information):** - OS: [e.g. iOS] - Browser [e.g. chrome, safari] - Version [e.g. 22] **Smartphone (please complete the following information):** - Device: [e.g. iPhone6] - OS: [e.g. iOS8.1] - Browser [e.g. stock browser, safari] - Version [e.g. 22] **Additional context** Add any other context about the problem here.