bug(core): age graph merge failed #668

New issue

Closed

opened 2026-03-28 04:27:13 +00:00 by mfreeman451 · 0 comments

mfreeman451 commented

2026-03-28 04:27:13 +00:00

Owner

Imported from GitHub.

Original GitHub issue: #2058
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/issues/2058
Original created: 2025-12-04T17:10:41Z

{"level":"info","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","component":"stats_aggregator","pruned_inferred_records":1,"fallback_remaining":0,"processed_records":50002,"cnpg_devices":50002,"time":"2025-12-05T00:45:15Z","message":"Pruned inferred registry records to reconcile with CNPG"}                               {"level":"warn","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","error":"error iterating rows: ERROR: Entity failed to be updated: 3 (SQLSTATE XX000)","age_sqlstate":"XX000","kind":"graph","batch_size":1,"attempt":1,"max_attempts":3,"queue_wait_secs":1.287284036,"backoff":212.684732,"time":"2025-12-05T00:45:15Z","message":"age graph: transient merge failure, will retry"}                                                                                                                                     {"level":"warn","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","error":"error iterating rows: ERROR: deadlock detected (SQLSTATE 40P01)",
"age_sqlstate":"40P01","kind":"graph","batch_size":128,"attempt":1,"max_attempts":3,"queue_wait_secs":5.313955279,"exec_secs":5.31395124,"queue_depth":2,"queue_capacity":256,"time":"2025-12-05T00:45:20Z","message":"age graph: merge failed"}                                                                                                                                {"level":"warn","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","error":"error iterating rows: ERROR: deadlock detected (SQLSTATE 40P01)","batch_size":128,"time":"2025-12-05T00:45:20Z","message":"age graph: failed to queue batch"}                                                                                            {"level":"info","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","component":"stats_aggregator","pruned_inferred_records":1,"fallback_remaining":0,"processed_records":50002,"cnpg_devices":50002,"time":"2025-12-05T00:45:25Z","message":"Pruned inferred registry records to reconcile with CNPG"}
{"level":"info","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","poller_id":"k8s-poller","services":2,"sweep_services":0,"payload_bytes":741,"time":"2025-12-05T00:45:27Z","message":"Status report summary"}                                                                                                                     {"level":"info","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","poller_id":"k8s-poller","device_id":"serviceradar:poller:k8s-poller","host_ip":"10.42.199.47","time":"2025-12-05T00:45:27Z","message":"Registering poller as device"}                                                                                            {"level":"warn","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","error":"error iterating rows: ERROR: Entity failed to be updated: 3 (SQLSTATE XX000)","age_sqlstate":"XX000","kind":"graph","batch_size":1,"attempt":1,"max_attempts":3,"queue_wait_secs":2.050268985,"backoff":217.688959,"time":"2025-12-05T00:45:29Z","message
":"age graph: transient merge failure, will retry"}                                                                                                                                     {"level":"info","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","poller_id":"k8s-poller","device_id":"serviceradar:poller:k8s-poller","time":"2025-12-05T00:45:29Z","message":"Successfully registered poller as device"}                                                                                                         {"level":"warn","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","error":"error iterating rows: ERROR: Entity failed to be updated: 3 (SQLSTATE XX000)","age_sqlstate":"XX000","kind":"graph","batch_size":128,"attempt":1,"max_attempts":3,"queue_wait_secs":4.296723462,"backoff":164.943612,"time":"2025-12-05T00:45:33Z","messa
ge":"age graph: transient merge failure, will retry"}

Imported from GitHub. Original GitHub issue: #2058 Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/issues/2058 Original created: 2025-12-04T17:10:41Z --- ``` {"level":"info","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","component":"stats_aggregator","pruned_inferred_records":1,"fallback_remaining":0,"processed_records":50002,"cnpg_devices":50002,"time":"2025-12-05T00:45:15Z","message":"Pruned inferred registry records to reconcile with CNPG"} {"level":"warn","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","error":"error iterating rows: ERROR: Entity failed to be updated: 3 (SQLSTATE XX000)","age_sqlstate":"XX000","kind":"graph","batch_size":1,"attempt":1,"max_attempts":3,"queue_wait_secs":1.287284036,"backoff":212.684732,"time":"2025-12-05T00:45:15Z","message":"age graph: transient merge failure, will retry"} {"level":"warn","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","error":"error iterating rows: ERROR: deadlock detected (SQLSTATE 40P01)", "age_sqlstate":"40P01","kind":"graph","batch_size":128,"attempt":1,"max_attempts":3,"queue_wait_secs":5.313955279,"exec_secs":5.31395124,"queue_depth":2,"queue_capacity":256,"time":"2025-12-05T00:45:20Z","message":"age graph: merge failed"} {"level":"warn","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","error":"error iterating rows: ERROR: deadlock detected (SQLSTATE 40P01)","batch_size":128,"time":"2025-12-05T00:45:20Z","message":"age graph: failed to queue batch"} {"level":"info","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","component":"stats_aggregator","pruned_inferred_records":1,"fallback_remaining":0,"processed_records":50002,"cnpg_devices":50002,"time":"2025-12-05T00:45:25Z","message":"Pruned inferred registry records to reconcile with CNPG"} {"level":"info","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","poller_id":"k8s-poller","services":2,"sweep_services":0,"payload_bytes":741,"time":"2025-12-05T00:45:27Z","message":"Status report summary"} {"level":"info","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","poller_id":"k8s-poller","device_id":"serviceradar:poller:k8s-poller","host_ip":"10.42.199.47","time":"2025-12-05T00:45:27Z","message":"Registering poller as device"} {"level":"warn","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","error":"error iterating rows: ERROR: Entity failed to be updated: 3 (SQLSTATE XX000)","age_sqlstate":"XX000","kind":"graph","batch_size":1,"attempt":1,"max_attempts":3,"queue_wait_secs":2.050268985,"backoff":217.688959,"time":"2025-12-05T00:45:29Z","message ":"age graph: transient merge failure, will retry"} {"level":"info","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","poller_id":"k8s-poller","device_id":"serviceradar:poller:k8s-poller","time":"2025-12-05T00:45:29Z","message":"Successfully registered poller as device"} {"level":"warn","trace_id":"e26aecea0ce1a34ab5296b98cb7b0ef8","span_id":"6d67ee9eb5cc3c84","component":"core","error":"error iterating rows: ERROR: Entity failed to be updated: 3 (SQLSTATE XX000)","age_sqlstate":"XX000","kind":"graph","batch_size":128,"attempt":1,"max_attempts":3,"queue_wait_secs":4.296723462,"backoff":164.943612,"time":"2025-12-05T00:45:33Z","messa ge":"age graph: transient merge failure, will retry"} ```

Rows
Columns