Blog/live stack elixir phoenix cnpg #2583

Merged

mfreeman451 merged 2 commits from refs/pull/2583/head into main

2025-12-16 21:14:43 +00:00

mfreeman451 commented

2025-12-16 21:14:11 +00:00

(Migrated from github.com)

Owner

Imported from GitHub pull request.

Original GitHub pull request: #2159
Original author: @mfreeman451
Original URL: https://github.com/carverauto/serviceradar/pull/2159
Original created: 2025-12-16T21:14:11Z
Original updated: 2025-12-16T21:15:43Z
Original head: carverauto/serviceradar:blog/live-stack-elixir-phoenix-cnpg
Original base: main
Original merged: 2025-12-16T21:14:43Z by @mfreeman451

User description

IMPORTANT: Please sign the Developer Certificate of Origin

Thank you for your contribution to ServiceRadar. Please note, when contributing, the developer must include
a DCO sign-off statement indicating the DCO acceptance in one commit message. Here
is an example DCO Signed-off-by line in a commit message:

Signed-off-by: J. Doe <j.doe@domain.com>

Describe your changes

Issue ticket number and link

Code checklist before requesting a review

I have signed the DCO?
The build completes without errors?
All tests are passing when running make test?

PR Type

Documentation

Description

Publishes architecture blog post documenting Phoenix/LiveView UI migration
Explains consolidation from Next.js + Kong + standalone SRQL to unified stack
Details Rustler-embedded SRQL NIF and CloudNativePG with TimescaleDB + Apache AGE
Includes OpenSpec change proposal and requirements specification

Diagram Walkthrough

flowchart LR
  A["Previous Stack<br/>Next.js + Kong<br/>+ Standalone SRQL"] -->|"Consolidate"| B["New Architecture<br/>Phoenix LiveView<br/>+ Rustler NIF<br/>+ CNPG"]
  B -->|"Unified Data"| C["CloudNativePG<br/>TimescaleDB + AGE"]
  B -->|"Orchestration"| D["Go Core<br/>Pollers & Agents"]

File Walkthrough

Relevant files

Documentation

2025-12-16-simplifying-observability-elixir-rustler-cnpg.mdx `Architecture blog post on stack consolidation` docs/blog/2025-12-16-simplifying-observability-elixir-rustler-cnpg.mdx New blog post documenting the architectural shift from React/Next.js to Phoenix LiveView Explains SRQL migration to Rustler NIF running in-process within Phoenix Details consolidation to CloudNativePG with TimescaleDB for metrics and Apache AGE for topology graphs Covers identity/access control moving from Kong to Guardian-based Phoenix sessions Discusses real-time streaming via `pg_notify` and BEAM's advantages for observability	+112/-0
proposal.md `Change proposal for architecture blog publication` openspec/changes/add-blog-elixir-rustler-cnpg/proposal.md Change proposal documenting the need for public-facing blog post Explains staging now runs Phoenix/LiveView with embedded SRQL and CNPG Specifies blog post slug, title, and key topics to cover Notes this is documentation-only with no runtime or API changes	+14/-0
spec.md `Requirements specification for blog post` openspec/changes/add-blog-elixir-rustler-cnpg/specs/docs-blog/spec.md Formal requirements specification for blog post publication Defines frontmatter requirements (slug, title, date, authors, tags) Specifies content scenarios covering shipped staging architecture Documents data consolidation explanation from Proton/ClickHouse to CNPG	+15/-0
tasks.md `Task completion checklist for blog publication` openspec/changes/add-blog-elixir-rustler-cnpg/tasks.md Task checklist for blog post content and validation Marks all content tasks complete (draft, architecture coverage, examples) Confirms validation tasks completed including OpenSpec validation Verifies accuracy against staging branch	+9/-0

Imported from GitHub pull request. Original GitHub pull request: #2159 Original author: @mfreeman451 Original URL: https://github.com/carverauto/serviceradar/pull/2159 Original created: 2025-12-16T21:14:11Z Original updated: 2025-12-16T21:15:43Z Original head: carverauto/serviceradar:blog/live-stack-elixir-phoenix-cnpg Original base: main Original merged: 2025-12-16T21:14:43Z by @mfreeman451 --- ### **User description** ## IMPORTANT: Please sign the Developer Certificate of Origin Thank you for your contribution to ServiceRadar. Please note, when contributing, the developer must include a [DCO sign-off statement]( https://developercertificate.org/) indicating the DCO acceptance in one commit message. Here is an example DCO Signed-off-by line in a commit message: ``` Signed-off-by: J. Doe <j.doe@domain.com> ``` ## Describe your changes ## Issue ticket number and link ## Code checklist before requesting a review - [ ] I have signed the DCO? - [ ] The build completes without errors? - [ ] All tests are passing when running make test? ___ ### **PR Type** Documentation ___ ### **Description** - Publishes architecture blog post documenting Phoenix/LiveView UI migration - Explains consolidation from Next.js + Kong + standalone SRQL to unified stack - Details Rustler-embedded SRQL NIF and CloudNativePG with TimescaleDB + Apache AGE - Includes OpenSpec change proposal and requirements specification ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Previous Stack Next.js + Kong + Standalone SRQL"] -->|"Consolidate"| B["New Architecture Phoenix LiveView + Rustler NIF + CNPG"] B -->|"Unified Data"| C["CloudNativePG TimescaleDB + AGE"] B -->|"Orchestration"| D["Go Core Pollers & Agents"] ``` <details><summary><h3>File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td>Documentation</td><td><table> <tr> <td> <details> <summary>2025-12-16-simplifying-observability-elixir-rustler-cnpg.mdx<dd><code>Architecture blog post on stack consolidation</code>                        </dd></summary> <hr> docs/blog/2025-12-16-simplifying-observability-elixir-rustler-cnpg.mdx <ul><li>New blog post documenting the architectural shift from React/Next.js to Phoenix LiveView <li> Explains SRQL migration to Rustler NIF running in-process within Phoenix <li> Details consolidation to CloudNativePG with TimescaleDB for metrics and Apache AGE for topology graphs <li> Covers identity/access control moving from Kong to Guardian-based Phoenix sessions <li> Discusses real-time streaming via <code>pg_notify</code> and BEAM's advantages for observability</ul> </details> </td> <td><a href="https://github.com/carverauto/serviceradar/pull/2159/files#diff-70acd289efff15274939e438d481027a1dc80a399803a2265563e7d66fd9f704">+112/-0</a>  </td> </tr> <tr> <td> <details> <summary>proposal.md<dd><code>Change proposal for architecture blog publication</code>                </dd></summary> <hr> openspec/changes/add-blog-elixir-rustler-cnpg/proposal.md <ul><li>Change proposal documenting the need for public-facing blog post <li> Explains staging now runs Phoenix/LiveView with embedded SRQL and CNPG <li> Specifies blog post slug, title, and key topics to cover <li> Notes this is documentation-only with no runtime or API changes</ul> </details> </td> <td><a href="https://github.com/carverauto/serviceradar/pull/2159/files#diff-4c7eba15bd07cd7bda09c3aed6744991d215178f1bf0c39dcbef5949df1268ca">+14/-0</a>    </td> </tr> <tr> <td> <details> <summary>spec.md<dd><code>Requirements specification for blog post</code>                                  </dd></summary> <hr> openspec/changes/add-blog-elixir-rustler-cnpg/specs/docs-blog/spec.md <ul><li>Formal requirements specification for blog post publication <li> Defines frontmatter requirements (slug, title, date, authors, tags) <li> Specifies content scenarios covering shipped staging architecture <li> Documents data consolidation explanation from Proton/ClickHouse to CNPG</ul> </details> </td> <td><a href="https://github.com/carverauto/serviceradar/pull/2159/files#diff-31c07735883a529a64a3a846f9dc3bb00dec11890833344c48317f138bca6d0f">+15/-0</a>    </td> </tr> <tr> <td> <details> <summary>tasks.md<dd><code>Task completion checklist for blog publication</code>                      </dd></summary> <hr> openspec/changes/add-blog-elixir-rustler-cnpg/tasks.md <ul><li>Task checklist for blog post content and validation <li> Marks all content tasks complete (draft, architecture coverage, examples) <li> Confirms validation tasks completed including OpenSpec validation <li> Verifies accuracy against staging branch</ul> </details> </td> <td><a href="https://github.com/carverauto/serviceradar/pull/2159/files#diff-1e97e4a9125ecb93b8e4fff845ea2f39fa17efec699c9209777a0e0b1341000e">+9/-0</a>      </td> </tr> </table></td></tr></tbody></table> </details> ___

qodo-code-review[bot] commented

2025-12-16 21:14:46 +00:00

(Migrated from github.com)

Author

Owner

Imported GitHub PR comment.

Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2159#issuecomment-3662407145
Original created: 2025-12-16T21:14:46Z

You are nearing your monthly Qodo Merge usage quota. For more information, please visit here.

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢	No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Passed Learn more about managing compliance generic rules or creating your own custom rules

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

Imported GitHub PR comment. Original author: @qodo-code-review[bot] Original URL: https://github.com/carverauto/serviceradar/pull/2159#issuecomment-3662407145 Original created: 2025-12-16T21:14:46Z --- _You are nearing your monthly Qodo Merge usage quota. For more information, please visit [here](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/#cloud-users)._ ## PR Compliance Guide 🔍  Below is a summary of compliance checks for this PR: <table><tbody><tr><td colspan='2'>Security Compliance</td></tr> <tr><td>🟢</td><td><details><summary>No security concerns identified</summary> No security vulnerabilities detected by AI analysis. Human verification advised for critical code. </details></td></tr> <tr><td colspan='2'>Ticket Compliance</td></tr> <tr><td>⚪</td><td><details><summary>🎫 No ticket provided </summary> - [ ] Create ticket/issue  </details></td></tr> <tr><td colspan='2'>Codebase Duplication Compliance</td></tr> <tr><td>⚪</td><td><details><summary>Codebase context is not defined </summary> Follow the <a href='https://qodo-merge-docs.qodo.ai/core-abilities/rag_context_enrichment/'>guide</a> to enable codebase context checks. </details></td></tr> <tr><td colspan='2'>Custom Compliance</td></tr> <tr><td rowspan=6>🟢</td><td> <details><summary>Generic: Comprehensive Audit Trails</summary> **Objective:** To create a detailed and reliable record of critical system actions for security analysis and compliance. **Status:** Passed > Learn more about managing compliance <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#configuration-options'>generic rules</a> or creating your own <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#custom-compliance'>custom rules</a> </details></td></tr> <tr><td> <details><summary>Generic: Meaningful Naming and Self-Documenting Code</summary> **Objective:** Ensure all identifiers clearly express their purpose and intent, making code self-documenting **Status:** Passed > Learn more about managing compliance <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#configuration-options'>generic rules</a> or creating your own <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#custom-compliance'>custom rules</a> </details></td></tr> <tr><td> <details><summary>Generic: Robust Error Handling and Edge Case Management</summary> **Objective:** Ensure comprehensive error handling that provides meaningful context and graceful degradation **Status:** Passed > Learn more about managing compliance <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#configuration-options'>generic rules</a> or creating your own <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#custom-compliance'>custom rules</a> </details></td></tr> <tr><td> <details><summary>Generic: Secure Error Handling</summary> **Objective:** To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. **Status:** Passed > Learn more about managing compliance <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#configuration-options'>generic rules</a> or creating your own <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#custom-compliance'>custom rules</a> </details></td></tr> <tr><td> <details><summary>Generic: Secure Logging Practices</summary> **Objective:** To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. **Status:** Passed > Learn more about managing compliance <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#configuration-options'>generic rules</a> or creating your own <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#custom-compliance'>custom rules</a> </details></td></tr> <tr><td> <details><summary>Generic: Security-First Input Validation and Data Handling</summary> **Objective:** Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities **Status:** Passed > Learn more about managing compliance <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#configuration-options'>generic rules</a> or creating your own <a href='https://qodo-merge-docs.qodo.ai/tools/compliance/#custom-compliance'>custom rules</a> </details></td></tr> <tr><td align="center" colspan="2">   </td></tr></tbody></table> <details><summary>Compliance status legend</summary> 🟢 - Fully Compliant 🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label </details>

qodo-code-review[bot] commented

2025-12-16 21:15:43 +00:00

(Migrated from github.com)

Author

Owner

Imported GitHub PR comment.

Original author: @qodo-code-review[bot]
Original URL: https://github.com/carverauto/serviceradar/pull/2159#issuecomment-3662410339
Original created: 2025-12-16T21:15:43Z

You are nearing your monthly Qodo Merge usage quota. For more information, please visit here.

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Clarify feature release status Clarify that the `pg_notify`-driven streaming feature is in staging and planned for the next release, as required by the PR's specification files. docs/blog/2025-12-16-simplifying-observability-elixir-rustler-cnpg.mdx [77-81] ### Real-Time Streaming with `pg_notify` + +Our real-time streaming architecture, now live in staging and rolling out in the next release, is built on `pg_notify`: - Triggers emit events. Inserts on hot tables (`logs`, `metrics`) raise `pg_notify` payloads. - Notification workers. GenServers using `Postgrex.Notifications` fan events into Phoenix PubSub topics; LiveViews subscribe and stream-insert rows. - Backpressure-aware. Large payloads fall back to ID-only messages plus Ecto hydration with per-tenant RBAC. For extreme volume, logical replication drops in without changing the LiveView consumers. Apply / Chat Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies a discrepancy between the blog post and the PR's specification files, improving the post's accuracy regarding the feature's release status.	Medium
General	Correctly attribute the database extension Correct the attribution of the `pg_textsearch` extension from Timescale to TigerData for accuracy. docs/blog/2025-12-16-simplifying-observability-elixir-rustler-cnpg.mdx [43] -With Timescale's [upcoming](https://www.tigerdata.com/blog/introducing-pg_textsearch-true-bm25-ranking-hybrid-retrieval-postgres) `pg_textsearch` extension (BM25 + hybrid retrieval), we can add document search without introducing another database. +With TigerData's [upcoming](https://www.tigerdata.com/blog/introducing-pg_textsearch-true-bm25-ranking-hybrid-retrieval-postgres) `pg_textsearch` extension for Postgres (BM25 + hybrid retrieval), we can add document search without introducing another database. `[To ensure code accuracy, apply this suggestion manually]` Suggestion importance[1-10]: 6 __ Why: This is a valid factual correction that improves the accuracy of the blog post by correctly attributing the `pg_textsearch` extension to TigerData instead of Timescale.	Low
More

Imported GitHub PR comment. Original author: @qodo-code-review[bot] Original URL: https://github.com/carverauto/serviceradar/pull/2159#issuecomment-3662410339 Original created: 2025-12-16T21:15:43Z --- _You are nearing your monthly Qodo Merge usage quota. For more information, please visit [here](https://qodo-merge-docs.qodo.ai/installation/qodo_merge/#cloud-users)._ ## PR Code Suggestions ✨  Explore these optional code suggestions: <table><thead><tr><td>Category</td><td align=left>Suggestion                                                                                                                                    </td><td align=center>Impact</td></tr><tbody><tr><td rowspan=2>General</td> <td> <details><summary>Clarify feature release status</summary> ___ **Clarify that the <code>pg_notify</code>-driven streaming feature is in staging and planned for the next release, as required by the PR's specification files.** [docs/blog/2025-12-16-simplifying-observability-elixir-rustler-cnpg.mdx [77-81]](https://github.com/carverauto/serviceradar/pull/2159/files#diff-70acd289efff15274939e438d481027a1dc80a399803a2265563e7d66fd9f704R77-R81) ```diff ### Real-Time Streaming with `pg_notify` + +Our real-time streaming architecture, now live in staging and rolling out in the next release, is built on `pg_notify`: - **Triggers emit events.** Inserts on hot tables (`logs`, `metrics`) raise `pg_notify` payloads. - **Notification workers.** GenServers using `Postgrex.Notifications` fan events into Phoenix PubSub topics; LiveViews subscribe and stream-insert rows. - **Backpressure-aware.** Large payloads fall back to ID-only messages plus Ecto hydration with per-tenant RBAC. For extreme volume, logical replication drops in without changing the LiveView consumers. ``` - [ ] **Apply / Chat**  <details><summary>Suggestion importance[1-10]: 7</summary> __ Why: The suggestion correctly identifies a discrepancy between the blog post and the PR's specification files, improving the post's accuracy regarding the feature's release status. </details></details></td><td align=center>Medium </td></tr><tr><td> <details><summary>Correctly attribute the database extension</summary> ___ **Correct the attribution of the <code>pg_textsearch</code> extension from Timescale to TigerData for accuracy.** [docs/blog/2025-12-16-simplifying-observability-elixir-rustler-cnpg.mdx [43]](https://github.com/carverauto/serviceradar/pull/2159/files#diff-70acd289efff15274939e438d481027a1dc80a399803a2265563e7d66fd9f704R43-R43) ```diff -With Timescale's [upcoming](https://www.tigerdata.com/blog/introducing-pg_textsearch-true-bm25-ranking-hybrid-retrieval-postgres) `pg_textsearch` extension (BM25 + hybrid retrieval), we can add document search without introducing another database. +With TigerData's [upcoming](https://www.tigerdata.com/blog/introducing-pg_textsearch-true-bm25-ranking-hybrid-retrieval-postgres) `pg_textsearch` extension for Postgres (BM25 + hybrid retrieval), we can add document search without introducing another database. ``` `[To ensure code accuracy, apply this suggestion manually]` <details><summary>Suggestion importance[1-10]: 6</summary> __ Why: This is a valid factual correction that improves the accuracy of the blog post by correctly attributing the `pg_textsearch` extension to TigerData instead of Timescale. </details></details></td><td align=center>Low </td></tr> <tr><td align="center" colspan="2"> - [ ] More  </td><td></td></tr></tbody></table>