Back to Portfolio
03 / Full-Stack Engineer • 2025

Prism LLM Trace & Cost Observatory

Streaming token-level latency waterfall, prompt regression diffs, and real-time cost attribution.

Next.jsStreaming RSCClickHousePythonVercel AI SDK
Prism LLM Trace & Cost Observatory
190msQuery P95 (10M spans)
31%Prompt Cost Saved
< 1msSDK Overhead
01 / The Engineering Challenge

Teams deploying multi-step AI workflows lacked visibility into which tool call or prompt revision triggered sudden token inflation and P95 latency spikes.

02 / System Topology
01
Agent Trace SDK
Zero-Blocking Span
02
Batch Ingest Queue
Backpressure Safe
03
Columnar Store
Sub-second Rollup
04
RSC Flamegraph
Streamed Analysis
03 / Architectural Solution

Created an asynchronous OpenTelemetry span collector feeding columnar rollups, surfaced inside a Next.js streaming Server Component dashboard with side-by-side prompt evaluation diffs.

04 / Key Trade-offs & Decisions
  • Buffered trace payloads in memory and flushed in 2-second micro-batches so telemetry never blocks user-facing LLM token streaming.
  • Pre-aggregated hourly cost cubes to serve instant dashboard views without full-table scans.
Next Case Study (04)
Veloce Headless Edge Commerce Engine