DGX-SPARK // LOCAL-AI-ENV

NVIDIA DGX Spark (GB10) · Grace Blackwell GB10 · unified CPU/GPU memory · NVFP4 tensor-core path

Fully local AI development
that survives the NDA.

Unified memory, quantized MoE architectures, and a native agentic harness turn a single NVIDIA DGX Spark into an unthrottled engineering workstation — every model, MCP server, and repository routed through one sovereign endpoint.

KEY THESIS — TOKEN ABUNDANCE

On fixed-cost hardware, increased token consumption ceases to be a bottleneck. No per-million-token bill means no incentive to compress workflows: aggressive multi-agent fan-out, continuous background verification, rich LSP/DAP/AST tooling, live stream inspection, and iterative self-correction loops all become free at the margin.

MARGINAL COST

0 ¢ / token

UNIFIED MEMORY

128 GB

CONTEXT WINDOW

262,144 tok

GATEWAY

1 endpoint

FIG. 01 — SYSTEM TOPOLOGY (INTERACTIVE)

DGX Spark service topologyThree swimlanes behind one ingress. Nginx Proxy Manager terminates TLS and publishes every lane under a *.dgx.lan host. The Inference lane routes model traffic through the LiteLLM gateway to two vLLM runtimes — Ornith 1.5 35B A3B NVFP4 and LFM3 1.7B NVFP4 — and to dgx-embedd, a python transformers service that serves the embedder, the reranker and the d3-nano decision head. The MCP / Memory lane holds Engram MCP, LightRAG with its Memgraph graph database, and FewShot MCP. The Workspace lane holds Gitea plus three containers: Isolated Shell, WebApps and Projects. Select a node to trace its path and inspect its details.EDGEsingle ingress for everyservice — TLS, access controland *.dgx.lan host publishingLEGENDdata flowingress01 · INFERENCEllm.dgx.lanGATEWAYRUNTIMESERVED MODELS02 · MCP / MEMORYrag.dgx.lan · engram.dgx.lan03 · WORKSPACEgit.dgx.lan · *.apps.dgx.lan · *.pro.dgx.lanNginx Proxy ManagerThe single ingress for the whole environment: TLS termination, access control and subdomain publishing (*.dgx.lan) in front of every internal service. It is driven declaratively by the agent over its token-authenticated JSON REST API rather than hand-edited nginx config.Nginx Proxy Managerreverse proxy · TLSLiteLLM Proxy:8000One OpenAI-compatible endpoint for every model call. Model-group aliases (default, smol, judge, embedding, reranker) resolve to the runtimes on the right, so no client ever binds to a port or a model id directly.LiteLLM Proxysingle point /v1:8000vLLM · Ornith 1.5:8009The workhorse. Mixture-of-Experts keeps only ~3B parameters hot per token, so NVFP4 weights fit in ~22 GB while the rest of unified memory becomes KV cache. Served by vLLM at :8009 and fronted by LiteLLM as `ornith` (plus `quick` — thinking disabled).vLLM · Ornith 1.535B A3B · NVFP4 · 262k:8009vLLM · LFM3:8100Sub-second model for high-frequency utility work: commit messages, fast fan-out under the `smol` role, and real-time TTSR rule evaluation. Feed-forward weights in NVFP4 (W4A4), attention/conv/embeddings in BF16.vLLM · LFM31.7B · NVFP4 · tri-hybrid:8100dgx-embedd:8010In-house Python application: FastAPI in front of Hugging Face transformers, fp16 on the GPU, loading and batching each model itself instead of delegating to a serving engine. Custom high-throughput service co-locating the primary embedder (BAAI/bge-small-en-v1.5, 384d), the secondary embedder (Qwen/Qwen3-Embedding-0.6B, 1024d), a cross-encoder reranker (ms-marco-MiniLM-L12-v2) and the d3-nano decision head. Canonical ids and aliases (`embedding`, `qwen3embed`, `reranker`, `decision`) resolved case-insensitively; per-task batched GPU inference.dgx-embeddpython · transformers · fp16:8010embedderPrimary embedder behind Engram and FewShot retrieval: BAAI/bge-small-en-v1.5, 384 dimensions, batched on the GPU. Aliased `embedding` at the gateway; a second 1024d embedder (Qwen3-Embedding-0.6B) is co-located in the same process for high-fidelity recall.embedderbge-small-en-v1.5 · 384drerankerCross-encoder that rescores the candidate set after vector/BM25 recall, exposed as a Jina-style /v1/rerank and aliased `reranker` at the gateway. Runs in the same GPU batch loop as the embedders.rerankerms-marco-MiniLM-L12-v2decisionDecision 3.0 checkpoint (Apache-2.0) co-located inside dgx-embedd. Answers state + questions with calibrated probability distributions (choice / yes-no / score) in one forward pass — no token-by-token decoding. Exposed as POST /v1/systemone and aliased `judge` at the gateway.decisiond3-nano · typed readoutEngram MCPCross-session episodic and architectural memory: agents retain project conventions, debugging lessons and environment specifics across restarts and context compactions.Engram MCPpersistent agent memoryLightRAGDocuments and notes indexed into a knowledge graph and retrieved by relational context rather than vector similarity alone. LLM and embedding calls go through the LiteLLM gateway; ingestion and querying run over a token-authenticated JSON REST API.LightRAGdocument knowledge graphMemgraphLightRAG's graph database: the entity/relation index lives here, so retrieval can traverse relationships between chunks instead of scoring isolated embeddings. Holds the document knowledge graph only — it is not the vector store.Memgraphgraph db · LightRAGFewShot MCPCurated code patterns: around 30,000 pre-verified pattern records across five language-routed knowledge bases, served over MCP and retrieved by hybrid BM25 + vector search. No external service is required.FewShot MCPcurated code patternsGiteaSovereign local git hosting for the stack's repositories and CI — local git :12000 (http) / :12222 (ssh).Gitearepos · CIIsolated ShellContainer the agent has unrestricted root access to. Long-running jobs, crontabs, prototypes and throwaway experiments run here instead of on the host, so nothing half-finished or irreversible touches the DGX itself.Isolated Shellroot sandbox · long jobsWebAppsContainer hosting a full web-application stack — nginx, MySQL, PostgreSQL and PHP — so applications can be exercised against real services and real SQL dialects instead of mocks.WebAppsweb app test stackProjectsSandbox container that acts as the playground for every other project: shared volumes, per-project toolchains and no risk to host state when an idea turns out badly.Projectssandbox · playground

SECTION 02 — INFRASTRUCTURE TOPOLOGY

Everything through one unified endpoint.

One gateway, one auth surface, one place to route, alias, and observe. The harness never talks to a model server directly — it talks to llm.dgx.lanon NPM, which is proxied to LiteLLM Proxy on port:8000, and the proxy owns the mapping from role names to model containers.

01 · EDGE · NGINX PROXY MANAGER

one ingress · *.dgx.lan

TLS termination, access lists, and subdomain publishing for the whole host. It is driven by the agent over its token-authenticated JSON REST API, so the network edge is just another control plane instead of a hand-edited nginx config.

  • llm.dgx.lan→ LiteLLM Proxy :8000
  • rag.dgx.lan→ LightRAG
  • engram.dgx.lan→ Engram MCP :7777
  • git.dgx.lan→ Gitea :12000
  • *.apps.dgx.lan→ WebApps container
  • *.pro.dgx.lan→ Projects container
SOURCE ↗

02 · INFERENCE

llm.dgx.lan → :8000

Ornith and LFM3 are served by vLLM. dgx-embedd is a Python application that loads its own models with Hugging Face transformers at fp16 — no serving engine, one process per batch loop.

No client ever binds to a port below: the gateway is the only address, and it owns the fan-out.

  • LiteLLM Proxy:8000 · single point /v1
  • vLLM · Ornith 1.5:8009 · 35B A3B NVFP4 · 262k
  • vLLM · LFM3:8100 · 1.7B NVFP4
  • dgx-embedd:8010 · python · transformers · fp16
  • └ embedderbge-small-en-v1.5 · 384d
  • └ rerankerms-marco-MiniLM-L12-v2
  • └ decisiond3-nano · typed readout
SOURCE ↗

03 · MCP / MEMORY

rag.dgx.lan · engram.dgx.lan

Engram MCP keeps episodic memory across sessions; FewShot MCP serves verified code patterns. LightRAG answers from documents by traversing a knowledge graph instead of scoring isolated chunks.

C#/.NET is wired one level deeper: FewShot is bound to omp's TTSR stream inspection, so verified patterns are injected mid-stream — including during the reasoning phase, not only when a tool call arrives.

  • Engram MCPpersistent memory · :7777
  • FewShot MCP~30k patterns · 5 languages
  • LightRAGdocuments → knowledge graph
  • └ Memgraphgraph database · in-memory

04 · WORKSPACE

git.dgx.lan · *.apps.dgx.lan · *.pro.dgx.lan

Sovereign git hosting plus three containers the agent uses instead of the host. Anything long-running, throwaway, or irreversible happens here, and each surface is published behind the same proxy edge.

  • Gitea:12000 http · :12222 ssh
  • Isolated Shellroot sandbox · cron · long jobs
  • WebAppsnginx · mysql · postgres · php
  • Projectssandbox · playground

model-group aliases at the router: `judge` → systemone · `d3-nano` → systemone · `embedding` → bge-small · `qwen3embed` → qwen3-embedding-0.6b · `reranker` → ms-marco-minilm-l12

SECTION 03 — MODEL STACK & INFERENCE SERVICE

Every gigabyte of unified memory earns its place.

NVFP4 quantization and 3B-active-parameter MoE architectures keep weights small enough that the rest of 128 GB becomes KV cache — which is what a 262,144-token context window and marathon agentic sessions actually consume.

Both vLLM engines publish their real vllm serve invocation. Expand LAUNCH CONFIG on a model to read the command and the reason behind every flag.

LANGUAGE MODELPORT 8009

Ornith 1.5 35B A3B (NVFP4)

Primary coding & reasoning

MoE · 35B total / ~3B active · NVFP4 (W4A4) · 262,144-token context · ~22 GB resident

The workhorse. Mixture-of-Experts keeps only ~3B parameters hot per token, so NVFP4 weights fit in ~22 GB while the rest of unified memory becomes KV cache. Served by vLLM at :8009 and fronted by LiteLLM as `ornith` (plus `quick` — thinking disabled).

vLLM LAUNCH CONFIGFLAG BY FLAG

Two vLLM servers share one 128 GB unified pool with dgx-embedd, so most of these flags are an allocation contract rather than benchmark chasing: deterministic memory partitions, a prefix cache that makes turn N+1 nearly free, FP8 KV that buys the 262k context, and kernels autotuned for GB10.

vllm serve "$ORNITH_MODEL" \
  --served-model-name ornith \
  --host 0.0.0.0 --port 8009 \
  --moe-backend b12x \
  --gpu-memory-utilization 0.40 \
  --speculative-config "$SPECCONF" \
  --kv-cache-memory 11542910976 \
  --enable-prefix-caching \
  --no-disable-cascade-attn \
  --max-num-seqs 10 \
  --enable-flashinfer-autotune \
  --max-model-len 262144 \
  --max-num-batched-tokens 8192 \
  --trust-remote-code \
  --enable-chunked-prefill \
  --kv-cache-dtype fp8 \
  --load-format fastsafetensors \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder
--served-model-name ornith
The alias is the contract, not the checkpoint path. LiteLLM upstreams, agent roles and scripts keep pointing at `ornith` when the weights underneath are swapped.
--host 0.0.0.0 --port 8009
All-interface binding is what lets the LiteLLM container reach the engine across the container bridge; the fixed port keeps the proxy upstream static.
--moe-backend b12x
Blackwell-tuned MoE kernel. For an A3B model the per-token cost is expert dispatch plus dozens of small grouped GEMMs; b12x fuses them instead of launching one GEMM per expert — the single largest throughput lever here.
--gpu-memory-utilization 0.40
A partition, not a tuning knob: ~48 GiB of the 128 GB pool for the primary model while LFM3 (0.07) and dgx-embedd (5.5 GB) co-reside. Unified memory has no swap — oversubscribing it takes the host down, not just the process.
--speculative-config "$SPECCONF"
Draft-model speculation: a cheap proposer emits candidate tokens, the 35B target verifies them in a single forward pass. Because only ~3B parameters are active per token, verification is nearly free and acceptance is high — inter-token latency drops with zero quality change, since accepted tokens are verified rather than approximated.
--kv-cache-memory 11542910976
An explicit 10.75 GiB KV pool (≈234k FP8-cached tokens) instead of a percentage. Cache size becomes deterministic, so TTFT and concurrency do not drift when a co-tenant changes its own reservation.
--enable-prefix-caching
Every agent turn re-sends the same system prompt, tool schemas and file context. Cached prefix blocks skip prefill entirely, so turn N+1 pays only for the new tokens — the dominant TTFT win in a coding loop.
--no-disable-cascade-attn
Keeps cascading attention enabled: shared prefix blocks are reused across query chunks instead of being recomputed, which is what makes a 262k cache bandwidth-affordable.
--max-num-seqs 10
Concurrency cap. Ten sequences matches real agent concurrency (main session plus subagents); a wider batch multiplies KV pressure and trades every stream’s latency for aggregate tokens/s nobody is waiting on.
--enable-flashinfer-autotune
Autotunes the FlashInfer attention/prefill kernels for GB10’s exact SM count and caches the plan. Generic kernel heuristics leave double-digit percent on the table on a part this unusual.
--max-model-len 262144
The full native 256k window, so long agent sessions never truncate mid-file. Affordable only because KV is FP8 and the pool is reserved explicitly.
--max-num-batched-tokens 8192
Scheduler step budget. 8k tokens per step keeps decode interleaved with a large prompt’s prefill, so a 200k-token context does not freeze streaming for seconds.
--trust-remote-code
Required: the checkpoint ships custom modeling code, and vLLM refuses to instantiate the architecture without it.
--enable-chunked-prefill
Splits long prompts into the 8k chunks above so decode tokens keep flowing while a large prefill is in flight.
--kv-cache-dtype fp8
FP8 E4M3 K/V halves cache bytes per token to 48 KiB (2 × 48 layers × 4 KV heads × 128 head_dim × 1 B). Attention tolerates quantization far better than weights do, and this is the single change that makes 256k context fit at all.
--load-format fastsafetensors
Parallel multi-threaded tensor loading: restarts take tens of seconds instead of minutes — which matters because every flag above was found by restarting.
--reasoning-parser qwen3
Splits the thinking channel from the answer as structured fields, so the harness can run TTSR over reasoning without reasoning text leaking into the reply.
--enable-auto-tool-choice --tool-call-parser qwen3_coder
Turns the model’s native tool-call grammar into typed arguments instead of prose the harness has to re-parse — fewer failed calls, less retry prefill.
DECISIONPORT 8010

TypeSafe System 1 (d3-nano)

Fast type/code verification · judge

2.21B (1.65B text + 0.33B vision) · single-pass typed decisions · non-decoding readout

Decision 3.0 checkpoint (Apache-2.0) co-located inside dgx-embedd. Answers state + questions with calibrated probability distributions (choice / yes-no / score) in one forward pass — no token-by-token decoding. Exposed as POST /v1/systemone and aliased `judge` at the gateway.

LANGUAGE MODELPORT 8100

LFM3 1.7B (NVFP4)

Ultra-fast smol · utility · routing

1.7B · tri-hybrid (conv + sliding-window attention) · NVFP4 feed-forward · ~1.8 GB resident

Sub-second model for high-frequency utility work: commit messages, fast fan-out under the `smol` role, and real-time TTSR rule evaluation. Feed-forward weights in NVFP4 (W4A4), attention/conv/embeddings in BF16.

vLLM LAUNCH CONFIGFLAG BY FLAG

The utility tier’s flags exist to keep it out of the way: a hard 7% memory ceiling, the same prefix cache and kernel autotuning as the primary model, and an identical OpenAI surface so a role can switch engines without changing a client.

vllm serve "$LFM3_MODEL" \
  --served-model-name lfm3 \
  --host 0.0.0.0 --port 8100 \
  --gpu-memory-utilization 0.07 \
  --enable-prefix-caching \
  --no-disable-cascade-attn \
  --max-num-seqs 10 \
  --enable-flashinfer-autotune \
  --max-model-len 65000 \
  --max-num-batched-tokens 8192 \
  --trust-remote-code \
  --enable-chunked-prefill \
  --kv-cache-dtype fp8 \
  --load-format safetensors \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder
--served-model-name lfm3
Stable alias for the `smol` and `commit` roles, independent of which 1.7B checkpoint is currently loaded.
--host 0.0.0.0 --port 8100
Reachable from the gateway container on a fixed port, so the LiteLLM upstream for `smol` never has to be re-pointed.
--gpu-memory-utilization 0.07
A hard ~8.5 GiB ceiling. The utility tier needs ~2 GB of weights plus KV for 65k tokens; anything more is memory the primary model’s KV pool cannot use. A co-tenant must not be able to starve the main engine.
--enable-prefix-caching
Commit-message prompts and TTSR judge calls repeat the same preamble thousands of times a day; cached prefixes turn each into a near-zero-prefill request.
--no-disable-cascade-attn
Same long-prompt attention reuse as the primary model — cheap to keep, and it prevents latency spikes when a large diff is summarised.
--max-num-seqs 10
Fan-out width for `smol` and `commit`. A 1.7B model already answers in milliseconds, so extra concurrency only adds queueing and KV churn.
--enable-flashinfer-autotune
Tuned FlashInfer kernels for GB10 and cached at startup; without it, the small model’s short sequences are dominated by launch overhead.
--max-model-len 65000
The model’s own window. Declaring more would only waste KV reservation on a tier that never sees long contexts.
--max-num-batched-tokens 8192
Bounded scheduler step so one large paste cannot stall the short utility calls queued behind it.
--trust-remote-code
The tri-hybrid conv/attention architecture ships custom modeling code.
--enable-chunked-prefill
Keeps decode interleaved while a long prompt prefills — the same behaviour as the primary engine, so latency is predictable across tiers.
--kv-cache-dtype fp8
Halves KV bytes per token. At 1.7B on short utility outputs the quantization cost is not measurable, and the saved memory goes back to the shared pool.
--load-format safetensors
The 1.8 GB checkpoint loads in about a second; the parallel fastsafetensors path buys nothing at this size, so the simpler default stays.
--reasoning-parser qwen3
Same structured thinking/answer split as the primary model, so TTSR reasoning scopes behave identically on both tiers.
--enable-auto-tool-choice --tool-call-parser qwen3_coder
Identical typed tool-call surface, which is what makes `smol` a drop-in substitution in a role fallback chain.
SERVICEPORT 8010

dgx-embedd

Embedding · rerank · decision service

FastAPI · OpenAI /v1/embeddings + Jina-style /v1/rerank + /v1/systemone · GPU (fp16)

Custom high-throughput service co-locating the primary embedder (BAAI/bge-small-en-v1.5, 384d), the secondary embedder (Qwen/Qwen3-Embedding-0.6B, 1024d), a cross-encoder reranker (ms-marco-MiniLM-L12-v2) and the d3-nano decision head. Canonical ids and aliases (`embedding`, `qwen3embed`, `reranker`, `decision`) resolved case-insensitively; per-task batched GPU inference.

Gitea (local) · TBD

FIG. 03 — UNIFIED MEMORY ALLOCATION

Unified memory · 128 GB

NVIDIA DGX Spark (GB10) — Grace Blackwell GB10 · unified CPU/GPU memory · NVFP4 tensor-core path

Stacked bar totaling 128 gigabytes of unified memory. Model weights are resident on the live host (Ornith 1.5 35B A3B NVFP4 22 GB, LFM3 1.7B NVFP4 1.8 GB, dgx-embedd (embed + rerank + d3-nano) 5.5 GB). The KV segment is a reservation of three 512K-token FP8 cache sequences (72 GB); the remaining 26.7 GB belongs to the operating system and the workspace containers.128 GBOrnith 1.5 35B A3B NVFP4 — 22 GBLFM3 1.7B NVFP4 — 1.8 GBdgx-embedd (embed + rerank + d3-nano) — 5.5 GBReserved for KV · 3 × 512K FP8 — 72 GBSystem + workspace containers — 26.7 GBKV reserved at 48 KiB/token (48 layers · 4 KV heads · 128 dim, FP8) → 24 GB per 512K sequence × 3weights measured on disk on the live host; the remainder serves the OS and the workspace containers

SECTION 04 — CONTEXT AUGMENTATION & MEMORY

Local models, elevated to senior-engineer reliability.

Weights alone don't know your codebase's conventions. Three layers close the gap: a curated pattern store against API drift, persistent agent memory against amnesia, and a graph-backed knowledge base over the team's documents and notes.

FewShot MCP

Curated code patterns

Dozens of battle-tested, verified code patterns paired with concise descriptions and architectural intent — retrieved by technical keyword at query time. It prevents models from hallucinating outdated APIs or reinventing solved internal patterns.

  • FastMCP STDIO server backed by embedded LanceDB (Tantivy BM25) or SQLite FTS5 — zero external services.
  • Around 30,000 pre-verified pattern records across five language-routed knowledge bases (the dotnet corpus is the largest).
  • Three-stage retrieval: token decomposition/expansion → high-precision AND tier → column-weighted OR tier → structural template dedup.
  • Dual-column BM25 surfaces real API names even when keywords omit them.

KNOWLEDGE BASES

dotnetpythonrustangulartypescript

Engram MCP

Persistent agent memory

Cross-session episodic and architectural memory: agents retain project conventions, past debugging lessons, and environment specifics across restarts and context compactions.

  • Local-first semantic memory: SQLite + LanceDB vectors, embeddings served locally — no cloud required.
  • Ingests notes, conversations, Drive, Notion, YouTube, Obsidian, audio; exposes a standard MCP server to any agent runtime.
  • 21 tools (recall, remember, relate, watch, pin, …) with hybrid semantic + FTS retrieval and time-decay ranking.
  • Elastic License 2.0 core; runs fully offline on the DGX (served at :7777 on the host).

LightRAG

KNOWLEDGE BASE

Documents and notes, indexed into a knowledge graph and retrieved by relational context rather than keyword match. The agent ingests and queries it through the same LiteLLM Proxy, so team knowledge becomes first-class retrieval for the models.

  • Graph-enhanced RAG — entities and relationships extracted from documents for multi-hop retrieval.
  • Backed by memgraph, an in-memory graph database, for fast entity-relationship traversal.
  • LLM and embeddings come from the same gateway (`quick` and `embedding`), keeping retrieval on sovereign hardware.
  • Agent-driven ingestion and querying over a token-authenticated JSON REST API.
GitHub ↗API :9621

THE LOCAL-MODEL FORGETFULNESS FIX — TTSR SYNERGY

FIG. 02

Smaller local models occasionally forget to query available MCP tools mid-task. Rather than bloating the system prompt with permanent reminders — which burn prefill tokens on every request — oh-my-pi's Time-Traveling Stream Rules (TTSR) solve it deterministically: the moment the model starts writing code in a relevant domain without having consulted FewShot MCP, the stream is intercepted and a Markdown rule is injected inline. Rules cost nothing until they trigger.

Time-Traveling Stream Rules — stream interruption timelineA token stream flows left to right. A watch band scans it; on a pattern match the stream halts, a rule is injected, and generation resumes from the same token.generation: normal reasoning + output tokens streamgenerationstream watch: regex · ast-grep · judge scan the live streamstream watchpattern match: writing code in domain X without querying FewShot MCPpattern matchhalt mid-stream: generation stops at the matching tokenhalt mid-streaminject rule: rule: query FewShot MCP firstinject ruleresume: generation continues from the exact tokenresumehover or focus a phase — the stream is intercepted, ruled, and resumed at the same tokenRULE INJECTEDquery FewShot MCP firstregex · ast-grep · judge0 tokens cost until triggeredinjected rule survives context compactioncut
TTSR inspects the live token stream — including reasoning. On a pattern match it halts mid-stream, injects the relevant Markdown rule, and resumes from that exact token. Rules cost 0 tokens until triggered and survive compaction.
EXAMPLE RULE · ~/.omp/agent/rules/suggest-dotnet-pattern.mdMARKDOWN + FRONTMATTER
---
description: Enforce querying xd://fewshot-patterns during C# and .NET tasks
scope: [thinking]
condition: '^(?![^]*(?:\bdef\s+\w+\s*\(|\bexport\s+(?:default|const|function)|:\s*(?:string|number|boolean)\b|\bimport\s+\{))(?=[\s\S]{150,})(?:[\s\S]*?\b(?:using\s+System|namespace\s+[\w.]+|public\s+(?:sealed\s+)?class|Task<|CancellationToken|dotnet\s+(?:build|test))\b){2}'
question: Is the task actual C#/.NET coding or C#/.NET code planning?
---
C# / .NET implementation or architectural planning detected.

Query xd://fewshot-patterns with "dotnet" plus 3-6 concrete keywords
(namespace, record, Span, DbContext, CancellationToken, ...) before answering.

One Markdown file per rule. Project-local rules live in .omp/rules/ and override the global set, so a repository can tighten the harness without touching the user profile. The condition above is shortened — the production rule enumerates ~40 language markers on each side.

OPTIONAL KEYS

astCondition
ast-grep pattern for edit/write streams — structural matching instead of text, e.g. catching a lock block that should have been an async primitive.
agents
Agent-name globs. main targets the top-level session only, so subagents can run a different rule set.
interruptMode
always · never · prose-only · tool-only — per-rule override of whether a match halts the stream or only warns.
globs · alwaysApply
The static escape hatch: a rule with no TTSR fields and a description joins the description-matched rulebook, and alwaysApply puts it in every prompt — the permanent-prompt cost TTSR exists to avoid.
RULE ANATOMY4 FIELDS
scope
Which token streams the rule watches. thinking is the reasoning channel, text the visible answer, tool:edit(*.cs) the argument stream of one tool on matching files. Omitted, it watches everything — the example narrows to reasoning on purpose, so the rule can fire before a line of code is ever emitted.
condition
A regex evaluated against the accumulating stream buffer, chunk by chunk: single-line patterns are tested on every delta, multi-line ones wait for the buffer to grow. It is the free first pass — no model call, no tokens until it matches. The example rejects Python/TypeScript markers with a negative lookahead, demands ≥150 characters of context, then requires two independent .NET markers.
question
A natural-language yes/no sent to the judge role — on this host d3-nano, whose single-pass typed readout answers without token-by-token decoding — once an in-scope output completes. It returns a yes-probability; ≥0.7 counts as a violation. This is the precise second pass: condition only gates whether the question is asked at all, so ambiguous phrasing costs one cheap judgment instead of a false halt.
body
Everything below the frontmatter is the text injected when the rule fires. Keep it imperative and short — it is paid for in prefill on every trigger, which is the exact cost static system-prompt reminders pay on every request.

SECTION 05 — THE AGENTIC HARNESS

The harness problem, solved for open weights.

Capable 30B–35B MoE models don't fail because they can't code — they fail because legacy harnesses force them through brittle, text-based editing tools. oh-my-pi (omp) is a high-precision agentic harness engineered around that failure mode: a Rust-native core, an IDE wired into the loop, and stream-level rule enforcement.

DAILY DRIVER — OH-MY-PI GUI

For day-to-day work I run the oh-my-pi GUI. It delivers a clean, responsive UI/UX with full control over the harness — every TTSR rule, hashline edit, runtime, and role is reachable from the panel — without giving up the Rust-native engine underneath.

STANDARD HARNESSES vs OH-MY-PI
ASPECTSTANDARD HARNESSESOH-MY-PI (OMP)
Edit mechanismsearch-replace blocks; exact multi-line reproduction, guessed line numbersHashline edits — reference content hashes, never re-type unchanged code
Context injectionStatic CLAUDE.md / AGENTS.md prompts; path-scoped rules; post-hoc hooksTTSR live stream inspection (incl. reasoning); rules cost 0 tokens until triggered
Code intelligenceRegex find-and-replace; console.log debuggingLSP renames/imports, DAP breakpoints, native Rust ast-grep/grep/glob/PTY
ExecutionStateless shells per call; context window fills with intermediate outputPersistent Python kernels & Bun workers; orchestration out of the LLM context
Model routingSingle model; provider lock-inRole-based (default / slow / smol / plan / commit / judge) with fallback chains
History controlMonotonic context growth; compaction loses nuancecheckpoint / rewind — restore clean states with a concise takeaway summary

SIX CORE PILLARS

01Time-Traveling Stream Rules (TTSR)

Unlike path-scoped rules or post-hoc PreToolUse hooks, omp inspects the live token stream — including the reasoning/thinking phase — in real time. When regex, ast-grep, or lightweight judge-model conditions match an emerging anti-pattern, generation halts mid-stream, the relevant Markdown rule is injected, and generation resumes from that exact token.

  • Rules consume 0 tokens until triggered — no bloated static system prompts.
  • Injected rules survive context compaction.
  • Hardware impact: preserved KV cache, minimal prefill latency on DGX Spark.
02Hashline File Editing

Standard harnesses force models to reproduce exact multi-line blocks or guess shifted line numbers — a destructive retry loop on fast 3B-active-parameter MoE models. omp tags every line with a short content hash; the model edits ranges by referencing hashes instead of rewriting unchanged code.

  • Drastically fewer failed edits ("search block not found" loops eliminated).
  • Less wasted prefill, less token burn — free locally, but still latency.
  • Deterministic anchors survive concurrent file shifts.
03Native IDE-Grade Intelligence (LSP · DAP · Rust AST)

Semantic refactoring executes via workspace/willRenameFiles — imports across the codebase update automatically instead of fragile regex replace. With DAP, the agent sets breakpoints and inspects runtime state directly rather than polluting files with print-statement guesswork.

  • LSP: symbol renames, file moves, cross-file import updates.
  • DAP: breakpoints and variable inspection — no console.log archaeology.
  • Rust-native core: ast-grep, grep, glob, PTY handling at native speed.
04Stateful Execution Runtime

Persistent Python kernels and Bun workers can invoke agent tools programmatically, enabling complex data transformations and multi-step orchestration without bloating the LLM context window with intermediate output.

  • Persistent kernels survive across turns — no cold-start per call.
  • Intermediate data lives in the runtime, not the context window.
  • Programmatic tool invocation for deterministic pipelines.
05Role-Based Model Routing & Local-First Ergonomics

Granular task routing across default, slow (heavy reasoning), smol (fast fan-out), plan, commit, and judge (TTSR evaluation) roles — with fallback chains and per-directory overrides — maps directly onto the models served by LiteLLM on this host.

  • Ornith 1.5 35B A3B for slow/default work; LFM3 1.7B for smol/commit.
  • d3-nano judge role for cheap stream-rule evaluation.
  • Per-directory overrides keep monorepo workspaces coherent.
06Deterministic Context Control (checkpoint / rewind)

Bookmark clean context states before exploratory debugging or dead-end branches, then rewind history back to the checkpoint with only a concise takeaway summary — keeping the KV cache lean during marathon coding sessions.

  • Dead ends cost a summary, not the whole exploration history.
  • Marathon sessions without unbounded context growth.
  • Deterministic restores — no "what did we try already" archaeology.

ENGINEERING CREDIBILITY — EXPLICIT TRADE-OFF

omp's rich native toolset trades roughly ~30% higher tool-call token overhead on frontier, metered models. In exchange it delivers dramatically higher first-pass accuracy on local / open-weights models. On DGX Spark — zero marginal token cost, fixed power envelope — that overhead is practically free.

SECTION 06 — THE DATASET FACTORY

While you sleep, the corpus grows.

oh-my-pi's persistent runtimes don't idle. Overnight it runs three dataset projects — typescript-dataset, dotnet-dataset, and python-dataset — each generating high-quality, tested, and proven coding patterns that feed the FewShot knowledge base.

Each project pins a target platform, then loops: pull the live answer from the web — official API docs, package metadata, real use cases — verify it against a zero-trust confidence gate, generate a unique pattern, syntax- and type-check it, dedup against a shared index, and append only what survives. No snippet is accepted on memory alone.

NIGHTLY RUNS

01typescript-dataset Angular 22 · TypeScript 6.0

A Principal Frontend Architect drives an Angular 22 + TypeScript 6.0 corpus. Every record is pinned to the released framework window, and any API, decorator, CLI flag, or compiler option below 90% confidence is re-verified against angular.dev, the Angular repositories, and the npm registry before it is written.

  • Pinned target — Angular 22.0.0, TypeScript 6.0.x, Node 26; version-sensitive behavior is stated explicitly.
  • 8 subagents in parallel on one shared dedup index — no two cover the same problem signature.
  • 14 pillars, from Signals & zoneless change detection through SSR routing, tooling, and schematics.
  • 3–14 sentences of prose followed by a self-contained snippet compiled under strict + strictTemplates + zoneless.
02dotnet-dataset C# / .NET 10

A Principal .NET Architect generates C# and .NET 10 patterns. NuGet packages, BCL signatures, and runtime features are pulled from Microsoft Learn and the dotnet/runtime repos — and re-verified below 90% confidence — so nothing in the corpus is a guessed API.

  • 12 pillars: zero-allocation performance, .NET 10 runtime innovations, the NuGet ecosystem, Kestrel/REST/GraphQL, and Linux internals.
  • 8 subagents running in batches, each emitting one single-line JSONL record.
  • 6–12 keyword tags, 3–14 sentences of prose, and a production-grade C# snippet with real type signatures.
  • Uniform pillar coverage and zero duplicate problem signatures enforced on every loop.
03python-dataset CPython 3.13 / 3.14

A Principal Python Architect builds a headless modern-Python corpus for CPython 3.13/3.14 — the language, stdlib, and PyPI, with no GUI, desktop, or TUI. Anything below 95% confidence is verified against docs.python.org, the PEP text, and the package docs before it is emitted.

  • Target: 5,000 distinct entries across 12 pillars — performance, concurrency, deep PyPI, RAG, and distributed systems.
  • Every snippet is syntax-checked with ast.parse before acceptance — no placeholders, no TODOs, no broken ellipsis.
  • 6–12 keyword tags, 4–14 sentences of prose, and a syntactically valid Python snippet.
  • Web-verified packages only; the exact API (module path, class, argument names) is named in the prose.

SECTION 07 — OPEN-SOURCE INDEX

Every tool, model, and service in one table.

The complete stack, its role, license, and where to find it. Slots marked TBD are reserved for private repository URLs pending publication.

COMPONENTCATEGORYROLELICENSELINKS
oh-my-pi (omp)HARNESSAgentic harness — TTSR, hashline editing, LSP/DAP, persistent runtimesMIT
LiteLLM ProxyGATEWAYSingle OpenAI-compatible endpoint with model-group aliasesMIT
vLLMRUNTIMEHigh-throughput serving for Ornith (NVFP4) and LFM3 (NVFP4)Apache-2.0
GiteaVERSION CONTROLSovereign local git hosting — :12000 http, :12222 sshMIT
FewShot MCPMCP SERVERKeyword-retrieved, pre-verified coding patterns (5 languages)Private
TBD
Engram MCPMCP SERVERCross-session episodic & architectural memoryElastic-2.0
dgx-embeddSERVICECo-located embedders + reranker + System One decision headPrivate
TBD
FastAPIRUNTIMEdgx-embedd service frameworkMIT
LanceDBRUNTIMEEmbedded vector + Tantivy FTS store behind FewShot & EngramApache-2.0
LightRAGSERVICEGraph-backed document knowledge base (memgraph)MIT
Nginx Proxy ManagerSERVICEAgent-managed reverse-proxy / TLS edge over JSON REST APIMIT

MODEL CARDS