LIBRAIN

Why this exists

Most RAG and AI-agent tutorials are written in Python. Production engineering organizations that run on .NET — banks, insurers, government — hear the message that "AI requires Python" and weaken the case for adopting LLM-driven features in their existing stack. LIBRAIN is a deliberate counter-example: a complete multi-agent retrieval pipeline built end-to-end in .NET 10 on Microsoft-blessed primitives (Semantic Kernel, Application Insights), Anthropic Claude, OpenAI embeddings, and a managed vector store.

Architecture

Two pipelines share a Reader Agent and a Qdrant vector store. The Phase 2 query path runs Synthesis to Evaluator on retrieved chunks. The Phase 2.5 Discovery path runs the Discovery Agent into three concurrent scorers (NoveltyScorer, Discovery Evaluator, ClaimValidator) under a single Task.WhenAll dispatch, then aggregates to a four-axis score. Every step emits a structured event with a correlation ID so any output is fully traceable from query back to source chunks.

Tech stack

Locked stack — chosen up-front, no swaps mid-build. Each pick has a stated rationale documented in the project plan.

API surface

Seven endpoints across Reader, Synthesis, Discovery, and Baseline tracks. Phase 1 ingest endpoints, the Phase 2 query endpoint that runs the full embed → search → synthesise → evaluate pipeline, the Phase 3 discovery endpoint for cross-domain hypothesis generation, and the Phase 5 baseline endpoints (Naive-RAG and Single-LLM ablations) that feed the companion paper's three-system comparison.

Pipeline walkthrough

Discovery Mode in action

Five end-to-end runs against an expanded 13-paper corpus spanning AI, drug discovery, climate, and neuroscience. Each card shows live POST /api/discover output: the generated hypothesis, the speculative portion flagged as novelClaim, citation-resolved supporting evidence from retrieved chunks, and 4-axis Discovery Evaluator scores (Claude Haiku, temperature 0.0).

A cross-run consistency study (N=5 per pair) confirms plausibility varies meaningfully across runs (std 0.14 in-corpus / 0.12 cross-domain), a genuine signal rather than a single-shot anchor. Structural coherence sits closer to a well-formedness baseline. Full empirical breakdown in the companion paper.

Inside the chunker

The recursive chunker is the foundation of citation tracking — every chunk's offset and length must be deterministic so the synthesis layer can map a hypothesis citation back to a verifiable source span. The fallback ladder ensures size limits are respected without ever splitting mid-sentence when a paragraph break is reachable.

private static int ChooseEnd(string text, int start)
{
    int remaining = text.Length - start;
    if (remaining <= MaxChars)
    {
        return text.Length;
    }

    int targetEnd = start + TargetChars;
    int maxEnd = start + MaxChars;

    // Prefer paragraph boundary in [target, max]
    int paraEnd = LatestMatchIndex(ParagraphBreakRegex(), text, targetEnd, maxEnd);
    if (paraEnd > start) return paraEnd;

    // Fall back to sentence boundary
    int sentEnd = LatestMatchIndex(SentenceBreakRegex(), text, targetEnd, maxEnd);
    if (sentEnd > start) return sentEnd;

    // Hard cut as last resort
    return maxEnd;
}

From LIBRAIN/Reading/RecursiveChunker.cs. Constants are TargetChars = 2000, MaxChars = 4000. Each emitted chunk also rewinds OverlapChars = 300 backward to preserve context across boundaries — the 15% overlap target from the original RAG paper.

Results

Smoke-tested end-to-end on 13 arXiv papers covering RAG, agentic AI surveys, drug discovery, climate forecasting, and computational neuroscience.

Papers in the corpus

The 13-paper corpus is built across two ingestion rounds. The first round established the RAG and agentic-AI foundation that validates the pipeline; the second round broadened reach into drug discovery, climate forecasting, and computational neuroscience so Discovery Mode can produce cross-domain hypotheses.

Seed corpus (5 papers, 218 chunks)

The foundational RAG paper, two agentic-AI surveys, and two long-form scientific-discovery surveys — chosen for pipeline depth (long surveys produce many chunks) and topical relevance (RAG, agents, retrieval-grounded generation).

Seed-corpus vector search smoke test: query "How does retrieval augmented generation work?" returned all 5 top hits from 2005.11401 — distinct chunks from pages 1, 2, 8, and 9, with cosine similarities 0.52–0.61. Semantic recall validated end-to-end against the corpus that planted RAG itself.

Domain expansion (8 papers, 392 chunks)

Drug discovery (3 papers), climate & weather (3 papers), and computational neuroscience (2 papers). Selected by popularity and topical recognition — well-known systems (GraphCast, Pangu-Weather, Aurora, ChemCrow, Centaur) plus comprehensive recent surveys.

What I learned

What's next

Phases 1 through 5 are complete: full Reader, Synthesis, and Evaluator pipeline, parallel Discovery Mode, a 13-paper corpus spanning AI, drug discovery, climate, and neuroscience, the static portfolio showcase you are reading right now, plus the three-system baseline (Naive-RAG and Single-LLM ablations), Anthropic prompt caching across all agents, parallelised scoring via Task.WhenAll, the claim-level hallucination mitigation, and a three-rater blinded replication of the AFTER-FIX pilot that confirms zero hallucinations across all fifteen outputs. Phase 6 (interactive frontend, cloud deployment, scaling the human-eval pilot beyond n = 15) is deferred and documented in the companion paper.

Get in touch

Open to senior .NET / AI-engineering roles globally — remote, on-site in Turkey, or relocating to the EU with visa sponsorship.