librain stats
- papers read
- 41
- chunks in Qdrant
- 1,766
- fabricated citations
67down to 0
gh repo view librain
- commits
- 105
- stars
- ★ 1
- files
- 316
- licence
- MIT
- first commit
- 1 May 2026
- last commit
- 19 Jun 2026
- C# 97.9%
- Shell 1.6%
- TeX 0.5%
01LIBRAIN
An open-source multi-agent RAG system in .NET 10 that reads scientific papers and proposes citation-grounded hypotheses.
A multi-agent RAG system that reads scientific papers and proposes new hypotheses, built in .NET 10.
- Reader Agent: extracts text from arXiv PDFs, chunks it, embeds it and stores it in Qdrant.
- Synthesis Agent: retrieves the top chunks and asks Claude for citation-grounded hypotheses.
- Evaluator Agent: LLM-as-a-Judge with a fixed rubric (plausibility, novelty, clarity).
- Discovery Mode: the part that goes beyond the sources is flagged as novelClaim.
POST /api/discover# CASE_STUDY.md
LIBRAIN
Problem
- Most RAG and AI-agent tutorials are written in Python, and teams that run on .NET hear that “AI requires Python”.
- And when an LLM connects research papers, it can cite sources that were never retrieved.
Approach
- A complete multi-agent retrieval pipeline built end to end in .NET 10: a Reader, then Synthesis, then an Evaluator.
- A parallel Discovery Mode puts the speculative part of an answer in a separate field, novelClaim, and a ClaimValidator labels each of its sentences GROUNDED, EXTRAPOLATED or RISKY.
- A citation contract drops any reference that does not resolve to the retrieved set.
- 41 papersarXiv
- Readerchunk + embed
- Qdrant1,766 chunks
- SynthesisDiscovery
- EvaluatorClaimValidator
- Hypothesescited
What I built
- Agents in C# on Anthropic Claude, OpenAI embeddings (text-embedding-3-small) and Qdrant; every step is logged with a correlation ID.
- Baseline agents (naive RAG, a single LLM) for a three-system comparison, an experiments CLI and an xUnit test suite.
- A companion paper on Zenodo.
Result
- papers read
- 41
- chunks in Qdrant
- 1,766
- fabricated citations
67down to 0
On a 41-paper cross-domain corpus, the contract-free baseline produced 67 fabricated citations. The contracted pipeline let 0 through.
What I learned
- Synthetic tests don’t replace a smoke test on real files: one real arXiv PDF exposed a word-spacing bug and gaps in section detection that seven passing unit tests had missed.
- Respect the real rate limit: sizing embedding batches to the actual tier limit worked better than retry logic.
The full story, step by stepSee how LIBRAIN thinks The paper (DOI)
LIBRAIN/Agents/ReaderAgent.csOpen on GitHub
1using System.Diagnostics;2using LIBRAIN.Embeddings;3using LIBRAIN.Reading;4using LIBRAIN.Storage;5 6namespace LIBRAIN.Agents;7 8public sealed class ReaderAgent(9 ILogger<ReaderAgent> logger,10 PdfTextExtractor extractor,11 RecursiveChunker chunker,12 OpenAIEmbeddingClient embeddings,13 QdrantPaperRepository repository)14{15 private readonly ILogger<ReaderAgent> _logger = logger;16 private readonly PdfTextExtractor _extractor = extractor;17 private readonly RecursiveChunker _chunker = chunker;18 private readonly OpenAIEmbeddingClient _embeddings = embeddings;19 private readonly QdrantPaperRepository _repository = repository;20 21 public async Task<IngestResult> IngestAsync(22 Stream pdfStream,23 string paperId,24 CancellationToken ct = default)25 {26 var correlationId = Guid.NewGuid();27 using var scope = _logger.BeginScope(new Dictionary<string, object>28 {29 ["correlationId"] = correlationId,30 ["paperId"] = paperId,31 });32 33 var sw = Stopwatch.StartNew();34 _logger.LogInformation("Ingest started for {PaperId}", paperId);35 36 var doc = _extractor.Extract(pdfStream);37 var chunks = _chunker.Chunk(doc.FullText, doc.Pages);38 if (chunks.Count == 0)39 {40 _logger.LogWarning("No chunks produced; nothing to persist");41 return new IngestResult(paperId, 0, correlationId);42 }43 44 var contents = chunks.Select(c => c.Content).ToArray();45 var vectors = await _embeddings.GenerateAsync(contents, ct);46 47 var title = string.IsNullOrWhiteSpace(doc.Title) ? paperId : doc.Title;48 var ingestedAt = DateTime.UtcNow;49 var records = new ChunkRecord[chunks.Count];50 for (int i = 0; i < chunks.Count; i++)51 {52 var c = chunks[i];53 records[i] = new ChunkRecord(54 PaperId: paperId,55 ChunkIndex: c.ChunkIndex,56 Content: c.Content,57 Embedding: vectors[i],58 Title: title,59 Authors: doc.Authors,60 Year: doc.Year,61 Section: c.Section,62 PageNumber: c.PageNumber,63 IngestedAt: ingestedAt);64 }65 66 await _repository.PersistAsync(records, ct);67 68 _logger.LogInformation(69 "Ingest completed: {ChunkCount} chunks in {ElapsedMs}ms",70 chunks.Count,71 sw.ElapsedMilliseconds);72 73 return new IngestResult(paperId, chunks.Count, correlationId);74 }75}LIBRAIN/Agents/SynthesisAgent.csOpen on GitHub
1using System.Diagnostics;2using System.Text;3using System.Text.Json.Nodes;4using Anthropic.SDK.Common;5using Anthropic.SDK.Messaging;6using LIBRAIN.Models;7using LIBRAIN.Storage;8using Microsoft.Extensions.Options;9using AnthropicTool = Anthropic.SDK.Common.Tool;10 11namespace LIBRAIN.Agents;12 13public sealed class SynthesisAgent(14 ILogger<SynthesisAgent> logger,15 AnthropicChatClient claude,16 IOptions<ModelOptions> modelOptions)17{18 private readonly ILogger<SynthesisAgent> _logger = logger;19 private readonly AnthropicChatClient _claude = claude;20 private readonly string _synthesisModel = ModelSelection.SynthesisModel(modelOptions.Value);21 22 private const string SystemPromptText = """23 You are a research synthesis assistant for a literature-review system. You are given24 a user query and a numbered list of source excerpts retrieved from academic papers.25 26 Form a single, conservative hypothesis grounded ONLY in the provided sources. Cite by27 source number. List any claim in your hypothesis that is NOT directly supported by a28 cited source in `unsupported_claims`; if all claims are supported, return an empty array.29 30 If the sources are insufficient to form any defensible hypothesis: set `confidence`31 below 0.3, populate `unsupported_claims` with what's missing, and write a hypothesis32 stating the limitation explicitly (e.g., "The provided sources do not address X").33 34 Always call the `submit_hypothesis` tool. Do not respond in plain text.35 """;36 37 // Schema is raw JSON because Anthropic.SDK.Messaging.Property in 5.10 has no Items field;38 // it can't express array<integer> via the typed builder. Function's (string, string, JsonNode)39 // ctor accepts a parsed schema directly.40 private const string ToolSchemaJson = """41 {42 "type": "object",43 "properties": {44 "hypothesis": { "type": "string", "description": "A single conservative claim grounded only in the provided sources." },45 "citation_indices": { "type": "array", "items": { "type": "integer" }, "description": "1-based indices into the SOURCES block, ordered by relevance." },46 "confidence": { "type": "number", "description": "Self-reported 0.0-1.0; lower if sources are sparse or contradictory." },47 "unsupported_claims": { "type": "array", "items": { "type": "string" }, "description": "Substrings of the hypothesis that are NOT directly supported by the cited sources. Empty array if all claims are supported." }48 },49 "required": ["hypothesis", "citation_indices", "confidence", "unsupported_claims"]50 }51 """;52 53 private static readonly AnthropicTool SubmitHypothesisTool =54 new Function("submit_hypothesis", "Submit a citation-grounded hypothesis.", JsonNode.Parse(ToolSchemaJson)!);55 56 public async Task<SynthesisResult> SynthesizeAsync(57 string query,58 IReadOnlyList<SearchHit> chunks,59 Guid? correlationId = null,60 CancellationToken ct = default)61 {62 var corrId = correlationId ?? Guid.NewGuid();63 using var scope = _logger.BeginScope(new Dictionary<string, object>64 {65 ["correlationId"] = corrId,66 ["chunkCount"] = chunks.Count,67 });68 69 if (chunks.Count == 0 || chunks.All(c => string.IsNullOrWhiteSpace(c.Content)))70 {71 _logger.LogWarning("Synthesis skipped: no usable source chunks");72 return new SynthesisResult(73 Hypothesis: "Insufficient sources to form a hypothesis.",74 Citations: Array.Empty<SynthesisCitation>(),75 Confidence: null,76 UnsupportedClaims: Array.Empty<string>(),77 CorrelationId: corrId);78 }79 80 var sw = Stopwatch.StartNew();LIBRAIN/Agents/ClaimValidatorAgent.csOpen on GitHub
1using System.Diagnostics;2using System.Text;3using System.Text.Json.Nodes;4using Anthropic.SDK.Common;5using Anthropic.SDK.Constants;6using Anthropic.SDK.Messaging;7using LIBRAIN.Storage;8using AnthropicTool = Anthropic.SDK.Common.Tool;9 10namespace LIBRAIN.Agents;11 12// Stage 2 of the claim-level hallucination mitigation prototype. After13// DiscoveryAgent produces novelClaim + extrapolation_basis, this agent runs a14// secondary Haiku-backed pass that scores each sentence of novel_claim against the15// retrieved chunks and classifies it as GROUNDED, EXTRAPOLATED, or RISKY using the16// rubric's hallucination definition. The aggregate risk is C#-computed as the max17// of per-sentence probabilities to keep the score halo-resistant (mirroring the18// Phase 2 Evaluator pattern).19public sealed class ClaimValidatorAgent(20 ILogger<ClaimValidatorAgent> logger,21 AnthropicChatClient claude)22{23 private readonly ILogger<ClaimValidatorAgent> _logger = logger;24 private readonly AnthropicChatClient _claude = claude;25 26 private const string SystemPromptText = """27 You are a claim-level factuality validator. You are given the novel_claim28 portion of a discovery hypothesis and the retrieved source chunks that were29 available to the synthesizer. Your job is to split novel_claim into its30 constituent sentences and classify each sentence under three statuses:31 32 - GROUNDED: the sentence is directly supported by at least one retrieved33 chunk. Cite the chunk(s) in supporting_chunks as "paper_id:chunk_index".34 - EXTRAPOLATED: the sentence is speculative or hypothesis-framed. It goes35 beyond the retrieved evidence but does NOT assert a specific factual claim36 as established. Examples: "X could enable Y", "this suggests Y", "future37 work may show Y". Speculation framed as hypothesis is EXTRAPOLATED, not38 RISKY.39 - RISKY: the sentence states a SPECIFIC FACTUAL CLAIM (a mechanism, a40 quantity, a named entity, a date, an attribution, or a process detail) as41 if it were established knowledge, AND that claim is not supported by any42 retrieved chunk. The hallucination signature is confident factual framing43 without retrieval support. Examples that ARE risky:44 * "EEG biomarkers determine drug-target binding in real time" (specific45 mechanism stated as fact, no retrieval support)46 * "GraphCast achieves 98% accuracy on protein folding" (fabricated stat)47 * "PharmAgents was released by DeepMind in 2024" (wrong attribution)48 Examples that are NOT risky (these are EXTRAPOLATED):49 * "Weather forecasting algorithms could plausibly transfer to protein50 folding via shared mathematical structure"51 * "Real-time biomarker integration may eventually inform drug discovery"52 Hedging language ("could", "may", "suggests", "is hypothesized") is the53 marker that distinguishes EXTRAPOLATED from RISKY. Confident factual framing54 of an unsupported specific claim is the marker that distinguishes RISKY.55 56 For each sentence also return factual_hallucination_probability in [0, 1].57 GROUNDED ≈ 0.0, EXTRAPOLATED ≈ 0.1–0.3, RISKY ≈ 0.6–1.0. Use the full58 continuous range; do not snap to round numbers.59 60 Provide a one-sentence rationale per claim describing why you chose the61 status. Also return an aggregate_risk in [0, 1] (the application will62 recompute the canonical aggregate in code; your aggregate is a sanity63 check).64 65 Always call the `submit_claim_validation` tool. Do not respond in plain66 text.67 """;68 69 private const string ToolSchemaJson = """70 {71 "type": "object",72 "properties": {73 "claims": {74 "type": "array",75 "minItems": 1,76 "items": {77 "type": "object",78 "properties": {79 "sentence": { "type": "string", "description": "Exact sentence from novel_claim." },80 "factuality_status": { "type": "string", "enum": ["GROUNDED", "EXTRAPOLATED", "RISKY"] },LIBRAIN/Program.csOpen on GitHub
1using LIBRAIN.Agents;2using LIBRAIN.Embeddings;3using LIBRAIN.Endpoints;4using LIBRAIN.Models;5using LIBRAIN.Reading;6using LIBRAIN.Storage;7using Microsoft.AspNetCore.Http.Features;8using Microsoft.Extensions.Options;9using Qdrant.Client;10using Scalar.AspNetCore;11 12const long maxUploadBytes = 50_000_000; // 50MB cap on PDF uploads (largest data/papers/ paper is ~9MB)13 14var builder = WebApplication.CreateBuilder(args);15 16builder.WebHost.ConfigureKestrel(options =>17{18 options.Limits.MaxRequestBodySize = maxUploadBytes;19});20 21builder.Services.Configure<FormOptions>(options =>22{23 options.MultipartBodyLengthLimit = maxUploadBytes;24});25 26var applicationInsightsConnectionString = builder.Configuration["ApplicationInsights:ConnectionString"];27if (!string.IsNullOrWhiteSpace(applicationInsightsConnectionString))28{29 builder.Services.AddApplicationInsightsTelemetry();30}31 32builder.Services.AddOpenApi();33 34builder.Services.Configure<AnthropicOptions>(35 builder.Configuration.GetSection(AnthropicOptions.SectionName));36builder.Services.Configure<OpenAIOptions>(37 builder.Configuration.GetSection(OpenAIOptions.SectionName));38builder.Services.Configure<QdrantOptions>(39 builder.Configuration.GetSection(QdrantOptions.SectionName));40builder.Services.Configure<ModelOptions>(41 builder.Configuration.GetSection(ModelOptions.SectionName));42 43// Qdrant.Client uses gRPC (port 6334), not REST (6333). Cloud clusters require https + apiKey;44// local Docker dev uses neither. Empty ApiKey selects local mode automatically.45builder.Services.AddSingleton<QdrantClient>(sp =>46{47 var options = sp.GetRequiredService<IOptions<QdrantOptions>>().Value;48 return string.IsNullOrWhiteSpace(options.ApiKey)49 ? new QdrantClient(options.Host, options.Port)50 : new QdrantClient(51 host: options.Host,52 port: options.Port,53 https: options.UseHttps,54 apiKey: options.ApiKey);55});56 57builder.Services.AddSingleton<AnthropicChatClient>();58builder.Services.AddScoped<OpenAIEmbeddingClient>();59builder.Services.AddScoped<LIBRAIN.Generation.OpenAIChatClient>();60builder.Services.AddScoped<LIBRAIN.Generation.OpenAiDiscoveryAgent>();61builder.Services.AddScoped<QdrantPaperRepository>();62builder.Services.AddScoped<PdfTextExtractor>();63builder.Services.AddScoped<RecursiveChunker>();64builder.Services.AddScoped<ReaderAgent>();65builder.Services.AddScoped<SynthesisAgent>();66builder.Services.AddScoped<EvaluatorAgent>();67builder.Services.AddScoped<DiscoveryAgent>();68builder.Services.AddScoped<DiscoveryEvaluatorAgent>();69builder.Services.AddScoped<ClaimValidatorAgent>();70builder.Services.AddScoped<NoveltyScorer>();71builder.Services.AddScoped<SingleLlmAgent>();72builder.Services.AddScoped<NaiveRagAgent>();73 74var app = builder.Build();75 76if (app.Environment.IsDevelopment())77{78 app.MapOpenApi();79 app.MapScalarApiReference();80}.secrets/key.txt
1# You opened the dotfiles. Most people don't.2# This one is yours to take:3 mainC#★ 1README.mdGitHub
