Skip to content

BrainO 2.0 — Platform Plan

Status: draft for discussion · Last updated: 2026-10-27

1. Vision

BrainO is an open-source EEG and fMRI analysis platform with:

  • Universal data import — every common EEG/MEG and MRI/fMRI format, normalised to BIDS.
  • Validated analysis — preprocessing, statistics, machine learning and connectivity built on established open-source neuroscience libraries.
  • AI agents — an analysis agent that plans and runs pipelines through real, executed tools and explains the results, with the researcher approving key decisions.
  • Interactive 3D visualisation — a Unity-powered brain viewer embedded in the application for surfaces, sensors, statistical maps and networks.

Decisions taken

Topic Decision
Project model Open source, developed by a single developer (contributors welcome later)
Users Research first, with a path to clinical use (design for PHI handling and audit from day one)
Product A desktop application that runs entirely on the user's own computer (Windows, macOS, Linux) and uses its resources. There is no BrainO server, cloud service or paid GPU, for users or for the project
AI Own local AI built on open-weight Qwen models — no Claude or other hosted LLM API. Runs on the user's computer, on the CPU by default and on a GPU if there is one; fine-tuned for BrainO's tools
Compute Everything runs locally. Minimum computer: no GPU, 16 GB RAM, on which the engine, the app and the agent must all work (the agent with a small model, more slowly). A consumer GPU or more RAM only makes the agent faster or allows a larger model
Development resources The office RTX 2080 Ti and the University of Memphis HPC (free) are used only for development: fine-tuning and large evaluation runs. Users never need them
Main application React + TypeScript UI in an Electron desktop app that starts the engine and the model runtime on the user's computer; one installer per operating system
3D viewer Unity 6 LTS, visualisation only, built for WebGL and embedded as a panel in the main UI
VR/AR Out of scope
First milestone EEG pipeline + agent, with results shown in 3D

2. Architecture

┌──────────────────────────────────────────────────────────────────┐
│ Desktop app (Electron)                                           │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Main UI — React + TypeScript                                 │ │
│ │  data browser · pipeline builder · EEG trace browser ·       │ │
│ │  2D plots · stats tables · agent chat + approvals · reports  │ │
│ │                                                              │ │
│ │   ┌────────────────────────────────────────────┐             │ │
│ │   │ 3D viewer panel — Unity 6 WebGL build      │             │ │
│ │   │ brain surfaces · sensors · stat maps ·     │◄─ JS bridge │ │
│ │   │ connectomes · volumes                      │  (commands, │ │
│ │   └───────────────────▲────────────────────────┘  selections)│ │
│ └───────────────────────┼──────────────────────────────────────┘ │
└──────────▲──────────────┼────────────────────────────────────────┘
           │ REST +       │ HTTP (binary meshes/arrays,
           │ WebSocket    │ fetched directly by the viewer)
┌──────────┴──────────────┴────────────────────────────────────────┐
│ Engine (Python, FastAPI on localhost) — started by the app        │
│  ┌────────────┐  ┌──────────────┐  ┌──────────────────────────┐   │
│  │ Agent      │→ │ Tool registry│→ │ Analysis modules          │   │
│  │ (local     │  │ typed tools, │  │ io · eeg · fmri · stats · │   │
│  │  Qwen LLM) │  │ sandboxed    │  │ ml · connectivity         │   │
│  └─────▲──────┘  └──────────────┘  └──────────────────────────┘   │
│        │ OpenAI-compatible API (localhost)                        │
│  ┌─────┴──────────────────────────┐                               │
│  │ llama.cpp (bundled): a Qwen    │  + knowledge base (RAG over   │
│  │ GGUF model on the CPU or GPU   │    MNE/nilearn docs, methods) │
│  └────────────────────────────────┘                               │
│  Job runner (local processes, sized to the machine's CPU and RAM) │
│  Provenance store · Scene service (meshes/arrays for the viewer)  │
└──────────▲────────────────────────────────────────────────────────┘
           │
┌──────────┴────────────────────────────────────────────────────────┐
│ Data: BIDS dataset + derivatives/ (BIDS-Derivatives layout)       │
└───────────────────────────────────────────────────────────────────┘

Principles:

  1. Each layer does one job. Python does the science, the React app does the workflow and the 2D views, and Unity only renders 3D. Unity contains no analysis code, file I/O or business logic.
  2. BIDS everywhere. Every import is converted to, or indexed as, BIDS. Every analysis output is written as a BIDS derivative.
  3. The agent never invents a number. Every value shown comes from an executed tool. The agent's job is to plan, choose parameters, interpret results and explain them.
  4. Reproducible by construction. Each analysis run exports a standalone script plus a provenance record (inputs, hashes, parameters, library versions).
  5. The engine works without the agent. All tools can also be used from the UI and a Python API, so the platform is useful without AI and can be tested on its own.
  6. Runs on the user's computer, within its limits. No server to run, no account, no network needed after installation and the one-time model download. The job runner uses the cores and memory available (it runs as many jobs at once as fit, shows progress and can cancel), results are cached so nothing is computed twice, and every feature must work on the minimum computer (no GPU, 16 GB RAM). For scale: the full 40-participant ERP CORE validation runs in about an hour on one desktop CPU.

Why Electron rather than Tauri

Tauri renders with the operating system's webview, which differs between platforms (WebView2 on Windows, WebKit on macOS, WebKitGTK on Linux), and WebGL support and performance vary between them. Electron ships its own Chromium, so the embedded Unity WebGL viewer behaves the same everywhere. We can revisit this if installer size becomes a problem.

3. Data formats

Modality Formats Library
EEG/MEG EDF/EDF+, BDF, BrainVision, EEGLAB .set, FIF, Neuroscan CNT, EGI MFF, GDF, XDF (LSL), Curry, Nihon Kohden, Persyst, BIDS-EEG MNE-Python, MNE-BIDS, pyxdf
MRI/fMRI DICOM, NIfTI-1/2, PAR/REC, Analyze, AFNI BRIK/HEAD, MGH/MGZ, BIDS dcm2niix, nibabel
Surfaces FreeSurfer surf/annot/label, GIFTI, CIFTI-2 nibabel, nilearn
Derived Connectivity matrices (CSV/MAT/NPY), source estimates, atlases numpy, scipy, MNE

The import step is one pipeline: detect → read → validate → de-identify → convert to BIDS → index. It produces a QC summary that both the UI and the agent read.

4. Analysis capability map

EEG (milestone 1 in bold) - Filtering, re-referencing, resampling, bad-channel detection (PyPREP), ICA artifact removal (ICLabel via mne-icalabel), epoching, autoreject - ERPs, power spectra (Welch/multitaper), time–frequency analysis (Morlet), band power - Sensor-level statistics: t-tests/ANOVA, cluster-based permutation tests, FDR/FWER correction - Source localisation (MNE/dSPM/eLORETA with fsaverage or an individual MRI) - Connectivity: coherence, PLV, wPLI, Granger/PDC (mne-connectivity), graph metrics - ML decoding: CSP+LDA, Riemannian methods (pyriemann), temporal generalisation, deep learning (braindecode) - Microstates, ERP component detection, sleep staging (YASA)

fMRI (phase 3) - Preprocessing with fMRIPrep (container, optional), confound handling - First- and second-level GLM (nilearn), contrasts, thresholding - Resting-state functional connectivity, ICA/dual regression, seed-based maps - MVPA/searchlight, ROI analysis on the existing BrainO atlases

Multimodal (phase 5): EEG-informed fMRI, joint source and connectivity views.

5. Agent design

5.1 Structure

  • One orchestrator agent running on a local, open-weight Qwen model with native tool calling. Nothing is sent to an external AI service.
  • Specialist sub-agents (QC reviewer, statistics reviewer) are added only when an evaluation shows they help.

5.2 BrainO's own AI: what "building it" means

Training a language model from scratch is not realistic for one developer. "Our own AI" is built in four layers on top of Qwen:

  1. Base model — an open-weight Qwen instruct model with tool calling, run by llama.cpp, which the desktop app bundles and starts, through its OpenAI-compatible API on localhost. The engine talks only to that API, so any compatible runtime (e.g. an Ollama the user already has) or model can be swapped in. Models are downloaded once, on first use, not bundled.
  2. Model tiers by the user's hardware (4-bit GGUF, approximate; the app detects RAM and GPU memory, proposes a tier, and the user can change it):
Tier User's computer Example Qwen model Expectation
Minimum (default) No GPU, 16 GB RAM Qwen3.5 4B (2.6 GB) Full agent on the CPU; slower answers. The design target
More RAM No GPU, 32 GB+ RAM Qwen3.6 35B-A3B, mixture of experts with ~3B active (19 GB) Better planning; MoE keeps CPU speed close to a small model
Consumer GPU 6–12 GB VRAM, or Apple silicon Qwen3.5 9B on the GPU, or the 35B-A3B with its experts in RAM Faster, and a larger model

The agent is designed and evaluated for the minimum tier: larger models make it better, but nothing may depend on them. On a CPU, reading the prompt is the slow part, so the agent sends only the tool schemas a step needs, keeps tool results short, and reuses llama.cpp's prompt cache between turns.

Development hardware (not needed by users): - Office workstation, RTX 2080 Ti (11 GB): runs the laptop tier (~8B, 4-bit) fully on the GPU for day-to-day development. A ~30B mixture-of-experts model also runs, with experts partly offloaded to CPU RAM by llama.cpp; it is slower but fine for testing the workstation tier. - University of Memphis HPC, V100 GPUs: fine-tuning (QLoRA) and large evaluation runs, submitted as batch jobs. Model serving for evals uses llama.cpp, which supports this GPU generation. Evaluations of the minimum tier run on the CPU (CPU nodes, or the office machine with the GPU unused), so the scores reflect what users get. - Both GPUs are pre-Ampere: no bfloat16 and no FlashAttention 2. Training uses fp16 mixed precision with loss scaling; inference uses quantised GGUF models.

  1. Knowledge (RAG) — a local search index over MNE-Python, nilearn and BIDS documentation, BrainO's own tool docs, and methods guidelines (e.g. COBIDAS / COBIDAS-MEEG). The agent looks these up instead of relying on memory.
  2. Fine-tuning (LoRA/QLoRA) — teach the model BrainO's tools and analysis conventions using tool-call examples: synthetic analysis sessions generated from validated pipeline templates on public datasets, then checked by running them. This specialises the model; it does not replace it. Training happens on the HPC; the result is merged and quantised to GGUF, so users download a ready model and never train anything. The small default model comes first, since it is the one most users run and the one with the most to gain.

Before fine-tuning anything, the agent is measured with the base model + RAG on the eval set (M1.7). We fine-tune only where the evaluation shows gaps.

5.3 Making a local model reliable

Local models are weaker planners than the largest hosted models, so reliability comes from the system design, not the model alone:

  • Constrained output: tool calls are forced to match their JSON schema (grammar-constrained decoding in llama.cpp). Invalid calls are impossible, not just unlikely.
  • Validated pipeline templates: for common analyses (ERP comparison, spectral comparison, resting-state connectivity) the agent fills in a tested template rather than inventing a pipeline from scratch. Free-form planning is reserved for unusual requests.
  • Plan check: before running, the agent's plan is checked by rules (required steps present, order valid, multiple-comparison correction chosen) and shown to the user.
  • Small, explicit tool results: tools return short structured summaries so the context stays within what a local model handles well.

5.4 Tool contract

Each tool is a typed Python function (Pydantic input and output schemas) that: - runs in the job runner, not inside the model, - returns a compact structured summary to the agent and writes full outputs to derivatives/, - logs a provenance entry, - declares a risk level: read, compute, or decision. decision tools (dropping channels or subjects, choosing a statistical model, correcting for multiple comparisons) require user approval in the UI.

Initial EEG tool set:

Tool Purpose
dataset.inspect Formats, subjects, sessions, channels, sampling rate, events, durations
eeg.qc_report Line noise, flat or noisy channels, artifact estimate
eeg.preprocess Filter, notch, reference, resample (configurable)
eeg.detect_bad_channels PyPREP; proposes channels to drop (decision)
eeg.ica_clean ICA + ICLabel; proposes components to remove (decision)
eeg.epoch Epoch by events, baseline, reject
eeg.erp Condition averages, peak and latency measures
eeg.spectral PSD and band power
eeg.tfr Time–frequency decomposition
stats.cluster_permutation Sensor/time/frequency cluster tests
stats.mass_univariate t-test/ANOVA with FDR/FWER correction
viz.show Show a result in the UI: a 2D plot, or a 3D scene in the viewer
report.generate HTML/PDF report and re-runnable Python script

5.5 Safety and data governance

  • No data leaves the machine. The LLM runs locally, so there is no external AI service to trust. The agent still only receives metadata, QC summaries and statistics (not raw signals), to keep its context small and its reasoning focused.
  • De-identification on import: EDF patient fields, DICOM PHI tags, and file names.
  • Statistical guardrails built into the tools: warn about circular analysis, ML train/test leakage (e.g. ICA or scaling fitted on all data), missing multiple-comparison correction and underpowered designs.
  • Full audit log of agent actions and user approvals (needed for any future clinical path).

6. User interface

6.1 Main UI (React + TypeScript)

  • Data browser: datasets, subjects and sessions, import wizard, QC status.
  • EEG trace browser: scrolling multichannel view with annotations and bad-channel marking (WebGL-accelerated canvas, so long high-density recordings stay smooth).
  • 2D plots: ERPs, spectra, time–frequency maps, topomaps, stats results (Plotly or similar).
  • Pipeline view: the steps of a run, their parameters and status; re-run from any step.
  • Agent panel: chat, the agent's plan, approval prompts for decision tools, links to outputs.
  • Reports: generated HTML/PDF, and export of the re-runnable script.

6.2 3D viewer (Unity 6, WebGL)

Shows only what the engine sends. It does no analysis or file I/O.

Built from scratch in Unity 6 (URP). Planned views: - Cortical surfaces (fsaverage, individual FreeSurfer) with per-vertex data - Head model with EEG sensor layout; animated topographies over time - Volume slices and glass-brain view for fMRI maps - Picking: clicking a sensor, vertex or region tells the main UI, which can open the related plot

Communication - Main UI → viewer (JS bridge): load a scene, change threshold, colour map or time point, highlight, capture a screenshot. - Viewer → main UI (JS bridge): selection events, camera state, ready/error. - Viewer ↔ engine (HTTP): large binary payloads (meshes, per-vertex and per-timepoint arrays) are fetched directly from the engine's scene service, so they do not pass through JavaScript.

Viewer architecture - No analysis, file parsing or menus in Unity: it only receives scenes and commands. - Dependency injection with VContainer and a small message bus driven by the JS bridge. - Data arrives as compact binary arrays (vertices, faces, per-vertex/per-timepoint values) from the engine's scene service, so the viewer has no dependency on neuroimaging file formats. - Atlas region coordinates and connectome views (nodes and edges with threshold bands) are rebuilt on the same scene format.

7. Repository layout (proposed)

BrainO/
├── engine/                 # Python package `braino`
│   ├── braino/io/          # readers, de-identification, BIDS conversion
│   ├── braino/eeg/         # preprocessing, ERP, spectral, TFR
│   ├── braino/fmri/        # (phase 3)
│   ├── braino/stats/
│   ├── braino/ml/
│   ├── braino/provenance/
│   ├── braino/agent/       # orchestrator, tool registry, templates, plan checks
│   ├── braino/llm/         # local model runtime (llama.cpp), hardware tiers, client, RAG index
│   ├── braino/scene/       # mesh/array export for the 3D viewer
│   ├── braino/server/      # FastAPI REST + WebSocket on localhost, for the desktop app
│   ├── braino/cli.py       # command-line interface
│   └── tests/
├── training/               # dataset generation, LoRA fine-tuning scripts, agent evals
├── app/                    # React + TypeScript UI, Electron shell, installers
├── viewer/                 # Unity 6 project (WebGL build), added in M1.6
├── docs/
└── examples/               # public sample datasets + notebooks

8. Roadmap

Order of work for a single developer

Four technology stacks (Python, local LLM, React/Electron, Unity) is a lot for one person, so work is ordered so that each step produces something people can use and the riskiest science comes first:

  1. Engine as a Python library + CLI — useful to researchers from day one, through scripts and notebooks, and the base for everything else.
  2. Agent in the CLI/notebook — proves the local-AI idea before investing in UI.
  3. Main UI — makes it usable without code.
  4. 3D viewer — the new Unity viewer, once the data it shows exists.

Milestone 1 — EEG pipeline + agent

Durations assume one developer working full-time; part-time roughly doubles them.

Step Deliverable Done when Approx. time
M1.0 Foundation Repo layout, Python package, CI (ruff, mypy, pytest), contribution docs CI green; pip install -e engine works 2–4 weeks
M1.1 Import EEG readers → BIDS, de-identification, dataset.inspect, QC report, CLI All listed EEG formats load from public test files 1.5–2 months
M1.2 Pipeline Preprocess, bad channels, ICA, epoching, ERP, spectral, TFR, cluster stats, provenance, script export Reproduces a published ERP effect on a public dataset 2.5–3 months
Release v0.1 Engine on PyPI with docs and example notebooks People outside the project can use it —
M1.3 Local AI Local model runtime and hardware tiers, LLM client, templates, plan checks, RAG index, approvals in the CLI "Compare conditions A vs B" runs end to end from the CLI on a computer with no GPU and 16 GB RAM 2–3 months
M1.4 Evaluation Agent eval set (tasks + expected outputs), scored per tier; first LoRA fine-tune (on the HPC) only if evals show gaps Eval pass rate tracked on the minimum tier; base vs fine-tuned compared 1–1.5 months, then ongoing
Release v0.2 Engine + local agent — —
M1.5 Desktop app Local engine service, Electron/React app (data browser, trace browser, plots, agent panel with approvals), installer per OS The M1.3 workflow runs entirely from the installed app on the minimum computer 3–4 months
M1.6 3D viewer New Unity 6 WebGL viewer, embedding, JS bridge, scene service; sensor topographies over time Results appear in the embedded 3D panel; clicking a sensor opens its ERP; smooth on integrated graphics 2–3 months
Release v1.0 Desktop app for Windows, macOS and Linux — —

Total for milestone 1: about 13–18 months full-time.

Milestone 1 build order

The detailed order of work within milestone 1. Near-term stages are specified in more detail than later ones; later stages will be refined as they come closer.

Progress (2026-10-27): Stage A done (items 1–4), the Qwen spike done, Stage B (M1.1 Import) done except the XDF reader, Stage C (M1.2 Pipeline, v0.1) done, awaiting publication. Next: Stage D. Plan changed on 2026-10-27: BrainO is a desktop application on the user's own computer only, with no server or paid GPU (minimum: no GPU, 16 GB RAM); Stages D and E were adjusted.

Stage A — Engine plumbing (before any EEG code, ~2–3 weeks). Every later tool depends on these, and they are cheapest to settle while only one tool exists.

  1. Run context in the tool contract. Tools receive a RunContext (BIDS root, derivatives/braino/ output folder, run ID, provenance logger) in addition to their arguments.
  2. Provenance store. Each tool call records input file hashes, parameters, library versions, outputs and timing as JSON next to the derivatives. Script export and the agent's audit log are both built from this record.
  3. Propose/apply for decision tools. A decision tool returns a proposal (e.g. channels to drop); a separate apply step commits it. The CLI, the UI and the agent then all handle approval the same way.
  4. Test-data setup. Small public sample files for each format, downloaded with pooch and cached in CI; network-dependent tests are marked. Scientific dependencies (mne, mne-bids, numpy) go in an [eeg] extra so the core install stays light.

Optional spike (2–3 days): serve a small Qwen model with llama.cpp on the RTX 2080 Ti and have it call system.info through registry.schemas(). This tests local tool calling end to end months before M1.3, while the tool contract is still cheap to change. Result (2026-10-07, spikes/llm_tool_call.py): Qwen3.5-9B Q4_K_M on llama.cpp (CUDA, --jinja, temperature 0) passed 9/9 cases: tool choice, arguments, chained calls, recovering from tool errors, a decision tool whose proposal was then approved, and no invented numbers. About 80 tokens/s using 6.4 GB of VRAM. The tool contract needed no changes.

Stage B — M1.1 Import.

  1. Format detection and readers: EDF/BDF, BrainVision, FIF and EEGLAB first; then CNT, MFF, GDF, Curry, Nihon Kohden and Persyst; XDF last (custom code on top of pyxdf). Done for all MNE-backed formats (braino.io); XDF remains.
  2. Normalised recordings: channel types, montages, and events/annotations. Every format stores events differently, so this is expected to be the hardest part of import. Done (braino.io.normalise): standard name spelling, types from names, positions from the file or a template only when the names identify one, trigger channels to annotations, all reported. Known gap: trigger codes with status bits set (e.g. BioSemi) are kept as recorded.
  3. De-identification: EDF patient fields, measurement info and file names, with tests that check the personal data is gone. Done (braino.io.deidentify): header fields removed; dates shifted per subject into 1902–1924 (keeping age, time of day and intervals); identifiers, dates, e-mails and phone numbers removed from event text, other free text listed for review; subject labels and shifts kept only in a subject key file that must be outside the dataset; provenance records files outside the dataset by hash, not path. For item 8: write converted files (never copy originals, whose headers still hold the data), and keep source paths out of recorded tool arguments.
  4. BIDS conversion with mne-bids. Subject, session and task labels come from CLI options or a mapping file; the result is checked with bids-validator. Done (braino.io.bids, braino.io.import_recording): the whole import pipeline; data always written fresh (BrainVision for EEG/iEEG, FIF for MEG), never copied; line frequency from the file or estimated from the data; template positions labelled as such; refuses data that are not de-identified. All MNE-backed samples pass the official validator (v3.0.2, run in CI), with only "recommended metadata" warnings. The CLI options and mapping file move to item 9, with braino import.
  5. dataset.inspect and eeg.qc_report tools, with CLI commands braino import, braino inspect and braino qc. Done. Import is a command, not an agent tool (it reads identifiable originals); its provenance keeps only the subject label, task/session/run, source hashes and outputs, with error messages scrubbed. A CSV mapping file imports many recordings. QC screens flat and noisy channels (robust z on 1–40 Hz data), line noise and artifact time, writes a per-channel table to the derivatives and returns a compact summary (at most 10 channels per list). QC checks EEG and iEEG channels; MEG checks come with MEG work.

Stage C — M1.2 Pipeline → v0.1.

  1. Choose the validation dataset first (ERP CORE is a candidate), so the pipeline is built against the published effect it must reproduce. Done: ERP CORE (Kappenman et al., 2021, NeuroImage 225:117465; CC-BY-4.0; 40 adults, BioSemi, 30 scalp channels). Targets, from the authors' per-participant scores:

    Component Effect Site, window Published
    P3 (first) rare − frequent Pz, 300–600 ms +6.29 µV (SD 3.39, dz 1.86, n = 34)
    N170 (second) faces − cars PO8, 110–150 ms −3.37 µV (SD 2.71, dz 1.24, n = 37)

    Reproduced means: (1) BrainO's 95% CI includes the published mean and the effect is significant (p < .001) in the published direction; (2) per-participant scores correlate with the published ones at r ≥ 0.9 (provisional, to confirm after the first full run). Exclusions are compared but not pass/fail. braino.datasets.erpcore fetches the BIDS data (P3 2.2 GB, N170 3.4 GB) with pinned SHA-256 hashes; braino.validation.erpcore.compare applies the criteria. The full run is an on-demand validation, not part of CI. 11. Tools, in dependency order: eeg.preprocess → eeg.detect_bad_channels (decision) → eeg.ica_clean (decision) → eeg.epoch with autoreject → eeg.erp, eeg.spectral, eeg.tfr → stats.mass_univariate, stats.cluster_permutation. Done (braino.eeg, braino.stats, tools in braino.tools.eeg, .eeg_measures, .stats). Each step writes new derivative files, never changing one in place; decisions made on an older version of an earlier step's output are refused. Changes from the order above and from library defaults, each found on ERP CORE data: - ICA before bad-channel detection works best. With bad channels first, PyPREP marks the channels near the eyes bad for eye activity, ICA then cannot separate the blinks, and the remaining blink activity cancelled most of the P3 (participant 007: 0.2 µV instead of 6.8). Either order is allowed; detection reads the ICA-cleaned recording if there is one. - Detection runs on an average-referenced copy, with flat channels found first and blinks left out (with mastoid references, PyPREP marked the mastoids bad). Reference channels cannot be marked bad. PyPREP's correlation criterion uses 5% of windows, not PREP's 1%, which marked 13 of 30 channels bad on a clean recording. - ICA is extended infomax solved with Picard: same solution, 230 s → 6 s on 2 min of 20-channel data. - TFR defaults start at 6 Hz, since 3-cycle wavelets at 4 Hz need 1.2 s epochs.

    First check against ERP CORE (P3, participants 001–010, default settings except the authors' 0.1–30 Hz filter, 256 Hz and P9/P10 reference): for the 7 participants the authors kept, r = 0.98 with the published scores, mean absolute difference 0.7 µV; participant 009, whom they excluded, also loses 77% of epochs here. Correct-response selection and the authors' own artifact rejection are not reproduced yet.

    Full runs (2026-10-25, braino validate erpcore, all 40 participants): both reproduced. P3: BrainO +7.04 µV [95% CI +5.36, +8.71], n = 34, published +6.29 µV; participant scores r = 0.949 over 30 (mean absolute difference 0.77 µV). N170: BrainO −3.68 µV [−4.61, −2.75], n = 40, published −3.37 µV; r = 0.967 over 37 (0.63 µV). The N170 first failed (−1.73 µV, r = 0.64): the authors' scripts move every event 26 ms later for their measured monitor delay, which matters for a 110–150 ms window. eeg.epoch now takes event_delay_s, and eeg.preprocess the authors' N170 reference (average_with_eog). The provisional r ≥ 0.9 criterion holds. 12. Pipeline spec and runner: steps declared in YAML/JSON, results cached, re-run from any step. These specs later become the agent's validated templates. Done (braino.pipelines, braino pipeline). Specs are checked against each tool's input schema when loaded. A step is reused when its tool, arguments and BrainO version are the same and every file it read and wrote still has the recorded hash, so re-running a step re-runs what depends on it without tracking dependencies by hand. Decision steps stop the subject until a person decides (a decision made outside the runner counts as new, so later steps re-run); --auto-approve is a command-line flag for the person running it, never part of a spec. Built-in specs erpcore-p3 and erpcore-n170; braino validate erpcore runs them and applies ERP CORE's exclusion rule (> 25% of trials rejected). 13. Script export from the provenance record. Done (braino.provenance.export, braino runs export, braino pipeline export). The script re-runs each recorded call, replays each decision with the reviewer's edits, checks input hashes first, and compares every number each step reported with the recorded result (relative tolerance 1e-4); files are not compared byte for byte because FIF files store the time of writing. Tested by replaying exports on fresh copies of a dataset. 14. Docs site, example notebooks and a PyPI release workflow → v0.1. Done, except publishing: MkDocs site (mkdocs.yml; tool and command references generated from the code; deployed to GitHub Pages by .github/workflows/docs.yml), two executed notebooks in examples/, CHANGELOG.md, and .github/workflows/release.yml (on a v* tag: checks, build, wheel smoke test, PyPI trusted publishing, GitHub release). Version is 0.1.0. To publish: add the trusted publisher on PyPI, enable Pages, push the tag.

Stage D — M1.3–M1.4 Local agent → v0.2.

  1. Eval harness first (tasks with expected outputs), so every agent change is scored, per hardware tier. Done (braino.evals, braino eval list|run|compare; agents work through braino.agent.Session: tool calls, questions, decision requests). 14 tasks in YAML (13 on synthetic data with planted answers, 1 on ERP CORE): inspection, QC, preprocessing settings, decisions (ask, rejection, edited approval, ICA), ERP, group statistics and correction, spectra, refusing to guess. Checks score tool use and arguments, step order, decisions requested, answer values and terms, and grounding: every number in the answer must come from a tool result. A scripted user answers and decides. Reports record agent, model, tier and hardware; compare lists fixed and broken tasks. The oracle agent (reference solutions) passes 14/14 and the null agent 0/14, and the oracle run is part of the test suite, so a tool change that breaks a task fails CI. Building the tasks found that global autoreject can reject a whole condition on short recordings; eeg.epoch now warns.
  2. Local model runtime: BrainO starts and stops llama.cpp itself, detects RAM and GPU memory, proposes a model tier, and downloads the chosen model once (checked against a pinned hash). A short benchmark tells the user how fast the agent will be on their computer. Done (braino.runtime, braino model hardware|list|setup|status|bench|serve|remove). Run for real on the office desktop (i7-8700, CPU build, 4B model): setup, hashes, server start (model loaded in 4 s) and benchmark all work; it reads 65 tokens/s and writes 9 tokens/s, so a first agent step takes about 45 s and later ones about 20 s ("usable"). The GPU builds have not been run yet. Pinned in catalog.json: llama.cpp build b11606 (2026-10-13; CPU, Vulkan, CUDA 12 and Metal builds for Windows, Linux and macOS, x64 and arm64) and three Apache-2.0 models: Qwen3.5 4B (minimum tier, 2.6 GB), Qwen3.5 9B (GPU tier, 5.3 GB) and Qwen3.6 35B-A3B (more-RAM and GPU tiers, 19 GB). Models are pinned by repository revision and SHA-256. Hardware detection needs no administrator rights (nvidia-smi, the Windows registry, Linux sysfs, Apple silicon). Downloads resume and retry. The server runs on 127.0.0.1 on a free port with a per-session API key, one slot and prompt-cache reuse. llama.cpp's own --fit places layers on the GPU, and a mixture-of-experts model's experts in RAM when the GPU is too small. The benchmark reads a prompt of BrainO's tool schemas and estimates first and later agent-step times. Tested with a stand-in server (start, health wait, API key, failure logs, timeout, benchmark, full setup); BRAINO_LLAMA_TESTS=1 runs the real build and model.
  3. LLM client for the OpenAI-compatible API, with schema-constrained tool calls, sending only the tool schemas each step needs (prompt processing is the bottleneck on a CPU). Done (braino.llm). Every step is one JSON action (call, load_tools, decide, ask, answer) constrained by response_format with a schema built from the loaded tools' own parameters, so llama.cpp's grammar makes invalid calls impossible. Native tool calls are not used: with Qwen3.5's tool-call format llama.cpp does not enforce the parameter schemas (it accepted filter_method: "chebyshev"). Compact schemas (no titles, optional fields as type: [X, null], $refs inlined) are 16% smaller and accept exactly the same values (checked with a JSON-schema validator). All 12 schemas are 6,500 tokens (100 s of reading on the CPU), so the system prompt has a one-line catalogue of every tool, the request's words preload up to 4 schemas, and the model loads others (load_tools, or simply calling the tool). A tool named in a result ("run eeg.preprocess first") is loaded automatically. The system prompt is identical for every request and the conversation only grows at the end (results as Qwen <tool_response> messages), so the prompt cache holds: later steps read 50–400 new tokens. Reasoning is off (it cost ~20 s per step). Works with any server that supports JSON-schema replies (llama_cpp=False). A throwaway loop over the eval suite with the real 4B model on the CPU found three problems the unit tests did not ($refs breaking when nested, prerequisites never loaded, repeated questions) and one habit (calling a tool before loading it), all handled: 4/13 → 8/13 tasks passed, mean score 0.52 → 0.86. What remains is for item 18: repeated identical calls and decision requests, a step budget for many-subject tasks (the group test took 30+ steps), reporting decisions and answers faithfully, and printing Unicode safely on a Windows console.
  4. Agent loop in the CLI, with approval prompts for decision tools.
  5. Templates and plan checks, built on the Stage C pipeline specs.
  6. RAG index over the MNE, BIDS and BrainO documentation, small enough to search on a laptop (a local embedding model on the CPU).
  7. Baseline evaluation on the minimum tier (CPU, ~4B model), then LoRA fine-tuning on the HPC only where the evals show gaps; fine-tuned models are published as GGUF files for users to download.

Stage E — M1.5 Desktop app.

  1. Local engine service: FastAPI on localhost exposing the tool registry (already schema-driven, so this layer stays thin), started and stopped by the app.
  2. Job runner that fits the user's computer: as many jobs at once as cores and memory allow, progress reported over WebSocket, cancellable, results cached.
  3. Electron/React app, in this order: data browser → plots → agent panel with approvals → trace browser (the hardest view, so it comes last).
  4. Installers for Windows, macOS and Linux, each bundling a Python runtime with the engine and llama.cpp; the model is downloaded on first use. Tested on a computer with no GPU and 16 GB RAM.

Stage F — M1.6 Unity viewer → v1.0.

  1. Scene service in the engine for binary meshes and arrays.
  2. Unity viewer: WebGL build, JS bridge, sensor topographies over time; meshes sized to run smoothly on integrated graphics.
  3. Picking: clicking a sensor in the 3D view opens its ERP plot in the main UI.

Later phases

  • Phase 2 — EEG depth: source localisation on cortical surfaces, connectivity, ML decoding, BrainO connectome view driven by computed connectivity.
  • Phase 3 — fMRI: DICOM/NIfTI import, fMRIPrep integration, GLM, resting-state connectivity, surface and volume rendering in the viewer. fMRIPrep runs on the user's computer (in a container, several hours per subject on a CPU), so jobs must survive closing the app and resume.
  • Phase 4 — Collaboration: group studies and shared projects without a BrainO server: projects, results and exported scripts are files people share by their own means (a shared drive, Git, OSF). A user who has their own cluster can run the exported scripts there.
  • Phase 5 — Multimodal and extensibility: EEG-fMRI fusion, plugin SDK for third-party tools.

9. Resources (open source, single developer)

Item Cost
All software: MNE, nilearn, scikit-learn, PyTorch, FastAPI, React, Electron, llama.cpp, Qwen weights Free (open source / open weights)
Servers and cloud None. BrainO runs on users' computers; the website and model downloads are hosted free (GitHub Pages and Releases, Hugging Face)
Unity 6 Free (Unity Personal; no revenue)
GitHub, CI for public repos, PyPI, Read the Docs Free
Development GPU Already available: RTX 2080 Ti (office)
Fine-tuning and large evaluations Already available: University of Memphis HPC (V100), free
Code-signing certificates for installers (macOS $99/yr; Windows varies) Optional; unsigned builds work but show security warnings
Test datasets Free (OpenNeuro, MNE sample data)

The main resource is developer time (section 8). Grants for open-source research software (e.g. NIH BRAIN Initiative informatics, NSF CSSI, Chan Zuckerberg Initiative EOSS) can fund it, and a paper in the Journal of Open Source Software (JOSS) helps adoption and citation.

10. Risks

Risk Mitigation
Scope is very large for one developer Library-first order with usable releases at each step; build depth before breadth; invite contributors after v0.1
Local model plans or reasons poorly Constrained tool calls, validated templates, rule-based plan checks, approvals, eval suite; fine-tune where evals show gaps
Most users have no GPU The agent is designed and evaluated for a ~4B model on the CPU (16 GB RAM); templates, constrained tool calls and plan checks carry reliability, not model size. A GPU or more RAM only adds speed or a larger model. The engine and UI work fully without the agent
Heavy analyses are slow on laptops Results cached so nothing runs twice; progress and cancel for every job; jobs sized to the machine's cores and memory; long jobs resume after the app is closed
Installer size (Python, scientific stack, model runtime) Models downloaded on first use, not bundled; one installer per OS
Agent produces wrong science Executed tools only, guardrails, approvals, eval suite, re-runnable scripts
WebGL memory and performance limits for large surfaces/volumes Engine-side decimation and level-of-detail meshes, streaming time series in chunks, 2D slice rendering in the main UI as a fallback
Unity WebGL build size and load time Keep the viewer small, compressed builds, load the viewer once and keep it alive
Licensing See open question 1. Model weights are downloaded by the user, not bundled; check each Qwen model's licence (most recent Qwen releases are Apache 2.0, some earlier sizes used a custom Qwen licence). FSL/FreeSurfer stay optional external tools, never bundled.
PHI and clinical regulation De-identify on import, everything local, audit log; clinical use needs a separate regulatory track (e.g. FDA SaMD / EU MDR)

11. Open questions

  1. Licence: keep GPLv3 or move to a permissive licence (MIT/BSD-3, common in the MNE/nilearn ecosystem)? All code is new, so this is your decision alone. Note that GPL code depending on the proprietary Unity engine is a known grey area; a permissive licence (at least for viewer/) avoids it.
  2. Real-time EEG streaming (LSL) for neurofeedback/BCI: in scope, and when?
  3. Which in-house datasets should drive milestone 1 validation, in addition to public ones?

Decided (2026-10-27): packaging is one installer per operating system bundling the Python runtime, the engine and llama.cpp, with the model downloaded on first use; the engine also stays available on PyPI for scripting.