How to Build an AI Second Brain (Free, Private, Self-Hosted)

How to Build an AI Second Brain (Free, Private, Self-Hosted)

An AI second brain is a folder of markdown files in a git repo that your AI tools read over MCP. Not a vector database you rent. You build the free local half in about 10 minutes with Claude Code, and if you want it reachable from claude.ai, your phone, or any AI agent, you self-host the open-source app.

A private git repo that holds who you are, how you write, and what you know. Claude Code that feeds it and answers from it locally. Optionally: a self-hosted app that puts the same repo behind MCP and REST with visibility tiers, an approval queue for every write, and a 6-second sync loop. Plus the side-by-side proof of what changes: the same prompt answered with and without the brain.

An AI second brain is a folder of markdown files in a git repo that your AI tools read over MCP or API, or directly from local files if you go the local version. That is the whole definition. Your identity in three files. One card per project. Dated notes for what you know: takes, stories, lessons, facts. A raw archive for provenance.

Most pages that rank for this term mean something else: upload your documents to a SaaS, let it embed them into a vector database, chat with the blob. That version has three problems. You cannot read what it stored. You cannot leave with it. And when the AI answers wrong, you cannot see why.

Files fix all three. You can open any note. You can git log every change ever made. And when an answer cites fact-2026-08-plausible-upgrade-16-seconds , you know exactly where the number came from, because you approved it going in. (That note is real, by the way: the 16 seconds is my published Plausible analytics upgrade measurement , and it shows up again later on this page.)

The system in this guide is BrainOutside : a free MIT template repo for the brain itself, and a free MIT app you self-host when you want the online half. I built it, I run my own brain on it , and every screenshot below is from a fresh install I timed for this page.

Ask any AI to write as you, and it writes as everyone. Generic replies, invented numbers, positions you never held. Not because the model is weak. Because it knows nothing about you, and it fills the gap with plausible.

Later in this page there is a side-by-side test . The short version: without the brain, the model wrote a confident reply that stated the opposite of my actual position. With the brain connected, the same prompt came back with my measured numbers and my actual take. The difference is not intelligence. It is access.

There is a second effect I did not expect to value this much. When the brain cannot ground an answer, it says so. The assemble-context tool returns a gaps field, and in my test it literally answered: "cannot answer the task's specific question. Answering would require inventing a position not backed by the mind." That sentence is a leash on hallucination, and it is worth more than most retrieval tricks.

Open the template repo and click Use this template , then Create a new repository . Name it whatever you like. Make it private . Your voice file and your unpublished takes will live here.

The template repo: github.com/hassancs91/brainoutside-template

What you get is a brain that is empty on purpose: a CLAUDE.md contract that tells any Claude Code session how the brain works, folder skeletons for identity, projects, knowledge and lenses, note templates with frontmatter, and two skills. mind-feeder is the only thing that writes. mind-reader is how anything reads. That contract is why this works with zero apps installed: the repo itself is the interface.

Clone your new repo, open the folder in Claude Code, and write three files by hand. This is the one manual part, and these twenty minutes pay for everything that follows.

Nearly every retrieval loads these three files first. A weak identity layer makes everything downstream sound like a polite assistant. Write the version of you that argues back.

Now feed it something real. In Claude Code, in your brain folder, type a plain sentence:

The feeder does not save your text as a blob. It proposes structure, and it does not write until you approve. Here is what my first feed actually produced:

Three things in that output are the whole philosophy. It split one thought into a story, a take, and a citable fact. It preserved my exact words as a VERBATIM quote. And it refused to invent the one number I did not give it. Feeds took 82 to 123 seconds each in my runs, 106 on average.

Reading is the same, in reverse:

The local brain is complete and useful with nothing installed beyond Claude Code. Everything from Step 4 on adds one thing: the same repo, reachable from claude.ai, your phone, and any MCP client, with tiers and an approval queue. If that is not your problem yet, bookmark this and come back.

The online half is one Docker Compose stack on one small VPS. I ran the whole build on a 2 vCPU, 4GB machine at $0.036 an hour, and I timed every segment on a stopwatch. 4GB is genuinely enough: across the entire day, including builds and AI indexing, host memory peaked at 2,067MB.

Any provider with Ubuntu images works. The budget pick I keep coming back to is Contabo : their entry plan costs about a third of the $14 a month I paid for this run's box, and it carries 8GB of RAM, nearly four times what the whole build peaked at.

Those are Contabo's EUR list prices, verified April 2026, and the advertised rate is also the renewal rate, which is rare at this end of the market. If you would rather shop around, the VPS Providers library is my audited comparison: real prices, refund policies, and the gotchas the marketing pages skip.

Create the VPS with Ubuntu 24.04, SSH in as root, and run Coolify's installer:

On my fresh box that took 2 minutes 51 seconds, and it installs Docker itself as part of the run. The dashboard was answering on port 8000 the moment the script finished. That one command is deliberately the whole install story on this page; DNS records, the firewall, and hardening the box without locking Coolify out get the full treatment in my Coolify install guide , the deep version of this step.

A fresh Coolify serves its registration page to whoever opens the URL first, and the first account owns the server. Register before you do anything else. While you are at it: Coolify's password rules want a symbol, which my first generated password did not have.

And this step scales past the brain. This exact VPS-plus-Coolify workflow is how I stopped paying about $700 a month in managed cloud bills: once the box exists, every extra app is just another Compose stack on hardware you already pay for. The full system, from choosing a VPS to backups, updates, and not breaking a running box, is what my Self Hosting 2.0 course covers.

In Coolify: your project, then New Resource , then Public Repository . Paste the app's repo URL and pick build pack Docker Compose :

Two things will try to trip you here. Both took me real minutes, so they get their own warnings.

Coolify defaults its Docker Compose Location to /docker-compose.yaml . The repo ships docker-compose.yml . You get a red "Docker Compose file not found" the moment you continue. Fix: set the location to /docker-compose.yml , save, and click Load Compose File.

Once the compose parses, Coolify shows the five services and offers domains. Set your domain on the web service only (for example https://brain.yourdomain.com ; the DNS A record should already point at the box). Then open Environment Variables and set exactly two values:

The container's healthcheck probes http://127.0.0.1:8000/healthz . If ALLOWED_HOSTS holds only your domain, Django answers that probe with 400, the container reports unhealthy forever, and Coolify's proxy never routes your domain to it. From outside it looks like the deploy failed. It did not: it is one env var. Domain, then localhost , then 127.0.0.1 , comma-separated, no spaces.

Click Deploy . Coolify clones the public repo and builds the image on your box: 2 minutes 45 seconds to running containers in my run, about 4 minutes to healthy with the TLS certificate issued. When the padlock shows on your domain, the hard part is over.

Visit your domain. The server has zero users, so every route redirects to /setup . No login box, no docker exec . The wizard's own copy tells you the important thing: this page is open to anyone who can reach the server until the account exists, so create it first.

Add up the stopwatch: 56 seconds for the VPS, 2:51 for Coolify, 4:12 for the deploy, 4:45 for the wizard. 12 minutes 44 seconds from nothing to a brain online , including my failed PAT attempt.

Zero to "your brain is online": 12 minutes 44 seconds

Measured on a fresh $0.036/hr 4GB VPS, 2026-08-05. Bars share one scale (seconds).

On the ops page, open API keys and mint one. Keys are tier-pinned: public , agents-only , or private . For your own assistant, agents-only is the normal choice: it can read your voice file and unpublished notes, but not the private core. The secret is shown once and stored as a hash.

Then drop this into any project as .mcp.json :

That is the whole client setup. Claude Code now sees nine tools: ping , get-index , list-notes , get-note , get-lens , get-identity , get-raw , assemble-context , and propose-feed . In my timing runs, get-index answered in 380 to 424 ms over the public internet.

The tool that earns the setup is assemble-context . Hand it a task and an agent on the server reads the tier-filtered index, opens the right notes, and returns a context pack, the list of entity ids it used, and the gaps. It is not instant: 17 to 40 seconds in my three runs, because it is a real agent reading real files. Here is the half-covered case, verbatim:

claude.ai's custom-connector dialog has nowhere to put an API key, so the credential rides in the URL path instead. The server treats that as a different, more dangerous kind of secret, and ships the whole surface off by default: without the flag, /mcp/k/<token>/ answers 404, not 401, so the address does not even admit it exists.

Turn it on deliberately. In Coolify, add one variable and redeploy (an env change means a rebuild; mine took about 6 minutes):

Then open Connectors on the ops page and mint a connector URL. It is tier-pinned like any key, rate-limited, and it expires on its own; a URL you paste and forget should age out. Paste it into claude.ai under Settings, Connectors, Add custom connector. Now your brain answers on your phone.

Feeding works from there too. I sent a propose-feed through the connector URL and it landed in the queue in 1.1 seconds, where it waited like every other write. Which brings us to the part that makes remote feeding safe at all.

The server never watches your repo. It pulls when GitHub tells it to. Set a webhook secret in the ops Settings page, then add a webhook on your brain repo: payload URL https://brain.yourdomain.com/webhooks/github , content type JSON, the same secret, push events only.

I measured the loop three times: commit locally, git push , poll the server. 6 seconds, all three runs , from push to the new note being served. Before I added the webhook, I pushed a commit and watched the server stay on the old HEAD: without the webhook you wait for the periodic pull, up to 15 minutes. The dashboard tells you this state honestly.

Two mechanisms carry the whole trust story, and both are enforced by the server, not by a prompt.

Tiers are structural. The server materializes one snapshot of the repo per tier, and a key physically reads its own snapshot. My demo brain held 17 entities: 13 public, 3 agents-only, 1 private. The agents-only key saw 16. The public key saw 13, and its identity payload had no voice file at all. And the private note? Both keys got "unknown entity", the same answer a nonexistent note gets. I verified all of it twice, over MCP and over REST: same counts, both doors.

Writes wait for you. Every write, from every door, lands in the same queue: pending, extracted into a proposed diff, and going nowhere until you click Approve. The proposal is rendered as a diff on purpose, because fed content is untrusted input. This is the "agents propose, you approve" rule with a UI around it.

The honest caveat, straight from the product's own landing page: on the self-hosted server, private notes are only as private as your VPS. Put the ops UI behind an IP allowlist or Tailscale, keep the repo private, and back up the state volume. The dashboard's health panel nags you about exactly this, and it is right to.

If you searched "memory MCP server", you found the other family: the official knowledge-graph memory server, Mem0, and a directory of 775 more. They solve session amnesia: an agent stores entities and observations as it works, so the next session remembers your project. For coding context, they are great, and I use that pattern too.

A second brain is a different animal on five axes. Files you can read instead of graph triples. Your identity and voice, not just facts about code. Human-approved writes instead of silent accumulation. Tier-filtered access instead of one pool. And git history instead of a database you hope is fine. The trade is real: a memory server writes itself, and a brain makes you the editor. For the thing that speaks as you in public, I want the editor's chair.

Two tests. Same Claude CLI, same model, same prompt, single run each, outputs unedited. The only difference: one directory had the .mcp.json from Step 7 and an instruction to ground through assemble-context .

Test 1: an X reply. The post being answered: "Self-hosting sounds nice until you become the unpaid sysadmin of your own life. I'll pay the $20/mo, thanks."

Read the left one again. It is fluent, reasonable, and it states a position I do not hold: that I pay for anything with auth attached. I moved all of it, auth included, onto one VPS. That is the quiet failure mode of ungrounded AI: not nonsense, but a confident stranger wearing your name. The right one cites 16 seconds, twice a year, $700 a month, and closes on my actual take. Every claim traces to a note I approved.

Test 2: a book chapter. The prompt asks for chapter 9's opening paragraph, calling back to "chapter 2's exact numbers" on self-hosting economics.

Without the brain, the model did the honest thing available to it: it refused to invent and handed back a form with five blanks. With the brain, chapter 9 opens with the same $8,400, the same 8GB comparison, the same 16 seconds that "chapter 2" holds, because both chapters draw from the same notes. Consistency is not a writing skill. It is a storage decision.

The cost of grounding: 49 and 70 seconds instead of a few. You are trading seconds for output you do not have to fact-check against your own life.

Everything below works today with the same two doors. I have not run controlled side-by-sides on these the way I did above, so they are listed as uses, not proofs:

A folder of markdown files in a git repo that your AI tools read over MCP: your identity, your voice, your projects, and dated notes of what you know. Not a vector database you rent, and not a SaaS you upload your life to. You can read every file, edit any of them, and leave with the folder at any time.

Claude's memory lives inside one product, on Anthropic's side, and you mostly cannot read it as files. A second brain is the inverse: files you own, readable by any client over MCP or REST, with git history, human-approved writes, and visibility tiers. Use both; they do different jobs.

No. The brain is small, structured, and indexed by frontmatter. Retrieval is an agent reading a tier-filtered index and opening the notes it needs, and assemble-context tells you which entities it used and what was missing. At personal scale, honest structure beats similarity search.

The repo is private, every key is tier-pinned, and a note above a key's tier answers "unknown entity", the same as one that never existed. I verified that over both MCP and REST. The honest part: on the self-hosted server, your private notes are only as private as your VPS, so lock the ops UI down and treat backups as part of the deal.

The local half costs nothing beyond your Claude plan. The online half ran on a $0.036/hr 4GB VPS with 2.2GB of RAM to spare, plus Claude usage on the server: my whole lab day cost $1.03 in API spend.

Yes. Both repos are free and MIT licensed: the brain template and the app. There is no hosted version and no account with anyone. You deploy from the public repo, and your brain never leaves machines you control.

Your brain outlives every model. Put it outside.

Hasan Aboul Hasan builds open-source tools and teaches solo developers how to build, host, and sell AI-powered products. Founder of LearnWithHasan.com , creator of SimplerLLM and PyRunner .

The exact building blocks I use to ship real products with AI — yours as a free PDF.

Have a question? Ask it in the community — it's tagged #guide and linked back here. Reading is open to everyone; posting needs a free account.

Recommended articles