Hardware and network: Tailscale + Cloudflare Tunnel

TL;DR — the 30-second summarytldr

The brain of the system is a Mac Mini M4 Pro with 48GB of memory (14-core CPU, 20-core GPU, unified memory) — the sweet spot: it runs 27B-class models comfortably with embeddings loaded alongside at all times, never touching swap

The iron rule of 48GB: only one big model resident at a time — not two, no matter what

Before installing anything at all, prepare the system's "blood vessels" first — the private network Tailscale (mesh VPN) + the door to the internet Cloudflare Tunnel The idea is simple: every machine talks through a secret tunnel, and opens to the outside world through a single door with a guard checking keys — no ports opened on the router at all

When to usewhen to use

What do you actually need?

Do this step as soon as you have 2 machines or more — and it must happen before the VPS install in chapter 3, because compose will call the Mac over its Tailscale IP

• Tailscale use case: machines in the house talking to each other (VPS -> Mac, NAS -> i9), both on the LAN and from outside — you get a safe IP that looks like 100.x.y.z
• Cloudflare Tunnel use case: exposing services for outsiders to call over HTTPS (the web chat, the bot API) without anyone seeing our home IP
• These two work together; they do not replace each other

How-tohow to

The real memory map on our machine (a 27B model in nvfp4 + embeddings):

48 GB total
- 23 GB   main 27B model nvfp4 (context 32768) — resident is bigger than the 18GB file
          because KV cache + runtime buffers live in the same unified memory
- ~1 GB   bge-m3 embeddings (+ nomic), small, kept warm
- ~8 GB   macOS + Ollama runtime + headroom
= ~16 GB left -> memory pressure green, swap = 0

If you respect the "one model at a time" rule, this table stays calm forever

The swap incident that taught us the hard way — once we kept a second (16GB) model resident with a 24-hour keep-alive. The result:

• memory jumped to 47.7GB + 3.8GB of swap
• TTFT (time to first character) ballooned from ~0.04 seconds to 28–60 seconds
• after evicting the second model (keep_alive: 0), at the 27-minute mark swapouts stopped dead

Since that day the whole kit has guardrails wired in: a 24-hour KEEP_ALIVE for one model only, health checks forbidden from sweeping the model back in, and a cron that warms only the main model

if you get stuck anywhere → chapter 9

Step 1 — sign up and install Tailscale on every machine (free for personal use):

1. Create an account at tailscale.com (Google/GitHub login works)
2. Install on the Mac: brew install --cask tailscale then log in
3. Install on the VPS (Debian/Ubuntu): the curl command from tailscale.com/download, then tailscale up
4. Install on Windows/others as usual, then approve every machine in the admin console
5. Check that everyone sees each other: tailscale status

Here is the real output from a machine in our fleet (some IP digits hidden for safety):

Your machines will get their own name+IP — write down the Mac's IP for chapter 3

Step 2 — set up Cloudflare Tunnel for 2 domain names (chat and llm):

1. Your domain's DNS must be hosted on Cloudflare first (the free plan is enough)
2. Install cloudflared on the VPS: follow the guide at developers.cloudflare.com/cloudflare-one/connections/connect-networks/
3. Create the tunnel and point 2 routes: chat.yourdomain.com → localhost:3000 and llm.yourdomain.com → localhost:4000
4. Enable the Cloudflare Access second gate (email OTP login) for admin pages — see the real screenshot below

if you get stuck anywhere → chapter 9

Config exampleconfig example

Speed actually measured on this machine (the benchmark commands live in benchmarks/ in the repo):

Why MLX is faster: the M4 Pro has ~273 GB/s of memory bandwidth — LLM inference is constantly reading model weights, so the faster the reads, the faster the answers (a regular M4 at 120 GB/s gets roughly half these numbers). See MLX in the glossary

The configuration principle to remember through the whole manual:

# BOMLLM's iron rules for the network
1. every service binds 127.0.0.1 only (except the Tailscale interface)
2. the public has exactly one way in = Cloudflare Tunnel
3. the Mac exposes Ollama to the Tailscale IP only (pf firewall open)
4. SSH uses a non-standard port + key only + ufw DROP by default

Source: docs/security.md — re-checked on every audit round

Pro tip — lessons from the fieldpro tip

FAQfaq

Q: Can I use a Windows PC + GPU instead of a Mac? — it can run LLMs, but a single 16GB card cannot hold a 27B model; you must shrink the model and trade away quality — we mostly use the PC as the "image/audio kitchen" instead (chapter 6)

Q: If the Mac goes down, do the bots die? — no. Tier 2 on the VPS takes over immediately (chapter 1)

For other questions see the FAQ

Q: Is Tailscale really free, and does data leak? — the personal plan is free for 100 devices, the connector is open source, and traffic is WireGuard-encrypted end to end — Tailscale's servers only handle the "matchmaking"; they never read your data

Q: No domain of your own? — a .com domain costs ~300–400 baht a year and Cloudflare's free plan is enough for the tunnel — the best-value spend in the whole system

See alsosee also

• Chapter 3 · Mac Mini and VPS — install Ollama, the model, and the storefront tier
• Chapter 6 · The Thai recipe — tuning parameters on this machine
• Chapter 9 · Security — the bind table for every service
• docs/hardware.md — the full English version

Figuresfigures

Real screen recording for this lesson · click play to hear the summary · not investment advice

▶ Watch the manual video: https://youtu.be/O4hM39cgDCY