# LOCAL-ENGINE-SETUP.md — run SDXL on YOUR machine

> Filename kept as `LOCAL-FLUX-SETUP.md` (historical) — the engine is **SDXL**,
> not FLUX. Everything below calls it what it is.

## 1. What this is, and why

PsykoMedia's AI art engine is **SDXL diffusion running on your own hardware**
via your sovereign API (`sdxl_api_scalable.py`). Your prompts go to a server
on your machine, the image renders on your GPU, and the finished PNG travels
back through your SSH tunnel to the site.

- Nothing touches a cloud API. There are no provider accounts and no third-party keys.
- Nothing leaves your hardware except the prompt you typed and the image it made.
- If your machine is off, the tunnel drops, or the model is still loading,
  jobs **queue** on the server and retry automatically — nothing fails
  silently, nothing gets faked.

Four engines. **Your sovereign SDXL API is the priority default** — then
ComfyUI, then the bundled standalone server, then the procedural engine
(which always works and needs nothing).

## 2. Engine 1 — Your sovereign SDXL API (priority default)

This is already written: `sdxl_api_scalable.py`, FastAPI on **port 8003**.
The backend's adapter is pre-filled from its contract — nothing to configure
unless your port changes.

### The contract (what the backend expects)

| Endpoint | What it does |
|---|---|
| `GET /health` | `200` = ready (body: `jobs_pending`, `jobs_processing` — shown in Settings). `503` = model still loading → the backend shows **"engine warming up"** and refuses to submit jobs until it's ready. |
| `POST /generate` | `{prompt (≤1000 chars), negative_prompt, num_inference_steps 1–100 (default 30), guidance_scale 1.0–20.0 (default 7.5), height/width 512–2048 (default 1024), seed (optional), return_base64: false}` → `{job_id, status: "pending"}`. Prompts are truncated to 1000 chars before sending. |
| `GET /generate/stream/{job_id}` | SSE. Parse `data: {...}` lines; the terminal event is `{"type":"update","data":"completed"\|"failed","image_base64":"...","seed_used":N}` — **the image always rides inline**, no second fetch needed. |
| `GET /status/{job_id}` + `GET /result/{job_id}` | Fallback only: if the SSE stream stalls, the backend polls status every 2s and fetches the PNG. |

### Rate limit — how we stay friends with it

Your server runs a token bucket: **8 burst, 1/sec refill per client IP** →
`429` with a `Retry-After` header when empty. The backend paces itself:

- The worker serializes submissions, guaranteeing **≥ 1.0s between
  `POST /generate` calls**.
- On a 429 it sleeps `Retry-After` and retries the POST once; if still
  limited, the job goes back to the queue with backoff.
- The Create page shows **"Queue position #N · paced at ~1/sec"** on queued jobs.

Don't hammer it from other clients on the same IP while jobs are queued.

### Seeds — reproducible pieces

The terminal SSE event includes `seed_used`. Every finished job stores it,
and the gallery shows a **"seed 123456 · remix"** button that pre-fills the
prompt + seed back into the form. Leave the seed blank for a fresh random one
(his server draws one from the GPU RNG).

### The tunnel

On **your** machine, with the SDXL API running:

```bash
ssh -N -R 127.0.0.1:8003:127.0.0.1:8003 root@209.46.125.134
```

Then the adapter base URL `http://127.0.0.1:8003` just works. Port moved?
Change the base URL in Settings (or server env `FLUX_BASE_URL`) — everything
else follows.

Keep it alive:

```bash
# autossh (reconnects on drops)
autossh -M 0 -N -R 127.0.0.1:8003:127.0.0.1:8003 root@209.46.125.134 \
  -o ServerAliveInterval=30 -o ServerAliveCountMax=3

# or plain ssh with keepalives
ssh -N -o ServerAliveInterval=30 -o ServerAliveCountMax=3 \
  -R 127.0.0.1:8003:127.0.0.1:8003 root@209.46.125.134
```

### Browser-direct (no tunnel)

If the tunnel is down but you're *at* your machine, the Create page can call
your SDXL API straight from the browser (`http://localhost:8003` — still
loopback, still your hardware) and streams the result via `EventSource`.

Your API must allow the site's origin. It reads the whitelist from the
`CORS_ORIGINS` env var — start it with the site included:

```bash
CORS_ORIGINS=https://psykomedia.com python sdxl_api_scalable.py
```

(Comma-separate more origins if you test from elsewhere.) Note: `EventSource`
can't run from a `file://` URL — open the site over http(s).

### Sanity checks

```bash
curl http://127.0.0.1:8003/health        # 200 {"status":"healthy",...} or 503 while loading
curl -X POST http://127.0.0.1:8003/generate \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"a red cube","num_inference_steps":5,"width":512,"height":512}'
```

## 3. Engine 2 — ComfyUI in API mode

The visual node editor for diffusion. More setup, finer control. The backend
talks to ComfyUI's `/prompt` API — you never touch the UI unless you want to.

1. Install ComfyUI portable and confirm the UI loads at `http://127.0.0.1:8188`.
2. Download into `ComfyUI/models/`:
   - `unet/flux1-schnell.safetensors` (or a GGUF quant — see section 5)
   - `clip/clip_l.safetensors`
   - `clip/t5xxl_fp16.safetensors` (or the fp8 variant on tight VRAM)
   - `vae/ae.safetensors`
3. Filenames differ? Set server env `FLUX_UNET`, `FLUX_CLIP_L`, `FLUX_CLIP_T5`, `FLUX_VAE`.
4. Tunnel it (same pattern, port 8188) and point the backend at it:
   server env `FLUX_COMFYUI_BASE_URL=http://127.0.0.1:8188`.
5. Sanity: `curl http://127.0.0.1:8188/system_stats` should return JSON with a `"system"` key.

Browser-direct needs CORS: `python main.py --enable-cors-header *`
(the tunnel route doesn't need CORS — server-to-server over loopback).

## 4. Engine 3 — Standalone flux-server.py (fallback)

One Python script in this repo (`local-flux/`), one model download, one command.
FLUX.1-schnell (the *model* — the mode is the bundled server), 4 steps,
sensible defaults. For when you want zero setup.

```bash
# Accept the license at https://huggingface.co/black-forest-labs/FLUX.1-schnell first
python3 -m venv .venv && source .venv/bin/activate
pip install -r local-flux/requirements-flux.txt
hf auth login   # read access is enough
python local-flux/flux-server.py --port 8188   # --model / --steps / --preload available
curl http://127.0.0.1:8188/health
```

`loaded: false` in the health reply is normal until the first render — the
model loads lazily so the health check stays instant (`--preload` to load at startup).
Point the backend at it with `FLUX_STANDALONE_BASE_URL`.

## 5. GPU / VRAM guidance — read before downloading models

**Your SDXL API** (the priority engine) is the one that matters here. From
its own code comments:

> *"Stream layers from RAM -> cuts peak VRAM below ~7GB"* (CPU offload), and
> the loader retries *"so the API auto-starts as soon as a card frees ~7GB"*.

| Setup | VRAM | Speed | Notes |
|---|---|---|---|
| SDXL, CPU offload on | ~7 GB+ peak | tens of seconds/image | His config — the default. |
| SDXL, no offload | ~12 GB comfortable | faster | Set `ENABLE_CPU_OFFLOAD = False` if you have the headroom. |
| CPU only | any | minutes per image | Works. Bring patience. |

**Bottom line:** your SDXL API handles its own model — this table matters
mostly for Engines 2–3 (ComfyUI GGUF quants: Q8 ≈ full on 12 GB, Q6 close,
Q5 visibly softer on 8 GB — honest numbers, not marketing).

## 6. Which port does each engine need?

| Engine | Default port | Override |
|---|---|---|
| SDXL sovereign API | `8003` (adapter base URL) | Settings → adapter base URL, or env `FLUX_BASE_URL` |
| ComfyUI | `8188` | Env `FLUX_COMFYUI_BASE_URL` (falls back to `FLUX_BASE_URL`) |
| Standalone | `8188` | Env `FLUX_STANDALONE_BASE_URL` (falls back to `FLUX_BASE_URL`), or `--port` |
| Procedural | none | Always on. |

The active engine is chosen in Settings (or server env `FLUX_MODE` =
`sdxl`/`comfyui`/`standalone`, which overrides the setting). The worker
only attempts the active engine.

## 7. Verify end-to-end

1. Your SDXL API answers locally: `curl http://127.0.0.1:8003/health` → 200.
2. Tunnel up (section 2).
3. Open `https://psykomedia.com/settings.html` → the SDXL card goes green
   with a latency reading and his queue counts (pending/processing);
   the active-mode highlight matches your choice.
4. Open `https://psykomedia.com/create/`, pick **SDXL · my machine** —
   the badge shows the active mode and its status — type a prompt, Generate →
   the job queues, your hardware renders it, the image appears in your gallery
   labeled **"SDXL · rendered on your machine"**, with a seed/remix button.

## 8. Offline behavior

- Your machine off / tunnel down → jobs stay `queued` with
  **"engine offline — will retry"** and backoff retries
  (1 min, 2 min, … capped at 15 min), against the active engine only.
- Model still loading (health = 503) → jobs stay `queued` with
  **"engine warming up — will retry"**. Nothing is ever submitted into a 503.
- Turn your machine on or re-open the tunnel → the worker picks the queue
  back up automatically. No button to press, no jobs lost.
- The procedural engine in Create works regardless — it never needed your machine.

## 9. Troubleshooting

- **SDXL unreachable but curl works locally** → the tunnel command isn't
  running, or base URL / ports don't match. Re-check section 2 — the adapter
  base URL must be the *tunneled* address (`http://127.0.0.1:8003`).
- **"Engine warming up" forever** → his API is still loading the model
  (retries with backoff for a long time on shared GPUs). Watch his server
  logs; `/health` flips to 200 when ready.
- **429s in his logs** → something else is spending your tokens — the
  backend already paces at 1/sec; check other clients on the same IP.
- **SSE stream breaks mid-render** → normal on flaky networks; the backend
  falls back to polling `/status` + `/result` automatically. Jobs live
  15 minutes server-side, so there's time.
- **422 on /generate** → a param is out of range (steps 1–100, guidance
  1.0–20.0, size 512–2048, prompt ≤1000). The job's error text says which.
- **Port clash** → either free the port or run on another one — then the
  tunnel forward and the base URL must match the new port on both ends.
- **Browser-direct button never appears** → your browser can't see
  `http://localhost:8003`. Either you're not on the machine running the
  API, or `CORS_ORIGINS` doesn't include the site's origin.
- **ComfyUI workflow errors** → check model filenames in `ComfyUI/models/`
  match `FLUX_UNET` / `FLUX_CLIP_L` / `FLUX_CLIP_T5` / `FLUX_VAE`.
- **OOM / CUDA out of memory** → his SDXL API already CPU-offloads to stay
  under ~7 GB; if it still OOMs, something else is holding the card.
- **Slow first render** → normal. Model load + (first time) download.
  Later renders are fast.