Local video generation on Apple Silicon

Why every tool fails on a Mac, and what we did instead

If you own an Apple Silicon Mac with real memory and you've tried to generate video locally, you've hit the same wall everyone does: you download 30 GB of curated weights, hit generate, and get

RuntimeError: Undefined type Float8_e4m3fn

This is a field report on why that happens, why it's not a Mac limitation the way it looks, and what we built once we understood it.

The wall: FP8 is a roach motel on Metal

Nearly every curated model bundle ships FP8 weights. FP8 exists for a good reason — it halves an FP16 model so a 22B video model fits in a 24 GB consumer NVIDIA card. It's the compression that makes local generation possible on gaming hardware.

PyTorch's Metal backend has no float8 kernels. That much is documented. What isn't documented is how completely closed the trap is. We tested every escape:

x = torch.zeros(8, dtype=torch.float8_e4m3fn, device="mps")

x.to(torch.bfloat16)     # RuntimeError: Undefined type Float8_e4m3fn
x.to("cpu")              # RuntimeError: Undefined type Float8_e4m3fn
x.view(torch.uint8)      # RuntimeError: Undefined type Float8_e4m3fn

You can put an FP8 tensor onto the GPU. You cannot convert it, copy it off, or even reinterpret its raw bytes. There is no userspace workaround — not a shim, not a monkeypatch, not a clever cast. We wrote all three before accepting it.

The failure is also path-dependent, which is why the situation looks so confusing from the outside:

So FP8 images work and FP8 video dies, on the same machine, in the same app. That inconsistency is why "does this run on a Mac?" gets contradictory answers.

The unlock: stop compressing

Here's the part that reframes everything.

FP8 is a workaround for not having enough memory. A Mac with 64–128 GB of unified memory doesn't have that problem. It can hold the model uncompressed.

The BF16 build of LTX 2.3 22B is a 46 GB file. On a 24 GB NVIDIA card that's a non-starter — hence FP8. On a 128 GB M5 Max, torch.mps.recommended_max_memory() reports 115 GB allocatable. It just… fits.

Requested to load LTXAV
loaded completely; 40053.34 MB loaded, full load: True
8/8 [01:00<00:00, 7.53s/it]

A 22-billion-parameter video model, full precision, resident in a laptop's memory, sampling at 7.4 s/step. No quantization, no shim, no compromise. The BF16 file is a drop-in for the FP8 one — the loader doesn't care.

The weights that don't work on Apple Silicon are the ones built for cards smaller than your Mac.

The second wall: attention doesn't care how much memory you have

Fixing precision got video generating. Then a large request killed the backend:

RuntimeError: Invalid buffer size: 456.35 GiB

The two-stage LTX pipeline refines at 2×, and the refine pass attends over roughly (W/32) · (H/32) · (frames/8 + 1) tokens. That's quadratic in resolution and linear in duration, so 1600 px × 11 s asks for a single 456 GiB allocation. No GPU satisfies that; it just fails later and uglier on CUDA.

Past about 45k tokens the allocation aborts. That's computable before you submit, so the right move is to check and refuse with an explanation, not to crash mid-queue.

Which leads to the most useful feature we built. The refine pass is ~60% of generation time, so making it optional gives you two modes:

Draft is ~2.7× faster and handles the 1600 px × 11 s request that crashes in Refined. Iterate in Draft, commit in Refined.

Flags that matter

PYTORCH_ENABLE_MPS_FALLBACK=1 \
PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 \
python main.py --use-split-cross-attention --fp32-vae

• HIGH_WATERMARK_RATIO=0.0 lifts the allocator cap — required for a 46 GB model.
• --fp32-vae is the first-line fix for MPS decode NaNs.
• Never --bf16-vae — it breaks the LTX audio VAE.
• Never --force-fp16 — black frames on recent macOS.
• SageAttention and FlashAttention are CUDA-only. torch.compile doesn't work on MPS.

Measured (M5 Max, 128 GB, macOS 26.4, torch 2.13, ComfyUI 0.29)

Published M1/M2/M3 benchmarks are a pessimistic floor for M4+: those chips emulate BF16, while M4 and later do it in hardware.

What we built

localVideo — a local image and video studio designed around these findings instead of fighting them.

• Picks BF16 weights automatically, and greys out anything your backend can't

run with the actual technical reason

• Draft / Refined and S / M / L, so speed is a dial rather than a surprise
• Computes the attention token budget before submitting and tells you the max

duration for your size instead of aborting the backend

• Offline by default: binds loopback only, zero telemetry, no CDN assets. Pull

the ethernet cable and it still works.

On phoning home

"Local" and "private" get used interchangeably in this space, and they aren't the same thing. A tool can run every model on your GPU and still narrate your session to a server somewhere.

So we audited what actually leaves the machine, and it's worth knowing what's normal in local-generation tools:

None of that is malicious — analytics answer real product questions, and update checks are a courtesy. But an opt-out beacon that fires on every generation is a strange default for software whose pitch is that your prompts never leave your computer, and a CDN font request means the UI doesn't fully work offline.

localVideo makes exactly zero outbound connections. Not fewer — zero:

• no analytics, telemetry, crash reporting, or usage beacons; there is no SDK to disable
• no update checks, no release pings, no license or entitlement calls
• no CDN fonts, scripts, or styles — every asset is served from your own machine
• binds 127.0.0.1 (LAN access is an explicit --lan flag, never a default)
• the only socket it ever opens is to your local ComfyUI

There's no privacy toggle in the settings because there's nothing to toggle. The verification is a one-liner: run it with the network down and nothing changes. Or put it on a firewalled network and watch it stay silent — we did both.

Don't take our word for it — grep the source. Every URL in the codebase:

$ grep -rhoE "https?://[a-zA-Z0-9.:_-]+" server.js lib/ public/ | sort | uniq -c
   5 http://127.0.0.1:8188      # your ComfyUI
   1 http://www.w3.org          # SVG xmlns namespace — an identifier, never fetched

That's the whole list.

On dependencies

We kept exactly one: ComfyUI, as the generation engine. It's genuinely good at being a graph executor, its node API is stable and introspectable, and reimplementing diffusion pipelines to avoid it would be vanity, not engineering.

Everything else is gone:

That third row is the one we're proudest of. Comparable tools require five or more third-party node packs, each an independent Python project that can fail to build — one of them (numba-based) simply doesn't compile on macOS + Python 3.12, which silently breaks an entire workflow family. Sticking to core nodes means the install is "clone ComfyUI, download weights, run."

Graphs are built against ComfyUI's own /object_info signatures, which is the authoritative description of what each node accepts. That also caught a real bug class: ComfyUI 0.29 types some inputs as unions ("FLOAT,INT"), and a builder that only recognizes single scalar types silently drops them — producing graphs the server rejects with a confusing required_input_missing.

A native MLX backend that removes the ComfyUI dependency entirely is interesting and plausible on this hardware. It is not what this is today.

Credit

Mix Studio by BlackMixture is what convinced us a local generation front end could feel genuinely good rather than like a node graph with a hat on. It's Windows/NVIDIA-only and doesn't run this hardware, so localVideo is an independent implementation written from ComfyUI's public API — no shared code — but the inspiration is real and worth naming.

Thanks also to the ComfyUI team, Lightricks (LTX-2.3), and Krea, whose models and published reference workflows made the rest possible.