BREAKING
'mac code' runs 35B model locally
0
B
params
0
GB
Mac mini RAM
0
tok/s
speed
How flash-paging works
1
Pin attention in RAM
↓
2
Stream FFN from SSD
↓
3
Load only active experts
↓
4
~5.5GB RAM used
IQ2_M vs Flash Streaming speed
IQ2_M
30
Q4 Stream
5.4
27B Stream
0.18
Trade-offs developers cite
Upsides
●
35B on a $600 Mac mini
●
Keeps full 4-bit quality
Constraints
●
Flash mode 0.18 tok/s
●
Needs mmap and pre-split
Repo just published, more tests ahead
AI NEWS BLITZ
A new open-source project lets a 35B AI coding agent run on Apple Silicon Macs.