An open-source project called llama-modchip has released a modified inference stack that its developer says can run a 284-billion-parameter mixture-of-experts model on a computer with 64GB of memory and no discrete graphics processor. The software is aimed at users who want local AI inference on CPU workstations rather than multi-GPU servers.
The project's repository presents support for a model identified as DeepSeek-V4-Flash and several 30-billion-parameter mixture-of-experts models. According to the developer's published benchmarks, the larger model can fit in less than 59GiB of RAM. The repository reports prompt-processing performance of 44.4 tokens per second and generation rates between 9.1 and 14.7 tokens per second, but those figures are project claims rather than independently reproduced results.
Llama-modchip's approach depends on custom model files and changes to the inference engine. The repository describes a block-64, low-bit tensor format with BF16 scales and assigns it the existing GGML type identifier 40. It also supplies specialized kernels for the supported architectures. That design means the models are not drop-in files for mainstream, unmodified installations of llama.cpp, Ollama or LM Studio. The developer warns that those programs will reject the tensors as an unknown type unless the provided patches or binaries are used.
The project offers a launcher that detects CPU cores, available system memory and, when present, GPU memory. It can then select a CPU and GPU split, download the project's chosen backend and open a local web interface. A separate precompiled Linux server binary is distributed alongside the model repositories, while the documentation also describes a Windows launch path.
Keeping the model fully in memory is central to the design. The repository says the large configuration avoids swapping model data to solid-state storage and can operate with a context window of up to 256,000 tokens. Those limits and speeds will depend on processor architecture, memory bandwidth, quantization choices and the exact workload, so prospective users would need to benchmark their own hardware before treating the headline results as typical.
The release is relevant to the growing effort to make capable language models usable on ordinary workstations, especially for offline or air-gapped tasks. CPU inference can reduce dependence on scarce accelerator memory, although it normally trades away some speed and may rely on aggressive quantization. Llama-modchip packages those trade-offs into a dedicated, incompatible toolchain rather than extending compatibility with every existing local-AI application.
For now, the evidence is the project's own code, documentation and benchmark claims. The repository invites users to run its benchmark command and submit machine scorecards to a community leaderboard. Broader testing will be needed to establish how consistently the reported performance carries across processors and operating systems.
