Lablup has open-sourced MLXcel, an inference engine for running AI models on Apple Silicon, releasing it under the permissive Apache 2.0 licence. The company frames the move around a goal bigger than the code: spreading the capacity to do AI inference more widely, rather than leaving it concentrated in a handful of clouds. The project is live on GitHub.

What it is

Inference is the model doing its job — answering a prompt, classifying an image — as opposed to the training that built it. MLXcel is built to run that workload efficiently on Apple's own chips, the M-series silicon in modern Macs. It joins a growing set of Apple-Silicon inference tools, but the Apache 2.0 licence is the part that matters: it lets anyone use, modify and ship the engine commercially without the strings that come with more restrictive terms, per Lablup's announcement.

Why decentralised inference is the point

Most AI compute today runs in a few large data centres. That is efficient and also a concentration of power, cost and dependency. Tools that run capable models on local hardware push against that concentration. A developer with a Mac can prototype, test and even serve smaller models without renting cloud GPUs. For privacy-sensitive work, keeping inference on the device rather than sending data to a provider is its own argument.

The caveat

On-device inference on a laptop does not replace a data centre for large models or heavy traffic. The ceiling is the hardware in front of you. What an engine like MLXcel changes is the floor: more people able to run useful models locally, and a healthier open ecosystem around Apple Silicon. For a developer deciding whether to use a cloud API by default, that new floor changes the calculation.

The hardware ceiling turned out to be moving, and upward

The ceiling is the hardware in front of you was the caveat, and it was stated as though the hardware were a fixed quantity. Over the following quarter the cost of that hardware moved sharply.

Memory went from about US$2.80 a gigabyte to US$12, and Google has confirmed price rises across the Pixel range on the strength of it. That is a consumer phone rather than a Mac, but the DRAM market is the same market, and unified memory is exactly what determines which models an Apple Silicon machine can hold.

Reading whether a machine has headroom is also less obvious than it looks. On a 17.2 GB laptop running a browser, a container stack and an editor, free memory read 0.06 GB with nothing wrong; the figures carrying information were the compressor at 7.82 GB and swap at 4,421 MB. Anyone sizing local inference off free memory is reading the wrong number.

None of this makes the decentralisation argument wrong. It does mean the entry price for it is set by a commodity market that has just repriced, and that a developer who already owns a capable Mac is in a materially better position than one who has to buy in now.

Inference did move to the edge, but not mainly onto laptops

The release implied that capable models on local hardware would push back against data-centre concentration. On-device inference did spread over the quarter, and it spread furthest where the constraint was never cost.

TDK's edgeRX Pro combines vibration, acoustic, magnetic, temperature and rotational sensing in a sealed node that decides locally. Condition monitoring has always been limited by data transport rather than by sensing, which is a different reason to run inference locally than either privacy or cloud bills.

The strongest case for on-device inference is not cost, but necessity: some workloads cannot usefully send their data anywhere. A permissive licence is most valuable to teams embedding an engine in a product they ship to run on that kind of workload.

The capital went the other way, decisively

Spreading the capacity to do inference rather than leaving it concentrated in a handful of clouds was the stated goal. Three months of capital allocation ran against it.

Five hyperscalers disclosed roughly US$1.2 trillion of lease obligations, US$725bn of it not yet commenced. Together AI booked up to 10,000 Nvidia B300s at a Chennai campus, its second buildout in a week.

An Apache 2.0 inference engine does not compete with that and was never going to. What it changes is who can build without asking permission, and the licence is the part of the release that will still matter in five years, because it is the only part nobody can withdraw.