H3-metal – Native MiniMax-H3 inference for Apple Silicon

A programmer in San Francisco has created a remarkably compact piece of software that lets Apple Mac users run a sophisticated Chinese AI system directly on their computers, no cloud connection required.

The project, called H3-metal, has caught the attention of the tech community after appearing on Hacker News earlier this week. It's the work of a developer who goes by the username 'antirez' on GitHub, the popular code-sharing website.

What it actually does

The software allows Macs with Apple Silicon chips — that's the M1, M2, M3 and newer models — to run something called MiniMax-H3 natively. MiniMax-H3 is an advanced AI language model from China that can understand and generate text, rather like ChatGPT but built by a different company.

What makes this interesting is the 'native' bit. Instead of sending your questions off to someone else's server farm and waiting for answers to come back over the internet, the AI runs entirely on your own machine. It's private, it works offline, and once you've downloaded it, there are no ongoing costs.

The 'metal' part of the name refers to Apple's Metal technology, which lets software tap directly into the graphics processing power built into modern Macs. This makes the AI run faster than it otherwise would.

Why people are excited

The project has generated considerable buzz, with 349 people voting it up on Hacker News and 76 comments discussing it over the past day or so. It's currently sitting at number four on the site's front page.

Part of the appeal is the sheer cleverness of the implementation. Getting complex AI systems to run efficiently on personal computers, rather than requiring massive data centres, is no small feat. The code is also reportedly quite compact and elegant, which programmers always appreciate.

For ordinary users, the attraction is simpler: the ability to use powerful AI tools without worrying about privacy, internet connections, or subscription fees.

The software is freely available on GitHub for anyone who wants to try it, though you'll need a bit of technical know-how to get it running.

Have you tried running AI on your own computer, or does this all sound like science fiction? Let us know in the comments.