The AI boom’s chip shortage has an unlikely hero: the blockchain

Blockchain and crypto have been criticized for years as a “solution in search of a problem.”

Now some industry entrepreneurs may have found the problem: A shortage of compute power needed to train AI models.

The solution: Researchers and startups that can’t get graphics processing units (GPUs) can use blockchain-based markets to quickly and easily access compute power for a fraction of what major providers charge.

Bitcoin miners and others spurred an arms race for GPUs needed to conduct calculations to keep systems running, but demand slumped when the digital currency market collapsed last year. Now instead of simply cannibalizing the struggling crypto industry, some AI companies are leaning on the blockchain to distribute those same GPUs.

Ishan Dhanani, a computer science graduate student at Columbia University, is an example of how upstarts are getting around GPU shortages through the blockchain. He wanted to start fine tuning Meta’s LLama2 open-source AI model to experiment on it, but ran into the reality that huge companies have swallowed up most of the compute power.

He couldn’t obtain any through market leader Amazon Web Services, and smaller providers were always sold out. Getting access through Columbia was also a headache.

That led him to the Akash Network, one of a handful of companies that have created protocols to allow owners of GPUs to rent them out on the blockchain, earning tokens for every minute the GPU is utilized. Dhanani was able to access a $15,000 Nvidia A100 for $1.10 per hour through Akash. It took him about seven hours to complete his work, for about the cost of a beer.

Companies like Akash can offer cheaper access partly because the protocols are set up to run on their own, like a version of Airbnb or Uber without those companies taking commissions. Community members on the blockchain, incentivized with tokens, handle the nuts and bolts of the operation. As a result, the costs are low, with nobody except for the owners of the GPUs earning any significant revenue on the transaction.

The experience spurred Dhanani and two friends to launch Agora Labs a few months ago to help what he calls “the GPU poor” more easily book time on GPUs via the blockchain. “The OpenAIs and Anthropics can’t be the only ones that have the power to train and host models like ChatGPT,” he said.

UK-based Gensyn, which recently announced a $43 million Series A funding round, represents a huge venture capital bet that blockchain has a future facilitating the sale of GPU time for the AI industry.

Gensyn is building a system that would vastly simplify the pricing model for training in AI, according to an interview with the company’s co-founders. Instead of paying for time on a GPU, Gensyn plans to estimate the overall time and cost of the training job and then spread the tasks around to computers all around the world, searching for the best prices.

That strategy involves tackling a thorny technological problem: The more spread out the compute resources are, the more complicated the training gets.

Gensyn co-founder Ben Fielding said when he was earning his PhD in deep learning, the scarcity of compute resources meant he was unable to fully complete his research in automating the development of AI models.

“The only people who could do that research were Google and Microsoft,” he said. “I realized if I was in that position, a lot of other people in the world were in that position, which meant we weren’t moving as quickly towards a machine-learning future as we could be.”

Fielding said that AI research has evolved around the world’s most powerful graphics processors. But he argued that if a massive network of GPUs were available on the blockchain, the types of AI models would adapt so they could be trained on a wider variety of processors.

As Gensyn readies its product, Akash Network says it will soon help facilitate the training of the first AI foundation model using GPUs on the blockchain. Startup Thumper AI is building a product that will allow artists to create their own AI models based on their personal work,and then sell access to those models. To do that, it needs access to GPUs to fine-tune its proprietary model using Stable Diffusion.

But Thumper CEO Logan Cerkovnik ran into a familiar problem: AWS would not give him access to the number of GPUs he needed to train the model. He also looked at some smaller providers and companies that resell GPU space. But he said the blockchain solution made more sense. With data center providers, he said, there are conversations with salespeople, price negotiations, and a vetting process. On Akash, the transaction is quick and easy, he said.

Some companies that provide compute power for crypto firms have begun offering up those resources to the AI industry, like Foundry. It repurposed its data centers, used primarily to mine Bitcoin and other cryptocurrency, and added additional GPU capacity to run AI models through another decentralized platform, Bittensor. It’s also allowing its GPUs to be rented out on the blockchain through Akash. “We asked ourselves ‘how else can we support the decentralized infrastructure thesis that Foundry holds as a whole?’” said Tommy Eastman, Accelerated Compute Engineering Lead.

While protocols like Akash are being used to train AI models, Bittensor is being used to run those models, a process known as “inference.” On Bittensor, users can play around with a chatbot similar to ChatGPT. The difference is that each prompt is sent to a wide network of entities on the blockchain, which get assigned the prompt based on the compute needs. The winning bid receives cryptocurrency as payment.

In tech, the people who build platforms often can’t imagine how they will ultimately be used. Steve Jobs probably never thought the iPhone would enable Uber. Mark Zuckerberg likely didn’t envision Facebook would lead to the creation of Zynga.

The idea behind the blockchain is to build the platform to end all platforms — the ultimate canvas for the development of new ideas, without a central owner, that can’t be corrupted and manipulated. Sure, it’s idealistic and possibly naive. And the blockchain has attracted plenty of unsavory people who saw an opportunity to make a quick buck.

And yet, here we are. Another platform with an unintended use case that seems, well, pretty useful. It’s ironic because the massive compute resources necessary to make the crypto industry work were viewed almost as the industry’s Achilles’ heel. Companies like Tesla stopped accepting Bitcoin because it was too energy intensive and thus a big contributor to climate change.

That Achilles’ heel may become crypto’s saving grace. Blockchain got really good at efficiently selling compute power to the highest bidder with very little fuss. And now, the generative-AI craze has spawned a new industry that is even more energy-hungry than crypto. And, unlike crypto, it is unlikely to slow down any time soon.

There’s another reason this could work: It’s not a get-rich-quick scheme. One of the biggest problems with crypto was that the financial speculation essentially spoiled all of its ideas. People invented “initial coin offerings” and the pump-and-dump schemes immediately tanked the idea. As soon as non-fungible tokens (NFTs) were created, speculators jacked up the prices so quickly that most people couldn’t participate.

This new model isn’t built for financial speculation. Most customers acquiring the GPU-time will be paying in regular currency, and the market for GPU-time will always be somewhat anchored to what cloud providers are charging. The tokens will mainly be used on the back end to facilitate the transactions and incentivize people to take part in the maintenance of the system.

And there are some interesting long-term possibilities here. (Sorry, but the following is super geeky). If the blockchain gets better at distributing compute power, significantly lowering latency, you could see something like a decentralized cloud. In that scenario, all of those powerful devices we carry in our pockets and have sitting around at home could be used more efficiently as part of powerful global computers, all meshed together into a communications network that gives us constant connectivity.

Some people argue that democratizing the ability to train AI models is a potentially dangerous development. People could make models that act maliciously, violate copyright laws, and potentially even develop into dangerous, out-of-control malicious actors.

• People are going to great lengths to find GPUs, as this New York Times story chronicles.

Comments attributed to a Gensyn co-founder were made by Ben Fielding. An earlier version of this story misattributed his comments to the company’s other co-founder, Harry Grieve.