The onslaught of products with artificial intelligence has begun. With it, comes the clamour of offerings from businesses new and old. Some applications are free (eg. chatGPT), others embedded into old products to bring them new life (eg. Bing). A variety of solutions exist for businesses (e.g. from AWS) to create AI embedded or powered products. Other companies sell components that make AI possible (e.g. Nvidia, Snowflake).
Nvidia has become a household name, assuming your household is interested in high growth tech stocks. Nvidia makes semiconductor or microprocessor computer chips, some for AI. Considering the level of hype, I wondered if AI chip is something truly unique and valuable, or just an ordinary thing repackage into the trend? Are they the grocery equivalent of ‘meat free’ bread and pasta, or Petri dish-grown, steak-resembling, synthetic food units?
The question of why it’s important sounds very business school: it comes down to the value chain. If AI chips are required to deliver AI products, which are very much in demand currently, then AI chips are very important if you are making AI products or investing in businesses that make AI components or AI products.
Regardless of the product or industry, understanding whether a business will prosper or default due to lack of supply or lack of profit requires knowing the starting materials, how the product is made, where it’s sold, who the customers are and how the seller makes money.
Consider an example of the value chain for an AI-based chat bot on a large retailers’ website:
- starting materials? knowledge of the retailer’s retail products, company sales and return policies, how to communicate with customers in the right way to make them happy and buy more stuff,
- how to make the product? AI programming, computer system to run it on
- where will it be sold? this type of product the retailer would likely outsource as a custom service from a company specializing in customized solutions for enterprise,
- who will buy? large retailers
- business model? SAAS – software as a service from a high tech firm, fee likely billed monthly
Parts of some value chains are quite ordinary. Two of the largest businesses today excel because they do the ordinary exceptionally well. Amazon delivers online orders fast. Faster than any other business. Doesn’t sound like rocket science, but to be able to do so makes them a leader in online retail. Walmart has the lowest prices. Cheap stuff isn’t that hard to come by, is it? Actually, it is, especially in volume and with a respectable quality. Walmart offers it better than most competitors.
Back to ‘are AI chips special?’. AI is an early growth industry so there is much commotion regarding which businesses have the best infrastructure products. It’s a value chain of delicious tension that venture capitalist enjoy.
The answer to whether AI chips are unique is: yes and no.
From my last-century undergraduate education, ‘chips’, semiconductors or microprocessors, revolutionized the computer industry by miniaturizing the ability of a machine to perform logic operations. Silicon-based. Hence the personal computer, laptop, mobile phone etc.
Chips, or those little green wafers you can find in a lot of physical things (like your phone, furnace, car or pet (do not attempt to find in your pet without medical assistance)) make the automated world go around. At a basic level, it’s all yes/no decisions, times a decision tree of millions of individual decisions. Or perhaps a billion decisions if the chip is sophisticated1.
An AI chip is one that is capable of logic-ing its way through billions of decisions, and crossreferencing them, really fast. How hard is that?
So, Nvidia:
- From the company themselves2, their A100 chips are faster (lower latency), smaller (0.15 vs 0.22 um) and have better communication capabilities (RAID/parallel processing). Overall: better because faster, more efficient and more convenient. Not revolutionary, just excellent at what they do, kinda like the difference between the iPhone 14 and the iPhone 6.
- However, they’ve distinguished their business in part with chips for the gaming industry, which require fast, complex computation. Sorta like AI. Hummm….
- It has been report that the language generative AI, chatGPT, runs on A100 chips from Nvidia3. Is part of the company’s magic – marketing a chip for all occasions – generative language AI, gaming, cryptocurrency mining, cloud computing and all of the above? Or is it that their excellent general function chips are bundled with a solution that includes other functionality that provide an ‘out of the box’ AI functionality for firms that want it (Nvidia’s H100).
At the moment, Nvidia underpins a revolution in technology4, by supplying a basic ingredient in the mix that delivers AI products and products that make AI products. But it’s more than being in the right place at the right time, even if they are. It’s about the math, according to this article5, doing supportive math calculations to fire processes in a more sophisticated way than other chips. This is more like the iPhone 3 compared to a landline. Yes, there is something special about Nvidia chips that allow AI to do its thing better. And there are reports that this was the intension all along from the dev. department at Nvidia – to make chips that facilitate deep learning and image processing of AI.6
Still I wonder if AI chips need to be just generally faster or specially faster? Does the GPU do the same thing faster by taking a different route (like providing power for cars via batteries rather than combustion of fossil fuel – same device new components, or is it a fundamental shift, same need, new device – like mobile telephony replacing landlines – new infrastructure to deliver same communication need).
This statement “tailor-made to efficiently perform specific calculations required by AI systems” 7 sounds like there is something special about AI requirements.
AI is a feat of computing. Just as rockets that take people to the moon are feats of propulsion. This quote sounds like less of a revolution than scale up: “modern AI techniques relies on computation on a scale unimaginable even a few years ago “8 The same source explains the key need with AI processes is parallel processing, rather than sequential. Mostly chip design, I’d say, which is anything but trivial.
I’m starting to get the picture of AI chips. They don’t do any particular new thing, they just do what chips do faster and more efficiently than other chips. Perhaps like engines. Those that lift mega-ton aircraft off the ground aren’t different than those that make a car roll down the road, except in a lot of ways.
An AI chip is important:
- to its manufacturers, as a source of income, and their clients as a building block for their products or customer’s products
- to businesses to help build better solutions to many things
- to venture capitalists and other investors – as a good investment, at least for now because of the broad demand
- to inventors and entrepreneur as an opportunity to make it better, faster, cheaper versions.
To me, it started as a mysterious sounding wave of innovation but turned out as skirting the line between better-faster-cheaper and innovative. Sure, making better-faster-cheaper requires innovation.
Nvidia has become the superhero of the supply chain. I’d say less so because they make chips that work really well for AI application that having the vision to anticipate the demand.
1 The way I understand this, a traffic light example: The basic logic control would be to turn from green [go] to yellow [warning of change] to red [stop]. Next step is to link it to stop light changes in the other direction. The logic is ‘if green east-west, then red north south’. Then, traffic lights should respond to the volume of traffic – ‘if five cars waiting at light, change red to green (which requires changing green to red in the other direction’. And, what about traffic in the same direction? Ideally, lights at sequential intersections are coordinated -> if light green on next block, then make light green on this block 3 minutes later. And multiply that coordination all over a city. AI might include information from each car on the car’s intended destinations to balance out traffic volume of various routes.
2https://www.nvidia.com/en-us/drivers/feature-single-chip/
3https://www.fierceelectronics.com/sensors/chatgpt-runs-10k-nvidia-training-gpus-potential-thousands-more
4that Nvidia chips are widely used in AI https://www.bbc.com/news/business-65675027
5https://qz.com/what-makes-nvidias-chips-so-special-during-this-ai-boom-1850487195
6https://www.wired.co.uk/article/nvidia-ai-chips
7https://cset.georgetown.edu/publication/ai-chips-what-they-are-and-why-they-matter/
8 Saif M. Khan Alexander Mann “AI Chips: What They Are and Why They Matter . An AI Chips Reference.” Centre for Security and Emerging Technology