Now when it comes A.I. it’s all well and good to be neutral and proscribe caution, and say there’s lots of good and bad on “both sides”. But let’s also consider physically, de facto the tangible artifacts of this pursuit. This is why I love EETimes (Engineers can be hostage to trends, and fashions as anyone else, but there’s also the tendency to want to measure and quantify as well). In this article the new AI Supercomputer “Inflection” is announced along with it’s particulars (NVidia GPUs, and lots of ’em), along with the attendant costs to acquire, much less RUN the thing. A relevant pull-quote:
When finished, Inflection’s installation will be 22,000 Nvidia H100 GPUs, making it both the largest AI cluster and one of the largest computing clusters in the world. All for a chatbot. Nvidia’s own AI supercomputer, Eos, a 4,600-GPU monster, is still in the bring-up phase, but will be dwarfed by Inflection’s cluster.
I’m a gen-Xer. So I grew up reading Douglas Adams at an impressionable age. The reason I say that is there’s a number of side/tangent stories throughout A Hitchiker’s Guide to the Galaxy. And the one most relevant to this is the idea that computers beget bigger computers and so on. From Deep Thought sprung the answer: 42. And from Deep Thought the Earth was created to find and understand the actual meaning of the original question, What is the meaning of Life the Universe and Everything.
Inflection is not as big as the Earth. But it is BIG. So this supercomputer is 5X bigger than the computer NVidia itself (who actually designs and builds the underlying GPU) uses for training and inference. In the article, the author estimates roughly the costs alone for the hardware: $800Million USD. Followed by costs to run the cluster for one day: $700,000 USD. So one has to ask, how the hell do you charge customers money, and make a profit having sunk all that capital into building and operating a behemoth like Inflection? Do you ever get to that point? When do you make money?
Or is it more like a Cold War mentality? Where competitors in the AI space attempt to cow all comers into submission. Meaning that having an asset at your disposal makes competitors less likely to “attack” because they cannot scale up to that size? And more so, costs of Nvidia GPUs are not going down the longer they manufacture them, Nvidia has a monopoly on it’s GPU technology, they’re never going to let that go (ever). So you won’t see anything like price competition for AI Supercomputers, ever. It’s just going and going, growing and floating on Venture Capital and investors, hoping like hell there’s a pay-off, sometime, somewheres in teh futures.
Carpet Bomberz Inc. 