I am a skeptic at heart, which can be a valuable characteristic at times. It can also be very problematic as I can personally attest to. The best proof of this is my life-long approach to investing in financial markets or other financial “games".”
I don’t gamble. Every year that our industry convenes in Las Vegas for ITC I roam through the casino floors with no desire to gamble my money because I know that this game is engineered to have the house win. Perhaps that is more of a utilitarian or logical approach, not a skeptical approach, but the example holds.
But apply my skepticism to investing in the stock market and I am a clear loser. Since my early twenties, when I started to have money to invest, I have been extremely skeptical of stock valuations. I have avoided equity markets for various reasons, not least the explosion of public and private debt which I thought was untenable even when our national debt was at a paltry $20 Trillion. Objectively, I have missed out on the longest bull run in equities in history - but I sleep well at night when it comes to inevitable stock market crashes.
I try very hard to force myself to park my inner skeptic when it comes to technology advancements, since that would not serve me very well as a person who makes their living in technology.
But when the three leaders of the largest frontier labs (who really dislike each other) all of a sudden declare in unison that AI might kill us all if we don’t slow down, my inner skeptic immediately emerges. (Note: Google/Gemini has stayed mostly quiet, which begs another question)
First of all, as the “AI Czar” David Sacks commented recently, if the leaders of the AI labs themselves have this existential fear of the product they are developing, then they should be doing something about it themselves. But they are instead calling for government regulation to “stop us before we kill!”
(Watch the 10 minute youtube clip to hear this argument in more detail)
So why would they be creating an AI panic and calling for the government to regulate and slow AI down? My strong hunch is that this is motivated by strong self-interest, which is the only scenario where I could see Sam Altman, Dario Amodei and Elon Musk in the same camp. While there are many other angles to explore, I call your attention to these two related motivations:
1) Regulatory Capture
These AI labs have raised hundreds of billions of dollars to fund an arms race in models that is extremely competitive. The competition, however, extends beyond these megaliths. Promoting a regulatory regime that would restrict AI advancement would disproportionately harm the next tier of competition which could harm the rents that they could extract from customers in the future. It could also restrain the development of highly competitive open-weight models from China. Open-weight models are free to use (aside from your cost of compute) and seem to always be one step behind the “frontier.” As an insurance executive, I wouldn’t need to pay top dollar for frontier models when an open weight model can perform as well as the frontier model did a mere three months ago.
2) Deflate the bubble before it pops
There is an estimated $1.5 trillion in capex that will be spent by the end of next year, with some estimates predicting that amount of investment annually by 2030. If you calculate the revenues that would be required for these AI labs to cover depreciation of the initial $1.5 trillion alone over a 5-year depreciation cycle ($300B annually) at a midpoint gross margin of 60% the number far exceeds even the rosiest projections of revenue.
Now picture these margins (which don’t even include costs of training which are typically booked as R&D) shrinking as most enterprise customers start to realize that open weight models are “good enough” for the task at hand.
These pre-IPO companies are riding a ferocious wave of capital markets interest in building more and more data centers for compute, and eye-popping headlines for planned investment have become a way of keeping score on who will win the race. In a classic prisoners dilemma, none of these leaders want to be the only one to blink and slow down investment. In fact, when Anthropic CEO Amodei dared point out this obvious risk earlier this year it was seen as a sign of weakness in contrast to Sam Altman’s insatiable desire for growth in spending.
Going back to my thesis that this is an intentionally spurred panic to drive self-interested motivations, it seems clear to me that the leaders of these run-away capital investment trains are seeing the crash ahead and need a way to slow the train down that doesn’t single them out as the “loser” ahead of their record-setting IPOs. If they can slow down under the cover of “we are avoiding human extinction” it betrays no weakness in their bubbly trajectory.
Collateral Damage
I would have said “unintended consequences” but I think the consequences of such doomsday pronouncements are quite intended, so as to spur action from a congress that famously does not act, especially when it comes to regulating tech.
Unfortunately, if the current dialog continues with a focus on the many ways that humans will die out, society (and insurance regulators) may take an increasingly negative, even adversarial view toward the increasing use of AI. Rampant data center buildouts in rural areas are already shaping up to be a major election issue. For sure, there are many serious risks to this latest technological revolution that need to be carefully considered, and major technological change tends to drive major political change, even violence. AI’s industrial tycoons should be leading the way responsibly and with accountability, not stoking panic for their own self-interest.