I expect Nvidia to boom for at least another 5 years. There will eventually be an answer to the L2 to L1 bottleneck problem with the present CPU architecture. I fully expect that answer will come in RISC-V. Real hardware takes 10 years from napkin to first consumer products in hand. The key catalyst was shortly after the Llama model weights leaked. Anyone that knows about fundamental computing history, like the relevance of Bell Labs historically and that Meta AI is headed by Yann LeCun, should see the relevance of the “leak” in shaping the future. It will eventually be obvious that this event shaped the world. This is Apache versus Sun Micro and OpenAI is Sun. Within a couple of months of the leak the inadequacy of present CPU architecture should have been obvious. In the past all CPU coprocessor topology schemes have failed next to a competing integrated architecture. It is well established and should have immediately sparked new designs to fix the issue. That is when the real clock started for 10 years before the real solution comes to market.

The real solution will likely be considerably slower in total speed and include many more logical cores with a new architecture that can throughput tensors efficiently. Anything that gets sold in the interm as hardware for AI is marketing wank. I doubt Intel AMD or Nvidia will have the real solution to this problem. The real solution will likely happen in a RISC-V ISA and from someone new like how traditional automakers are unlikely to succeed with EVs because the change is too radical of a rethink and requires throwing out too much of the supply chain while investing massively into areas like a software first design when these companies are wholly incompetent with even the most basic software on present and late model ICE vehicles. Intel and AMD have failed to keep parity with Nvidia and the GPU, while Nvidia has no experience with traditional architectures. I think all will be made irrelevant by a future monolithic replacement, but that has quite a ways to go before it happens. I also find it funny that the whole e-core zen-5 energy optimization push for hardware is cutting out the very instruction set required for the most efficient tensor math that x86 can support. The existence of e-cores and how the CPU scheduler and software works means that even the performant cores that have the underlying tensor math instructions cannot use them because e-cores exist. There is even speculative circumstantial evidence that Intel’s solution to people discovering that the AVX instructions for the P-cores was disabled in microcode only and enabling it with the server CPU microcode is the main issue that caused the massive issue with 13-14th gen processors. It is all happening when, if they were prepared and optimised for LLM inference, all p-core consumer processors with the most advanced AVX instructions to load 512 bit wide words in one cycle would have made a massive difference. There are even ridiculously useless Intel processors that have all e-core designs. Both Intel and AMD showed they were not ready for this shift at all, so to me that means they are not likely to have a real monolithic replacement for x86 in the works either. It would not surprise me if China were to realize the political power of open source as a socialist rally and push back against the West and roll out a RISC-V solution. I kinda hope they do. Restricting their access to Nvidia is pushing them in this direction.