I see zero evidence that logic has a path to being solved or even just improved from the train wreck logic LLMs currently have as scaling up did not have the impact the investments anticipated. Yes, most other AI metrics really did improve with scale, but not logic as is obvious to anyone using AI for heavy mental lifting like codin. I call this the Cognitive Gap and despite the effort, it is clear no AI company has any advantage here. Certainly not OpenAI based on their behaviour alone nor anyone else as the first to crack it, will have a massive advantage and will likely see the birth of AGI and the elusive self improvement takeoff. Clearly no one is there today.
John Carmack said a few years ago on the Lex Fridman YouTube video titled “The code for AGI will be simple”
“It seems to me this is the highest leverage moment for a single individual potentially in the history of the world … I am not a madman in saying that the code for artificial General intelligence is going to be tens of thousands of lines of code, not millions of lines of code. This is code that conceivably one individual could write, unlike writing a new web browser or operating system.”
This deeply resonated with me as it is likely true. In fact I am sure many in IT can relate as anyone working on large IT systems likely have a story that goes like this. Performance of a new Enterprise system that was developed in-house will require millions of dollars of compute to meet the SLA performance metrics which is well above the original budget. As hardware was being begrudgingly acquired a smart developer (usually 1 person) realized there was a better way to code a key component and it gained 100x in performance making the need for all that extra millions in hardware null and void. This did not happen to me once, but many times over my 30 year IT career. You can brute force, but you can also find elegant solutions.
In my firm opinion there is a strong possibility that John is right and if this comes to pass, it will suddenly mean that AGI is much more efficient than the data centres we have built for it, thus making those investments pointless.
Of course it may not roll that way and instead we realize it is big code and big data Centers needed and one company or country ends up Dominating all, but my money is on an individual discovering and then making it open source and decentralized and you should hope for this too as the alternative is ugly for all. Centralization = Corruption and Exploitation.
Continuing to invest and scale up AI Data Centres without a clear path on how to improve LLMs logic is a fool’s errand.
I don’t think this meets the criteria of not mattering at all, it matters a whole lot
I see zero evidence that logic has a path to being solved or even just improved from the train wreck logic LLMs currently have as scaling up did not have the impact the investments anticipated. Yes, most other AI metrics really did improve with scale, but not logic as is obvious to anyone using AI for heavy mental lifting like codin. I call this the Cognitive Gap and despite the effort, it is clear no AI company has any advantage here. Certainly not OpenAI based on their behaviour alone nor anyone else as the first to crack it, will have a massive advantage and will likely see the birth of AGI and the elusive self improvement takeoff. Clearly no one is there today.
John Carmack said a few years ago on the Lex Fridman YouTube video titled “The code for AGI will be simple”
“It seems to me this is the highest leverage moment for a single individual potentially in the history of the world … I am not a madman in saying that the code for artificial General intelligence is going to be tens of thousands of lines of code, not millions of lines of code. This is code that conceivably one individual could write, unlike writing a new web browser or operating system.”
This deeply resonated with me as it is likely true. In fact I am sure many in IT can relate as anyone working on large IT systems likely have a story that goes like this. Performance of a new Enterprise system that was developed in-house will require millions of dollars of compute to meet the SLA performance metrics which is well above the original budget. As hardware was being begrudgingly acquired a smart developer (usually 1 person) realized there was a better way to code a key component and it gained 100x in performance making the need for all that extra millions in hardware null and void. This did not happen to me once, but many times over my 30 year IT career. You can brute force, but you can also find elegant solutions.
In my firm opinion there is a strong possibility that John is right and if this comes to pass, it will suddenly mean that AGI is much more efficient than the data centres we have built for it, thus making those investments pointless.
Of course it may not roll that way and instead we realize it is big code and big data Centers needed and one company or country ends up Dominating all, but my money is on an individual discovering and then making it open source and decentralized and you should hope for this too as the alternative is ugly for all. Centralization = Corruption and Exploitation.
Video referenced: https://youtu.be/xLi83prR5fg