Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI | Lex Fridman Podcast #416

I think you might find Lex Fridmans interview with Yann LeCun interesting[1]. It discusses exactly this, how LLMs just mimmick intelligent behaviour but have no understanding of the world at all. It also discusses other approaches we should look at instead of current LLMs. [1] [VIDEO]

> A 4 year old child has 16k wake hours x 3600 s/hour x 1^6 optical nerve fibers x 2 eyes x 10 bytes/s = 1^15 bytes (approximation by Yann LeCun). Unfortunate typo. You meant 10^15 bytes at the end. Thanks to your citation I was able to find a podcast transcript [1] with Yann LeCun's explanation: > If you talk to developmental psychologists and they tell you a four-year-old has been awake for 16,000 hours in his or her life, and the amount of information that has reached the visual cortex of that child in four years is about 10 to 15 bytes. The transcript is missing "the" (10 to the 15 bytes). The corresponding timestamp in the podcast on YouTube is 4:48. [1] [LINK] [2] [VIDEO]

These are all facts coming from publicly available information, such as the Llama 3 paper and recent interviews. I'd dig up the specific timestamps, but I'm working at the moment and can't spend the next hour scrolling through long interviews. The interviews are: "Mark Zuckerberg - Llama 3, $10B Models, Caesar Augustus, & 1 GW Datacenters": [LINK] "Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI | Lex Fridman Podcast #416 ": [VIDEO] The total FLOPS used in training is in bunch of places, I can't find a good link at the moment.

The Unitree is what made me think we are headed for a bubble. The machines themselves will become commodity hardware. Tesla bot will push the hype side. I just don’t believe current AI systems have what it takes to operate in the real world. It doesn’t have to be AGI at all, but it needs a bunch of capabilities not present in existing models, some of which cannot be created by training on large corpuses from the internet like text and images. [1] Also people can’t afford to spend $20k on a robot just to do their dishes. My good friend tried to make a robot that could pick up dog poop from your back yard, and he went to HAX accelerator and everything, and through extensive research he found that the addressable market for dog poop robots is just too small to fund a company. Now he makes farming robots. (I do too as it happens, on a separate project.) The point is a humanoid robot worth the price of that robot needs to be very useful, and I don’t believe current approaches can get us there. Maybe some uses will be found, just as current AI systems have their uses, but I say it will be a hype bubble because companies will fund raise on massive promises they will never achieve, and investment will move on to something else after 5 years or so. Could still be robotics, but I actually think modular purpose built machines make way more sense than general purpose humanoids. What if instead of a humanoid to load the dishwasher you had a dishwasher that cleaned dishes one at a time. Yo

>The findings reveal that even slight changes in the phrasing of questions can cause major discrepancies in model performance that can undermine their reliability in scenarios requiring logical consistency. Relevant conversation with Yann Lecun: [VIDEO]