Tesla AI Day 2021

The newly announced Tesla humanoid bot is around 2:05:11. Expected prototype in 2022. The demo was an obvious troll but it looks pretty exciting. I'd rather have resources put into actual innovation than learning how to increase the likelihood someone will click a button by 0.01% [VIDEO]

Elon's been giving some lip service to closer-to-GI capabilities anyways, e.g. when describing how it would be optimal to be able to control Tesla's humanoid robot by saying "please go to the store and get me the following groceries"... [VIDEO] Whether Tesla wants to fund serious work towards that may be a different question (but they do seem to be serious about at least building the robot).

They do. Karpathy spoke about it in on Tesla AI day. They use it for transforming image space to a vector space. See: [VIDEO] (timestamp 54.40 onwards)

In Karpathy’s recent AI day presentation he specifically stated they use transformers. But not on the raw camera input — they use regnets for that. The transformers come higher up the stack: [VIDEO] Transformers mentioned on the slide at timestamp 1:00:18.

Can you provide a more specific timestamp? 54.40 doesn't seem to mention anything about transformers, and "onwards" is two hours. I'd be really surprised if they use transformers due to how computationally expensive they are for anything involving vision. EDIT: Found it. 1h: [VIDEO] Fascinating. I guess transformers are efficient.

They use the key-value lookup/routing mechanism from Transformers to predict pixel-wise labels in bird view (lane, car, obstacle, intersection etc.). The motivation here is that some of the predictions may temporarily be occluded, so for predicting these occluded areas it may be particularly helpful to attend to remote regions in the input images which requires long-range dependencies that highly depend on the input itself (e.g. on whether there is an occlusion), which is exactly where the key-value mechanism excels. Not sure they even process past camera frames at this point. They only mention that later in the pipline they have an LSTM-like NN incorporating past camera frames (Schmidhuber will be proud!!). Edit: A random observation which just occurred to me is that their predictions seem surprisingly temporally unstable. Observe, for example, the lane layout wildly changing while it drives makes a left-turn at the intersection ( [VIDEO] ). You can use the comma and period keys to step through the video frame-by-frame.

> […] AI has a massive value to every aspect of the company's work. That's also just wrong. During the recent "Tesla AI Day", when asked during Q/A, Elon Musk specifically mentioned that they intentionally use machine learning only for very few cases: Q: "Is Tesla using machine learning within its manufacturing, design or any other engineering processes?" Elon: "I discourage use of machine learning, because it's really difficult. Unless you have to use machine learning, don't do it. It's usually a red flag when somebody is saying 'We wanna use machine learning to solve this task'. I'm like: That sounds like bullshit. 99.9% of the time you don't need it." [VIDEO]

The Q&A section on their compiler and software that the author links to is very interesting: [VIDEO] It sounds like they're going to have write a ton of custom software in order to use this hardware at scale. And, based on the team being speechless when asked a follow up question, it doesn't sound like they know (yet) how they're going to solve this. Nvidia gets a lot of credit for their hardware advances, but what really what their chips work so well for deep learning was the huge software stack they created around CUDA. Underestimating the software investment required has plagued a lot of AI chip startups. It doesn't sound like Tesla is immune to this.