WE MUST ADD STRUCTURE TO DEEP LEARNING BECAUSE...

Interesting, I thought it was proven Transformers cannot do recursion[0] and aren't actually able to be equivalent to a turing machine due to lack of expandable memory, and are rather finite state automata. [0]: [VIDEO]

Do you have any citation on lack of funding for foundational research? From the looks of things, it seems like there is insane amount of money in AI and while maybe majority of it goes to "projects with high probability of success", whatever that means, fringe ideas seem to generate good amount of capital as well. Even (what I would consider to be) academic projects seems to get good funding [0]. On top of that, it seems like you're not saying that Transformers might be a dead end but Neural Networks itself might be one. At this point, NNs already gives a lot of value. If someone's goal is to just get products out that say diffusion models already helps you achieve, how is that even a dead end? Do you mean dead end for AGI? In which case, there seems to be research on alternatives there as well? [1] [0] [VIDEO] [1] [LINK]

Symbolica.ai | London, Australia | REMOTE, INTERNS, VISA We're trying to apply the insights of category theory, dependent type theory, and functional programming to deep learning. How do we best equip neural nets with strong inductive biases from these fields to help them reason in a structured way? Our upcoming ICML paper gives some flavor [LINK] ; you can also watch [VIDEO] ; but there is a lot more to say. If you are fluent in 2 or more of { category theory, Haskell (/Idris/Agda/...), deep learning }, you'll probably have a lot of fun with us! Check out our open positions at [LINK]