ChatGPT, DALL-E-2 and other AGI tools are all based on NLP (Natural language processing). Transformers are a type of neural network model that have recently been making significant advancements in the field of natural language processing.
They have been used in a variety of applications, such as machine language translation, conversational chatbots, and improved search engines. The attention mechanism, a key component of transformers, allows the model to reference relevant words in the input as it generates new text, making it more capable of understanding context and generating more coherent responses.
Transformers were first introduced in 2017 by a team at Google Brain, and are increasingly the model of choice for NLP problems and models. The transformer architecture, as described in the 2017 paper “Attention is All You Need,”(that was published by google) was able to demonstrate improved performance on machine translation tasks compared to previous models.
A video by The A.I Hacker – Michael Phi demystifies the novel neural network architecture with step by step explanation and illustrations on how transformers exactly work.