Developing machine understanding
Language teaching methods based on grammar rules have long been criticized. Language pedagogues suggest reading, listening, and writing excercices rather than grammar exercises for a better comprehension of a foreign language.
As a person who studied many foreign languages, I also find the second method more practical and logical than the grammar rules-based method.
It turns out that artificial intelligence models agree with me as well. Traditionally, algorithms of natural language processing (NLP) models were told to analyze sentences by sequences: adjective, noun, verb, adverb… This was a laborious task and even though it worked well enough, today we do have a better model for text analysis: GPT3 by OpenAI.
How does it work?
The basis of the GPT3 algorithms technology is transformers that are self-supervised algorithms. It means that they are not told what to look for among the language but instead, they analyze lots of data to understand how sentences are built, which words follow the others, and which make the most sense. They don’t require labeling.
Differently from other transformers models, GPT 3 is trained not only with articles but sentence and paragraph patterns. So, the same way a beginner language learner would understand much easier a single sentence or a paragraph instead of a whole article, the GPT 3 model developed a good understanding of the natural language thanks to this training method.
Before self-supervised algorithms, supervised technics used which separated sentences into nouns, adjectives, and verbs… Same as when I was told to study only grammar rules to learn a foreign language, these technics were limiting the language understanding.
What it can change?
Thanks to its specific training model, GPT 3 creates human-like texts but also they can compose music, write poetry, translate and code. It’s also difficult to differentiate either the outputs are created by a human or algorithms.
This single evolution can facilitate web development, text translation, content creation, image recognition, and everything else that works with data and algorithms…
Today, a GPT 3 model contains 175 billion parameters which is a lot more than its predecessor GPT2. Even though this amount is huge, compared to a human brain with around a hundred trillion synapses, there is still quite a way to go for machines to reach a human brain’s capacity.