Last year OpenAI (California), backed by Sam Altman and Elon Musk amongst other tech entrepreneurs, revealed to the world the largest language model ever with overwhelming results. Our research team got early access to the technology and has been reflecting on what the future of human-artificial intelligence cooperation might look like.
Recently, our team got also access to a downstream language model from this technology - a model that translates natural language into source code in the programming language of your choice. Details are presented below.
Research must advance towards models learning from unlabeled data - GPT-3 is a great example - also aligned with Yan LeCun's (a notable computer scientist) vision: "The next AI revolution will not be supervised". This unsupervised learning has several benefits such as larger amounts of databases, no label bias is inherited or no need of microworkers on data tagging processes.
Model Interface
GPT-3's most astounding aspect is that it is a meta-learner, meaning it has learned how to learn. You can ask it to execute a new task in normal language, and it will "understand" what it has to do in a similar way to how a human would.
OpenAI released a beta API that works with a natural language interface. Developers condition the model to their particular case by means of text - called a prompt. Thanks to its meta-learning capacity, the model is able to understand the pattern of the task and generate an answer accordingly.
Unsupervised learning is a learning environment where model learns patterns from unlabeled data.
This is GPT-3's natural language interface. A prompt (bold text) provides some examples of the task to be solved: English to French translation.
Finally, the model is able to translate What's going on tonight? to French thanks to these few samples.
This is a particular case in which we - Batou - are very interested and we are also developing our own tools.
Conclusions
This post’s main goal has been to explain our experience with one of the most outstanding technologies in the AI industry.
Basically, humans are responsible for coming up with new ideas and it is AI's job to understand and turn those ideas into applications, not the other way around.
Augmenting human capabilities should be the ultimate goal of AI.