With all the recent hype around AI chatbots, it’s easy to forget that these systems aren’t quite fully human. Murati hopes GPT-4 can eventually do more “brain-like things” than its predecessors, which would require a deeper understanding of language and thinking.
At its core, GPT-4 is a simple neural network. Yet somehow it manages to capture some of the complexities of language (and thinking).
1. It’s a natural language model
ChatGPT is a natural language model that uses machine learning to understand and interpret human speech and text. NLP is one of the main branches of artificial intelligence, and it can be used to perform a variety of tasks, such as speech recognition, translation, and text processing.
While it may seem complicated, the underlying principles of NLP are actually quite simple. The basic idea is that a neural network is trained to recognize patterns in language, and then programmed to develop algorithms that represent these patterns. The end result is an intelligent system that can carry out specific tasks, such as responding to user prompts.
To train GPT-4, it is given a large amount of existing text that it can use to learn what words mean and how they are used in different contexts. Then, it is fine-tuned using a set of guidelines provided by human reviewers, who provide conversations and rank the model’s responses. Once it is trained, ChatGPT can then start generating text on its own to respond to user prompts.
When a user provides a prompt, ChatGPT analyzes the text and divides it into smaller units known as tokens. Each token represents a word or a subword. Then, the model applies a machine learning algorithm to each token and generates a new sequence of text that sounds most likely to match the user’s prompt.
The NLP system then assembles the sequence into a complete response and transmits it to the user through a chat app. Users can then give the model feedback by rating the response with a thumbs-up or thumbs-down icon, and the model will continue to improve its output based on this data.
2. It’s a deep learning model
The generative model in ChatGPT is the engine that drives its responses to your questions. It takes a prompt in the form of a sequence of tokens and uses its knowledge base to produce a response that will be most likely to satisfy the question.
This process is aided by context, which helps the model understand the meaning and intent of a particular word or phrase. This is a crucial part of the process, because even if the machine can understand your question, it will struggle to respond unless it has an understanding of what the question is asking for in its entirety.
Unlike a predictive text algorithm on your phone, which is essentially bluntly guessing the next word to be produced, ChatGPT can attempt to construct full sentences, paragraphs and stanzas that are fully coherent with your prompt. This is a result of its pretraining with massive amounts of existing text data from the web and other GPT-4.
The model uses the transformer architecture, which is a type of neural network designed specifically to process sequential data like text. During pretraining, the model learns to predict the next word in a sentence by looking at what was said before it and the contextual environment in which it was spoken.
One thing to note about this step is that it can be an iterative process, which means that you can continue to feed the model more and more training data so that it continues to improve. Ideally, this would allow the model to get closer and closer to the correct answer for any given prompt, but there’s a chance that it may never reach a perfect answer.
3. It’s a generative model
One of the most impressive things about ChatGPT is its linguistic quality, apparent nuance and ability to respond to a wide range of prompts. For those without significant experience with large language models or generative AI, it can be a bit confusing to try and figure out what exactly is going on inside of such sophisticated (albeit inevitably somewhat arbitrary) “engineering choices”.
What’s inside ChatGPT is an artificial neural network that uses a specialized type of learning algorithm called a transformer to generate text responses to your prompts. The GPT-4 in the name stands for Generative Pre-trained Transformer, and the “gen” part refers to a set of generative rules that it’s learned from the training data it’s been exposed to.
A transformer is a specialized form of neural network that has a special capability for finding long-range patterns in sequences of data. It is this sort of pattern recognition that allows it to generate more natural-sounding text than a more traditional neural network.
The exact number of parameters or variables that are in the GPT-3 model isn’t publicly known, but OpenAI has said it has at least 175 billion. Once it has your prompt, it runs the transformer against this vast repository of previously generated text to find a match.
As it does so, it “reads” the entire sequence of tokens that have come before it, including those it has written itself. Unlike many typical computational systems, such as the brain or current computers, this setup means that it doesn’t internally “have loops” and reuse results. Instead, each time it “reads” the input sequence to produce a new output token, it reads all of it at once.
4. It’s a reinforcement learning model
GPT-4 is a large language model that can generate various types of text. It can write scripts, poems, essays, and even computer code. It can even pass the Turing test, which determines whether a machine is intelligent. It has many potential applications, from game and app development to college-level essay writing and medical disease diagnosis. However, the technology raises several ethical concerns. Some people fear that it could replace their jobs.
The generative model is trained by using reinforcement learning, which is a type of machine learning that uses reward functions to train a policy. The model is given feedback from human trainers who provide input-output pairs. Based on this feedback, the model adjusts its parameters to improve performance. This process is iterated until the model reaches acceptable accuracy.
To produce text, the model begins by tokenizing the input into words or phrases. It then calculates the probability that a word will follow another, and then adds or subtracts from these probabilities to create the final output. This process is iterated over time, adjusting the probability of each word and the overall text based on user feedback.
As a result, the system learns how to generate a text with the highest probability of being accepted by human reviewers. The system also learns how to correct mistakes, such as misspellings or grammatical errors.
Although GPT-4 has been successful at generating natural-sounding text, it is still limited by its training data. The training data used by the model can contain bias and may skew the results. This can lead to misinformation and inaccurate information. It also has trouble with phrasing that is offensive or discriminatory, and it does not understand the meaning of some words. These limitations will need to be addressed as the generative AI model is widely adopted.
5. It’s a recurrent neural network
ChatGPT uses a recurrent neural network to generate natural, conversational responses. This type of model learns from patterns and information gathered during training. It can ask follow-up questions to clarify your intent or better understand your needs, and it can provide personalized answers that take into account the context of the conversation.
During pretraining, the model is fed large quantities of existing text data from websites, books and other sources. The text data is tokenized into small units called “tokens” that represent words or parts of words. Then the model analyzes this input and determines a probability distribution for each token. This information is then used to predict the next token in the sequence, as well as the overall context of the text.
The model then combines these probabilities to generate the correct response to your question or statement. For example, if you ask, “How is the weather today?”, the model might respond with, “The weather is sunny and clear.”
Once the system is fully trained, it’s fine-tuned using real human feedback. This is an important step because the training data can contain biases that might be reflected in the model’s output. This is especially true of chatbots that use a limited amount of knowledge to write plausible-sounding responses, which can perpetuate stereotypes and cause discrimination.
Once the GPT-4 system is ready, users can interact with it by asking questions and giving a thumbs up or down to the responses. This feedback will be used to train the model and improve its performance. In addition, users can also report issues or errors with the system. This helps to ensure that the software is accurate, fair, and ethical.
