Large language models, or LLMs, are powerful AI models specially designed to understand, use, and generate human language. LLMs exist where AI and linguistics meet, in a subfield known as natural language processing, or NLP. (In fact, LLMs have completely revolutionized the field.)
Today’s language-processing AI models are mostly deep learning neural networks, which often use a novel approach called transformer architectures. In this article, we’ll provide a brief overview of each of these terms, proceeding from general to specific.
What is deep learning?
Deep learning is a type of machine learning that involves using many layers of computation to enable more complex problem-solving. These many layers are able to work together thanks to their arrangement in a “neural network,” which we’ll discuss in more detail later in this article. Where a basic machine learning model might only have two layers—input and output—neural networks have additional hidden layers in between. By leveraging many of these layers, deep learning models can better capture complex relationships in data; each layer is responsible for one task, and each successive layer can build upon the discoveries of the previous layers.
What exactly are neural networks?
This type of multi-layered deep learning is enabled by something called an artificial neural network: a computer network inspired by the structure of the human brain. Each layer of a neural network is made up of several nodes (or “neurons”) that perform specific functions. Each neuron has an associated weight and is connected with every neuron in the previous layer. Each neuron receives input from the previous layer, performs its own computation, and passes the output to the neurons in the next layer. With this structure, the entire deep learning model becomes capable of discovering sophisticated patterns and relationships—making them especially good at tasks like the ability to process human language.
What are transformers?
Transformers are a particular type of neural network architecture (of which there are many). Transformers excel at language-processing tasks by making use of “self-attention mechanisms,” which enable them to model semantic relationships from an input (i.e. a sequence of text) and understand the relationships between that input’s words and phrases.
Transformers essentially assign attention weights to each word (or “token”) based on its relevance to other tokens in a sequence. That means words in a sentence are processed contextually—in relation to one another—as opposed to in isolation. This also means that transformers are able to more efficiently process inputs, because they can process an entire sequence in parallel rather than sequentially.
Attention mechanisms also afford transformers a more robust “memory” to better remember dependencies (i.e. previous parts of a conversation) and carry them forward in a dialogue, using them to inform the model’s continued outputs or responses. This also opens the door for more use cases, as some LLMs can process up to ~100,000 tokens in a single input, enabling a model to respond to much more technically complex prompts.
Summary: key points about LLMs
- State-of-the-art LLMs are machine learning (specifically deep learning) models—meaning they pass information between many different layers to achieve complex computations.
- To accomplish this multi-layered computation, LLMs make use of artificial neural networks (transformer architectures in particular, because they’re so well-suited for language tasks).
- All this, of course, falls under the much larger umbrella of AI in general (and also within the field of natural language processing).
How are LLMs developed and trained?
Generally speaking, LLMs are trained the same way as any other NLP-capable machine learning model (which we covered in more detail elsewhere). But to summarize, LLMs learn to understand human language by training on vast sets of textual data. LLMs are trained to predict the next word in a sequence. Thanks to this learning, LLMs can model language, and thus develop an “understanding” of the complex relationships, statistical patterns, grammar, and semantics of language.
The main difference between an ordinary language model and a large language model is that LLMs have more trainable components or variables (called “parameters”). As a consequence of having more parameters, LLMs tend to utilize more training data. They also tend to rely on network architectures (e.g. transformer networks) that enable more efficient processing, and the ability to model better long-term relationships in the data. (It’s also worth noting that larger models require more resources than smaller ones—both in terms of costs associated with training, and with deployment and everyday usage.)
What makes a language model “large?”
As their name suggests, the key characteristic of LLMs is their size. The size of an LLM boils down to the number of parameters it has. The parameters of an LLM include its weights (and biases), which are values associated with the neurons of a network, and how much those neurons influence the output per layer. Throughout the training process, the model will iteratively adjust the values of parameters to optimize for the desired output.
Language models come in many different shapes and sizes, and can be fine-tuned for specific purposes, but the most impressive and powerful models have been trained on vast amounts of data, and have billions of parameters. For example, OpenAI’s GPT-3 model (of ChatGPT fame) was trained on ~45 terabytes of data (comprising ~300 billion tokens), and has ~175 billion parameters. Other notable LLMs include Meta’s LLama, Anthropic’s Claude, and Google’s PaLM (which powers the Bard chatbot).
What are LLMs used for?
LLMs power the most sophisticated virtual assistants/chatbots, ones capable of communicating with humans via natural-language interfaces. Through these interfaces, LLMs can answer questions, generate novel content, and more.
But LLMs also excel at a variety of other natural language processing tasks, such as:
- Summarizing articles, videos, or other content
- Boosting search engine functionality
- Increasing readability in word-processing programs (e.g. autocorrect/spellcheck)
- Translating text between languages
- Filtering important or potentially dangerous emails based on their content
- Analyzing sentiment from content like reviews, comments, and surveys
Basically, LLMs represent the state of the art in NLP; they’re capable of handling very complex natural language tasks, and providing results that—just a few years ago—would have been unthinkable.
That said, large language models are only as good as the data they’re trained on, which is why it’s important to use high-quality training data. If you’re interested in building or working with LLMs, check out the Brave Search API as a high-quality data source. The Brave Search API gives you access to the Brave Search index, which contains billions of webpages backed by real human visits as a proxy for quality content. The API offers a variety of plans, including ones tailored specifically to AI use cases.
