What are large language models (LLMs)?

A large language model (LLM) is a deep learning algorithm that can perform a variety of natural language processing (NLP) tasks. Large language models use transformer models and are trained using massive datasets — hence, large. This enables them to recognize, translate, predict, or generate text or other content.

Large language models are also referred to as neural networks (NNs), which are computing systems inspired by the human brain. These neural networks work using a network of nodes that are layered, much like neurons.

In addition to teaching human languages to artificial intelligence (AI) applications, large language models can also be trained to perform a variety of tasks like understanding protein structures, writing software code, and more. Like the human brain, large language models must be pre-trained and then fine-tuned so that they can solve text classification, question answering, document summarization, and text generation problems. Their problem-solving capabilities can be applied to fields like healthcare, finance, and entertainment where large language models serve a variety of NLP applications, such as translation, chatbots, AI assistants, and so on.

Large language models also have large numbers of parameters, which are akin to memories the model collects as it learns from training. Think of these parameters as the model's knowledge bank.

A transformer model is the most common architecture of a large language model. It consists of an encoder and a decoder. A transformer model processes data by tokenizing the input, then simultaneously conducting mathematical equations to discover relationships between tokens. This enables the computer to see the patterns a human would see were it given the same query.

Transformer models work with self-attention mechanisms, which enables the model to learn more quickly than traditional models like long short-term memory models. Self-attention is what enables the transformer model to consider different parts of the sequence, or the entire context of a sentence, to generate predictions.

Large language models are composed of multiple neural network layers. Recurrent layers, feedforward layers, embedding layers, and attention layers work in tandem to process the input text and generate output content.

The embedding layer creates embeddings from the input text. This part of the large language model captures the semantic and syntactic meaning of the input, so the model can understand context.

The feedforward layer (FFN) of a large language model is made of up multiple fully connected layers that transform the input embeddings. In so doing, these layers enable the model to glean higher-level abstractions — that is, to understand the user's intent with the text input.

The recurrent layer interprets the words in the input text in sequence. It captures the relationship between words in a sentence.

The attention mechanism enables a language model to focus on single parts of the input text that is relevant to the task at hand. This layer allows the model to generate the most accurate outputs.

There are three main kinds of large language models:

• Generic or raw language models predict the next word based on the language in the training data. These language models perform information retrieval tasks.
• Instruction-tuned language models are trained to predict responses to the instructions given in the input. This allows them to perform sentiment analysis, or to generate text or code.
• Dialog-tuned language models are trained to have a dialog by predicting the next response. Think of chatbots or conversational AI.

What is the difference between large language models and generative AI?

Generative AI is an umbrella term that refers to artificial intelligence models that have the capability to generate content. Generative AI can generate text, code, images, video, and music. Examples of generative AI include Midjourney, DALL-E, and ChatGPT.

Large language models are a type of generative AI that are trained on text and produce textual content. ChatGPT is a popular example of generative text AI.

All large language models are generative AI¹.

A large language model is based on a transformer model and works by receiving an input, encoding it, and then decoding it to produce an output prediction. But before a large language model can receive text input and generate an output prediction, it requires training, so that it can fulfill general functions, and fine-tuning, which enables it to perform specific tasks.

Training: Large language models are pre-trained using large textual datasets from sites like Wikipedia, GitHub, or others. These datasets consist of trillions of words, and their quality will affect the language model's performance. At this stage, the large language model engages in unsupervised learning, meaning it processes the datasets fed to it without specific instructions. During this process, the LLM's AI algorithm can learn the meaning of words, and of the relationships between words. It also learns to distinguish words based on context. For example, it would learn to understand whether "right" means "correct," or the opposite of "left."

Fine-tuning: In order for a large language model to perform a specific task, such as translation, it must be fine-tuned to that particular activity. Fine-tuning optimizes the performance of specific tasks.

Prompt-tuning fulfills a similar function to fine-tuning, whereby it trains a model to perform a specific task through few-shot prompting, or zero-shot prompting. A prompt is an instruction given to an LLM. Few-shot prompting teaches the model to predict outputs through the use of examples. For instance, in this sentiment analysis exercise, a few-shot prompt would look like this:

Customer review: This plant is so beautiful!
Customer sentiment: positive

Customer review: This plant is so hideous!
Customer sentiment: negative

The language model would understand, through the semantic meaning of "hideous," and because an opposite example was provided, that the customer sentiment in the second example is "negative."

Alternatively, zero-shot prompting does not use examples to teach the language model how to respond to inputs. Instead, it formulates the question as "The sentiment in ‘This plant is so hideous' is…." It clearly indicates which task the language model should perform, but does not provide problem-solving examples.

In addition to these use cases, large language models can complete sentences, answer questions, and summarize text.

With such a wide variety of applications, large language applications can be found in a multitude of fields:

• Tech: Large language models are used anywhere from enabling search engines to respond to queries, to assisting developers with writing code.
• Healthcare and Science: Large language models have the ability to understand proteins, molecules, DNA, and RNA. This position allows LLMs to assist in the development of vaccines, finding cures for illnesses, and improving preventative care medicines. LLMs are also used as medical chatbots to perform patient intakes or basic diagnoses.
• Customer Service: LLMs are used across industries for customer service purposes such as chatbots or conversational AI.
• Marketing: Marketing teams can use LLMs to perform sentiment analysis to quickly generate campaign ideas or text as pitching examples, and much more.
• Legal: From searching through massive textual datasets to generating legalese, large language models can assist lawyers, paralegals, and legal staff.
• Banking: LLMs can support credit card companies in detecting fraud.

Get started with Generative AI in Enterprise. Watch this webinar and explore the challenges and opportunities of generative AI in your enterprise environment.

With a broad range of applications, large language models are exceptionally beneficial for problem-solving since they provide information in a clear, conversational style that is easy for users to understand.

Large set of applications: They can be used for language translation, sentence completion, sentiment analysis, question answering, mathematical equations, and more.

Always improving: Large language model performance is continually improving because it grows when more data and parameters are added. In other words, the more it learns, the better it gets. What’s more, large language models can exhibit what is called "in-context learning." Once an LLM has been pretrained, few-shot prompting enables the model to learn from the prompt without any additional parameters. In this way, it is continually learning.

They learn fast: When demonstrating in-context learning, large language models learn quickly because they do not require additional weight, resources, and parameters for training. It is fast in the sense that it doesn’t require too many examples.

Popular large language models have taken the world by storm. Many have been adopted by people across industries. You've no doubt heard of ChatGPT, a form of generative AI chatbot.

Other popular LLM models include:

• PaLM: Google's Pathways Language Model (PaLM) is a transformer language model capable of common-sense and arithmetic reasoning, joke explanation, code generation, and translation.
• BERT: The Bidirectional Encoder Representations from Transformers (BERT) language model was also developed at Google. It is a transformer-based model that can understand natural language and answer questions.
• XLNet: A permutation language model, XLNet generated output predictions in a random order, which distinguishes it from BERT. It assesses the pattern of tokens encoded and then predicts tokens in random order, instead of a sequential order.
• GPT: Generative pre-trained transformers are perhaps the best-known large language models. Developed by OpenAI, GPT is a popular foundational model whose numbered iterations are improvements on their predecessors (GPT-3, GPT-4, etc.). It can be fine-tuned to perform specific tasks downstream. Examples of this are EinsteinGPT, developed by Salesforce for CRM, and Bloomberg's BloombergGPT for finance.

The arrival of ChatGPT has brought large language models to the fore and activated speculation and heated debate on what the future might look like.

As large language models continue to grow and improve their command of natural language, there is much concern regarding what their advancement would do to the job market. It's clear that large language models will develop the ability to replace workers in certain fields.

In the right hands, large language models have the ability to increase productivity and process efficiency, but this has posed ethical questions for its use in human society.

To address the current limitations of LLMs, the Elasticsearch Relevance Engine (ESRE) is a relevance engine built for artificial intelligence-powered search applications. With ESRE, developers are empowered to build their own semantic search application, utilize their own transformer models, and combine NLP and generative AI to enhance their customers' search experience.

Supercharge your relevance with the Elasticsearch Relevance Engine