AI Explained – Model Training, Fine-Tuning, and Grounding

See part 1 of this series here: AI for the Business User – Part 1 – Marquee Insights

Hi everyone! I hope you enjoyed Microsoft Build and other recent Microsoft conferences. We loved listening to Arun Ulag and Jeff Teper talk about how Microsoft’s AI Copilot will revolutionize the way we work. Some of my fellow audience members weren’t crystal clear on the AI terms used in the presentations. This led to a number of people asking about data security and “Is Microsoft using my data to train their model?” Don’t worry, we’re here to explain what those terms mean and how Copilot will use grounding techniques protects your data. Let’s ensure you have all the information you need to use AI with confidence.

Let’s start with model training, fine-tuning, and grounding, three important concepts in machine learning that help us create intelligent AI systems that can learn from data and perform various tasks.

Model training

So, what is Model training? It is the process of teaching an AI model how to make decisions based on data. An AI model is a program that uses mathematical algorithms to recognize patterns and make predictions. For example, an AI model can learn to identify cars in images by looking at many examples of labeled car images.

To train an AI model, we need two things: a large amount of data and a loss function. Data, in this example, would be the car images needed to train a model. This could require millions of images, depending on the model and level of desired capabilities. A loss function is a way of measuring how well the model’s predictions match the actual outcomes or labels. For example, if the model predicts that an image is a car, but the label says it is a truck, the loss function will give a high penalty to the model.

The goal of training is to find the best values for the parameters of the model that minimize the loss function. Parameters are the numbers that control how the model processes the data. Neural networks are a type of AI model, composed of layers of artificial neurons (or nodes). Parameters are the weights and biases used to connect the neurons. For example, GPT-3 has 175 billion parameters.

Model training can be a long and expensive process. GPT-4 that powers ChatGPT costs $100 million to train. The base model training costs incent most companies to build upon and use pretrained models.

Model fine-tuning

You may have heard the term “fine-tuning” and thought, isn’t that a radio term? In AI, model fine-tuning is a technique for adding a new capability to a model.

Sometimes, we do not have enough data, time, or money to train an AI model from scratch. We instead use a pre-trained model, already trained on a large and general dataset, then adapt it to our specific needs. For example, we want to use our pretrained car model above to recognize trucks.

Fine-tuning involves copying all or some of the parameters from the pre-trained model and updating them based on our new data. We can also add new layers or components to the model to customize it for our task. For example, if we want to use our car image recognition model for a truck recognition task, we can add a new output layer that predicts different trucks based on the images.

An example of AI model fine-tuning is natural language processing (NLP). NLP is the field of AI that deals with understanding and generating natural language, such as text and speech. There are many NLP tasks, such as sentiment analysis, machine translation, text summarization, etc. A large language model (LLM), such as GPT-4, is already pre-trained on a huge amount of text for different NLP tasks. We can fine-tune GPT-4 by adding a task-specific layer and updating its parameters with our data.

Realistically, fine-tuning is another expensive process, performed mostly by software companies that sell AI services. They are using the pre-trained models from OpenAI and others and extending them for their own use. Even though we are extending the model above to recognize trucks, this may not be the most economical path forward.

Model grounding

Model grounding is a much more economical technique for connecting an AI model’s output to real-world facts or context. For example, I want to have our car model know everything there is to know about Ford trucks. We would ground the model with Ford truck specific photos.

AI models are stateless and do not have a way to extend their learning automatically the way humans do. Therefore, grounding allows us to securely provide our information to the model for use. This information is not incorporated into the model’s training itself, eliminating the concern that Microsoft is co-opting your data.

The benefit to us is that we want our AI models to not only generate outputs based on data but also verify them against external sources or user input. This helps ensure our models are reliable and accurate and avoid errors or hallucinations.

Hallucinations are another term that gets thrown about, so what exactly is it? Hallucination is the term used when the AI model provides a response that is not supported by its data. You’ve likely experienced this if you’ve ever asked your 4-year-old child to tell you a story about a subject on which they have little knowledge. The child will be thrilled to tell you a story, filled with great sounding but completely made-up elements.

Thankfully, later models like GPT-4 are much less likely to hallucinate. It is recommended that you use a combination of prompts that can direct the model how to respond when certainty is not high and “human in the loop” review techniques like that used by Microsoft Viva Topics, to reduce and catch hallucinations when they occur. It would be very bad if I were chatting with my health insurance AI chatbot and it told me a treatment was 100% covered when it wasn’t.

Hallucinations are not necessarily bad if you use them in the proper context. Rory Sutherland, Vice Chairman at Ogilvy UK, was discussing with Jason Calacanis on the This Week in Startups podcast that using AI hallucinations as inputs for brainstorming may lead to better idea generation. Humans tend to self-edit their brainstorming suggestions based on group social norms. Ideas generated by the AI don’t carry the same stigma, leading to more interesting suggestions. It’s a use case we’ll be exploring further in a later post, especially after we give rosemary Dr. Pepper a try, as suggested by ChatGPT on the podcast.

Grounding can guide or constrain the model’s output. For example, if we want our AI models to answer questions based on Ford Truck facts or knowledge bases, we can ground the output by checking it against the Ford Truck sources we provide.

An example of AI model grounding is natural language understanding (NLU). NLU is the field of AI that deals with interpreting and extracting meaning from natural language. There are many NLU tasks, such as answering questions, chat systems, information extraction, etc. We can ground our NLU models by linking their output to real-world entities or actions and using them to execute commands or queries. For example, if we want our NLU models to help users buy a Ford truck, we can ground their output by connecting it to a database or an API that provides the relevant information about the available trucks.

Summary

Hopefully, you have found this explanation of model training, fine-tuning, and grounding to be useful. In a future post, we’ll examine another emerging technology, AI Orchestration. It enables you to tie multiple AI technologies together to create an end-to-end workflow. We are extremely excited by this technology. We also realize that you have to be careful not to create “chaos at ludicrous speed” within an organization.