Training AI

Now that we have covered the importance of selecting the right training data and architecture, it is time to see how it all comes together. The final step is the process of training the AI. You built the building, you bought the furniture, and now it is time to arrange it. In this chapter we will discuss several relevant topics regarding training AI. In the context of GAs we will explain the importance of fitness functions and also relate this more broadly to reinforcement learning. After that we will look into the difference between supervised and unsupervised learning.

While there are some differences between these 2 concepts, they have been grouped together since the AI learns from interacting with an environment. Fitness functions are prominent in GA as the solutions need to be ranked in some way so the parents can be selected. It is therefore important to consider what exactly the most effective fitness function is.

Let’s pretend for a moment that Google maps used an AI for finding the shortest path. What would the fitness function be? Would it be the shortest distance? Least amount of fuel consumption? Least amount of time to get to the destination? There are always trade-offs – it is rather simple to calculate the distance, but if you want to calculate the time you need to consider the maximum speed of each road, traffic lights, road traffic, road closures, etc. This might take longer for the system to score the solutions and slow down the evolution process. It is important to have an appropriate fitness function, however, as it determines the results you will get. The answers for fastest path and fuel efficiency might even be different.

For example if you are training a chess AI and you tell it the number of pieces on the board are the fitness function, you might get checkmated despite not losing any pieces. That is why it needs to be carefully considered what fitness function to use.

Reinforcement learning is related in some ways to this as it includes fitness functions. It is prominent in simulated learning – where the AI is in a simulated environment and is attempting to achieve some goal. The concept of reinforcement learning is that the AI gets rewarded for doing the right thing and punished for doing the wrong thing. For example if the AI is a car in a racing game, it gets more points for reaching the finish line faster, it may also lose points for going in the wrong direction. In this way, every next generation the better solutions are kept and the worst ones do not reproduce. Once again the function used to score the AI should be carefully crafted.

The AI should be given enough room to figure things out as it may find a better way than you anticipate, but if you are too strict with the fitness function then it will find the solution you want which you can find yourself. In the same example of the racing game, if you give it points for going tightly around the corners you are being strict and not allowing it to experiment. It may be the case that going far from one corner and maintaining speed is more effective in the context of the whole race.

GAs, fitness functions, and reinforcement learning are all related since they use interactions with an environment to train the AI through telling it what is scored higher as a result. The other paradigm is not telling the AI whether it is doing well but rather just train it on a bunch of data and see what happens.

In supervised learning, despite the name, there is no human directly monitoring the training process. The difference comes from the type of data being fed to the AI. Supervised learning is when the data is labeled (usually by humans). This means that someone has provided the “correct answers,” much like studying for an exam by using the solutions to previous exam questions. This is beneficial when there is a clear result that is expected, and it produces AI models which are more predictable and less volatile (have less variance in accuracy and consistency).

Unsupervised learning is when the data is not labeled, and the AI scans for patterns within the data itself rather than making connections between the data and the labels. This is useful for when the expectations are more subjective, like in written English – there are many ways to say the same thing and none of them are inherently wrong. In this case it is impractical to label the data and is more effective to allow the AI to discover patterns on its own.

Now that we know the distinction between supervised and unsupervised learning is the type of data being given to the AI, let’s consider the applications of both. In advanced models like LLMs (ChatGPT) both are used to give the AI more capability, but what is each useful for?

The distinction is important as either type of learning comes with particular traits. Supervised learning may be expensive as it requires the data to be labeled – this is an intensive process due to the volume of data needed to train a complex model. It comes with the advantage that the AI will know the expected output as it is well defined and clearly structured. Unsupervised learning, on the other hand, can use any data you give it and try to construct patterns from that. It therefore is less explicit and structured in the results but is easier to train as it does not require special data. It is good at finding small nuances and patterns that are otherwise not obvious.