Machine learning (ML) is a type of algorithm that automatically improves itself based on experience, not by a programmer writing a better algorithm. The algorithm gains experience by processing more and more data and then modifying itself based on the properties of the data.
Types of machine learning
There are many varieties of machine learning techniques, but here are three general approaches:
- reinforcement learning: The algorithm performs actions that will be rewarded the most. Often used by game-playing AI or navigational robots.
- unsupervised machine learning: The algorithm finds patterns in unlabeled data by clustering and identifying similarities. Popular uses include recommendation systems and targeted advertising.
- supervised machine learning: The algorithm analyzes labeled data and learns how to map input data to an output label. Often used for classification and prediction.
Let’s dive into one of the most common approaches to understand more about how a machine learning algorithm works.
Neural networks
An increasingly popular approach to supervised machine learning is the neural network. A neural network operates similarly to how we think brains work, with input flowing through many layers of “neurons” and eventually leading to an output.
Diagram of a neural network, with circles representing each neuron and lines representing connections between neurons. The network starts on the left with a column of 3 neurons labeled “Input”. Those neurons are connected to another column of 4 neurons, which itself connects to another column of 4, and those neurons are labeled “Hidden layers”. The second hidden layer of neurons is connected to a column of 3 neurons labeled “Output”.
Training a network
Computer programmers don’t actually program each neuron. Instead, they train a neural network using a massive amount of labeled data.
The training data depends on the goal of the network. If its purpose is to classify images, a training data set could contain thousands of images labeled as “bird”, “airplane”, etc.
A grid of images in 10 categories (airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck).
Images from the CIFAR10 training data set. Image source: CIFAR10
The goal of the training phase is to determine weights for the connections between neurons that will correctly classify the training data.
A diagram of a neural network classifying an image of a plane. Parts of the image are fed into the first layer of neurons, those neurons lead to a middle layer, and those neurons lead to a final layer of neurons. Each edge between neurons is labeled with a question mark, denoting an unknown weight.
The weights between the neurons are unknown (labeled with a “?” here), and the neural network wants to find weights that will result in classifying each image correctly.
The neural network starts off with all the weights set to random values, so its initial classifications are way off. It learns from its mistakes, however, and eventually comes up with a set of weights that do the best job at classifying all of the training data.
A diagram of a neural network classifying an image of a plane. Parts of the image are fed into the first layer of neurons, those neurons lead to a middle layer, and those neurons lead to a final layer of neurons. Each neuron has a weight (from 0 to 1). In the final layer, the neuron labeled “plane” has the highest weight.
Each of the connections between neurons is assigned a weight (represented by shades of green). A neuron multiplies each connection weight by the value of the input neuron, and sums up all of those to come up with a single number (shown on each neuron). The neuron will only send its value to the next layer if it’s above a threshold.
Using the network
When the neural network is asked to classify an image, it uses the learned weights and outputs the possible classes and their probabilities.
Diagram of a neural network, with circles representing each neuron and lines representing connections between neurons. The network starts on the left with an image of a fox. The image is broken into 4 parts, and those parts are connected to column of 4 neurons, which itself connects to another column of 4. The second column is connected to 3 possible outputs: “Fox (0.85)”, “Dog (0.65)”, and “Cat (0.25)”.
Accuracy
The accuracy of a neural network is highly dependent on its training data, both the amount and diversity. Has the network seen the object from multiple angles and lighting conditions? Has it seen the object against many different backgrounds? Has it really seen all varieties of that object? If we want a neural network to truly understand the world, we need to expose it to the huge diversity of our world.
Companies, governments, and institutions are increasingly using machine learning to make decisions for them. They often call it “artificial intelligence,” but a machine learning algorithm is only as intelligent as its training data. If the training data is biased, then the algorithm is biased. And unfortunately, training data is biased more often than it’s not.
In the following articles, we’ll explore the ramifications of letting machines make decisions for us based on biased data.