Ensemble Learning, AI, and Weak Vs. Strong Learners

Ensemble learning refers to a machine learning approach that focuses on using multiple machine learning algorithms to achieve a better predictive performance than could have been obtained through the utilization of single algorithms on its own. This being said, ensemble learning is predicated on the concept of “the wisdom of the crowd”, where a large group of people that may have an average amount of knowledge concerning a specific topic are still able to provide certain answers and insights regarding this topic, due in large part to the sheer number of people that are contributing to the end goal, as well as the diversity of these people. What’s more, this group can also perform more effectively than a smaller group of highly qualified experts.

How does ensemble learning work?

To illustrate the ways in which ensemble learning works, consider a software developer that is looking to create a machine learning algorithm that can automatically detect human faces within video recordings with 95% accuracy. To accomplish this goal, the developer in question might utilize four different machine learning models, including an artificial neural network, a support vector machine, a decision tree, and a linear regression model. However, after this software developer has finished training these four different algorithms, they may have all failed to reach the 95% accuracy threshold that was established at the beginning of the development process. Likewise, these four algorithms would be classified as weak learners in the context of ensemble learning.

This being the case, as opposed to creating a 5th machine learning algorithm, the software developer in this example could instead combine the 4 algorithms they have already developed into a single model. Next, the developer could run their inputs through this ensemble of algorithms, and then compute the average of the results. To this point, this developer could use this ensemble training approach to break the 95% accuracy threshold they established at the onset of the development process, without the need to obtain additional training data, much less develop another algorithm that could perform better than the first four algorithms had initially. Subsequently, this process is made possible because of the differing nature of the machine learning algorithms that were used to create the ensemble.

In going back to the example of a large group of people with an average amount of knowledge outperforming a small group of experts on a particular task, the diversity of the former means that they will cover each other’s weaknesses, while the specialized knowledge of the latter will inevitably mean that certain factors are not considered at all. For example, an artificial neural network may perform better on a certain data set than a decision tree would have. While these weaknesses become apparent when these algorithms are being used on their own, these blind spots can be covered when these algorithms are brought together in an ensemble, where one hand essentially washes the other.

Challenges of ensemble learning

Despite the numerous advantages of ensemble learning, there are a few notable drawbacks that are also associated with such an approach. Most notably, creating 4 different machine learning algorithms can be an extremely time-consuming and expensive process, as many software developers will not be able to afford such pursuits. Conversely, another issue that arises when using the ensemble learning approach in machine learning is transparency and explainability. Otherwise known as Black-box AI, it can be difficult to determine why a machine learning algorithm that has been trained using the ensemble approach has arrived at a particular decision, as combing a decision tree, an artificial neural network, a support vector machine, and a linear regression model would make it almost all but impossible make sense of the logic that has led to a particular result or decision.

As is the case with many other techniques and methods that are currently being utilized within the realm of machine learning, ensemble learning is perhaps best viewed as a tool that can be used to solve problems that are extremely nuanced and complex. Even before considering the time that is necessary to train 4 different machine learning models, the resources that are needed to fund such development would be out of reach for the overwhelming majority of the general population. Nevertheless, for those who can afford to support such an approach, ensemble learning can give software developers the extra push they need to accomplish goals that would otherwise seem to be out of reach.