What is a Modular Neural Network? Weak vs. Strong Learners

Modular neural networks refer to artificial neural networks (ANNs) that are comprised of several different neural networks that are connected together in conjunction with an intermediary. To illustrate this point further, consider a consumer that owns multiple smart devices, such as a smartphone, a smartwatch, and a tablet such as an iPad, in addition to a laptop or desktop computer. Despite the different capabilities of these respective devices, they will all be connected to a modem or router that will enable the users of said devices to access online and mobile services in a fast and effective manner. On top of this, this online connectivity also enables users to combine the functionality of their various devices to achieve a particular goal, such as streaming a popular television program or making a phone call to a friend or family member, among other things.

With all this being said, modular neural networks give software developers the ability to leverage the power of individual neural networks in a more cohesive and efficient way. To do this, each neural network within a larger modular neural network will be used to solve a portion of a particular problem or issue. On this point, an intermediary known as an integrator will be used to organize and analyze these numerous modules to create the final output of the neural network. Through this configuration, simple neural networks can be implemented in a more sophisticated and complex fashion, and some common applications of these networks include image processing, high-level input data compression, and stock marketing forecasting software, just to name a few.

Ensemble learning

The concepts and ideas that have formed the foundation for the creation of modular neural networks were first theorized in the 1980s, and led to the development of a machine learning method that is referred to as ensemble learning. This method is predicated on the idea that weaker machine learning models can be combined together to create a single stronger model. Moreover, this collective approach can be used to produce more substantial results than those that could have been obtained through the use of a single deep learning model. Another way to conceptualize this process is the divide and conquer approach, where a large issue or problem is addressed by breaking said issue or problem into smaller parts that can be solved in a simpler or more viable way.

The structure of modular neural networks

In keeping with the idea of neural networks being based on the numerous functions and capabilities of the human brain, the human brain is comprised of a hierarchy of networks made up of millions of neurons, with each network being specialized to perform specific a job in relation to the functioning of the human body. To provide an example of this, consider 3 different feed-forward neural networks that have been trained to tackle problems regarding pattern recognition, such as license plate recognition. For any myriad of reasons, these 3 networks may struggle to analyze and identify license plates within videos or images on a consistent basis. As such, a software developer could combine the output of these three models to create a single output, with the goal of creating a single model that is more accurate than the previous 3 models.

Alternatively, software developers can also use modular neural networks to break down a machine learning problem itself into smaller more manageable portions. Going back to the example of license plate recognition, this problem could be broken down into three sub-problems, such as identifying license plates within images, identifying license plates within videos, and identifying license plates within videos that depict inclement weather, such as heavy rain or snowfall. Through these combined insights, a developer could create a deep learning algorithm that would be able to identify the occurrence of license plates with both images and videos that depict various weather conditions that a driver might experience when passing through a busy intersection or street.

The benefits of modular neural networks

Utilizing modular neural networks comes with a wide range of advantages and benefits. For instance, the combination of multiple neural networks that have been trained to accomplish a particular task can greatly cut down on the training time that is often needed to train neural networks, as well as the associated costs. Conversely, this approach also allows software developers to combine different types of machine learning models and approaches that could not have been possible using traditional methods. Finally, as has been stated, the combination of multiple neural networks can lead to the development of a single neural network that is able to perform in a much more accurate and efficient manner than any of the previous networks had been able to on their own.

Due to the fact that artificial neural networks, in general, have become a hot topic for both research and software development in the past few years, it is fitting that modular neural networks are an approach that is also increasingly in popular use. What’s more, as the cost of training machine learning models can be prohibitive in practice, modular neural networks allow software developers to create models in a much more cheaper and sustainable way. This being said, modular neural networks will continue to be utilized in the years to come, as approaches that had been viewed as inadequate in previous scenarios can be combined to create a single viable solution.