Transforming AI with the Power of Generative Networks
Dr. Emile-Reyn Engelbrecht and Professor Johan du Preez showed that generative networks used for semi-supervised learning and for recognising unseen categories work in the same way.
Read the original paper
The evolution of machine learning techniques continues to push the boundaries of what is possible with artificial intelligence. In a groundbreaking study, researchers Professor Johan du Preez and Dr. Emile-Reyn Engelbrecht from Stellenbosch University have bridged the conceptual gap between two significant areas of machine learning: semi-supervised learning (SSL) and generative open-set recognition (OSR). Their findings, published in Scientific African, reveal a profound connection through the use of generative adversarial networks (GANs), which could lead to more cost-efficient and effective machine learning models.
At the heart of their research lies the innovative use of GANs, a dynamic tool that traditionally pits two neural networks against each other: one to generate data and one to evaluate it. The study delves into how these networks can be utilized not just for SSL, where only part of the data is labeled, but also for OSR, which requires the identification of novel, previously unseen categories during the testing phase.
The researchers hypothesized that the key to linking SSL and OSR lies in the generation of what they term ‘bad-looking’ samples—data points that are intentionally crafted to be ambiguous or misleading. These samples populate the ‘complementary space,’ a conceptual area in the data spectrum that lies between known categories. By training classifiers with these samples, the models can not only recognize but also appropriately categorize novel inputs that they weren’t explicitly trained on.
Dr. Engelbrecht explains, “By extending what we understand about the complementary space in SSL to OSR, we’ve found that our models can effectively generalize this open space, significantly enhancing their ability to deal with unexpected data.” This revelation is crucial for applications where encountering unknowns is common, such as in automated diagnostic tools and self-driving car technology, where a misclassification could have serious, if not fatal, consequences.
The study conducted extensive comparisons between foundational SSL-GANs and state-of-the-art OSR-GANs under identical experimental conditions. The results were strikingly similar, thereby substantiating the researchers’ theory that the underlying mechanisms governing both SSL and OSR are interconnected through their treatment of the complementary space.
Furthering this line of inquiry, the team experimented with various GAN models to determine which configurations offer optimal performance in SSL-OSR scenarios. Among the models tested, Margin-GANs stood out, providing superior results thanks to their refined approach to defining and exploiting the complementary space.
The implications of this research are vast, suggesting that the integrated framework of SSL-OSR not only simplifies the training process but also enhances the functionality of machine learning systems, making them more adaptable and efficient in real-world applications. As the field of artificial intelligence continues to evolve, studies like this pave the way for more robust, versatile systems capable of handling the complexities and unpredictabilities of real-world data.
About the researchers
Journal reference
Engelbrecht, E.-R., & du Preez, J.A. “On the link between generative semi-supervised learning and generative open-set recognition.” Scientific African, 2023. DOI: https://doi.org/10.1016/j.sciaf.2023.e01903




