In the past, one way AI researchers have tried to deal with this problem has been to use the data multiplication approach. Using an image algorithm as an example again, in cases where there is not much material to work with, they will try to deal with the problem by creating “distorted” copies of what is available. Distortion, in this case, means cropping an image, rotating or flipping it. The idea here is that the network does not see the same image twice.
The problem with that approach is that it can lead to a situation where GAN learns to reflect those distortions, rather than creating something new. Nvidia’s new Adaptive Discrimination Multiplication (ADA) approach still uses data augmentation, but adapts accordingly. Instead of distorting images throughout the entire training process, GAN avoids excessive material because it is selective and adequate.
The potential impact of Nvidia’s approach makes more sense than you might think. Teaching AI to write new Text based adventure game Easy, because the algorithm has a lot of material to work with. This does not apply to many tasks that researchers may turn to GANs for assistance. For example, teaching a method to diagnose a rare neurological brain disorder is difficult precisely because of its rare cause. However, a GAN trained with Nvidia’s ADA approach may come around to that problem. As an added bonus, doctors and researchers can easily share their findings because they work from an AI-generated image base, not patients in the real world. Nvidia will be sharing more information about its new ADA approach in the near future Neurips Conference, Which starts on December 6th.
