Sharp palm fronds, cracks in asphalt, and reflections from the sun on car roofs: GTA V’s virtual Southern California still looks impressive years after the video game launched, especially in the PC version. At the same time, the simulated world in the game is still easy to distinguish from the real world.
Scientists at US chip maker Intel want to change that with image enhancement software. Researchers have a software model (PDF), Which uses artificial intelligence (AI) and makes the big streets of Los Santos look more realistic.
The team uploaded a video to YouTube Which indicates the improved graphic. Using the AI filter, asphalt looks more authentic, cars reflect light more beautifully, and trees look as if they were photographed with a camera from inside the car.
Stefan Richter worked on the project with two colleagues for two years. “Our goal was to make the game look as real as possible,” an Intel researcher said in an interview with SPIEGEL. “Often the simulations of games do not look realistic enough: the graphics are not yet sufficient to portray the real world.” A lot of work went into the project before the images looked what they do now.
The program forks up the information for the graphics card
In order to create clean images, the researchers performed a double check: on the one hand, the full game image is transmitted through an AI filter. However, at the same time, the program also uses so-called G buffers. This contains information for the graphics card to correctly calculate engineering details such as the distance between buildings and the camera and the position of the light source.
The researchers forked G-buffers to show the AI not to place trees in the sky. Finally, the resulting image is subject to a reality check. The software compares the result with street shots and assigns points of authenticity.
In order to map reality as accurately as possible, the program captures it, among other things Training photos of cityscapes back. This photo database contains streets from 50 predominantly German cities such as Erfurt, Cologne and Stuttgart. The pictures clearly indicate if the object is a pedestrian, traffic light or bus stop.
Not all is well
While the roads, trees, and cars really look more realistic, image enhancement reaches its limits with a few things – also because roadside palm trees are somewhat rare in this country. The program sometimes interprets the jacks as traffic lights and park benches are converted into wraparound umbrellas. According to a report by the scientists, pedestrians are also “less persuasively addressed”.
Despite these problems, the researchers believe the technology could bring more realistic graphics to games. “I can imagine engine developers using this program as a reality filter,” Richter says. “GTA is only a proxy here. It can be applied to many games.”
However, in its current form, the program is still very slow. Even with Nvidia’s top-of-the-line GeForce RTX 3090 graphics card, image optimization only yields the result after half a second. According to Richter, this is currently a prototype that “cannot be connected directly to a game at the moment”.
