Recognize gestures or how to interact with a computer or screen without touching it

A UCLA researcher has just perfected a technique that allows air gestures to be recognized. With a simple movement of the hand, we can control our screen or read a digital book.

The app may seem useless at first, but it is not. In the medical field, not touching equipment or even in shopping malls not having to touch the screen to find a store is more than useful, especially in times of Covid.

Imagine asking for your computer, mouse, looking for a good place to eat, wanting to browse your photo galleries or even like a Facebook post, all without touching anything, just by gesture.

Well, you are not dreaming. Arthur Slaughters, aspiring FNRS engineer in UCLouvain’s Department of Information Technology Engineering explains:For example, I love this turtle video, and by moving my finger towards the screen I can play it. If I move my finger left or right, I can move it forward or backward, and always with a simple gesture, I can even increase the volume. Simple how it works. I will make gestures over a sensor, and it will transmit this information to my computer, which, by receiving it, will decode the gesture I made and show the result on the screen.“

His mentor believes that it is an innovative technology that can revolutionize our daily lives.

Jan Vanderdonkt, Professor of Computer Science and the Human-Machine Interface at UCLouvain, details: “All these devices, these sensors, these sensors, these cameras are there, but they don’t recognize gestures. What they do is photograph the human body and it is up to us to interpret the signal (data) sent by the sensor to recognize the gestures and send commands to the computer. This is what the new machine does.“

At first glance, it might seem like a gimmick, but it isn’t. In the ophthalmology clinic of Saint Luke’s University Clinics, virtual recognition by gesture can bring real benefits.

For example, during certain laser operations, where it is necessary to place a magnifying glass over the eye and go and check the screen further, arteries are mapped to precisely position the laser in the right place.

As Demet Yuksel, Head of the Ophthalmology Clinic at Saint Luke University Clinics explains: “If we can continue our gesture without having to move with the handcuffs of our hands which must remain immobilized to hold the lens. With the other hand at a medium distance, it would be very nice to be able to process the X-ray image for a more accurate gesture.“

The ophthalmologist takes us to the operating room, where she explains that sometimes having to touch the screen with pictures of the patient hanging from his arm is complicated: “Right now we’re trying to muster what we can to try to put our medical file on the screen, protect the screen in a sterile way, and we have to deal with file items with a touch screen whose resolution isn’t always perfect. “

There again, there is no need to draw, arranging the images with a hand gesture will allow it to be faster and more accurate.

Mark Mosher, a famous American DJ followed by millions of fans, actually creates his songs by touching, cleaning or even moving smart cubes of all colors. The result was amazing.

Through the interlaced video, Mark Mosher, DJ set in Denver, USA, gives us a simple demonstration: “This means that I can control the music through this cube by making gestures and movements. And then the two cubes can talk to each other, so with these two cubes I can orient as I want, something will be generated.“

You don’t have to worry about the computer keyboard anymore, the artist can rearrange everything as he pleases, just by moving as the DJ explains to us: “We are more expressive and creative, these cubes give us the ability to control a lot of parameters at the same time.“

In the movie Minority Report, science fiction has already demonstrated virtual recognition by gesture. Twenty years later, reality has joined fiction.