Virtual reality technologies are advancing in more and more areas. From training pilots in a flight simulator to spatial studies, for example in architecture, to increasingly vibrant video games: simulations of environments, combined with technologies such as virtual reality goggles, offer nearly limitless possibilities. However, virtual reality systems are not used much in everyday life. “Nowadays, virtual reality is mainly used for content consumption. “Technologies have not yet established themselves in productive activities as classic office work,” says Christian Holz, a professor at the Institute for Intelligent Interactive Systems at ETH in Zurich. There is also great potential here: the content is no longer tied to a screen, users can act in 3D, very flexibly and intuitively with their hands.
Every finger vibrates differently
What is hindering this development? Christian Holz sees the central problem in the interaction between people and technology. Most VR applications today are powered by either handheld controllers or by hand in the air, the position of which is recorded by the camera. Most of the time, users are also on the service. “If you always have to keep your arms raised, it quickly becomes tiring. The normal work process, that is, interacting with the system over a period of several hours, is unimaginable,” says Holz. Another problem arises when typing on a virtual keyboard. The fingers move very little, and the cameras can’t capture the depth of motion as accurately as a mechanical keyboard can. In addition, the usual tactile feedback is missing when working in the air.
For the team of researchers led by Christian Holz, it is therefore clear that passive surfaces are still important for the productive use of virtual reality. This can be the classic table top, but also a wall or your body. In order to make the best use of it, they developed a sensor principle called “TapID”, which they used at the conference IEEE VR It will be presented at the end of March. The principle is based on several accelerometer sensors located on the wrist, which are included in an ordinary rubber bracelet. They are recorded when the hand touches a surface and which finger is being used. The researchers found that their innovative sensor design could resolve subtle differences in the vibration coil at the wrist that each finger fires differently upon movement. A specially developed machine learning pipeline processes the measured data in real time. In combination with the camera system of the VR glasses, which records the position of the hands, a very precise insert can be made. The researchers demonstrated this in some of the applications they programmed to develop, such as a virtual keyboard and piano (see video).
