This software turns your smartphone into portable AR device

Researchers have developed a new software system that turns smartphones into Augmented Reality (AR) portals, enabling users to place virtual building blocks, furniture and other objects into real-world backdrops, and use their hands to manipulate those objects as if they were really there. The new system, called Portal-ble, could be a tool for artists, designers, game developers and others to experiment with AR, said researchers from Brown University.

“We wanted to make something that made AR portable so that people could use anywhere without any bulky headsets. We also wanted people to be able to interact with the virtual world in a natural way using their hands,” said Jeff Huang, assistant professor of computer science at Brown.

The idea for Portal-ble’s “hands-on” interaction grew out of some frustration with AR apps like Pokemon Go. AR apps use smartphones to place virtual objects (like Pokemon characters) into real-world scenes, but interacting with those objects requires users to swipe on the screen. The software makes use of a small infrared sensor mounted on the back of a phone.

The sensor tracks the position of people’s hands in relation to virtual objects, enabling users to pick objects up, turn them, stack them or drop them. It also lets people use their hands to virtually “paint” onto real-world backdrops.

Huang and his students plan to continue working with Portal-ble — expanding its object library, refining interactions and developing new activities. The team is set to present the work later this month at the ACM Symposium on User Interface Software and Technology (UIST 2019) in New Orleans.