Artificial Intelligence: Microsoft’s new ‘Fetch’ app can identify dog breed with image
The app also features, scrapbook where you can keep track of all your pictures and results. There’s also an impressive list of breeds that contains information such as disposition, size, coat and what types of families are best suited for each.
Man’s best friend has inspired a new app – Fetch! Using your iPhone camera or photo library, it can identify and classify dogs by breeds and tell you what kind of human personality fits best with specific breeds.
If the dog’s breed is unknown, the app will show a percentage of the closest breed. Tapping the percentage rosette leads to the top five breeds that could be in the dog. Clicking on the arrow in the corner leads you to more information on the breed.
Microsoft claims that no two pictures yield the same result. You could resemble a Doberman Pinscher in one photo (sunglasses, no makeup) or a Pekingese (no glasses, makeup) in another.
Two years ago, in July 2014, a demonstration of the technology behind Fetch! appeared on stage at the 15th annual Microsoft Research Faculty Summit.
Even then, the technology was able to tell the difference between people and dogs, with Cortana saying, “I believe this is not a dog.”
The team has also developed a website, What-Dog.net that has photos you can play with to find out about the breeds, and you can also submit your own photos and share them.
The app also features, scrapbook where you can keep track of all your pictures and results. There’s also an impressive list of breeds that contains information such as disposition, size, coat and what types of families are best suited for each. The team consults several dog experts and kennel clubs to curate and update the information about dog breeds, not relying solely on gathered datasets.
“What makes this unique is the combination of machine intelligence and expert data about dog breeds, which helped us be more accurate. We invested in dog experts in making sure we had correct images for each breed,” said Javier Alvarez-Valle, senior software development engineer on the team that built Fetch!.
For Alvarez-Valle, the only member of the team who hasn’t had a dog, the app tuned him into looking at dogs while he was out and about, and trying to guess what breed they were. “This is the kind of app you’re going to take out when you’re with your friends,” he says. “It’s a lot of fun to learn about what dog you look like, and it comes up with surprising comparisons. It will also make for good discussions when I post on Facebook. I think people will find many more uses than we can anticipate now.”
Microsoft has also posted a video to showcase the app.
Released through the Microsoft Garage, this mobile app demonstrates the potential for Microsoft researchers’ continued advances in artificial intelligence, which have already appeared in other playful ways through Microsoft Project Oxford-powered experiences such as HowOld.net, TwinsOrNot.net, MyMoustache.net and Mimicker Alarm. In Fetch!, Project Oxford works together with some powerful new machine learning technology to deliver interesting results for all kinds of photos.
“There was an interest in creating a framework that would allow you to take a domain – in our case, dogs – and recognize numerous classes, such as breeds. We were interested in enabling an app to allow you to make object recognition extraordinary, fun and surprising,” says Mitch Goldberg, a development director at Microsoft Research whose Cambridge, U.K based team built the experience. His team works at the intersection of user experience, machine learning, computer vision and more recently, intelligent cloud services. He’s also had two German shepherd dogs, though now he has a cat.
