I recently wrote that the reason we can now get some real AI results is that we have a large body of data available.  One area in which this applies is visual search.  There are already some fantastic examples of what can be done with a large number of images in the work that has been done on pulling images from Flickr to assemble detailed views.  There are also a couple of visual search engines already out there, such as Riya.  The great thing for teaching a lot of these algorithms is that there are not just a lot of images now, but in many cases there is useful meta data, such as tags and rankings / preferences (both explicit and implicit based on user behavior).

Take face recognition, for example.  Facebook has a huge number of images that are tagged with the names of the people in them.  This far outstrips anything ever previously available - I recall how many early face recogniton efforts had data sets in the low hundreds of faces and images.  Facebook’s data will make it possible to create a system that identifies people in images based on the many images that have already been tagged.

In shopping, I am not sure how important visual search will be for branded products (although some folks really like like.com), but it could be hugely helpful for Etsy, which has lots of unique items.  Visual search on Etsy could combine image analysis vectors with preferences.  For instance it could figure out color, complexity and shape of objects that someone likes or has bought in the past and then find similar objects along those dimensions.

The key to the ultimate success of “visual search” I believe is to make it be just “search."  By that I mean the use of visual characteristics has to be embedded in the regular search process (where applicable) and not be separate functionality.  This is not unlike say "stemming” in search today.  You don’t realize that’s what the search engine does under the hood, it simply gives you back better results.  In fact, Google has had a close analogy in text search for a long time.  Results from a search have a link for “similar pages.”

I recently wrote that the reason we can now get some real AI results is that we have a large body of data available.  One area in which this applies is visual search.  There are already some fantastic examples of what can be done with a large number of images in the work that has been done on pulling images from Flickr to assemble detailed views.  There are also a couple of visual search engines already out there, such as Riya.  The great thing for teaching a lot of these algorithms is that there are not just a lot of images now, but in many cases there is useful meta data, such as tags and rankings / preferences (both explicit and implicit based on user behavior).

Take face recognition, for example.  Facebook has a huge number of images that are tagged with the names of the people in them.  This far outstrips anything ever previously available - I recall how many early face recogniton efforts had data sets in the low hundreds of faces and images.  Facebook’s data will make it possible to create a system that identifies people in images based on the many images that have already been tagged.

In shopping, I am not sure how important visual search will be for branded products (although some folks really like like.com), but it could be hugely helpful for Etsy, which has lots of unique items.  Visual search on Etsy could combine image analysis vectors with preferences.  For instance it could figure out color, complexity and shape of objects that someone likes or has bought in the past and then find similar objects along those dimensions.

The key to the ultimate success of “visual search” I believe is to make it be just “search."  By that I mean the use of visual characteristics has to be embedded in the regular search process (where applicable) and not be separate functionality.  This is not unlike say "stemming” in search today.  You don’t realize that’s what the search engine does under the hood, it simply gives you back better results.  In fact, Google has had a close analogy in text search for a long time.  Results from a search have a link for “similar pages.”