Human Vs. Machine 2

This is a continuation of yesterday’s post. Those of us who have been around computers for a long time have lived through many letdowns when it comes to AI.  The promise was frequently grand (cf various Herbert Simon quotes) but the actual results were disappointing to say the least.  But a funny thing has happened over the last decade on the web.  The grand promises have (mostly) disappeared and yet the results are beginning to show up!  We might not be noticing them because they are not even labeled as AI, but I believe that they qualify.

A simple example is Netflix’s recommendation system.  Based on your ratings of movies, the sytem recommends other movies that you might like.  It does so with fairly good accuracy already and several teams have been working on significant improvements.  In my book this is a type of AI – the machine is doing something that if a human did we would consider to be intelligent.  I belong to the “if it walks like a duck and talks like a duck, then I don’t care if it’s mechanical” school of AI.

One can easily see how the Netflix recommendation system could one day become a subsystem of the kind of AI that we have envisioned in the past.  For instance, as a first step, if you were to combine it with all the movie knowledge contained in IMDB, then you would have something that could easily pass as an expert on movies.  Yes, it still could not watch and understand a movie, but I don’t think that’s a requirement.  Certainly plenty of humans hold opinions about movies that they have not seen solely on the basis of reading someone else’s review.

Why is this happening now?  The answer is “It’s the data, stupid."  AI had been madly focused on algorithms, but algorithms without data are like fish without water – they don’t swim very far.  What the web has done is put a wealth of data out there that algorithms can make use of.  When there is a trade-off, it appears that more data beats better algorithms (of course better algorithms with more data will always be superior).