I just heard about this program that participated in one of the conversational Turing Test competitions, through this article in The New Yorker by Gary Marcus.
Nobody who seriously works on any aspect of AI would be genuinely surprised by this. However, it is a useful reminder of exactly ‘how much’ AI one needs in any practical application. There remain many hard problems, e.g., natural language understanding at a human-competitive level. However, there are many applications where the bar is actually really low, e.g., as Gary Marcus notes in the above article,
If Goostman can fool a third of its judges, the creation of convincing computer-based characters in interactive games—the next generation of Choose Your Own Adventure storytelling—may be a lot easier than anyone realized.
Over the years, I have been surprised, and a tad disappointed, at how many applications that could be so stimulating for AI research are actually cracked by simplistic and naive methods, with just a bit of clever wrapping (like the game AI comment above). I wonder if there are any genuinely intermediate level problems – something not as trivially solvable by naive methods (e.g., the game AI in many apps today), yet not as steep as ‘full’ NLU?