Through some conversations at AAAI last week, I came to know about Winograd Schemas and a challenge involving their interpretation that is considered by many to be a more fitting one than the conversation bot versions of the Turing Test that have repeatedly been shown to be easy to crack without really solving the main problem. A Winograd schema is based on pairs of sentences that differ in only one or two words which make a radical difference in the meaning, thus requiring deeper understanding of ‘commonsense’ in order to resolve the meaning.
This challenge has now been picked up by Nuance, who see substantial benefits from this level of AI to their own products. This is a positive thing.
I wonder if we can also define equally crisp and well defined challenges that bring out the need for intelligence in the area of robotics. The nice thing about the Nuance Winograd challenge is that it gets at the heart of the issue in a bite-sized fashion, allowing anyone with a keen mind and some bandwidth to potentially contribute to the issue. In contrast, most robotics challenges out there today are as much a test of people’s political and management skills in assembling the resources to compete as it is a test of scientific ideas – and even then, they often seem to be set up to not really address these kinds of core AI issues. It’d be a lot of fun to find more bite-sized yet deep and significant open problems within robotics.
Interesting article on why assistive robotics might make sense:
How is it that goal keepers manage to ever save any penalty shots, beating the striker in the incredibly little time available?
This brief video in the online version of The Economist outlines a few quite different attributes to their thought process, not to mention the not so explicitly conscious reflexes. Even at this cursory level, it is interesting how many different modalities are involved – learning from the striker’s historical kicks, randomised strategies in the spirit of game theory, face processing to extract subtle cues, psychological intimidation, etc.
My student, Aris Valtazanos, and I wondered about this problem in one of our papers associated with our football playing robots, but clearly we are unable to capture this whole variety of interactive intelligence. It would be cool when one day we have agents that can actually function at this level!
This is a nice book review written by Garry Kasparov, insightful in part because of the major role he played in the area. Some interesting snippets below.
On human-computer team play:
The chess machine Hydra, which is a chess-specific supercomputer like Deep Blue, was no match for a strong human player using a relatively weak laptop. Human strategic guidance combined with the tactical acuity of a computer was overwhelming.
The surprise came at the conclusion of the event. The winner was revealed to be not a grandmaster with a state-of-the-art PC but a pair of amateur American chess players using three computers at the same time. Their skill at manipulating and “coaching” their computers to look very deeply into positions effectively counteracted the superior chess understanding of their grandmaster opponents and the greater computational power of other participants. Weak human + machine + better process was superior to a strong computer alone and, more remarkably, superior to a strong human + machine + inferior process.
On the methodology of chess agents:
Like so much else in our technology-rich and innovation-poor modern world, chess computing has fallen prey to incrementalism and the demands of the market. Brute-force programs play the best chess, so why bother with anything else? Why waste time and money experimenting with new and innovative ideas when we already know what works? Such thinking should horrify anyone worthy of the name of scientist, but it seems, tragically, to be the norm.
I just heard about this program that participated in one of the conversational Turing Test competitions, through this article in The New Yorker by Gary Marcus.
Nobody who seriously works on any aspect of AI would be genuinely surprised by this. However, it is a useful reminder of exactly ‘how much’ AI one needs in any practical application. There remain many hard problems, e.g., natural language understanding at a human-competitive level. However, there are many applications where the bar is actually really low, e.g., as Gary Marcus notes in the above article,
If Goostman can fool a third of its judges, the creation of convincing computer-based characters in interactive games—the next generation of Choose Your Own Adventure storytelling—may be a lot easier than anyone realized.
Over the years, I have been surprised, and a tad disappointed, at how many applications that could be so stimulating for AI research are actually cracked by simplistic and naive methods, with just a bit of clever wrapping (like the game AI comment above). I wonder if there are any genuinely intermediate level problems – something not as trivially solvable by naive methods (e.g., the game AI in many apps today), yet not as steep as ‘full’ NLU?
Samsung has released an open platform for wearable sensing, the Simband: http://www.bbc.co.uk/news/technology-27615445.
I think this is a very nice direction for the field, setting people up to explore more interesting consequences at the level of what can be done now that data acquisition like this is becoming easy.
The Longitude Prize is an interesting concept. Way back in 1714, it was an award for the person(s) who best solved the difficult problem of determining the ship’s longitude – in that day and age, a real grand challenge, one that literally determined the safety and lives of plenty of people!
Today, the problems of clocks and chronometers may seem quaint, but the notion of defining some big problems that need solving remains interesting. All the more so, in my local context, given the short-termism of so many sources of funding that are realistically available to researchers.
So, the announcement today that there is a new version of this prize is interesting indeed. They are all big societal issues, but one of them stands out in particular as having something to do with problems I am professionally interested in. There is a challenge associated with Dementia – How can we help people with dementia live independently for longer?
Assisted living advocates, including within the robotics and AI communities have built systems associated with this problem before, but can all that be lifted up to the standards of the longitude prize? What are the substantial questions that still remain un answered in this area? Useful things to ponder…