“Beauty” as a Search Heuristic?

Through my colleague, Prof. Andrew Ranicki, I came upon this interesting interview with another distinguished colleague, Sir Michael Atiyah: https://www.quantamagazine.org/20160303-michael-atiyahs-mathematical-dreams/. The interview contains interesting reflection upon many things including the notion of beauty in mathematics. Indeed, Atiyah has co-authored a paper based on a very interesting neuroscience study on neural correlates of beauty: http://journal.frontiersin.org/article/10.3389/fnhum.2014.00068.

The key conclusion of this paper is that,

… the experience of mathematical beauty correlates parametrically with activity in the same part of the emotional brain, namely field A1 of the medial orbito-frontal cortex (mOFC), as the experience of beauty derived from other sources.

This in itself is cool, but it made me wonder – if we have a parametric signal in our brains, after a certain level of expertise has already been acquired, which correlates with how ‘beautiful’ something is, then is this useful as a heuristic in the search for new such objects? One of the limitations of most computational intelligent systems is precisely that they perform so poorly when it comes to exploratory search in open ended domains. Does this signal then help us prune search in our heads, a bit along the lines of José Raúl Capablanca’s famous, “I see only one move ahead, but it is always the correct one”?

Action Priors and Place Cells

My former student, Benji Rosman, and I worked on an idea that we called action priors – a way for an agent to learn a task-independent domain model of what actions are worth considering when learning a policy for a new sequential decision making task instance. This is described in a paper that has recently been accepted to the IEEE Transactions on Autonomous Mental Development.

This has been an interesting project in that, along the way, I have found several quite interesting connections. Firstly, I had always been curious about the famous experiment by Herb Simon and collaborators on board memory in chess – does that tell us something of broader relevance to intelligent agents? For instance, watch the clip below, paying special attention to the snippet starting at 1:25.

The kind of mistake he makes – involving only those two rooks – suggests his representation is not just based on compressing his perception (these rooks are by no means the least salient or otherwise least memorable pieces), but is intrinsically driven by the value – along very similar lines, we argue, as action priors.

Subsequently, through a nice kind of serendipity – i.e., conversations with my colleague Matt Nolan, I have come to know about some phenomena associated with place cells as studied by neuroscientists. Apparently, there are open questions about how place cells seem to represent intended destination in goal directed maze tasks. The question we are now trying to figure out is – does the concept of action priors and this way of re-representing space give us a handle on how place cells behave as well?

I’ll give a talk on this topic at the upcoming Spatial Computation Workshop, abstract below.

Priors from learning over a lifetime and potential connections to place and grid cells
– Subramanian Ramamoorthy (joint work with Benjamin Rosman)

An agent tasked with solving a number of different decision making problems in similar environments has an opportunity to learn over a longer timescale than each individual task. Through examining solutions to different tasks, it can uncover behavioural invariances in the domain, by identifying actions to be prioritised in local contexts, invariant to task details. This information has the effect of greatly increasing the speed of solving new problems. We formalise this notion as action priors, defined as distributions over the action space, conditioned on environment state, and show how these can be learnt from a set of value functions.

Applying action priors in the setting of reinforcement learning, using examples involving spatial navigation tasks, we show the benefits of this kind of bias over action selection during exploration. Aggressive use of action priors performs context based pruning of the available actions, thus reducing the complexity of lookahead during search. Additionally, we show how action priors over observation features, rather than states, provide further flexibility and generalisability, with the additional benefit of enabling feature selection.

An alternate interpretation to these priors is as a re-parameterisation of the domain within which the various tasks have been defined. In this sense, these prior distributions bear a resemblance to attributes, such as spatial distribution and firing patterns, of place cells. We conclude by discussing this connection using a variation of the above experiments.

Intention prediction among goalkeepers

How is it that goal keepers manage to ever save any penalty shots, beating the striker in the incredibly little time available?

This brief video in the online version of The Economist outlines a few quite different attributes to their thought process, not to mention the not so explicitly conscious reflexes. Even at this cursory level, it is interesting how many different modalities are involved – learning from the striker’s historical kicks, randomised strategies in the spirit of game theory, face processing to extract subtle cues, psychological intimidation, etc.

My student, Aris Valtazanos, and I wondered about this problem in one of our papers associated with our football playing robots, but clearly we are unable to capture this whole variety of interactive intelligence. It would be cool when one day we have agents that can actually function at this level!


Lighthill on AI, and some questions that remain open!

As an AI researcher, and a faculty member at Edinburgh where some of the most interesting episodes in the history of AI were enacted, I find the Lighthill report to be quite fascinating and I have occasionally come back to reading it when thinking about the big questions.

One interesting point is that many of the things he criticised so strongly, e.g., machine translation, have turned out to be the areas where in subsequent decades AI has made the most progress. Some of his other critiques, such as his scepticism about data-driven approaches (spoken like the applied mathematician he was), have turned out exactly the other way – as hallmarks of a methodology that has come to define our age.

There is one observation, however, that he rightly makes, that continues to remain a blind spot for the research community:

We must remember, rather, that the intelligent problem solving and eye-hand co-ordination and scene analysis capabilities that are much studied in category B represent only a small part of the features of the human CNS that give the human race its uniqueness. It is a truism that human beings who are very strong intellectually but weak in emotional drives and emotional relationships are singularly ineffective in the world at large. Valuable results flow from the integration of intellectual ability with the capacity to feel and to relate to other people; until this integration happens problem solving is no good because there is no way of seeing which are the right problems. These remarks have been included to make clear that the over-optimistic category-B-centred view of AI not only fails to take the first fence but ignores the rest of the steeplechase altogether.

He is right, and we still don’t pay nearly enough attention to this. Perhaps it is time, especially given the remarkable new opportunities created by new advances in allied areas ranging from experimental neurosciences to cognitive psychology?

Neuroscience of choice

This is a nice, short, introduction to some key issues at the interface of neural aspects of human motor control and higher cognitive decisions – problems of choice. It concisely explains why people in areas such as robotics are excited about some new developments, but it also brings out a couple of things that should be better known within the machine learning community, e.g.,

Placing motor and cognitive decisions in a common theoretical framework brings into sharp relief one apparent difference between them. Researchers have long argued that humans and animals mostly make choices in sensorimotor tasks that are nearly optimal – in the sense of approaching maximal expected utility – or complying with principles of statistical inference. In contrast, work in traditional economic decision-making tasks often focuses on situations in which participants violate the predictions of expected utility theory, for instance by misrepresenting the frequency of rare events or due to interference with emotional factors (Kirk et al., 2011).