“Beauty” as a Search Heuristic?

Through my colleague, Prof. Andrew Ranicki, I came upon this interesting interview with another distinguished colleague, Sir Michael Atiyah: https://www.quantamagazine.org/20160303-michael-atiyahs-mathematical-dreams/. The interview contains interesting reflection upon many things including the notion of beauty in mathematics. Indeed, Atiyah has co-authored a paper based on a very interesting neuroscience study on neural correlates of beauty: http://journal.frontiersin.org/article/10.3389/fnhum.2014.00068.

The key conclusion of this paper is that,

… the experience of mathematical beauty correlates parametrically with activity in the same part of the emotional brain, namely field A1 of the medial orbito-frontal cortex (mOFC), as the experience of beauty derived from other sources.

This in itself is cool, but it made me wonder – if we have a parametric signal in our brains, after a certain level of expertise has already been acquired, which correlates with how ‘beautiful’ something is, then is this useful as a heuristic in the search for new such objects? One of the limitations of most computational intelligent systems is precisely that they perform so poorly when it comes to exploratory search in open ended domains. Does this signal then help us prune search in our heads, a bit along the lines of José Raúl Capablanca’s famous, “I see only one move ahead, but it is always the correct one”?

Ockham’s razor in different fields

Someone participating in an ongoing exchange within a public newsgroup made the following observation which I believe is very pertinent to areas I work in as well:

Ockham’s razor was a terrible heuristic for planetary science for about 1500 years, not with respect to replicating the position of the planets in the night sky (big data ‘prediction’ problem as it would currently be called now) but instead as a model to actually understand planetary motion, which became the foundation for mechanics and modern physics.

It would perhaps be considered unfashionable to knock Ockham in today’s age of data-driven science, but I think that reaction only arises from an underestimation of how valuable a good theory is. Throughout graduate school I was just in awe of concepts like the variational formulation of mechanics which were not only the very foundations of physics but also slowly crept under to provide the foundation for the most efficient methods for data-driven inference in graphical models and so on! It would be a real shame if researchers interested in intelligence, be it AI or neuroscience, gave up on the search for these principles…

Look for the longest threads!

In my free time, I am beginning to watch the lecture series by Richard Feynman entitled The Character of Physical Law. I have sort of read bits and pieces of it before but this time my focus is different. I am keen to understand the nature of physical law in terms of how people think about it as they go about discovering it, and what they expect of these laws. After all, in principle at least, that is what we aspire to in AI.

I’ll say more about all this later but already, from the first lecture, the following final quote stands out as very important food for thought:

Nature uses only the longest threads to weave her patterns, so that each small piece of her fabric reveals the organization of the entire tapestry.

To what extent do people pay attention to this aspiration when talking about unsupervised learning, computational discovery, etc.?!

Sociology of ideas

Related to my previous post, this was in my inbox today (thanks to ACM TechNews):

Computers Intersect With Sociology to Sift Through ‘All Our Ideas’
Princeton University (07/19/10) Emery, Chris

Princeton University researchers have developed a new way for organizations to solicit ideas from large groups and to have them vote on the merit of the ideas generated by the group.  The system, called All Our Ideas, combines the concepts of sociology and computer science to enable an organization set up a Web site where people can contribute and rank ideas.  The researchers say the system could help governments tap into public opinion and provide sociologists with a new research tool.  “This is a hybrid method that combines the quantitative power of surveying with the power of focus groups to generate ideas,” says Princeton professor Matthew Salganik.  All Our Ideas allows survey creators to present respondents with a question and two possible answers.  Respondents can either pick one of the answers or submit a new one, which joins the pool of answers that are voted on in pairs.  The researchers designed All Our Ideas to avoid the pitfalls that have plagued other systems, such as stopping people from voting for their favorite entry multiple times by programming the system to randomly select two voting choices.  The system also accounts for some ideas being in the system longer than others, preventing older entries from dominating the rankings because they have been voted on more times.

The long slow conversation, among machines

Science is a long slow conversation. As a doctoral student, I was very impressed by this simple claim which, at the time, was offered as advice regarding research strategy. I was quite taken by the idea that there are many different ways to influence and shape the conversation, that the loudest voice is typically not the most interesting one, that influence plays out at many different time scales.

Years later, as a practicing scientist interested in the elusive question of how an autonomous agent should learn in open ended scenarios, I draw a different set of lessons from this notion. If we take the metaphor seriously then just as it is hard for a single person to have an interesting conversation, it is likely that a lot of interesting learning must come out of different perspectives that get weaved together. Moreover, the most interesting thing about good conversations is that it seems to have a flow that is determined by multiple, often inconsistent, views brought in by participants – the overall quantum of ‘knowledge’ consists of a proper synthesis of the consituent theories and philosophies. Wouldn’t it be nice if we could have the same happening with autonomous agents that operate in large worlds? What might one get if the thousands of autonomous robots that roam around various microworlds (individually trained using standard state of the art algorithms and techniques) could ‘compare notes’ and discuss strategies – that would lead to the elusive `step change’ in capability of these machines. Of course, we don’t fully understand how to achieve this sort of open ended transfer and generalization.

In this context, I was quite intrigued when a colleague told me about the semantic science project (http://www.semantic-science.org/). The idea is to put up machine understandable theories (and data) on the web so that this sort of synthesis may reveal new theories. The project seems to be in its infancy but I hope it does take off. Mostly, I am curious to see the conceptual foundations behind this style of knowledge aggregation and formation.

This isn’t just idle speculation from the sidelines. I am trying to work on some aspects of this problem myself. I suppose I should really say that I am trying to understand the groundwork in this area, in order to be able to eventually make progress on these questions. This doesn’t seem easy – the groundwork ranges from ways to reason about knowledge and ways in which one can assign and compute with probabilities over structured fragments of knowledge for learning from experience all the way through to the methodology of discovery, e.g., Lakatos’ famous little book on Proofs and Refutations. Let us see where all this takes us…