A recent study on the utility of concrete examples in mathematics education has been gaining significant attention. In a nutshell, the researchers arrived at the seemingly counterintuitive result that students who were taught a particular concept using specific examples (e.g., filling water in a jug) did worse than students who were taught in terms of abstract symbols (such as “meaningless” geometric shapes being used as labels) – when measured on a new problem that tests whether they understood the underlying idea (in this case, that of a group).

Personally, I am not very surprised. I have always felt that the mere use of examples do not achieve much unless the examples are really well chosen. It is nothing short of a fine art to pick a concrete story that actually delivers a nontrivial appreciation for a deep concept. Also, I think that I would have used a different measure of “successful learning” – I’ll come back to this point shortly.

Of course, some concepts just take time and effort to get across. For instance, I have been taught the concept of eigenvalue (and its use) in numerous courses – ranging from my second year undergraduate math course (at which time, I doubt that I understood the concept at all) to a nonlinear functional analysis course I audited during the last year of my PhD studies. And I still arrive at a new(er) appreciation for the concept as I continue to learn, e.g., when I try to read bits and pieces from works such as this.

I guess what I am trying to get at is this – if you teach a student the abstract version of a concept then she learns the rules and consciously thinks about the rules each time a new problem requires their use (as opposed to a slightly more naive version of pattern matching that happens when the same concept is explained using a few limited examples). Eventually, after numerous attempts, the student gains sufficient dexterity with the concept that she begins to ‘understand’. More often, as has been famously noted by Von Neumann, one may just come to terms with a tricky concept before (possibly, eventually and really) understanding. Sometimes, the specific examples are used to short circuit the hard work that it takes to arrive at this state – which is not a great idea. There are surely instances where the use of specific examples yield quicker learning in narrow domains but there must be a careful balance with the general idea if the student is to have any hope of eventually reaching a deeper understanding. In this sense, I would not measure success by speed of learning – but instead I’d like to see how far the student manages to “stretch” based on what she has been taught.

One excellent example of a successful attempt to convey a subject area primarily using concrete examples is a book by Jeff Weeks called The Shape of Space (perhaps this is a nice counterexample to the study mentioned above). It was an absolute pleasure to read Weeks’ explanations of concepts like orientability, curvature and the structure of three-manifolds. He does a careful job of picking examples that are actually proxies for the more abstract concepts – so that a student who first reads his book and then goes on to a more serious course will surely just say, “Oh yeah, that’s just like…”! One example that particularly stands out is his explanation of the difference between one-sidedness and orientability. I can easily imagine someone in graduate school using the same basic explanation, perhaps aided by more fancy symbols. In this sense, I think that Weeks has done a much better job than more glamorous popular science books that just replace the concepts with wishy-washy analogies.

I wonder whether the same study, repeated with Weeks’ examples and corresponding problems, would yield the same results…

1. Kevin Hogan

Argh!!! My first response, based on the methodology described in the NYT article, is “well, duh!”. My second response, based on the conclusion drawn, is that too many uninspired “teachers” will use this as an excuse not to motivate the abstractions they teach with compelling examples.

Let me explain. The power of mathematics is simply the power of abstraction – i.e., the ability to recognize that the same rules may apply to a variety of disparate applications. If these applications are varied, numerous and important enough, then it makes sense to study the implications of the rules themselves, rather than simply the context of a particular situation with its peculiar bits of trivia that are not relevant to the particular problem you want to solve.

Thus, the successful solution of a problem with mathematics involves at least 4 steps:
#1 – recognize that a set of rules/symbols with which you have particular skill applies to the situation at hand
#2 – translate the problem to its abstraction in your system of rules/symbols
#3 – manipulate the symbols (the more skilled you are at the system, the quicker this is)
#4 – translate the result of your symbol manipulation back to the problem at hand.

#2 and #4 can often be reduced to rather mechanical processes, which means that the real skill comes from being good at #1 and #3. To be good at #1, it helps to have experience translating a bunch of examples into a common rule/symbol form. To be good at #3, well – you do have to spend some time with the rules and symbols in abstraction to see what they do. (If I just tell you what the rules and symbols are, rather than practicing how to manipulate them, then step #3 is much harder). Furthermore, the amount of practice needed to get good at #3 is often more than the practice needed to get good at #1 – thus I can see how a curriculum which focuses solely on #3 produces some results, and a curriculum that severely skimps on #3 will produce poorer results. HOWEVER – notice that the entire sequence of steps is necessary in order to be successful! Therefore, a course of instruction which skips #1 altogether is missing an important piece of the puzzle. (One exception might be a class of students who already have some experience w/ the rules/symbols in question, are already good at #1, and just want to get better at #3 – but this is one class, not an entire regimen of instruction from kindergarten through graduate school)

Furthermore, there’s one more very important piece that must be considered. Notice that there’s a heck of a lot of formal systems! Very few people have the time and resources to study them all (well, at least assuming they want to eat, too…) Thus, a student also needs to do some pruning in choosing at least their initial course of instruction – and unless an introductory instructor does a good job of convincing said student that a set of formal rules is useful and powerful in many situations, they may lose the student – either forever or for a significant period of time. At best, the student may ignore the subject for seemingly more important courses of study, losing valuable time until they reach a point where they see the utility of the subject matter at hand. A savvy teacher in a society which places a high premium on practicality would do well to include well-chosen examples that quickly show off the power and utility of a formal method.

Finally, it seems that one can interpret the results of the study according to the considerations above.

#1 – the course which taught nothing but how to solve a small number of examples, with zero abstraction, failed for the obvious reason – the methods which the students invented to solve the particular examples were not abstract enough to apply to a seemingly unrelated problem. If this method of instruction (all concrete examples, no abstraction) was effective, then the development of mathematics would have zero utility – we could all learn everything we need solely from concrete experience without any need for ideas painstakingly developed over centuries. However – are there _really_ any courses of math instruction which spend _zero_ time on abstraction??

#2 – the course which focused primarily on examples, presenting abstract symbols as an afterthought, failed for a more subtle reason. The students invented their own, less abstract solutions to the examples first, giving them a significant emotional investment in their own methods. At this point, they are apt to be even less convinced of the power of a more abstract solution (remember, they need to devote energy to learning that new solution, as well) UNLESS they are simultaneously presented with a set of examples to which the abstraction applies but their own personal solution does not. Hence, the students didn’t see the utility of the abstraction and did not learn it. As well, did the students also see a clear demonstration of how the abstractions applied to the concrete problems at hand? If not, then the time spent working on concrete examples was indeed wasted work. I also expect that significantly less time was spent in working with the abstraction in this group as in group #3 (the solely theory group). Hence, even this group was at a loss in step #3 of the 4-step process due to lack of practice with the abstraction.

#3 – the course which focused solely on abstraction. This course directly addressed the work in step 3 of the four step process above – meaning that, assuming students were successful at steps 1 and 2, they were well prepared to tackle the problem. Hence, some of these students did better than those students who received no instruction at all (group #1). However, those students who struggled with the process of abstraction most likely received little benefit from this course of instruction. As well, those students who were successful in translating the given problem to the abstraction might yet struggle in ultimately creating their own abstractions, and may struggle to have the confidence to invent their own abstract solutions to problems in the future. In addition, even this partial success was biased, in the sense that the participants already possessed some motivation to take part in the study (whether monetary remuneration or other), and were told that the abstract concepts they were learning would apply to a final problem presented at the end. This guaranteed them with an expected short-term benefit to learning the abstraction. On the other hand, the investment required in most mathematics courses is much longer than the few hours or days spent in this study, and the expected real benefit often happens much further in the future (I neglect the benefit of receiving a good grade on a test) – hence there is less intrinsic motivation to learn the subject at hand. If the students were instead simply placed against their will in such a classroom for a longer period of time, it is likely that more of the students would have tuned out of the discussion, and gained little benefit from the class. (i.e. – lack of motivation becomes a problem over a longer term, rather than over a short term, such as a few hours of days)

What the researchers did NOT do, unfortunately, was to integrate the use of well chosen examples _in parallel_ with the teaching of an abstraction to motivate the concepts as well as to help students practice with the process of abstraction. I expect that if this had been done, these students would have done better than all 3. As I stated at the beginning, I’m afraid that this omission will result in more “teachers” using this as an excuse to remove any sense of motivation from their work – a truly appalling and counterproductive result…

In summary – I think I agree with your blog post, but not with the conclusions of the study. Concrete examples, with only wishy-washy attempts at making abstractions, are a poor excuse for true mathematical training. However, well chosen and diverse examples, which prove both the utility and the power of a particular abstraction, are an important and – in my opinion – essential ingredients of an optimal course of instruction. I also vouch for some element of “free” problem solving, where the student is encouraged to invent their own solution to free form problems. This final method cannot form the entirety of an efficient course of instruction, but if it is neglected in entirety, it still leaves students as half-formed intellectuals.

• Very insightful observations!
One other point that occurs to me as I read your comments – a lot of university programs spend most of their time on steps 2 – 4 of your list but interviews for really interesting and meaningful technical jobs place a lot of emphasis on step 1 (something students do not get much training on)…

2. Kevin Hogan

Bleah – I missed the graphic which accompanied the NYT article – and this, I’m afraid, tells the whole story – i.e., that an _incredibly poorly_ chosen example can act worse than no example at all. The point here is that the measuring cups do not, cognitively, map neatly onto the symbolic representation – most people viewing the measuring cups will not automatically view part full cups as _completely different abstract symbols_, at least not as different as circles are from diamonds, or ladybugs from vases. Instead, they’ll view them as different _amounts_. Ditto w/ pizza, clocks, etc. I expect that if the children given the _entirely different symbol_ representation were told to figure out the measuring cups (i.e. amounts), then they wouldn’t get it any better than they would with no intervention at all – just as those who were presented w/ the measuring cups (amounts) couldn’t figure out the symbols.

So yeah – double bleah – if you teach me an abstract concept and ask me to solve a practically identical and silly abstract problem, of course I can get it. The point here is not the examples (_all_ concepts are taught w/ examples!) – the point is that there is a cognitive distance between examples. Examples which are “farther” from each other cognitively require more abstraction in order to link them together.

What’s more, the response that “teaching 1+1=2” gets the same result as the symbolic intervention is just as useless – 1,2,3 are equally interpreted as symbols and as amounts. Identically shaped measuring cups and pieces of pizza will be interpreted simply as amounts. Completely dissimilar shapes and pictures will be interpreted as symbols. The distance within each category is small – the distance between categories is large. Drawing the conclusion that concrete examples should not be used in mathematics instruction is completely missing the point – because the _true_ measure of abstraction would be the ability to leap between symbols and amounts, and this was _never_ tested!

(Notice that a course of instruction, as I outlined above, which used a wider variety of examples – i.e. from both groups, would have taken longer but produced a stronger result!)

What frightens me is that such a seemingly flawed course of research may end up influencing curriculum design.