A year and a half ago I wrote a post on my ideas about using “structured proofs” to improve understanding in the Analysis I course.

I duly tried this approach, and felt that it made some difference, though perhaps not as much as I had hoped.

Now in this month’s Notices of the AMS there is a fascinating article by a team of educators from Loughborough University in the UK.  Entitled Understanding and Improving Undergraduate Proof Comprehension, the article discusses as three-stage effort to help undergraduate students gain a deeper understanding of proofs – the same issue that my “structured proof” software was intended to address.

In fact, the authors started with a system called “e-Proof” which has a similar underlying idea as “structured proof” but which supplemented it with a number of other student-friendly features, making the proofs accessible in a multimedia audio-visual context.  It seems clear to me that “e-Proof” is a much better implementation of the “structured proving” idea than mine was.   And student feedback was strongly positive.  Nevertheless, when the researchers measured actual outcomes, they found that students’ understanding of material learned via e-Proof in fact was less lasting than the understanding that a control group learned through a conventional textbook.  “It seemed likely”, they write, “that students using e-Proofs felt good about their learning because they were able to understand without too much effort, but that this very fact meant that the understanding that they had acquired was less robust in the longer term.”

The authors went on to try to study student understanding of proofs on a more basic level.  They did this by using infrared cameras to track the eye movements of novice and expert readers confronted with mew purported proofs, some of which were true and some false.  On page 746 of the article (the fifth page of the PDF) , for example, is a fascinating “heat map” showing where experts (top) and novices (bottom) focused their attention when reviewing a fallacious proof of the (true) statement that there are infinitely many primes congruent to 1 modulo 4.  The fallacious line

Every number that leaves remainder 1 when divided by 4 is divisible by a prime that also leaves remainder 1 when divided by 4

attracted a lot of “heat” (attention) from the experts, but comparatively little from the novices – presumably because it involved only words, not mathy-looking formulae.

The news from the researchers is not all bad though.  In a third study they taught students a form of self-explanation training: how to “talk themselves through” a proof and ask questions. Students who had had this training, they found, were more able  to give good explanations, and eye tracking methods showed that they made significantly more “between-line saccades” (a proxy for checking reasons and inferences) when reading new proofs.  It seems as though self-explanation training has a strong positive effect.The authors end cautiously but optimistically:

With these comments in mind, we believe that the success of self-explanation training across our three studies is encouraging not only because it appears to be effective but also for two further reasons. First, self-explanation training is extremely light touch: it is generic, it does not rely upon time-intensive adaptation of existing resources, and students can work through it independently in about 15–20 minutes (as noted above, the training is available at www.setmath.lboro.ac.uk for readers who might wish to use it). Second, self-explanation training does not require more work from the student; it encourages more effective independent work by simply teaching students to make better use of their existing knowledge and reasoning skills. Studies in education research often highlight what students cannot do, so it is cheering to be able to present positive results based on things that they can.