## Wednesday, 5 January 2011

### The Reinfeld Experiment

Back in 1969, when I was a student, I devised a learning technique that I used when I had a particularly difficult piece of mathematical physics to learn.  Mostly, I did not need this technique - I reserved it for use when I knew I was going to have serious trouble.  The technique was very simple.  I would thoroughly learn the material in question, wait a week, revise it, wait another two weeks and revise it again, wait another four weeks and revise it again, and so on doubling the interval between successive revisions.  Since the intervals kept doubling, they soon became very large.  In practice, I either passed the exam, and did not have to worry any more, or I was using the material so often that I no longer needed a revision programme.

Could I apply my old method to chess tactics?  I thought so.  For the learning phase, I could solve a batch of problems three times.  For the revision phase, I could solve them again after one week, two weeks, four weeks etc.  It all looked feasible, so I decided to give it a try.  I divided Reinfeld’s 1,001 Winning Chess Sacrifices and Combinations into ten batches, which I labelled A to J.  My original schedule was:

Week 1:  3A
Week 2:  3B + A
Week 3:  3C + B
Week 4:  3D + C + A
Week 5:  3E + D + B
Week 6:  3F + E + C
Week 7:  3G + F + D
Week 8:  3H + G + E + A
Week 9:  3I + H + F + B
Week 10: 3J + I + G + C

In Week 1, I would solve A three times (which I have written as 3A).  In Week 2, I would solve B three times and revise A for the first time.  In Week 3, I would solve C three times and revise B for the first time.  In Week 4, I would solve D three times, revise C for the first time, and A for the second time.  I planned to carry on in this way, with A due for its first revision in Week 10.  (You will notice that taking on a new batch of problems every week results in an additional repetition each week every time the number of batches doubles.)  I started solving a new batch of problems each Monday, and took Thursday and Sunday off.  My daily schedule was:

Week 1:  A1  A2  --  --  A3  --  --
1   2   3   4   5   6   7
Week 2:  B1  B2  A4  --  B3  --  --
8   9  10  11  12  13  14
Week 3:  C1  C2  B4  --  C3  --  --
15  16  17  18  19  20  21
Week 4:  D1  D2  C4  --  D3  A5  --
22  23  24  25  26  27  28

(Where A1, A2, A3, A4 and A5 are the first, second, third, fourth and fifth repetitions of A, and similarly for B, C and D.)  In reality, my schedule was much messier, because I conducted a number of experiments to find out which repetition intervals worked best.

In my very first attempt, I had A4 a week later, and was horrified to find that my performance had dropped sharply from A3, so I hurriedly amended my schedule to that above.

The cognitive psychology experiments quoted previously suggest that doing the first three repetitions on Monday, Tuesday and Wednesday should be less good than doing them on Monday, Tuesday and Friday, or doing them on Monday, Wednesday and Friday, but I did not notice an obvious difference.

I tried doing the third repetition for only those problems which had caused me trouble (i.e. those where I did not get the entire solution in the book, before looking at it, on one or both of the first two repetitions).  This turned out to be a bad idea.  The troublesome 20% of the problems took 50% of the time, and not doing the easier 80% slowed down subsequent repetitions and made them more difficult, so I probably was not saving any time.  I did another repetition of the problems that I had missed to fix the problem.

When I got to the fifth repetition, it felt harder than the fourth repetition, which is not very surprising if you look at my daily schedule.  The intervals between successive repetitions were 1, 3, 5, 17, so there was a big jump in going to the fifth repetition.  Moving the fifth repetition forward a week would have avoided this problem, but it was too late for that, so I moved the sixth repletion forward a week as a quick fix.   If I had done the fifth repetition a week earlier, the intervals would have been 1, 3, 5, 10.  That is getting close to 1, 2, 4, 8, which I suspected would work better.

When I got to batch G, I decided to carry out a detailed test of the repetition intervals 1, 2, 4, 8...  With this schedule, numbering from the day of the first repetition, the repetitions take place on days 1, 2, 4, 8, 16...  (As noted in the earlier section: Introducing the Expanding Repetitions Method, this schedule is the reverse of the 7 Circles schedule, where the repetition intervals are 64, 32, 16 … 2.)  The n th repetition takes place on day 2 ^ (n-1).  I attempted a complete solution of each problem before looking at the solution in the book, and scored my success on a five point scale:

0 points:  Wrong!
1 point:   Right idea
2 points:  Right first move
3 points:  Proved the combination sound
4 points:  Complete solution, as in the book

When I found a winning combination that was less good than the one in the book, I gave myself 2.5 points.  I gave myself full marks when I found a better solution than the one in the book - but no bonus points for that - or for spotting other mistakes in the book!  My progress so far is summarised below:

Repetition:        1    2    3    4    5    6    7    8    9
Percent score:    85%  93%  95%  95%  97%  97%  95%  92%  90%
Minutes/problem:  3.5  2.7  2.0  1.5  1.3  1.3  1.2  1.0  1.2

In this table, minutes/problem is the total number of minutes of the session (including checking the solution) divided by the number of problems tackled.  This schedule was so successful that I decided to use for the remaining batches, and convert the other batches over to it.  (When moving to the tighter schedule, the repetitions that have already been done will be a little late according to the new schedule.  This makes the intervals between the repetitions more equal than they would have been if the new schedule had been used from the outset.  We saw in the cognitive psychology section that making the repetition intervals more equal has little effect on long term memory retention - but performing a repetition too late makes that repetition harder - so it is better to use the tighter schedule from the outset.)

When I got to batch H, I found that it was difficult to squeeze all the repetitions into my schedule.  I solved this problem by taking on a new batch every two weeks rather than every week.  (I also had another week in which I did not take on a new batch, to move the previous batches over to the schedule of batch G.)

At the time of writing, I have completed the early repetitions on all the batches, and am I up to seven repetitions on the earlier batches (or eight if you count my failed attempt at the 7 Circles).  I found this method to be much more effective than the 7 Circles at teaching me to solve these problems quickly and reliably (and to retain that capability) - but it is to early to assess the impact on my ability to solve fresh problems, or my chess generally.  Watch this space!

[Repetition 8 added to the table on 15 March 2011.  Repetition 9 added to the table on 19 July 2011.]

1. very interesting to read. Awesome job.

2. Awesome blog; and pointing right at what is fundamentally wrong with the manza-circles

3. This is ground breaking work. I read your prior posts and look forward to your future findings. This is one of the big questions of tactical training: What is the best methodology to learn tactics ?

Most Knights in the past found it helpful to break the problems into manageable sized sets and progress from easier to harder problems.

You might find this Wired article about repetition,training and learning interesting.

http://www.wired.com/medtech/health/magazine/16-05/ff_wozniak

4. Interesting, and very pertinent to chess improvement. I found you via the Chess Blog Carnival, by the way.
You may want to view GM Davies short post on this ( http://chessimprover.com/2011/02/05/finding-the-time-to-study/ ) where he also points to this research ( http://projects.ict.usc.edu/itw/gel/EricssonDeliberatePracticePR93.pdf).

My issue is that I don't enforce my study enough, so don't have the reinforcement effect. I am probably among those who hope to improve by just having chess books lying around :)
I will say though, that when I do make the effort to focus study it has a positive effect on my chess, and I do use the methods you outline here ( although not so rigorously ).
I feel it is regular repetition that has the most use.

5. Thank you very much for your comments and the links. I do not doubt that Wozniak is right in saying that learning systems that show rapid apparent progress are more popular than those showing slow real progress! Becoming a strong chess player is slow and requires a lot of work. Kasparov improved by only 100 rating points per year:

http://temposchlucker.blogspot.com/2005_02_01_archive.html

6. Excellent post! I was inspired to come up with a problem-solving schedule for chesstempo.com based on your ideas:

http://www.chessvideos.tv/forum/viewtopic.php?f=16&t=8192&p=63605#p63605

7. I am always pleased to hear that my efforts are appreciated. The main update that I can offer here is that on simpler problems with repetition on days 1.9^(n-1), rounded to the nearest whole number, my performance rises to a maximum on day 8, and declines slowly thereafter, despite the repetitions. Missing out the repetition on day 2 (which contributes little to long term memory) should give essentially the same results after many repetitions, but the early repetitions will be harder. If you want better performance retention, you can use a ratio of less than 1.9. However, I am constantly taking on more problems, so my loss in performance on the previous ones does not appear to matter too much, and I can always add extra repetitions later on. Good luck!

8. I'm very curious about the evolutions of your experiments and eventual conclusion. perhaps this post can attract your attention back to it :)

9. The method appears to work best for simple problems. As with all training methods, diminishing returns eventually set in. I am currently solving lots of harder new problems. Nonetheless, no training has any lasting effect without memory retention, and memory retention requires repetition.

10. What do you do when your solution is better than Reinfeld's?

http://chessskill.blogspot.com/2008/08/reinfeld-busts.html

11. As I said in the article: "I gave myself full marks when I found a better solution than the one in the book - but no bonus points for that - or for spotting other mistakes in the book!"