Empirical Rabbit

Weteschnik’s Understanding Chess Tactics

2012-02-01T01:02:00.000-08:00

I first read Martin Weteshnik’s Understanding chess Tactics about a year ago, and found it both helpful and engaging. I have since been through most of the examples several times. Weteschnik does not claim that his examples are original, and I found many of them in other books. The general level of his examples is about the same as those in Fred Reinfeld’s 1001 Winning Chess Sacrifices and Combinations.

Weteschnik says that the club players he was coaching had great weaknesses in their tactical play, and that this was not fixed asking them to solve a huge number of problems. Nonetheless, with his coaching, they improved by an average of 100-200 rating points, and were promoted to higher leagues twice in three years. Clearly, this is not a scientific trial. An exacting critic would say that his students might have improved by as much or more if Weteschnik had used different training methods; or might not be impressed by an improvement of 100-200 points over three years. Nonetheless, its a good inspirational story!

The main emphasis of the book is on understanding chess tactics, rather than just solving problems. Weteschnik says “do not exercise what you do not understand.” Nonetheless, he fully accepts that you need to practice once you do understand, see:

http://www.westmichiganchess.com/reports/Reports/Martin%20Weteschnik.aspx

The main theme of Weteschnik’s book is what he calls “status analysis,” by which he means studying the position to find clues for possible tactics. The book addresses this topic better than any other that I have seen.

Understanding tactics is an important part of the learning process, but that it is not enough in itself. You need to practice until your thought processes become automatic. Nonetheless, as sportsmen say: “practice does not make perfect - perfect practice makes perfect.” Practicing finding tactics badly is not a promising approach.

Reading explanations can be helpful, but working things out for yourself is better, and the examples in this book are only a subset of those that a strong player needs to know. Sooner or later, you are going to have conduct your own investigations. You need to be able to spot most simple tactics (including the uncommon ones) and most common tactical possibilities (including the complicated ones) on sight.

The diagrams in this book are large and clear but a little non-standard. Nonetheless, I did not find them difficult to use. The book was not intended to be used as a problem book. Nonetheless, I like to study a position before I read the commentary, to see what I can work out for myself. The publisher has not been helpful in this regard. None of the diagrams say who is to move. You often have to read a long way through the text to find out, and in so doing, are told the solution. In addition, a sizable proportion of the diagrammed positions are not suitable as problems - either because a bad move has to be made before the problem position arises - or because the sacrifice is speculative and does not win against the best defence. This fault could easily have been remedied. Prior to reading this book in detail, I marked each diagram to indicate who was to move, and whether the position was suitable for use as a problem.

Overall, this is an excellent book, and deserves its high reputation. Will it greatly improve your chess? Perhaps not by itself, but I it could make an important contribution.

Empirical Rabbit Timer Source Code

2012-02-01T01:01:00.000-08:00

I have been asked to publish the Java source code for the Empirical Rabbit Timer, so here it is:

/*
* Empirical Rabbit Timer
*/
import javax.swing.*;
import java.awt.event.*;
import java.awt.*;
import java.io.*;
import java.text.DecimalFormat;

public class RabbitTimer extends JFrame implements KeyListener {
int ProblemIndex = 81, ClumpSize = 9, ProblemInc = 90;
int ProblemNumber = 0, ProblemMax = 10000;
int Buckets = 12, BucketWidth = 5;
int[] Bucket = new int[Buckets+1];
String[] ProblemId = new String[ProblemMax];
int State = 1;
long StartTime;
float SolutionTime;
JLabel xlabel = new JLabel("Input File Error", JLabel.CENTER);
JTextField xtext = new JTextField("0% Hit the . key or s for stats", 80);

RabbitTimer() {
super("Empirical Rabbit Timer");
setSize(600,300);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
xtext.addKeyListener(this);
BorderLayout bord = new BorderLayout();
setLayout(bord);
xlabel.setFont(new Font("Times New Roman", Font.PLAIN, 24));
add(xlabel, BorderLayout.CENTER);
add(xtext, BorderLayout.SOUTH);
setVisible(true);
try{
// Open the file
FileInputStream fstream = new FileInputStream("rabbitin.txt");
// Get the object of DataInputStream
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String strLine;
//Read File Line By Line
while ((strLine = br.readLine()) != null) {
if(ProblemNumber < ProblemMax)ProblemId[ProblemNumber++] = strLine;
}
// Close the input stream
in.close();
} catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
ProblemNumber = ProblemMax+1;
}
if(ProblemNumber <= ProblemMax)
xlabel.setText(ProblemId[ProblemIndex] + ". Ready?");
// Update the time taken
int delay = 1000; //milliseconds
ActionListener taskPerformer = new ActionListener() {
public void actionPerformed(ActionEvent evt) {
if(State == 2) {
SolutionTime = (System.currentTimeMillis() - StartTime) / 10;
SolutionTime /= 100;
DecimalFormat twoPlaces = new DecimalFormat("0.00");
xlabel.setText(ProblemId[ProblemIndex] + ". " +
twoPlaces.format(SolutionTime) + " sec. Move?");
}
}
};
new Timer(delay, taskPerformer).start();
}

public void keyTyped(KeyEvent input) {
char key = input.getKeyChar();
switch (State) {
case 1:
// State = Ready?
String outstring;
switch(key) {
case 's':
outstring = "" + Bucket[0];
for (int i = 1; i < Buckets+1; i++)
outstring = outstring + "," + Bucket[i];
System.out.println(outstring);
break;
case '.':
if(ProblemIndex < ProblemNumber) {
State = 2;
xtext.setText("Hit any key");
StartTime = System.currentTimeMillis();
xlabel.setForeground(Color.red);
xlabel.setText(ProblemId[ProblemIndex] + ". " +
"0.00 sec. Move?");
}
break;
default:
Toolkit.getDefaultToolkit().beep();
}
break;
case 2:
// State = Move?
if(key == KeyEvent.VK_ENTER) {
SolutionTime = (System.currentTimeMillis()-StartTime) / 10;
SolutionTime /= 100;
State = 3;
xlabel.setText(ProblemId[ProblemIndex] + ". " +
SolutionTime + " sec. Score?");
xlabel.setForeground(Color.black);
xtext.setText("Hit 0, 1, r=redo or b=back");
} else Toolkit.getDefaultToolkit().beep();
break;
case 3:
// State = Score?
switch (key) {
case '0':
case '1':
System.out.println(ProblemId[ProblemIndex] + "," +
SolutionTime + "," + key);
if (key == '0') SolutionTime = 999;
for (int i = 0; i < Buckets; i++) {
if (SolutionTime >= i*BucketWidth &&
SolutionTime < (i+1)*BucketWidth) Bucket[i]++;
}
if (SolutionTime > Buckets*BucketWidth)
Bucket[Buckets]++;
if(ProblemIndex < ProblemNumber){
ProblemIndex++;
if (ProblemIndex % ClumpSize == 0)
ProblemIndex += ProblemInc - ClumpSize;
}
case 'r':
State = 1;
if (ProblemIndex < ProblemNumber)
xlabel.setText
(ProblemId[ProblemIndex] + ". Ready?");
else {xlabel.setText("The End");
ProblemIndex = ProblemNumber;}
int Percent = ProblemIndex * 100 / ProblemNumber;
xtext.setText(Percent +
"% Hit the . key or s for stats");
break;
case 'b':
State = 2;
xlabel.setText
(ProblemId[ProblemIndex] + ". Move?");
xlabel.setForeground(Color.red);
xtext.setText("Hit any key (t for time)");
break;
default:
Toolkit.getDefaultToolkit().beep();
}
}
}

public void keyPressed(KeyEvent txt) {
}

public void keyReleased(KeyEvent txt) {
}

public static void main(String[] args) {
RabbitTimer rab = new RabbitTimer();
}

}

Rethinking Chess Problem Server Ratings

2012-01-01T01:03:00.000-08:00

The purpose of this article is to investigate improvements to the rating calculations used by the online chess problem servers, e.g. Chess Tactics Server (CTS), chess.com and Chess Tempo. For these servers, whenever you tackle a problem, you are given a score based on whether you succeeded (or how completely you succeeded in the case of chess.com) - and, if you are successful, the time that you took to solve the problem. These scores are used as input to the Glicko rating system, with the problems being treated as opponents.

With CTS, if you fail to solve a problem correctly, your score for that problem is 0. If you solve a problem correctly within 3 seconds, your score is 1, and after that your score decreases linearly to 0.5 at 10 seconds. If the problem rating is greater than your rating, the score remains at 0.5 for 1 second * (the problem rating - your rating) / 20 Elo rating points. Your score then decreases exponentially:

For chess.com, you receive a “move score” of 1 if you solve a problem correctly, or a fractional move score depending on how many moves you got right. The move score is then multiplied by a factor that depends on how your solution time compares with the average (correct or partially correct) solution time for that problem:

With Chess Tempo, if you fail to solve a problem correctly, your score for that problem is 0. If you solve a problem correctly in less than the average correct solution time for that problem, minus one standard deviation, your score is 1.25. If your correct solution time lies between the this value, and the average correct solution time, your score is 1. If you solve the problem correctly in more than the average correct solution time, your score is: the average correct solution time for that problem / your correct solution time.

There are inherent difficulties with this approach. The first difficulty is that the longer you take to solve a problem, the more likely you are to get it right. Giving problems fixed ratings makes no sense unless there is a fixed time to solve the problem. The creators of CTS appear to have recognised this fact: “Due to the fact, that the problems are invariant to time control, their rating will decrease systematically if the time regime is relaxed. The ratings of tacticians will increase vice versa.” Nonetheless, they still give a longer time limit to more difficult problems. It makes no sense to give a player more time on the clock than has opponent, and then fail to take this into account in the rating calculation! The other tactics servers have the same problem. They all have a single rating per problem, and none of them imposes a fixed time for solving these problems, or tries to compensate for not having done so.

A second difficulty with this approach is that although the servers take account of the time taken to solve problems successfully, they ignore the time taken on unsuccessful attempts. For my first pass through Coakley, my average time to solve a problem incorrectly was nearly 50% longer than my average time to solve a problem correctly, and the problems that I failed to solve at took me nearly three times longer (the time limit), see: A Three Parameter Model.

A third difficulty with this approach is that there is that no clear rationale is offered for the scoring methods.

A chess game can be viewed as a sequence of problems, one for each move. Your playing strength depends on the degree of success you achieve in solving these problems, and the total time taken. The simplest approach to scoring is to allocate points for solving each problem, and add up these points to give a total score, as in a “How Good is Your Chess?” magazine article. For simplicity, I will assume that one point is allocated per problem, as for Chess Tactics Server and Chess Tempo. I will also assume the we measure the score as the total number of problems solved correctly, divided by the number of problems attempted. I believe that the most appropriate measure for the time taken is the average time taken per problem (including those that were solved incorrectly or not solved at all). (N.B. This approach is an improvement on that used by the tactics servers, but is less than ideal. There are difficulties in relating to probability of winning a game to the probability of solving a problem, as discussed in my earlier article Rating Points Revisited.)

Clearly, we cannot ensure that all players take the same time solving a problem, or that all problems are solved in the same time. However, we can, in principle, use the problem scheduling to ensure that all players take the same average time per problem, and that all problems have the same time spent on them, averaged over all the players that attempt them. We can make the players take the same average time per problem by serving up harder problems whenever their average time is less than the standard average time, and easier problems when their average time is greater. Similarly, we can ensure that the same average time is spent on each problem by serving them up to stronger players when the average time is less than the standard time, and to weaker players when it is greater than the standard time. We can then update the rating for the player or problem concerned when the average time is very close to the standard time. If the average time deviates significantly from the average time, we can simply accumulate the data for later use.

This simple method should work very well for a server that does not allow the user to choose which problems are scheduled. The main limitation for such a server is that forcing all the players to take the same average time per problem is not likely to be popular. Some of players will want to practice with a faster time limit than others. If we have enough traffic, it should be possible to offer a choice of average times, each with its own rating system and different ratings for both the problems and the players. Since some players will be better at blitz and others with a slower time limit, different ratings for different speeds makes a lot of sense.

An alternative approach is to adjust the scores of both the players and the problems to what they would have been if their average times were equal to a standard time. My earlier article, Rating, Time and Score suggests a possible calculation. This calculation is based on the assumption that each doubling of a player’s speed adds K points to his rating. K appears to be about 200 Elo points for human players (see An Important Discovery and Rating vs. Time on the Clock). For this calculation, we adjust the players’ scores to what they would have been if their average time per problem was equal to the standard time. We also adjust the problems’ scores (using the unadjusted players’ scores) to what they would have been if the average time per problem was equal to the standard time. We can then apply a standard rating calculation to update the ratings of both the players and the problems. (It is also possible to use this approach in conjunction with the previous method to correct for small deviations from the standard time. The calculation does not have to be particularly accurate to work well in this case.)

We can use the data collected by the server to optimise the value of K. One possible method is to calculate two sets of ratings for both the players and the problems. The first set uses data from only those sessions where the average time per problem was below the average. The second set uses data from only those sessions where the average time per problem was above the average. We can assess the accuracy of the fit by calculating the sum of the squares of the differences between the corresponding ratings for the players and problems in these two sets. We can carry out this calculation for various values of K, and find the value of K that provides the best fit to the data.

The optimum value of K may be both problem and player dependent. Players with good calculation skills are more likely to benefit from extra time. Extra time may also be more advantageous for problems where the candidate moves are obvious, but significant calculation is needed. Given sufficient data, it should be possible to fit different K values to different players and problems, but a “one size fits all” value of K may be adequate. The mathematical relationship between speed and rating may not precisely match my assumption: K points for each doubling in speed. Nonetheless, that relationship is a good starting point. In principle, we can find the precise relationship by collecting enough data, and adjusting the calculation accordingly.

I believe that the methods of this article are a significant improvement on those used by the popular servers. From the standpoint of accuracy, the most promising approach is to use problem scheduling to equalise the average times. Equalising these times by calculation allows more user features to be supported, but is likely to be less accurate. Nonetheless, this method should still be more accurate than the methods currently in use. Combining both methods may be the best approach.

Improving on Least Squares

2012-01-01T01:02:00.000-08:00

Initially, I used the method of least squares to fit exponential curves to my chess tactics training results. See my previous articles: An Important Discovery, and A Three Parameter Model. This method served me well, but does have some limitations. Firstly, it does not treat all the results symmetrically. The shortest solution time contributes to all the data points on the cumulative distribution, whereas the longest solution time that is within the time limit contributes to only one. More importantly, perhaps, because of their much greater numbers, the shorter solution times are weighted much more heavily than the later solution times. The least squares curves, as a result, often do not fit the tail of the distribution well. It would also be more satisfying if the calculation was based on quantities that are more directly meaningful.

It turns out to be possible to calculate the parameters a and b of the cumulative probability distribution P = a*(1 - exp(-t/b)) from the average correct solution time and the number of problems solved correctly. The probability distribution corresponding to P is:

dP/dt = a*exp(-t/b)/b

If we multiply this expression by t and integrate by parts from 0 to t, we get:

a*(b - (b + t)*exp(-t/b))

The average correct solution time is therefore:

t_av = a*(b - (b + t)*exp(-t/b)) / (a*(1 - exp(-t/b)))

which simplifies to:

t_av = b - t * exp(-t/b) / (1 - exp(-t/b))

We also have:

1 - exp(-t/b) = P/a

and

exp(-t/b) = 1 - P/a

Eliminating the exponentials gives:

t_av = b - t*(1 - P/a) / (P/a) = b - t*(a - P) / P

which can be rewritten as:

b = t_av - t*(a - P)/P

This is a simple linear equation connecting a and b. (For P = 1 - exp(-t/c), this equation becomes c = t_av - t*(1 - P)/P. For my data, this value of c matches the value given by least squares very closely indeed.) Eliminating b gives:

exp(-t/(t_av - t*(a - P)/P)) = 1 - P/a

We can solve this equation numerically using Newton’s method, using P as the first approximation for a.

f(a) = exp(-t/(t_av - t*(a - P)/P)) - 1 + P/a

f'(a) = (t / (t_av - t*(a - P)/P))^2 * exp(-t / (t_av - t*(a - P)/P)) / P - P/a^2

h = -f(a) / f'(a)

a' = a + h

where a’ is the next approximation for a.

(Note that f(a) can be rewritten as:

f(a) = exp(-1/(t_av/t-(a - P)/P)) - 1 + P/a

The solution for a (and therefore b) therefore depends on t_av/t, rather than on t_av and t separately.)

For a fixed time limit t, we can calculate a, b and b/a from two clearly meaningful quantities:

t_av = average of the correct solution times within the time limit, and

P = number of correct solutions found within the time limit (applied to each problem individually), divided by the total number of problems in the problem set.

This calculation involves all the solution times symmetrically and matches the tail of the distribution well. It is also easy to carry out with a spreadsheet:

N.B. An average successful solution time of zero gives numerical problems here, as does a success probability of zero, but these should not be realistic scenarios. The calculated value of a can be more than 1 in some cases. Here is a plot of the P against t_av for a = 1 and a = 1.1, with t = 100 seconds:

The value of a is more than 1 when P is too large in relation to t_av.

A Three Parameter Model

2012-01-01T01:01:00.000-08:00

My earlier article, An Important Discovery, described a two parameter model for the distribution of correct solution times for solving chess problems. This article extends that model to three parameters, to enable the distribution of failures to be modelled accurately. I will illustrate this model using the results from my first pass through all ten problem batches in the Blue Coakley Experiment. The graph below shows the cumulative distributions of solution times (in blue), correct solution times (in green) and failed solution times (in red):

The least squares best fit to the cumulative distribution of solution times (correct or incorrect) is P_solved = 1 - exp(-t/27.1), where t is the time in seconds (this curve is shown in grey). 27.1 seconds represents my average time to find a solution, if we extend the curve to infinity. I will call this value c. The least squares best fit to the cumulative distribution of successful solution times is P_correct = 0.764*(1 - exp(-t/21.0)). This curve is also shown in grey. The values 0.764 and 21.0 represent my success rate and average correct solution time, if we extend the curve to infinity. I will call these values a and b respectively. The cumulative distribution of failures (shown in red) is obtained by subtracting the other two distributions, as its theoretical curve (shown in grey).

In my earlier article, An Important Discovery, I proved that the two parameter model estimated my average time to find a solution that is always correct as b/a = 21.0 seconds / 0.764 = 27.5 seconds. However, according to the blue curve above, my average solution time, with failures, is 27.1 seconds, which is only marginally smaller than b/a. This small difference would not be not enough to allow me to complete all my failed solutions correctly. There is clearly something wrong with the two parameter model.

The average values of my solution times within the time limit give an important clue. My average time to solve a problem (correctly or incorrectly) within the time limit was 20.1 seconds. My average time to solve a problem correctly within the time limit was 18.1 seconds, and my average time to solve a problem incorrectly within the time limit was 29.2 seconds. (The values calculated from the theoretical (grey) curves on the graph above closely match these numbers.) I was clearly taking longer on the harder problems that I was getting wrong, than on the easier problems that I was getting right.

I decided to divide the 60 second time limit into 5 second intervals, and work out the number of problems that I solved within each successive interval, and the corresponding number of successes:

Clearly, the number of problems that I solved within each successive time interval fell steadily. Overall, I got 660*100%/805 = 82% of my solutions right, but my success rate declined. The next chart shows my success rate for each of the five second intervals:

It is very clear from this graph that my success rate fell with time. The average value for the 5 second bands is 75.5% which is roughly the same as the value of a (76.4%) shown by the horizontal red line.

The mathematical form of my success rate curve is already determined the two cumulative probability distributions:

P_solved = 1 - exp(-t/c)
P_correct = a*(1 - exp(-t/b))

The corresponding probability distributions are:

dP_solved/dt = exp(-t/c)/c
dP_correct/dt = a*exp(-t/b)/b

My success rate is therefore given by:

(dP_correct/dt) / (dP_solved/dt) = a*c*exp(-(1/b-1/c)*t)/b

(This calculation divides the 60 seconds into an infinite number of infinitely narrow intervals, rather than twelve 5 second intervals above.) This calculation indicates that my success rate dropped off exponentially with time.

Why was b/a nearly as large as c in this experiment? This happened because the problems that I solved incorrectly took me almost as long as the two parameter model allows for me to solve them correctly. If I had continued with these problems, I would have eventually have got them all right, and they would then appear on my cumulative distribution of successful solutions, but with larger values than this model implies. The tail end of this cumulative distribution must rise less rapidly than this model predicts. Nonetheless, I solved 73% of the 900 problems correctly within the time limit, and the simple model is a good match up until the time limit. The remaining 27% of the problems would take longer to solve than the two parameter model predicts.

The cumulative probability distributions P_solved and P_correct can be used to calculate the average time that I used in solving problems up to any time t. The average time used on the problems that I had solved (correctly or incorrectly) up to time t is the integral of t*exp(-t/c)/c from 0 to t. Integrating by parts gives:

c - (c + t)*exp(-t/c)

The average time used on the problems that I had not solved by time t is:

t*(1 - (1 - exp(-t/c)))

The average time T used per problem up to time t is the sum of these expressions:

c - (c + t)*exp(-t/c) + t*(1 - (1 - exp(-t/c))) =

c - c*exp(-t/c) - t*exp(-t/c) + t - t + t*exp(-t/c) = c - c*exp(-t/c)

T = c*(1 - exp(-t/c))

The fraction of the problems that have been solved correctly by time t is:

s = a*(1 - exp(-t/b))

exp(-t/b) = exp[(-t/c)*(c/b)] = [exp(-t/c)]^(c/b) = (1 - T/c)^(c/b)

so that:

s = a*[1 - (1 - T/c)^(c/b)]

The graph below plots s against T (in green), and compares the results with the corresponding experimental values (in red, see my earlier article Rating vs. Time on the Clock):

s = 0 when T = 0, and s = a when T = c. a/c represents the fraction of the problems solved per unit time up to time c.

ds/dT = -a*(c/b)*(1 - T/c)^(c/b - 1)*(-1/c) = (a/b)*(1 - T/c)^(c/b - 1)

When T = 0, this becomes a/b, which represents the fraction of the easier problems solved per unit time. The curve for s becomes horizontal when T = c. Solving for T gives:

T = c*[1 - (1- s/a)^(b/c)]

The three parameter model based on a, b and c provides a more accurate picture than the simpler model based on a and b. It is clear from the three parameter model that b/a does not accurately represent the average time per problem to solve all the problems correctly.

Rating, Time and Score

2011-12-07T08:03:00.000-08:00

In the previous two articles, An Important Discovery and Rating vs. Time on the Clock, I discussed the possibility that the rating of human chess players increases by a fixed amount K whenever the time on their clock is doubled, as it does for computer programs. If this hypothesis is correct, it is possible to derive a simple equation relating a player’s expected score to the ratio of his speed to that of his opponent. Let:

r = player's speed / opponent's speed

If my hypothesis is right, the player should be K*log₂(r) points stronger than his opponent. According to the Elo formula, this rating difference should also be -400*log₁₀(1/s-1), where s is his expected score (see my earlier article Rating Points Revisited).

K*log₂(r) = -400*log(1/s-1)

-K*log(1/r) / log(2) = -400*log(1/s-1)

Let k = K / (400*log(2))

k*log(1/r) = log(1/s - 1)

r^-k = 1/s -1

s = 1 / (1 + r^-k)

The graph below plots the player’s expected score s against his speed ratio r for K = 200 Elo points:

If the player’s speed is equal to that of his opponent, his score is 0.5. If he is infinitely slow, his score is 0. If he is infinitely fast, his score is 1. This graph can also be interpreted as showing how much the player’s score improves if he is given more time on the clock than an equal opponent. The equation can be rewritten as:

r^k = s / (1 - s)

The right hand side of this equation is the player's score divided by his opponent's score. Interestingly, when k = 1, which occurs when K = 400*log(2) = 120, the speed ratio is equal to the score ratio; so if you are twice as fast, you score twice as many points.

Suppose that a player takes time t_s to achieve a score s, when an equal opponent has time t. This is equivalent to both players having equal time, but with the speed ratio:

r = player's speed / opponent's speed = t / t_s

(t / t_s)^k = s / (1 - s)

t / t_0.5 = 0.5 / (1 - 0.5) = 1

t_0.5 = t

(t_0.5 / t_s)^k = s / (1 - s)

t_0.5 = t_s * [s / (1 - s)]^1/k

Given the player’s time and his score, this equation gives the time that he would need to score 0.5. Having calculated t_0.5, we can calculate the score for any time t_s using:

s = 1 / (1 + (t_0.5 / t_s)^-k)

N.B. In this and the earlier article, I have used the logistic distribution (used by the USCF and others) rather than the normal distribution (used by FIDE).

Rating vs. Time on the Clock

2011-12-05T11:07:00.000-08:00

In my previous article, An Important Discovery, I speculated that for human chess players, each doubling of the time of the clock results in a gain of about 200 Elo rating points in playing strength. After reading the resulting discussion (in comments to the previous article and on Chess Tempo), it occurred to me that I had the data to test this hypothesis for problem solving. I sorted the results for my first pass through Coakley into ascending order of solution time, and calculated:

(1). The total time taken on all the problems that I had solved (correctly or incorrectly) within each of these solution times.

(2). The time taken on all the problems that I had not solved within each of these solution times (i.e. the solution time multiplied by the number of unsolved problems).

(3). The average time taken per problem for each of these solution times, i.e. (1) + (2) divided by the total number of problems.

(4). My score, as measured by the fraction of the problems that I solved correctly within each of these solution times (i.e. the number of problems solved correctly divided by the total number of problems).

The graph below plots my score against the average time taken per problem (in seconds):

I made almost linear progress here. I calculated the Elo point rating difference -400 * log(1/score - 1) for each point on the graph above (see my earlier article Rating Points Revisited). The graph below plots these rating point differences against the average time taken per problem:

From this graph we get:

Seconds Points Increase
6 -242 -
12 -62 180
24 170 232

An increase of 200 Elo rating points for each doubling of the total solution time taken appears to be about right, judging from this example.

An Important Discovery

2011-12-01T01:03:00.000-08:00

While puzzling over the results of the Blue Coakley Experiment, I stumbled across an important and unexpected discovery. When I plotted the cumulative distribution of (correct) solution times, aggregated over the first passes through all the Coakley problem batches, I noticed that it looked like an exponential curve. (N.B. The cumulative distribution of solution times gives the probability that the solution time will be less than any chosen time value.) I tried fitting an exponential curve to my results:

I had not expected this! I thought that the distribution of chess problem solution times would be highly dependent on the distribution in difficulty of the problems. The problems at the beginning of Coakley’s book are significantly easier than those at the end, but that does not appear to have affected the result. The green line on the graph is the exponential curve P = 0.764*(1 - exp(-t/21.0)), where t is the time in seconds. I got much the same result with the Woolum Experiment:

For this graph, the green curve is P = 0.775*(1 - exp(-t/7.22)). The problems in Woolum vary from mates in one, to mates in three with more than one variation and the King in the middle of the board. Again, the distribution of difficulty does not appear to have affected the result.

In general, the cumulative distribution of correct solution times appears to be of the form P = a*(1- exp(-t/b)). If I continue solving an infinite set of problems, in the same way, until I have (correctly or incorrectly) solved them all, a represents the fraction of the problems that I solve correctly, and b represents my average correct solution time. (N.B. The mathematics of the exponential distribution is summarised by the Wikipedia article http://en.wikipedia.org/wiki/Exponential_distribution.)

I constructed a simple mathematical model to explain the graphs above. I assumed that I had a fixed probability per unit time 1/b of solving a problem (irrespective of the time that I had already spent on that problem). It follows that the probability that I will solve the problem (correctly or incorrectly) within time t is given by 1 - exp(-t/b). I also assumed that I had a fixed probability a that my solution was correct. The probability that I will find a correct solution within time t is then given by a*(1 - exp(-t/b)).

If I do not stop at the time limit, but continue until I have (correctly or incorrectly) solved all the problems, my average solution time will be b, but a fraction 1-a of the problems will have been solved incorrectly. If I try again, with these remaining problems, my average solution time will again be b, and a fraction (1-a)^2 of the problems will remain to be solved correctly at the end. If I repeat this process over and over again, until all the problems have been solved correctly, my average solution time will be:

b + b*(1-a) + b*(1-a)^2 + b*(1-a)^3 +…. = b /(1 - (1-a)) = b/a

This model therefore estimates my average time to find a solution that is always correct as b/a. (N.B. I have assumed that I have perfect memory of my previous attempts to solve a problem, and I do not benefit unfairly from knowing when my solution is wrong.) I have assumed that the tail of the distribution continues to follow the same exponential curve. A more detailed study of my results shows that this assumption is over optimistic. Nonetheless, the model appears to be reasonably accurate for most of the problems in my problem sets.

The expression 1 - exp(-a*t/b) can be interpreted as the probability that I will find a solution that is always correct within t seconds. Equivalently, it can be viewed as the cumulative probability distribution of the difficulty of the problems as measured by the time that it takes me to solve them correctly. With this perspective, my failure to solve a problem is simply a consequence of not spending enough time on it.

If I work on randomly selected fraction a of an infinite set of problems until I solve them all correctly, and ignore the others, the probability that I will solve a problem correctly within t seconds is again given by a*(1 - exp(-t/b)). (N.B. Ignoring a randomly chosen fraction a of the problems reduces both the number of problems that I solve by a fraction a, and the time that I take to solve them by a fraction a.) This is the same result as I got when I tackled all the problems, with a probability a of getting each one right. Indeed, because there is fixed probability per unit time that I will solve a problem, I get the exactly same result however I allocate my time between the problems. Time management is futile here! My time management does, however, affect the distribution of failures. I can give up solving a problem whenever I choose, and greatly affect the failure distribution, without affecting the distribution of successful solutions. The distribution of solutions (correct or incorrect) is the sum of the distribution of successes and the distribution of failures, so it too is affected by my time management.

For computer programs, each doubling of the calculation speed increases the Elo rating by about 70 points, but I have not been able to find a corresponding number for human players. My guess is that each doubling of the time that a human player has on the clock increases his rating by about 200 points (up to about the standard time limit at least). The only evidence that I was able find is a clock simultaneous match between Topalov and the four player Irish team: http://2seeitlive.co.uk/icu/ This match was drawn with a rating difference of 367 Elo points, so having a quarter of the time on the clock appears to have reduced Topalov’s rating by about 400 points. This result is very interesting, but there were only four games, and data for 2000-2200 players playing clock simultaneous matches against four players rated 400 points lower would be relevant to the average player.

If my time to solve a set of problems correctly goes down from t₁ to t₂, the number of times t₂ needs to be doubled to get to t₁ is log₂(t₁/ t₂). This number can be fractional. If each doubling corresponds to 200 rating points, my improvement is 200 * log₂(t₁/ t₂) rating points. Let t_x be the average time that players rated x take to solve the problem set correctly. If I take time t, my performance is 200 * log₂(t_x/ t) + x = -200 * log₂(t) + 200 * log₂(t_x) + x (N.B. log₂(t) = ln(t) / ln(2).)

Consider two players, who are identical except that player 2 is 20% faster than player 1. Suppose that their distributions of (successful) solution times are given by P1 = 0.8*(1-exp(-t/20)) and P2 = 0.8*(1-exp(-t/16) respectively. The Elo formula gives the difference between their respective ratings and the rating of the problem set as -400*log(1/P1 - 1) and -400*log(1/P2 -1). (See my earlier article Rating Points Revisited.) Here is a graph showing these rating point differences for solution times between 5 and 60 seconds:

At first sight this graph looks balmy! The rating difference between the players and the problem set increases as the time limit increases - but this makes sense, because the players’ scores increase when they have more to think. More surprisingly perhaps, the difference in the players’ ratings relative to the problem set is at a maximum at around 30 seconds and smaller for both longer and shorter time limits. Again, this makes sense. If the players both have all day to think, but do not check their solutions any more thoroughly, they are both going to get 80% of the problems right. It is also not too surprising that if they are both given very little time to think, their scores will be closer. This method can give useful results, but it is less than ideal.

I used the method of least squares to fit exponential curves to my data points (Pi, ti). I tested the accuracy of the fit by calculating:

E = Sum_i { (P_i - a*x_i)² }

where x_i = 1 - exp(-t_i / b)

For a fixed value of b, E is at its minimum when:

dE/da = Sum_i { 2 * (P_i - a*x_i) * (-x_i) } = 0

Solving for a gives:

a = Sum_i { x_i * P_i } / Sum_i { x_i² }

I used a spreadsheet to calculate this value of a (and the corresponding value of E) whenever I entered a value for b. I adjusted b manually until I found the value that minimised E. This method worked better than the other curve fitting methods that I tried, and I used it to produce the graphs above.

The Blue Coakley Experiment

2011-12-01T01:02:00.000-08:00

For my next experiment, I used Jeff Coakley’s Winning Chess Exercises for Kids. The main part of this book has 900 problems, with nine to a page. I divided these problems into ten batches. The first batch was pages 1, 11, 21..., and he second batch was pages 2, 12, 22..., and so on. My schedule was:

Sa Mo Fr Fr We
Week 1: A1, A2, A3 Days: 1-7
Week 2: B1, B2, B3, A4 Days: 8-14
Week 3: C1, C2, C3, B4 Days: 15-21
Week 4: D1, D2, D3, C4, A5 Days: 22-28
Week 5: E1, E2, E3, D4, B5 Days: 29-35
Week 6: F1, F2, F3, E4, C5 Days: 36-42
Week 7: G1, G2, G3, F4, D5 Days: 43-49
Week 8: H1, H2, H3, G4, E5 Days: 50-56
Week 9: I1, I2, I3, H4, F5, A6 Days: 57-63
Week 10: J1, J2, J3, I4, G5, B6 Days: 64-70

Where A1, A2, A3.… are passes 1, 2, 3... of batch A, and similarly for the other batches. Passes 1 to 5 were spaced at the same intervals as in the previous two experiments, but I delayed Pass 6 for a week. I modified the Empirical Rabbit Timer to cope with problems occurring in clumps (pages in this case). Incorrect solution times were counted as more than 60 seconds irrespective of the actual time spent. I stopped the clock as soon as I thought that I had found the solution, and counted my solution as correct if I got the right idea and the right first move. Here is a comparison of my performance aggregated over the first passes through all ten batches with the corresponding results from the Susan Polgar and Ivaschenko 1b Experiments:

Coakley appears to be at a similar level of difficulty overall as Stage 5 of Ivashchenko (but the range of difficulty in Coakley is wider). Here is a chart for my first pass through each of the problem batches:

There is no clear evidence of improvement here. Any improvement that I did make has been swamped by the random variation in the results. The time limit method described in my earlier article Rating Points Revisited gives the table:

Sec Gain SD Gain/SD
0-5 67 59 1.14
0-10 17 38 0.44
0-15 32 50 0.65
0-20 10 43 0.24
0-25 16 39 0.41
0-30 39 35 1.11
0-40 38 35 1.08
0-50 23 43 0.53
0-60 23 43 0.53

It is possible that I made a worthwhile improvement, but any improvement is about one standard deviation at best, so we cannot draw any reliable conclusions.

Coakley’s Winning Chess Exercises for Kids

2011-12-01T01:01:00.001-08:00

I used Jeff Coakley’s Winning Chess Exercises for Kids for my next experiment, so I will give it a brief review.

This is the hardest of the four Coaklies and has a blue cover to distinguish it from the others. These books are not cheap, but the cost per problem is reasonable for this book. The diagrams are very clear, but it is difficult to get the pages to lie flat. A hard back would be a significant improvement, but no doubt more expensive! Dan Heisman says this is possibly the best intermediate problem book for all ages - “don't be fooled by the ...for Kids part of the title.” He says that he receives more positive feedback on this book than any other book that he recommends. He also says that the book is suitable for players rated 1300-1650. Nonetheless, it should also be useful for significantly stronger players, but they will need harder books too.

The main section of the book contains 100 pages of problems with nine problems per page. The problems increase in difficulty throughout the book. The first three problems on each page are checkmate problems, and the next three are combinations to win material. The final three are: a defensive problem, a general problem and an endgame problem. The general problem can be anything, but the objective is usually to secure a positional advantage. Coakley says that the three checkmate problems on a page share a theme, as do those for winning material. Fortunately he does not follow this too rigorously. I found that, in most cases, the previous problems did not give too much of a clue. Each page also has a tenth bonus question - e.g. “How many Knights are needed to checkmate a King in the middle of the board?” - but I did not bother with these. The problems in this book are all White to move, which is not good, but it is not a serious disadvantage either. The combinations that are labelled as “Material” all win more than a pawn - which makes them more dramatic - but winning humble pawns can also be important!

My statistics suggest that the level of difficulty of this book is about the same as Sergey Ivashchenko’s Chess School Stage 5, but the range of difficulty is wider. Coakley’s combinations are typically five moves deep. The overall level of difficulty is similar to Reinfeld’s 1,001 Winning Chess Sacrifices and Combinations - but Reinfeld contains a much higher proportion of both trivial problems and difficult problems. Reinfeld also has many errors, whereas I did not find any in Coakley, except for those noted in the very short Errata on his website, see: http://www.coakleychess.com/ The range of difficulty in Coakley is about twice as wide as in Susan Polgar’s Chess Tactics for Champions and Chess School 1b - but Coakley has almost twice as many problems as either of these books.

I did not notice any duplicate problems. Coakley says that the majority of his problems are original compositions, and that the others were taken from various sources and altered, usually beyond recognition. He says that these changes were made in order to clarify the solution, eliminate needless complications, adjust the level of difficulty, or otherwise make the problems more instructive. However, practical chess does not always offer neat and tidy solutions! I found that a significant proportion of the problems were either the same as problems that I have seen in other books or very similar.

Coakley recommends that the reader write down his answers for each page before looking at the solutions, and says that the book is intended to be at least a years study. His solutions are very detailed, showing why alternative moves do not work, and sometimes discuss many related positions. There are 103 pages of solutions for 100 pages of problems. My speed training did not do the book justice, and it would repay more leisurely study.

Coakley also has twelve small sections of “Lilly’s Puzzlers” - named after Lilly Pawn. These are typically completely crazy checkmate compositions. I must confess that I have not tried these, but they look amusing. I did not find the cartoons in this book quite as amusing as in Coakley’s green book - Winning Chess Strategy for Kids - but I liked the one in which a white rat informs us that no animals were harmed during the making of this book!

Overall, this is an excellent book. Coakley’s books provide a sound grounding in chess strategy, with a whopping 3,000 problems. The endgame treatment is more detailed in many respects than Alburt and Krogius’s “Just the Facts!” - which claims to cover all that is needed up to Master level. Coakley has trained a string of national junior champions, so his material is battle proven. I do not know of any series of books by a single author to compete with this one.

A Year of the Rabbit

2011-11-01T01:03:00.000-07:00

The Empirical Rabbit is one year old this month! It is time to summarise the highlights of the year, and indicate any changes to my perspective.

2011 Chinese
Year of the Rabbit

Lessons from Cognitive Psychology Nov 2010
This article summarises findings of cognitive psychology that are relevant to repeatedly solving chess problems. When the repetitions are closely spaced, performance increases rapidly, but long term memory retention is poor. When the repetitions are widely spaced, performance increases slowly, but long term memory retention is much better. Practical learning systems (e.g. those for learning foreign languages) space the initial repetitions closely, and gradually increase the intervals between successive repetitions.

Once Through vs. Repetition Nov 2010
This article remains a fair summary of the relative merits of solving an infinite stream of new problems, versus solving a smaller set of problems repeatedly. In practice, a compromise has to be made between practicing on as many problems as possible; and learning (and retaining) all the lessons that these problems have to teach. Another practical consideration is the limited supply of good problem sets - an infinite stream of problems is not available in the real world - so the real question is when to repeat!

7 Circles Dec 2010
This article remains a good summary of the limitations of Michael de la Maza’s 7 Circles method. MDLM solved 1,000 tactical chess problems seven times using the CT-ART 3.0 tactical trainer, halving the time interval between each repetition: 64 days, 32 days, 16 days, 8 days, 4 days, and 2 days. My experience was that by that by the time I had worked my way through Fred Reinfeld’s 1,001 Winning Chess Sacrifices and Combinations, and returned back to the beginning, I did not remember much. My accuracy had improved a little, but I was not obviously faster.

The Reinfeld Experiment Jan 2011
I decided to use Reinfeld’s 1,001 to experiment with expanding repetition intervals. After much experimentation, I decided to solve each batch of problems repeatedly at intervals of 1 day, 2 days, 4 days, 8 days, 16 days... My progress is summarised by the table:

Repetition: 1 2 3 4 5 6 7 8 9
Percent Score: 85% 93% 95% 95% 97% 97% 95% 92% 90%
Minutes/Problem: 3.5 2.7 2.0 1.5 1.3 1.3 1.2 1.0 1.2
Day Number: 1 2 4 8 16 32 64 128 256

This method was clearly highly effective at improving my performance at solving the problems that I was practicing, and helped me to refine my training methods.

The Bain Experiment Mar 2011
Inspired by Dan Heisman’s Novice Nook articles, I decided to apply my method to timed tests of the 388 simple tactics problems from John Bain’s Chess Tactics for Students. Again, I improved at the problems that I was practising - but what about problems that I had never seen before? Here is a histogram of the solution times for my first passes through three equally difficult problem sets:

I was astonished! The time limit method (see my October article Rating Points Revisited) estimates my improvement as 346 Elo points with a standard deviation of 134 Elo points, for a time limit of 5 seconds. Unfortunately, this very large apparent improvement appears to have been partly due to a high proportion of problems that were near duplicates.

Tactics Performance Measurement Apr 2011
This article discusses some important issues concerning tactics performance measurement. It has become clear that only reliable method of measuring tactics performance is to compile statistics for a large number of reliably rated players. The methods based on giving problems ratings and treating them as opponents (e.g. those used by the online tactics servers and my time limit method) can give only a rough indication.

The Woolum Experiment May 2011
For my next experiment, I used 792 problems from the next book on Dan Heisman’s list, which was Al Woolum’s Chess Tactics Workbook, repeating the problems on days 1, 3, 5, 7, 14, 26, 50 and 96. Here are the results for my first passes through six equally difficult problem batches:

The time limit method estimates my improvement as 136 Elo points with a standard deviation of 39 Elo points, for a time limit of 5 seconds. The proportion of near duplicates in Woolum is much more realistic than for Bain, so this result is more convincing.

The CHP Experiment Jul 2011
My next experiment used the next three books on Dan Heisman’s list: Jeff Coakley’s Winning Chess Strategy for Kids, Dan Heisman’s Back to Basics Tactics, and Bruce Pandolfini’s The Winning Way. I used the same repetition schedule as for the Woolum Experiment. Here are the results for my first passes through six equally difficult problem batches:

The time limit method estimates my improvement at the 583 problems in Heisman + Pandolfini as 71 Elo points with a standard deviation of 44 Elo points, for a time limit of 5 seconds. This result was not as convincing as for Woolum, but it was still encouraging.

Time to Move Up a Gear Sep 2011
This article discusses the adjustments that I made to speed up my tactics training. Most importantly, I decided to finish Woolum at Pass 8, and omit Pass 2 from my schedule for my next two experiments. (I later decided to end CHP at Pass 8, and to end the next two experiments at what was now Pass 7.)

The Susan Polgar Experiment Sep 2011
My next experiment was based on Susan Polgar’s Chess Tactics for Champions. I divided the 570 problems into just four problem batches to save time, but at the cost of limiting my chances of accurately measuring my progress. Nonetheless, the time limit method estimates my improvement as 64 Elo points with a standard deviation of 33 Elo points, for a time limit of 5 seconds. To avoid schedule overload, I had to extend the interval between Pass 6 and Pass 7 from six weeks to eight weeks. Nonetheless, here are my results for Passes 2 to 7 of the first batch of problems:

There is some evidence of a drop off in the number of these problems that I was able solve in under 5 seconds, but the number that I failed to solve within 40 seconds reduced steadily. (It may be significant that I was tackling harder problem sets in later experiments between Passes 6 and 7 of this experiment.)

The Ivashchenko 1b Experiment Oct 2011
This experiment was based 539 problems from Sergey Ivashchenko‘s Chess School 1b. I divided the problems into just four problem batches, which again limited my chances of obtaining a clear result. The time limit method estimates my improvement as about 70 Elo points at the longer time limits, but the standard deviation is at least as large - so we cannot draw any firm conclusions here.

Beyond the Blue Coakley

2011-11-01T01:02:00.000-07:00

I have already said that Jeff Coakley’s Winning Chess Exercises for Kids (the hardest Coakley) is the subject of my next tactics training experiment, but what is there for me to consider after that?

Richard Palliser’s Complete Chess Workout - 1,200 problems:
http://www.chesscafe.com/text/review621.pdf
http://www.chessville.com/reviews/TheCompleteChessWorkoutVsChessGems.htm
This book gets good reviews on Amazon and elsewhere, and is reasonably priced with a good cost per problem. The introduction to the book says that it was written for club players. I have bought a copy, and it looks very good. I was put off buying this book for a long time by the publishers statement that it is suitable for all levels of player. Fortunately, that is not true.

Ray Cheng’s Practical Chess Exercises - 600 problems:
http://www.chesscafe.com/text/review595.pdf
http://www.chessville.com/reviews/PracticalChessExercises.htm
I have had a copy of this book for some time. I will have to get round to using it! The book gets good reviews on Amazon and elsewhere. 600 problems is not a lot, but the book is cheap and the price per problem is good.

Maxim Blokh’s Combinative Motifs - “Many more than” 1,205 problems:
http://www.chesscafe.com/text/review535.pdf
I had a chance to look at a copy of this book at a local tournament. It is sorted by motif, with numbers in circles indicating the difficulty of the problems. I would have expected more than a small paper back for the money, but the diagrams are not too bad, and the price per problem is reasonable. This book is the paper version of the computer based trainer CT-ART 3.0, which has a good reputation. The latest version of this trainer, CT-ART 4.0 has 2,200 basic exercises and 1,800 auxiliary exercises:
http://www.chesscafe.com/text/review730.pdf
I have not seen any praise of the new problems added in CT-ART 4.0, and the paper version is more convenient.

Sergey Ivashchenko’s Chess School 2 - 1,188 problems:
http://www.chesscafe.com/text/review546.pdf
I expect that the format of this book is the same as for Chess School 1b, which I found to be excellent. The book is expensive, but the price per problem is reasonable. The computer version of this book is Chess Tactics for Intermediate Players:
http://www.chesscafe.com/text/review453.pdf
I found a web post saying this program is harder than CT-ART. Again, the paper version is preferable.

Chessimo - More than 6,000 problems:
http://www.chessimo.com/trainer/chessimo-en-1
It has tactics, strategy and endgame modules, with built in repetition. The endgame modules have a particularly good reputation, and the strategy modules look good too. Chessimo is not cheap, but it does appear to offer a lot for the money.

Yakov Neishtadt's Improve Your Chess Tactics - 700 problems:
http://lousyatchess.blogspot.com/2011/05/review-improve-your-chess-tactics.html
I have a copy of the forerunner of this book, Neishtadt’s Test Your Tactical Ability, which is very good. The new book contains more problems, and omits the very hard “Do You Know the Classics?” chapter. Kasparov’s coach has been quoted as saying that it is the best tactics book ever. I also had chance to look at this book at the tournament. The diagrams are tiny! My optician says that I have excellent corrected vision, but I do not believe that I could cope with this book. I will stick to the old book!

Other Possibilities
Unfortunately, most tactics books are claimed to be suitable for beginners to super GMs, which is not what I want at all! The Ultimate Chess Puzzle Book, Giant Chess Puzzle Book and Quality Chess Puzzle Book all appear to fall into this “something for everyone, but not much for anyone” category. However, as with the Palliser book, this may just be misguided marketing. None of these books are cheap, but the price per problem is reasonable. Sharpen Your Tactics (Lein) is widely recommended, but is out of print. Second hand copies are usually very expensive after shipping from the US. One review on Amazon said that the problems were 75% checkmates, which is not ideal.

Chess Combinations for Club Players

2011-11-01T01:01:00.000-07:00

I downloaded a copy of Chess Combinations for Club Players (CCFCP), when it was being offered at a discount and the price per problem looked good. CCFP does indeed have 2,056 problems, but they are very repetitive. I counted what appeared to be three exact duplicates in six problems on one occasion! Even when the problems were not exact duplicates, they were usually very similar to others in the set.

Convekta, the publisher the CCFCP software, says that it is suitable for club players and intermediate players, 1400–1800 Elo. By comparison, Convekta says Chess School 1b is suitable for Elo 1200-1500. This suggests that CCFCP should about 300 points harder than Chess School 1b - but most of the problems are about 300 points easier! The problems in CCFCP are grouped into folders labelled Level 3 to 17, with the problems at the higher Levels given higher ratings. Nonetheless, I found that I scored 98%, irrespective of the Level, which suggests that there is no significant difference in difficulty between the Levels. Here is an example of a problem rated 2800:

(N.B. You can enlarge the diagrams by clicking on them.) The solution to this problem is obvious, given the hint “annihilation of an important square defence”. Black plays 1...Bxa3 and meets 2.ba with 2...Nxc2+ forking the king and rook. There are problems harder than that in Bain! If that is a 2800 problem, I am Magnus Carlsen! Does that mean that all the problems in CCFCP are trivial? Not entirely. The only continuation on which you are tested is the game continuation, and with very few exceptions, that is trivial. However, for about 10% of the problems, there is a defence that is more problematic. Here is an example:

The only continuation on which you are tested here is 1...Qc6 Qxc6 2.Nxe7+ Kh8 3.Nxc6, but what about the other defences? 1...Qa7 is trickier, but defeated by 2.Qe6+ (if 2...Rf7 3.Nh6+). 1...Qd7 is also trickier, but is defeated by 2.Ne5. Nonetheless, given the hint “decoy to a knight fork,” what else could the solution be? This problem is at Level 10, but there are problems that are at least as tricky at the easiest Levels. Nonetheless, these problems do not have higher ratings than completely trivial problems in the same folder!

The program’s scoring was rather generous, giving me credit for finding the remaining moves when I failed to find the first move. Nonetheless, I found it hard to score more than 98%, partly because of the occasional tricky problem; and partly because only the game continuation is counted as correct, and there is often more than one way of capturing a piece, for example. The games from which the problems are are sometimes between lowly rated or unrated players, so the game continuation is not always the best. Nonetheless, in Learn mode (an untimed test), I was awarded a totally meaningless rating of 2290.

A minor gripe is that when the problem has Black to move, Peshka (the interface used by CCFCP) displays Black at the bottom of the board and White at the top. There is an icon on the lower left hand side of the board to flip the board round; but there is no box that I can check to say that I always want standard diagrams with White at the bottom. Another minor gripe is that there is no easy way of selecting a particular problem by number. It is possible to select a particular problem using the row of buttons just above the board. (In the diagram above, these buttons are green for the problems that I have completed, and grey for the others.) However, there is no space to number the buttons, and you have to click on the button to see the problem (AKA Task) number. It would be a big improvement if hovering the mouse over a button brought up a bubble showing the problem number.

By clicking the Play icon, it is possible to enter moves that are not part of the solution and receive replies from Crafty, the onboard computer engine. However, there is no way of entering alternative defences, and getting Crafty to show you how to defeat them. However, even if there was a way of making Crafty switch sides, that would still be more time consuming than simply being given the complete solution as in a problem book. Having to type in the solution makes sense if you are tested on all the reasonable defences, but not if you are only tested on the least challenging one.

The problems in CCFCP are sorted by:

* Annihilation of defence of a piece.
* Annihilation of an important square defence.
* Decoy of a king into a mating net.
* Decoy to a knight fork.
* Intercepting a piece defending another piece.
* Distraction of a piece defending another piece.

The “decoy of a king into a mating net” problems are all trivial mates in 2, in which the king is decoyed to a square on which it can be mated. The “intercepting a piece defending another piece” problems are mostly trivial removal of the guard problems. The “distraction of a piece defending another piece” problems are mostly trivial problems in which the line of defence of a piece is intercepted. Clearly, this list is only a subset of the tactical motifs that are included in most elementary tactics books. Given that most of the problems are so trivial, it would be better if they were in random order of motif.

The CCFCP software labels the problem folders as “Ctart for beginners2,” but beginners need proper solutions, so I do not believe that this product is suitable for beginners. (Indeed, the absence of proper solutions is likely to cause some difficulty even for strong club players.) Beginners would also need the problems to be reliably sorted into order of difficulty. They also need to become accustomed to standard chess diagrams (with White at the bottom), and used to spotting simple tactics from both sides of the board. Strong club players are not likely to want a set of highly repetitious problems that are mostly at Bain level, with hints, and without proper solutions for the few tricky ones. I cannot recommend this product to anyone!

More positively, having bought CCFCP, I did work my way through all the problems, and I believe there was some value in that. I also got plenty of practice at solving problems with the diagram “the wrong way round,” which seems to be a necessary survival skill for the online servers. I was also able try out the Peshka interface.

I am sorry to have had to give my first negative review, but the tactics books that I have recommended are all very much better than this product.

The Ivashchenko 1b Experiment

2011-10-01T01:03:00.000-07:00

My next experiment was with Sergey Ivashchenko‘s Chess School 1b, which comprises Stage 4 and Stage 5 of Chess School. I divided the book into four batches, which I labelled A to D. Batch A was the 1st, 5th, 9th problem, and so on. Batch B was the 2nd, 6th, 10th problem, and so on. Batches C and D were constructed in the same way. I found that I had bitten off more than I could chew here. The endgame studies at the end of Stage 5 caused me particular trouble, so I set them aside, which left me with 539 problems (134 or 135 problems per batch). The early part of my schedule was the same as for the Susan Polgar Experiment:

Sa Mo Fr Fr We Mo
Week 1: A1, A2, A3 Days: 1-7
Week 2: B1, B2, B3, A4 Days: 8-14
Week 3: C1, C2, C3, B4 Days: 15-21
Week 4: D1, D2, D3, C4, A5 Days: 22-28
Week 5: D4, B5 Days: 29-35
Week 6: C5 Days: 36-42
Week 7: D5 Days: 43-49
Week 8: A6 Days: 50-56
Week 9: B6 Days: 57-63

Where A1, A2, A3.… are passes 1, 2, 3... of batch A, and similarly for the other batches. For the first nine passes, the day on which each pass takes place was again given by the table:

Pass: 1 2 3 4 5 6 7 8 9
Day: 1 3 7 14 26 50 96 185 355

I again used the Empirical Rabbit Timer to time my solutions and collect the results. Incorrect solution times were counted as more than 60 seconds irrespective of the actual time spent. I counted my solution as correct if I got the right idea and the right first move. Here is a cumulative graph of my first pass through each of the batches:

The results show a rising trend for time limits of 30 seconds and more. Here is a comparison of my performances for my first pass through the batches of the Susan Polgar Experiment, Stage 4 and Stage 5:

(N.B. I have aggregated the results over all the problem batches here, for each problem set. 0-5 denotes 0-4.999... seconds, and similarly for the other “buckets”.) It is clear that I found Stage 4 harder than SPolgar, despite any improvement that I made between the corresponding batches for the two experiments. It is also clear that I found Stage 5 harder than Stage 4. Here are the results for my first pass through each batch for Stage 4:

I appear to have made good progress here, particularly for the longer time limits. The results for my first pass through each batch for Stage 5 were less tidy:

I appear to have made good progress here too, for time limits over 30 seconds, except for batch D, which appears to have been harder then the others. For Stage 4, the problems are divided into sections according to the problem type. My method of dividing the problems into batches ensures that each of the four batches contain (nearly) equal numbers of each type of problem, which helps to equalise the difficulty of the batches. In contrast, the problems in Stage 5 are in random order, which does not help here.

Applying the method of calculation that I described in my earlier article, Rating Points Revisited, to the whole of 1b, gives the table:

Sec Gain SD Gain/SD
5 46 55 0.84
10 8 43 0.19
15 1 38 0.03
20 -13 38 -0.34
25 -24 44 -0.55
30 41 47 0.87
35 49 54 0.91
40 59 91 0.65
45 68 65 1.05
50 78 84 0.93
55 94 106 0.89
60 71 114 0.62

Where Sec is the time limit, Gain is the Elo points gain, and SD is the corresponding standard deviation. The accuracy here is one standard deviation at best, so we cannot draw any reliable conclusions, but it is likely that I made a significant improvement. The same method of calculation suggests that Stage 4 is about 100 Elo points harder than SPolgar, and Stage 5 is about 50 points harder still.

Sergey Ivashchenko’s Chess School 1b

2011-10-01T01:01:00.000-07:00

For my next experiment, I used Sergey Ivashchenko’s Chess School 1b, so I will give it a brief review. This book turned out to be excellent for my purposes. Ivashchenko wrote the first three books of the Chess School series:

http://www.chesscafe.com/text/review519.pdf
http://www.chesscafe.com/text/review532.pdf
http://www.chesscafe.com/text/review546.pdf

Over 200,000 copies of the previous edition of this book were sold in the Soviet Union in the late 1980's.

Chess School 1a and 1b form the basis of the misleadingly titled Chess Tactics for Beginners training software; and Chess School 2 forms the basis of the Chess Tactics for Intermediate Players.

Chess School 1b book provides 580 problems at a good price. I found the problems to be both interesting and instructive, with the level of difficulty in a reasonably tight range. The solutions are not many moves deep, but I often found them difficult to see. My statistics show that I was significantly slower at solving these problems than those in Susan Polgar’s Chess Tactics for Champions. I recognised some of the problems from other books, but most of them were new to me. The diagrams are large and clearly printed, but the font is a little non-standard. I soon got used to the diagrams - but standard diagrams would have been better. The book wastes no space on words. The only words are titles and headings in four languages. The book is very well bound with a hard (or fairly hard) cover. It is much more durable than a Western paperback.

The book is split into two roughly equal sized sections: Stage 4 and Stage 5. The problems in Stage 4 are further divided according to the type of problem, e.g. “Win a queen,” “Win a bishop” or “Draw.” I found these clues to be less helpful than the traditional ones (which is good). The title of section of Stage 4 is “How to proceed?,” which gives no clue at all (which is better). There are no checkmates in Stage 4. (More accurately, any checkmate in Stage 4 can be avoided at the cost of material, with the exception of an unintended mate in 2, see below.) The problems in Stage 5 are all “How to proceed?,” and are mostly checkmates. Both Stage 4 and Stage 5 conclude with endgame problems. The endgame problems in Stage 4 are either simple tactics or tests of basic endgame knowledge; but the endgame problems in Stage 5 are more difficult, and I found some of the solutions hard to follow (the lack of words was a disadvantage here). The problems under each heading in both Stages appear to be in random order of difficulty. My statistics show that I was significantly slower at solving the problems in Stage 5 than those in Stage 4.

There are some mistakes in the book. Two of the problems at the end of a page should be under the heading of the next page: 929 “Gain a knight” should be “Draw”, and 959 “Draw” should be “How to proceed.” The solution to 838 “Gain a rook” works, but 1.Bc6+ mates in 2. The solution to 1240 is given as 1.Bf4 Qh5 2.Qc4+-, but 2... g5! stops the mate, see:

http://dunnechess.dyndns.org/review.Ivashchenko.The_Manual_of_Chess_Combinations_2.html

This link gives the better solution 1.Bxe7+ Rxe7 (Kg7 2.Bf8++ Kh8 3.Qg7#) 2.Rd8 Ne8 (Re8 3.Qf7#; Kg7 Qxe7+ +-) 3.Rxe8+ Kxe8 (Rxe8 4.Qf7#) 4.Qc8#. None of these mistakes detract from the usefulness of the book, however.

Overall, this is an excellent book. My only real criticism of this book (and by extension the other books in the series) is that there are not enough problems at each Stage to take you from one Stage to the next.

Rating Points Revisited

2011-10-01T01:00:00.000-07:00

In my previous article, How Many Rating Points Is That?, I introduced a method for estimating my tactical rating point improvement from my improvement in solution times. After applying this method to the results of my tactics training experiments, it has become clear that the method can be improved upon.

In my earlier article, I used a scoring graph that closely followed that used by Chess Tactics Server (CTS). With CTS (and other tactical servers), the problems are given ratings and treated as opponents. Solving a problem quickly counts as a win for the user, and a failure or a slow success counts as a loss. For a correct solution, the scoring graph provides a score between 0 and 1, depending on the time spent solving the problem:

I found that I got similar results when, in place of this scoring graph, I simply scored 1 whenever I solved the problem in under 5 seconds and 0 otherwise. Superficially, it appears that just counting the number of solution times that fall within a time limit should be less accurate than making use of the precise values of all those solution times. However, in practice, the standard deviations given by the simple time limit method were often smaller (in relation to the rating improvement) than those obtained using the scoring graph. The main problem with the CTS method (and those used by other tactics servers) is that the resulting score does not relate directly to what happens in a real game. The score given by simple time limit method, on the other hand, does have a direct relationship with what happens in a real game.

The score given by the simple time limit method estimates the probability that you will find the tactic within the time available. If there is a single win or lose tactic per game at the level of the tactics problem, this probability is the same as your probability of winning the game (provided that the time limit matches the time available in a game). In practice, it is more likely that if you fail to spot a tactic, you will lose (or fail to gain) half a point. If there is only one such tactical chance per game, the score given by the time limit method will, in this case, over estimate your game score. However, if there are two tactical chances per game (attacking or defensive), and spotting each tactic in time earns you half a point, the time limit method gives a realistic estimate of the probability of winning the game.

(Suppose that there is a probability p that you will spot a tactic to earn half a point. There is then a probability 1 - p that you will not spot the tactic. Suppose also that there is the same probability p that you will spot a second tactic to earn another half point. The probability that you will earn two half points is p ^ 2, the probability that you will earn one half point is 2 * p * (1 - p), and the probability that you will earn no points is (1 - p) ^ 2. On average, you will gain p ^ 2 + p * (1 - p) = p points.)

If there is more than one tactical chance per player per each game, the time limit method underestimates the rating benefit of spotting those chances. The number of tactical chances per game will clearly depend on how sharp the positions are. You can get a feel for the numbers here by analysing your own games on a computer, or simply by playing against a computer. Taking the score given by the time limit method as the average number of points that you can expect to win tactically per game (at the level of tactical difficulty of the problems concerned) is likely to be conservative for lower rated players.

Previously, I converted my scores for solving chess problems into rating points using the English Chess Federation (ECF) method. This was adequate when the scores were near to 0.5, but the Elo method gives good results over a wider range:

http://en.wikipedia.org/wiki/Elo_rating_system

For this method, your expected score s is given by:

s = 1 / (1 + 10 ^ -(d/400))

Where d is your rating minus that of that of your opponent (or the problem set here).

(The ECF method approximates this curve with a straight line from the bottom left hand corner (-400,0) to the top right hand corner (400,1).) Solving the Elo equation for d gives:

d = -400log(1/s - 1)

In this context, the score s is taken to be the fraction of the problems that you were able to solve within the time limit. To use this result, we can:

(1). Time ourselves solving a series of equally difficult problem batches.
(2). Calculate the values of s for each batch.
(3). Calculate the values of d for each batch.
(4). Plot the values of d on a graph.
(5). Fit a straight line to the graph.

(N.B. I assume that we time ourselves on our first pass through each problem batch, that there are no duplicate problems, and that the batches are all representative of the tactics that we will meet in real games.) Here is the graph for the Ivashchenko 1b Experiment, with a time limit of 55 seconds:

The line extends from first problem of batch A to the last problem of batch D, and the red dots are at the mid point of each problem batch. The graph suggests that I improved by about 100 Elo points, but the standard deviation is also about 100 Elo points (due to the large scatter), so we cannot draw any firm conclusions here.

(N.B. In my experiments, I stop the clock as soon as I believe that I have found the solution. This protocol enables me to estimate the number of problems that I can solve at different time limits; but I would get a higher score if I continued checking until the time limit concerned expired. However, the resulting underestimation of my performance probably is not significant, given all the other uncertainties.)

I believe that this method is an improvement on my previous one, and on those used by the online tactical servers. However, it is clear from the discussion above that all these methods have serious limitations. The only really sound approach here is to test a large number of accurately rated players at solving the problem set, as discussed in my earlier article: Tactics Performance Measurement.

The Susan Polgar Experiment

2011-09-01T03:01:00.000-07:00

My next experiment was with Susan Polgar‘s Chess Tactics for Champions. This book contains 570 problems. I divided them into four batches (of 142 or 143), which I labelled A to D. Batch A was the 1st, 5th, 9th problem, and so on. Batch B was the 2nd, 6th, 10th problem, and so on. Batches C and D were constructed in the same way. The early part of my schedule was:

Sa Mo Fr Fr We Mo
Week 1: A1, A2, A3 Days: 1-7
Week 2: B1, B2, B3, A4 Days: 8-14
Week 3: C1, C2, C3, B4 Days: 15-21
Week 4: D1, D2, D3, C4, A5 Days: 22-28
Week 5: D4, B5 Days: 29-35
Week 6: C5 Days: 36-42
Week 7: D5 Days: 43-49
Week 8: A6 Days: 50-56
Week 9: B6 Days: 57-63

Where A1, A2, A3.… are passes 1, 2, 3... of batch A, and similarly for the other batches. This schedule is the same as for the Woolum and CHP Experiments, but with the omission of the third pass in those experiments. For the first nine passes, the day on which each pass takes place is given by the table:

Pass: 1 2 3 4 5 6 7 8 9
Day: 1 3 7 14 26 50 96 185 355

From Pass = 3 onwards, the pass takes place on Day = 1.92 ^ Pass, rounded to the nearest whole number. I used the Empirical Rabbit Timer to time my solutions and collect the results. Incorrect solution times were counted as more than 40 seconds irrespective of the actual time spent. I counted my solution as correct if I got the right idea and the right first move. Here is a comparison of my performance on my first pass through batch A with corresponding performances in previous experiments:

Where H+P denotes Heisman + Pandlofini (in the CHP Experiment). (0-5 denotes 0-4.999..., and similarly for the other “buckets”.) SPolgar was clearly harder than the previous problem sets, but I nonetheless did better on my first pass through SPolgar than on my first pass through Bain, which is much easier. Here are the results for the first six passes through batch A:

Despite reducing the number of passes in the first eight days from four to three, I made good progress here. Not surprisingly, I improved at the problems I was practicing, but what about problems that I had never seen before? Here is my performance on my first passes through batches A-D:

(0-5 denotes 0-4.999... seconds, and similarly for the other “buckets”.) Again, I appear to have made progress, but a large number of difficult problems in batch D rather rained on my parade! (A poor performance on the day does not appear to be the explanation here. On the first pass, 33% of the problems that took me over 40 seconds were in batch D. On the second pass 34% of the problems that took me over 40 seconds were in batch D. Batch D appears to be harder than the others.) The rough calculation that I presented last month gives the graph:

SPolgar: Rating Difference vs. Problems Learned

This graph, in conjunction with that for Heisman + Pandolfini (H+P), suggests that SPolgar is about 130 Elo points harder than H+P. The graph also suggests that my improvement during the SPolgar Experiment was about 25 Elo points. (The corresponding standard deviation is about 14 Elo points, so there is a high level of uncertainty in this number.) 25 points may not sound much, but 25 points in one month would be excellent progress! My graphs also suggests that, between starting the first batch in the Bain Experiment and finishing the last batch of SPolgar, I improved by about 100 Elo points + the rating difference between Bain and SPolgar. I expect that SPolgar is a good 200 Elo points harder than Bain. I do not have any good way of checking this, but someone who is unfamiliar with both problem sets could time themselves solving both, and estimate the rating difference.

Susan Polgar’s Chess Tactics for Champions

2011-09-01T02:52:00.001-07:00

For my next experiment, I used Susan Polgar’s Chess Tactics for Champions, so I will give it a brief review. This book turned out to be excellent, and served my purpose very well.

The book provides 570 problems at a good price. The problems are both interesting and instructive, with the level of difficulty in a tight range.

My statistics show that I was significantly slower at solving these problems than those in Dan Heisman’s Back to Basics: Tactics.

The problems are not difficult, but the general level is a step up from that of elementary tactics books. I would have thought that this was squarely in the mass market, but had difficulty finding problem books at this level.

The book is split into chapters according to theme, e.g. forks and double attacks, pins. These chapters cover all the main types of combination. Each chapter has an introduction showing several illustrative examples. These introductions are good, but no better than those in other popular tactics books. The final two chapters: Sibling Positions and Twenty-five Famous Combinations do not provide any additional problems. The problems at the beginning of each chapter are easier, but apart from that, they appear to be in random order of difficulty. (It would have been better for me if the problems within each chapter had been reliably sorted into order of difficulty, which would have made my problem batches of more equal difficulty.)

I did not notice any exact duplicate problems, and there are few near duplicates. Three of the problems: p126 #17, p238 #18 and p276 #14 are labelled as White to move, but they should be Black to move. I did not notice any significant errors in the solutions - but I was using the book for speed training - and I am not a Grand Master! I recognised some of the problems from other books, but most of them were new to me.

The diagrams in this book are small, but very clearly printed in the standard chess font on matt finish paper. I found them easy to use for speed training despite their size. The book has is lot of white space with just three (or in some chapters two) diagrams per page. The book has 347 numbered pages (plus some additional pages), so you get a lot of paper for your money. The book also has a quality feel for a paperback.

Overall, the book is excellent, but would have been twice as good if the white space, chapter introductions and the final two chapters had been cut to make space for twice as many problems!

Time to Move Up a Gear

2011-09-01T02:48:00.000-07:00

It was time to review my progress, and look for ways to speed up my tactics training. There were a number of issues to address.

Redo problems that give me trouble? In the Woolum Experiment, I repeated the 25% of the problems that had given me the most trouble on Pass 6. I repeated them half way between Pass 6 and Pass 7, to maximise their effect on long term memory. (N.B. Passes 1 to 8 took place on days 1, 3, 5, 7, 14, 26, 50 and 96.) The chart below shows my results aggregated over all six problem batches for Passes 3 to 6:

(0-5 denotes 0-4.99... seconds, and so on.) Not surprisingly perhaps, repeating the harder problems did not noticeably slow the decline in the number of problems that I was able to solve in under 5 seconds. It also did not clearly reduce the number of problems that took me over 20 seconds, which does not appear to have been a problem anyway. I repeated this measure on the first three batches in the CHP Experiment (which had the same schedule), and it did not show any benefit there either. I decided that the peak at Pass 4, in the number of problems that I could solve in under 5 seconds, was probably the result of doing too much work in the first 8 days, rather than too little subsequently.

Omit Pass 3? In the CHP Experiment, I found that repeating the 25% of problems that gave me the most trouble on Pass 1 (half way between Pass 1 and Pass 2) had almost no benefit a week later. I also found that restricting Pass 3 to the 25% of problems that gave me the most trouble on Pass 2 had no discernible effect on my results nine days later. Since replacing Pass 3 with something ineffective did not have any lasting effect, it seemed highly likely that I could omit Pass 3 entirely without any lasting effect. Having to work harder as a result of omitting Pass 3 might even improve the learning process, particularly for harder problems. It was worth a try! The resulting schedule was remarkably close to my original conception for the Reinfeld Experiment, but with more precise timings.

Move to harder problems? In the CHP Experiment, the problem set from Jeff Coakley’s Winning Chess Strategy for Kids appeared to have been ineffective. I exceeded my best performance in the Bain Experiment from the outset, and did not measurably improve. This result suggests that I should move on to more difficult problems. The next three books on my list were: Susan Polgar’s Chess Tactics for Champions, Sergey Ivashchenko’s Chess School 1b and Jeff Coakley’s Winning Chess Exercises for Kids. The Susan Polgar book looked the easiest of the three, and the other two looked to be of a similar level of difficulty, with Coakley probably the harder. Coakley had the most exercises, and split well into ten problem batches, which looked likely to be of roughly equal difficulty. The other two books looked best suited to short experiments to adapt my methods to harder problems. (I decided to use only four problem batches for Polgar and Ivashchenko to speed things up, at the cost of making it more difficult to measure progress.)

Continue with speed training? Another issue that I had to address here was whether to continue with my speed training in which I gave myself full marks for finding the right first move and the right idea. This is ideal for developing my ability to spot tactics quickly, but potentially encourages impatience and sloppiness. However, that is only one cause of mistakes - the more important cause is failure to see moves. (N.B. When speed training, I stop the clock when I think that I have found the solution, but then I check my solution. If I find a mistake, I have another try at finding the correct solution. If I fail to find a solution within the time limit, I have another quick look at the position. It is surprising how often the solution pops up as soon as the time pressure has gone! If even that fails, I always take the trouble to study the solution carefully.)

When to stop? My pattern matching model suggests that Woolum was sampled from about 1,000 patterns (see my earlier article A Pattern Matching Model). When I get to Pass 9 of Woolum, I will have solved about 3,000 other problems one or more times since beginning Pass 8. These 3,000 problems should contain about 95% of the patterns in Woolum (see my earlier article Distinct Random Selections). There will then be little point carrying out Pass 9. Any fall in my performance at Woolum should mostly be the result of fading memory of the positions, rather than forgetting the underlying patterns. (If you take on problems at a slower rate, you may have to carry out extra repetitions to reach this stage.) If I had missed out Pass 3, I would have needed only seven passes to reach this stage. After Pass 1, subsequent passes take about half as long, so the level of effort in doing seven passes is about four times that of doing one pass.

In summary, decided to:

* Not use extra passes for problems that were giving me trouble.

* Remove Pass 3 from my schedule.

* Move on to Polgar, Ivashchenko and then to Coakley.

* Continue with speed training.

* Finish Woolum at Pass 8.

[I later decided to also stop CHP at Pass 8.]

Dan Heisman’s 7-10 Basic Tactics Books

2011-08-01T01:17:00.000-07:00

Dan Heisman suggests that there may be about 2,000 basic tactics patterns, and recommends 7 tactics books that he says together “may” contain about 97% of these tactical patterns:

Chess Tactics for Students - John Bain
The Chess Tactics Workbook - Al Woolum
Winning Chess Strategy for Kids - Jeff Coakley
Back to Basics: Tactics - Dan Heisman
The Winning Way - Bruce Pandolfini
Winning Chess Traps - Irving Chernev
Bobby Fischer Teaches Chess - Bobby Fischer

He also suggests that if you need more basic patterns, “throw in”:

Starting Out: Chess Tactics and Checkmates - Chris Ward
Checkmate for Children - Kevin Stark
The Art of the Checkmate - Georges Reynaud & Victor Kahn

The purpose of this article is to review these books with regard to their suitability for learning basic tactical patterns. (See my earlier articles Dan Heisman’s Basic Tactics Training, Discrete Random Selections and A Pattern Matching Model for discussion of pattern recognition. The number of simple common tactical patterns clearly depends on how simple and common they have to be to qualify. If 2,000 patterns are randomly sampled “with replacement,” we need to sample about 7,000 to achieve 97% coverage. For 1,000 patterns, we need to sample about 1,600 to achieve 80% coverage, which looks more realistic for these books.) In this article, I will assume that you already know how to find basic tactics; but want to become faster and more accurate, using learning by repetition.

I am writing from Yorkshire, and Yorkshire folk are famous (some would say notorious) for their reluctance to part with their money. I would be letting my adopted county down if I did not fully consider value for money. The relevant statistics are summarised by the table below:

Book Cost Problems Per Problem
------------------------------------------------------
Bain £9.20 388 2.4p
Woolum £9.99 792 1.3p
Coakley £20.65 205 10.1p
Heisman £10.67 434 2.5p
Pandolfini £8.80 150 5.9p
Chernev £13.31 300 4.4p
Fischer £3.56 275 1.3p
------------------------------------------------------
Ward £12.74 150 8.5p
Stark £12.99 150 8.7p
Reynaud & Khan £8.99 80 11.2p
------------------------------------------------------

(I do not have copies of Ward or Stark, so I have had to estimate the number of problems in these two books, see below. The costs include delivery, and are what I paid for each book, or prices from Amazon or the London Chess Centre, as appropriate.)

Bain is the easiest of these books, and passed the practical test in the Bain Experiment. The book does not appear to have a British supplier, but I managed to get a second hand copy from a supplier on Amazon (.com) for $1.25 plus $12.49 USPS shipping! My copy is not the new 10th Anniversary Edition, but as far as I know, the new edition is not materially different. My edition has 390 problems that are not exact duplicates, but this includes two duds, so I got 388 problems for my money, which is reasonable value. The diagrams are large and clear, but not as good as those in Woolum. A large proportion of Bain’s problems are either simple examples from Reinfeld, or simplified versions of more complicated ones. In addition to the exact duplicates, Bain also has many problems that are the same as another problem within the book, but on a different part of the board, or with one move less at the beginning. Bain does not provide solutions, as such, but the hints are so detailed that a solution is hardly necessary. Susan Polgar’s World Champion’s Guide to Chess has been suggested on the web as a more widely distributed alternative to Bain.

Woolum is the star as far as value for money is concerned, and it passed the practical test in the Woolum Experiment. The book is not expensive, and has 792 problems in the main section. The problems in the “Canadian Corner” probably duplicate those in the Coakley books (and the triple Lloyds and mazes are not suitable for speed training anyway), so I have not included them in the total. Woolum is heavy on checkmate problems, but Bain and Heisman are both light on them. The book is well presented with large and exceptionally clear diagrams. There are a significant number of exact duplicate problems and near duplicates, but less as a proportion than in Bain. I recognised a number of problems from Reinfeld and other books. There are some errors in the diagrams, and many errors in the solutions, but my Woolum Errata should help here. Woolum is harder than Bain. Nonetheless, most of the problems are easy, but some are not at all easy!

Coakley scores poorly if you are just looking for problem book. It is an expensive book, and has only 205 problems that are clearly numbered as exercises. The smaller diagrams are good, but I had some difficulty with the larger diagrams. Coakley failed the practical test in the CHP Experiment. The problems appear to have too easy for me at that stage, and I did not make a measurable improvement at solving fresh problems from the book. Perhaps there just were not enough problems for me to show a measurable improvement, but the book loses either way. Nonetheless, Coakley is a great book if you are looking for an entertaining introduction to the basics of chess strategy, with some simple tactics examples thrown in. How can you dislike a book which opens a section with “rooks are usually glad when pieces and pawns are exchanged,” or with GM Potatowoski offering to “help you think like a potato?” The illustrations are also very amusing. I particularly like the one where a bishop (you can tell from his hat) is fiddling with a safe. The caption reads: “I wish I could remember the combination to this safe - I left my sandwiches in there last week!” This book is a real hoot. The lessons are basic, but the examples are very good, and Coakley does an excellent job of showing how tactics arise out of strategy. This is a great book for its intended purpose, but I do not believe that it is good value as a source of tactics problems.

Heisman is another star book. It is not expensive, and contains a reasonable 434 numbered exercises. It also passed the practical test in the CHP Experiment. The book is harder overall than Woolum. The diagrams are good, but not as good as those in Woolum. The book includes many simple pieces of tactics that do not seem to find their way into other books.

Pandolfini is primarily an opening traps book, and is of questionable value for money as a problem book. It is not expensive, but contains only 150 diagrammed positions that can be used as problems. The diagrams are large and clear. The book appeared to pass the practical test in the CHP Experiment, but with only 150 problems, I cannot claim any degree of statistical significance here. I thought that the problems were very easy, but my statistics suggest that the book is at about the same level of difficulty as Heisman. Pandolfini’s problems are mostly very simple, but you have to look at the whole board to find them, which takes vital seconds. Pandolfini is a reasonable choice here, but I expect that there are better value problem books out there. If you make horrible blunders in the opening or your opponents do, it might be useful to use the book as the author intended.

Chernev is another opening traps book. It is out of print, but I managed to get a battered second hand copy from a supplier on Amazon (.com) delivered for about the same price as a new one sourced locally. The book contains 300 problems. These problems are all available free online, see http://wwwu.uni-klu.ac.at/gossimit/c/chess.htm, but the book is more useful. The book is an old classic, but the diagrams are small and fuzzy, and some of the combinations are more than ten moves deep. I do not believe that this book is suitable for speed training. I expect that it will come in useful one day, but for now, it has been assigned to the bookshelf.

Fischer is seriously cheap and contains 275 problems. I did not have any difficulty with the diagrams, despite the negative reviews of the print quality on Amazon. Unfortunately, the problems in this book (which are all back rank problems) are nearly all very easy, and do not represent a progression from Heisman. The format of the questions (e.g. can White mate?) is not very suitable for rapid pattern recognition training. This book too has taken up residence on the bookshelf.

Ward is primarily a “how to” book rather than a problem book, see:
http://www.chessville.com/reviews/StartingOutChessTacticsandCheckmates.htm
http://www.chesscafe.com/text/review569.pdf
There appear to be ten or so problems at the end of four chapters and 100 quick-fire puzzles, perhaps 150 problems in all. I expect that Ward is a good book, but it does not appear to offer many problems for the money.

Stark also appears to be primarily a “how to” book rather than a problem book, see:
http://www.newinchess.com/Checkmate_for_Children-p-941.html
There appear to be about 24 pages of checkmate problems at six to the page, perhaps 150 in all. Again, it looks like a good book, but does not appear to offer many problems for your money, and the problems that you do get look very elementary.

Renauld & Kahn is an outstanding old classic and I have had a copy for many years. This book is essentially a “how to” for checkmates, with some instructive and well annotated games and 80 exercises. I have marked up the examples in my copy so that too they can be used as exercises. Even so, you do not get many problems for your money.

The Rabbit’s Choice for the task in hand is Bain, Woolum and Heisman. Susan Polgar’s World Champion’s Guide to Chess may be a good alternative to Bain. Sergey Ivashchenko’s Chess School 1a is worth considering as a supplementary source of problems. Susan Polgar’s harder book, Chess Tactics for Champions forms the basis of my next experiment. As a follow on from that, I have not found anything better than Chess School 1b, followed by Jeff Coakley’s harder book Winning Chess Exercises for Kids.

How Many Rating Points Is That?

2011-08-01T01:08:00.000-07:00

I have been asked how many rating points I have improved. As I have said, unless you make a very large improvement, it is not possible to estimate your rating improvement with a useful degree of accuracy from the results of a practicable number of games. For most players, year to year rating variations are mostly random. However, given the large volume of data that I now have, I can now roughly estimate the number of rating points by which I have improved at spotting simple tactics quickly.

My first clue on how to estimate my improvement was the rating system used by Chess Tactics Server (CTS). With CTS, the problems are given ratings and treated as opponents. Solving a problem quickly counts as a win for the user, and a failure or a slow success counts as a loss. For a correct solution, CTS assigns the user a score between 0 and 1, depending on the time the spent solving the problem:

See: http://chess.emrald.net/time.php. (You can click on the diagrams to enlarge them.) I approximated the CTS scoring graph above with exp(-0.099021*(t-3)) for t>3:

Smoothed CTS Scoring Function

This graph closely tracks that used by CTS. The CTS link above also says that the time for which the result is 0.5 is extended at the rate of 1 second for each 20 Elo points difference between the rating of the problem and the user. I used a fixed 30 second time limit in the Bain, Woolum and CHP experiments, so the unmodified graph looks appropriate. Extending the graph for higher rated problems is highly questionable anyway. It would make no sense to give me extra time on the clock in a game against a stronger opponent, and then fail to take this into account when working out my rating! The precise shape of the scoring graph does not appear to matter very much. I get similar results if I score 1 whenever I get the solution in under 5 seconds and 0 otherwise. Other tactical servers use very different graphs, e.g. see: http://www.chess.com/tactics/help.html#rating and http://chesstempo.com/user-guide/en/tacticRatingSystem.html#blitzRating.

How do we convert these scores into rating points? My clue here was the calculation used by the English Chess Federation (ECF) rating system. In this system, your rating is calculated by adding 50 points to your opponent’s rating if you win, adding nothing to it if you draw, and subtracting 50 points if you lose. Your rating for next year is then the average of these values for this year’s games. There are some refinements to this system that need not concern us here, see: http://en.wikipedia.org/wiki/Chess_rating_system.
With my simplifications, your new rating is the average rating of your opponents, plus a rating difference, which is the average of 50 points whenever you win, no points whenever you draw, and -50 points whenever you lose. Of course I do not know the average rating of the problems that I am solving, but this cancels out when we calculate rating differences. (N.B. The ratings of the problems will depend on the time limit that I impose. If I reduce the time limit, the problems become more to difficult to solve within that time limit, and their ratings will therefore be higher.) We can find my score for each problem in a problem batch using the graph above, work out my average score, multiply it by 100 and subtract 50, to give an ECF rating point difference. We can convert this to Elo points by multiplying by 8. Here are my results for the first pass through each batch for the Bain Experiment:

Bain: Rating Difference vs. Problems Learned

The horizontal axis of this graph is the number of problems learned, and the vertical axis is the rating difference, calculated as above. The red dots represent the rating differences for each of the problem batches, and the green line is the least squares best fit to the data. Each red dot represents my average score for 65 problems, and is positioned at the mid point of these problems on the horizontal axis. It is reasonable to assume that my improvement started with the first problem that I learned and continued until the last problem, so I have extended the line to the first problem in the first batch and the last problem in the last batch. The graph suggests that my ability to spot simple tactics (very simple tactics in the case of Bain) quickly improved by about 300 Elo points in the Bain Experiment. (This improvement was in my ability to solve problems that I had never seen before, not the problems that I was practicing.)

For the Bain Experiment, I removed all the problems that were exact duplicates, but many near duplicates remained. The remaining level of pattern duplication is still looks larger than that in tactics randomly selected from real games, or indeed from a large collection of problem books. However, my pattern matching model puts the level of remaining pattern duplication in Bain at about 40%, and the level of pattern duplication in Woolum at about 30%. The duplication in Bain is more blatant and annoying than in Woolum, but perhaps it is not as bad as it appears. Nonetheless, any excess pattern duplication in Bain will show up as a spurious improvement on this graph. See my earlier article Tactics Performance Measurement for further discussion. Here are the corresponding results for the Woolum Experiment:

Woolum: Rating Difference vs. Problems Learned

This graph is less dramatic, but roughly 100 points in 42 days still looks impressive! The drop from +200 at the end of Bain to 0 at the start of Woolum suggests that Woolum is about 200 points harder than Bain. However, I believe that this drop is partly a reflection of the larger number of patterns sampled by Woolum. (I would not have done as well on new patterns, even if the problems containing them were no harder.) Here are the results for Heisman + Pandolfini from the CHP Experiment:

Heisman+Pandolfini: Rating Difference vs. Problems Learned

I again appear to have improved by roughly 100 points in 42 days. The drop of about 80 points from the end of Woolum to the start of Heisman + Pandolfini suggests that Heisman + Pandolfini is about 80 points harder than Woolum.

How accurate are these numbers? Of course, these graphs are just estimating my performance improvement at spotting simple tactics quickly, not my improvement at the game as a whole. The numbers here are also subject to random variation. For Bain, my estimated rate of progress is about three standard deviations (according to the standard formula based on the least squares residuals). For Woolum, it is about two standard deviations. For Heisman + Pandolfini, the standard formula puts my estimated rate of progress at 1.2 standard deviations. The larger scatter on this graph appears to be due to chance variations in my performance. Nonetheless, I cannot claim a good level of accuracy for this problem set.

Can we just add my improvements together? That would be too optimistic. My pattern matching model suggests that the patterns in Bain were selected from a pool a about a third as big as that for Woolum. This suggests that my 300 point gain for Bain would be diluted to about a 100 point gain for the Woolum problem set. (My pattern matching model also suggests that I learned about 200 patterns from Bain, and about 300 from Woolum, so a 100 point gain looks reasonable from this point of view.) It is also possible that my improvement at Woolum might not be fully reflected in my improvement at Heisman + Pandolfini. There are many uncertainties here, but an overall improvement of 200-300 points looks likely for solving problems at this level. [See my later article Rating Points Revisited for the improvements that I made to this method.]

The CHP Experiment

2011-07-01T04:18:00.000-07:00

Following on from my success with the Bain and Woolum Experiments, I decided to conduct a similar experiment with the next three books on Dan Heisman’s list:

Chess Strategy for Kids - Jeff Coakley
Back to Basics: Tactics - Dan Heisman
The Winning Way - Bruce Pandolfini

Since Coakley-Heisman-Pandolfini is a bit of a mouthful, I have abbreviated it to CHP - not to be confused with Combined Heat and Power! I spit the problems into six batches, which I labelled A to F. I took the books in the order Heisman, Coakley, Pandolfini, and made batch A the 1st problem, 7th problem, 13th problem and so on. Batch B was the 2nd problem, 8th problem, 14th problem and so on. The other batches were constructed in the same way. Coakley presented some difficulties here. I used only the problems that were clearly numbered as exercises, and discovered too late that some of the problems at the end could be used both with White to move and with Black to move. I used only the White to move versions. I missed out the first problem in Heisman to make the number of problems divisible by six. I used 433 problems from Heisman, 185 problems from Coakley and 150 problems from Pandolfini. There were 768 problems in all, i.e. 128 problems per batch. In Pandolfini, the page header gives the solution, so I had to make a mask to cover it up. See the previous section for the modifications that I made to the Empirical Rabbit Timer to cope with multiple books, irregular numbering systems, and problems scattered about within the books. The early part of my schedule was the same as for the Woolum Experiment:

Sat Mon Wed Fri Fri Wed Mon
Week 1: A1, A2, A3, A4 Days: 1-7
Week 2: B1, B2, B3, B4, A5 Days: 8-14
Week 3: C1, C2, C3, C4, B5 Days: 15-21
Week 4: D1, D2, D3, D4, C5, A6 Days: 22-28
Week 5: E1, E2, E3, E4, D5, B6 Days: 29-35
Week 6: F1, F2, F3, F4, E5, C6 Days: 36-42
Week 7: F5, D6 Days: 43-49
Week 8: E6, A7 Days: 50-56
Week 9: F6, B7 Days: 57-63

Where A1, A2, A3.… are passes 1, 2, 3... of batch A, and similarly for the other batches. As with Woolum, I did my first four passes of each batch at two day intervals on a Saturday, Monday, Wednesday and Friday. I again did the fifth pass of each batch on the next Friday, and the sixth pass two Wednesdays later. For the first ten passes, the day on which each pass takes place was again given by the table:

Pass: 1 2 3 4 5 6 7 8 9 10
Day: 1 3 5 7 14 26 50 96 185 355

From Pass = 4 onwards, the pass takes place on Day = 1.92^Pass-1, rounded to the nearest whole number. I had a heavy schedule on a Monday at this point, and decided to make Monday easier by repeating on the Sunday, the 25% of the problems that had given the most trouble on the Saturday. (This did make Monday easier, I do not believe that it was not a good idea. Although I did better with CHP than with Woolum on Pass 4, this advantage had almost completely evaporated a week later. This result confirms that repetitions that are close together do very little to improve long term performance. It is also possible that having to work a little harder on a repetition makes increases its instructional value.) As with the previous experiments, incorrect solution times were counted as more than 30 seconds irrespective of the actual time spent. For the first three batches, I also repeated the 25% of the problems at which I had done worst on Pass 6 half way between passes 6 and 7. These problems roughly equated to those that I got wrong or took me more than 5 seconds to solve. I had already done this in the Woolum Experiment, where it seemed to help. Here is a comparison of my performances on my first passes through the first band last batches of the Bain and Woolum, and on my first batch of CHP:

(0-5 denotes 0-4.999... seconds, and similarly for the other “buckets”.) I did better on CHP A1 than I did on Woolum A1, but worse than on Woolum F1. I believe that the reason for this is simply that CHP was harder overall. Here is my performance on my first passes through batches A-F:

The overall picture was still one of improvement, but this chart looks less convincing than that for Woolum. However, the results do look more convincing when we look at the individual books separately - see below. With batch F, I omitted the partial pass on the Sunday, and restricted Pass 3 to the 25% of the problems at which I did worst on Pass 2, which roughly equated to those that I got wrong or took more than 10 seconds to solve. This experiment proved to be successful in that my performance on batch F was essentially the same as that for the earlier batches:

Could I have omitted Pass 3 entirely? Probably. Pass 3 may not have a significant effect after I have completed several passes at wider intervals. Here are the results of my first passes through each of the batches of CHP, for the Coakley problems:

(The cumulative chart presents a clearer picture here with the smaller number of problems per batch.) I bettered my best performance on Bain from the outset here, but subsequently made no discernable improvement. Here are the results of my first passes through each of the batches of CHP, for the Heisman problems:

This chart presents a more favourable picture than that for CHP as a whole. Here are the results of my first passes through each of the batches of CHP for the Pandolfini problems:

Again, this chart looks more favourable than that for CHP as a whole. Here are the results for Heisman + Pandolfini:

This chart too is more favourable than that for CHP as a whole, but perhaps not quite as favourable as for Heisman alone. Coakley appears to have been too easy for me at this stage, and masked my progress by adding random variation to the results. I have a good case here for discounting the Coakley results, but not the Pandolfini results, so the Heisman + Pandolfini chart above is probably the best indication of my progress.

Rabbit Timer Revamped

2011-07-01T04:08:00.000-07:00

The problems in my next three books were scattered about with irregular problem numbering systems. The Empirical Rabbit Timer was not up to the job of telling me where to find every sixth problem, so I decided to modify it read free text instructions from a file. The Rabbit Timer could then simply read every sixth line of the file, prompting me on where to find each problem. For my next experiment, I used a simple code for the problem references:

H33 2
C39 1
P22 1

In this example, “H33 2” means Heasman page 33 problem 2, “C39 1” means Coakley page 39 problem 1, and “P22 1” means Pandolfini page 22 problem 1. I also added a percentage progress indicator to the Ready screen. I could now experiment with repeating only those problems at which I did worst. I could do this by pasting the output file from the Timer into a spreadsheet, sorting it, extracting what I wanted, sorting it back into the original order, pasting it into Notepad, and saving it as text file.

I found that the original Rabbit Timer occasionally had a problem with double hits on the Enter key. If I double hit the Enter key when I was on the Start screen, the Timer would jump straight to the Score screen, recording a fraction of a second as my solution time. I could fix this problem by hitting the r (redo) key, but I occasionally failed to spot the double hit immediately, and then I had to guess the solution time. I fixed this problem by modifying the Rabbit Timer so the “.” key has to be hit to progress to the Move screen, and the Enter key has to be hit to proceed to the Score screen. These keys are together on the numeric key pad, which is rather convenient. Pressing any other key results in a beep.

I also modified the Rabbit Timer to update the Move screen every second to show the time that I had spent so far on the current problem, which made the “t” command redundant.

2011-07-01T04:06:00.000-07:00

Wetescick's Understanding Chess Tactics .......... Feb 2012
Empirical Rabbit Timer Source Code ............... Feb 2012
Rethinking Chess Problem Server Ratings .......... Jan 2012
Improving on Least Squares ....................... Jan 2012
A Three Parameter Model .......................... Jan 2012
-----------------------------------------------------------
Rating, Time and Score ........................... Dec 2011
Rating vs. Time on the Clock ..................... Dec 2011
An Important Discovery ........................... Dec 2011
The Blue Coakley Experiment ...................... Dec 2011
Coakley's Winning Chess Exercises for Kids ....... Dec 2011
A Year of the Rabbit ............................. Nov 2011
Beyond the Blue Coakley .......................... Nov 2011
Chess Combinations for Club Players .............. Nov 2011
The Ivaschenko 1b Experiment ..................... Oct 2011
Sergey Ivashchenko's Chess School 1b ............. Oct 2011
Rating Points Revisited .......................... Oct 2011
-----------------------------------------------------------
The Susan Polgar Experiment ...................... Sep 2011
Susan Polgar's Chess Tactics for Champions ....... Sep 2011
Time to Move Up a Gear ........................... Sep 2011
Dan Heisman's 7-10 Basic Tactics Books ........... Aug 2011
How Many Rating Points Is That? .................. Aug 2011
The CHP Experiment ............................... Jul 2011
Rabbit Timer Revamped ............................ Jul 2011
-----------------------------------------------------------
Learning Chess Tactics ........................... Jun 2011
A Pattern Matching Model ......................... Jun 2011
Distinct Random Selections ....................... Jun 2011
The Woolum Experiment ............................ May 2011
Woolum Errata .................................... May 2011
Woolum's Chess Tactics Workbook .................. May 2011
Update on the Bain Experiment .................... Apr 2011
Empirical Rabbit Timer ........................... Apr 2011
Tactics Performance Measurement .................. Apr 2011
-----------------------------------------------------------
The Bain Experiment .............................. Mar 2011
Dan Heisman’s Basic Tactics Training ............. Mar 2011
Lessons from SuperMemo ........................... Feb 2011
Scheduling Expanding Repetitions ................. Feb 2011
The Reinfeld Experiment .......................... Jan 2011
Reinfeld’s 1,001 Winning Chess Sacrifices... ..... Jan 2011
-----------------------------------------------------------
7 Circles ........................................ Dec 2010
Once Through vs. Repetition ...................... Nov 2010
Lessons from Cognitive Psychology ................ Nov 2010
About the Author ................................. Nov 2010
Introducing the Expanding Repetitions Method ..... Nov 2010
Welcome .......................................... Nov 2010

Learning Chess Tactics

2011-06-01T05:27:00.000-07:00

In this post, I present some chess positions that illustrate how being able to solve simple problems faster helps with solving more difficult problems.

Diagram 1 - Black to win

This is one of my favourite positions from Fred Reinfeld’s 1001 Winning Chess Sacrifices and Combinations (#264). It is a very simple example, three moves, with no sacrifices. The underlying motifs are elementary, but the solution is not easy to see: 1...Bxh3 Kxh3 2.Qg1 and Black wins the B or N. We have three motifs here: line clearance, removal of the guard and double attack.

Diagram 2 - White to win

This position from Al Woolum’s Chess Tactics Workbook looks suspiciously like a simplified version of the previous example. Here you just have to spot the queen fork on g8. There is, however, also the easier to see queen fork on d6, which wins just a pawn. Woolum gives the d6 fork as an alternative solution, but it is really a false trail. If you do not see the fork on g8 very quickly, you have very little chance of solving the previous example over the board. However, if you spot the queen fork almost instantly, you should find the solution to the previous example. You should certainly always look at all your captures, and your opponent’s captures in response. You then just have to see that the g file has opened, and that target square for the queen and the knight have both lost their defence. Bingo!

Diagram 3 - Black to win

I believe that this example comes from CT-ART. It is very simple, but again tricky to see. The important things to note are that the White queen is defending c2 against a knight fork, and the White N on e4 is loose. 1...Qd5 attacks both the defender of c2 and the loose knight. At first sight, it looks as though White can neutralize both threats by exchanging queens, but the recapture exd5 hits the loose knight, and the defence has gone from c2. In his book Understanding Chess Tactics, Martin Weteschnik calls an exchange which replaces one attacking piece by another a reloader. The replacement of the Black queen by the Black pawn as the attacker of the White knight is an example of a reloader.

Diagram 4 - Black to win

Here is a simpler example which forms the conclusion of a famous Tal combination (Tal - Evans Amsterdam 1964). The full combination is discussed in Weteschnik’s book. The important thing to note here is that the White queen is defending the White rook against the attack by the Black rook. Black plays 1...Qf6+ forking the White queen and king. Again, at first sight, it again looks as though White can neutralize the double attack by exchanging queens and pick up Black’s hanging rook. However, as in the previous example, the recapturing pawn reloads, rechecking the king and gaining the tempo Black needs to capture the now hanging White rook. This example is very similar to the previous one. We can say that they both have the same underlying pattern If you can solve one of these problems very quickly, you should be able to solve the other quickly.

Diagram 5 - Black to win

Here is another of Weteschnik’s examples (Bellon Lopez - S.Garcia, Cienfuegos 1976). Black plays 1...Rg2. If the White king takes the rook, the Black knight forks king and queen. Otherwise, 2...Nf4# is threatened and White can defend against this threat only by giving up his queen for the knight. How do you spot this one? Lots of practice with problems where you leave pieces hanging, and lots of practice with rook and knight side file mates is going to help. If you are playing a slow time limit game, it is always a good idea to make a quick check of all your alternative moves, no matter how silly they look. If you do that, and you can spot the rook and knight mate almost instantly, you should find the solution. (It is also a good idea to make a quick check of all your opponent’s replies before making your move, which should stop you falling down this particular hole!)

Diagram 6 - Black to win

Here is an example from Dan Heisman’s Back to Basics: Tactics. In the previous example, we were able to leave a rook hanging because of a knight fork. Here we can leave a knight hanging because of a discovered attack. Black plays 1...Ne5. White cannot take the knight on d4 because of 2...Nxf3+, and has to leave the defence of the pawn on c4.

Daigram 7 - White to win

Here is another example from Heisman’s book. The Black bishop and h7 pawn are both attacked, but if White takes either, he looses his knight. There is a simple solution: defend the knight with 1.h4. We now have a double attack on the bishop and are threatening another rook and knight side file mate. This is a very nice example of a defensive move creating a double attack. Even simple examples have a lot to teach!

Diagram 8 - White to win

My final example is the spectacular conclusion of another problem in Reinfeld’s book (#407). The solution to the position above is very simple, but not easy to see from several moves earlier. White plays 1.Rb7+ Kxb7 (other moves drop the queen). He follows this up with 2.Bc8+, which is double check, so the king must move. If Black takes the bishop, he cuts off the defence of the queen, if he does not, the bishop cuts off the defence of the queen! If you cannot spot the solution quickly from the diagram above, you have very little chance of solving the problem in Reinfeld’s book!

A Pattern Matching Model

2011-06-01T05:16:00.001-07:00

The basic idea behind this section is a simple one. If tactics patterns are selected at random from a small collection in a bucket, returning them to the bucket as we use them, we will soon learn them all. If the patterns are selected at random from a large collection, it will take us a long time to learn them all. If we make some reasonable assumptions, it is possible to estimate the number of patterns from our actual rate of progress. We are going to be able to make only a rough estimate here, but a rough estimate is better than none at all, or somebody’s wild guess!

I will assume that the patterns are sampled with replacement (see the previous section) from a fixed number of such patterns, and that there is one pattern per problem. I will also assume that if I know a pattern to the required standard, I can solve p% of problems based on such patterns in under five seconds at the first attempt; and if I do not know the underlying pattern, I will not be able to solve any of the problems in under five seconds. Clearly, the value of p is going to depend on the difficulty of the problem set. In the Woolum Experiment, I was able to solve about 75% of the problems in under five seconds when I was practicing them intensively. This dropped to 67% in some cases when the repetitions became further apart, and I would not do so well with new problems sharing the same underlying patterns. I believe that a reasonable value for p here is 60%.

Suppose that I am able to solve q% of the problems in under five seconds at the first attempt. The fraction of the patterns that I know is then q% / p%. If M selections are made from a bucket containing N patterns, the previous section tells us that the fraction of the patterns that I know will also be given by:

1 - (1 - 1/N)^M

On my first pass through batch A, I got 29.55% of the problems right, and on my first pass of batch F, I got 44.70% of the problems right. Let M be the number of problems that I had already learned when I started batch A. I would then have learned M+5*132 problems when I started batch F. We get two equations:

1 - (1 - 1/N)^M = 29.55% / 60% = 0.4925

1 - (1 - 1/N)^M+660 = 44.70% / 60% = 0.7450

Solving these equations gives M = 650 and N = 959. The fraction of the patterns that I knew after learning all the 6*132 problems in Woolum is given by:

1 - (1 - 1/N)^M+792 = 0.7778

The number of the patterns in Woolum’s bucket that I knew at the start of the experiment is therefore 959 * 0.4925 = 473, and the number that I knew at the end is 959 * 0.7778 = 746, so Woolum taught me 274 patterns. The number of distinct patterns in Woolum is given by:

N*(1 - (1 - 1/N)⁷⁹²) = 539

The percentage of distinct patterns is 539 * 100% / 792 = 68%. These numbers all look plausible, so this simple model is probably close to the truth. However, it is possible that some of my improvement was not pattern specific, which would increase the estimated value of N.

The problems in Bain are easier than those in Woolum, so I believe that p = 75% is more reasonable for Bain. I got 17.19% right on my first pass through batches A+B, and 49.23% right on my first pass through batches E+F. An analogous calculation to that above gives M = 74 and N = 332. The number of patterns that I knew at the start and end of Bain come out at 74 and 248 respectively, so I learned 174 patterns. The number of distinct patterns in Bain comes out at 226. The percentage of distinct patterns is 226 * 100% / 388 = 58%.

It is reasonable to assume that all the patterns that I learned from Bain’s bucket are also present in Woolum’s bucket, and that I initially knew the same proportion of patterns in both buckets when I started Bain. The Bain data then implies that I knew 394 patterns from Woolum’s bucket when I finished Bain, which compares reasonably well with the 473 patterns implied by the Woolum data, so the numbers are about as consistent as we can expect here.

The main conclusions of this section are that the Woolum results can be explained if the patterns underlying the problems were randomly selected from about 1,000 such patterns; and the Bain results explained if the underlying patterns were randomly selected from about a third of these patterns.

Distinct Random Selections

2011-06-01T05:09:00.000-07:00

In this section, I derive a mathematical result that enables us to estimate the number of problems that we need to solve to achieve a given level of coverage of the underlying tactical patterns. If you find that your eyes glaze over, just skip down to the graph!

Suppose that we make M selections from N distinguishable items, such that for each selection, it is equally likely that any of the N items will be selected. This is called sampling with replacement, because whenever one of the N items is extracted, it is replaced with another identical item, so that item can be selected again. Since items can be selected again, it is possible that some of the selected M items will duplicate one another. If we remove the duplicates, on average, how many distinct items will remain?

(1). The first item selected will always be distinct. After this item has been selected, the number of distinct items will be 1.

(2). There is a 1/N chance that the second item will match the first. There is therefore a 1 - 1/N chance that the second item will add one to the total of distinct items. On average, the second item will add 1 - 1/N to the total of distinct items.

(3). There is a 1/N chance that the third item will match the first item and a 1/N chance that it will match the second item. There is therefore a 1- 1/N chance that the third item will not match the first item and a 1 - 1/N chance that it will not match the second item. There is therefore a (1 - 1/N) * (1 - 1/N) chance that the third item will not match either the first item or the second item. If the third item does not match either item it will add 1 to the number of distinct items. If the third item matches either or both items, it will not add anything to the total of distinct items, irrespective of whether they duplicated one another. On average, the third item will add (1 - 1/N)² to the total of distinct items.

(4). Similarly, the fourth item will, on average, add (1 - 1/N)³ to the total of distinct items.

The average number of distinct items selected is therefore:

1 + (1 + 1/N) + (1 - 1/N)² + (1 - 1/N)³ +….+ (1 - 1/N)^M-1

The sum of this geometric series is N*(1 - (1 - 1/N)^M). (N.B. I have checked this important formula with a Monte Carlo simulation.) The average fraction of the items that will have been selected after M tries is given by 1 - (1 - 1/N)^M = 1 - {(1 - 1/N)^N}^M/N. For large N, this expression becomes 1 - e^-M/N.

In a chess context, 1 - (1 - 1/N)^M estimates the fraction of the N patterns that we will have encountered after solving M problems (assuming that there is one pattern per problem). The graph below plots this fraction (as a percentage) against M for three values of N:

Is sampling with replacement a realistic model to apply to the patterns underlying the problems in elementary tactics books?

* Firstly, does this model over estimate the number of problems that we need to solve? For this to happen, the authors would need to have carefully studied the rival books and had a high rate of success at avoiding repetition of the patterns in those books. In reality, most of these authors are not at all good at avoiding exact and near duplication within their own books, and frequently reuse problems from other books. They also usually claim to offer comprehensive tactics course, which is not compatible with missing out all the common patterns that occur in other books.

* Secondly, does this model over underestimate the number of problems that we need to solve? For this to happen, the level of duplication in elementary tactics books would have to be larger than the level of duplication that would arise from sampling with replacement. This is possible, but I think that sampling with replacement is a reasonable approximation here.

The Woolum Experiment

2011-05-01T10:20:00.000-07:00

Encouraged by my success with the Bain Experiment, I decided to conduct a similar experiment with the next book on Dan Heisman’s list: Al Woolum’s Chess Tactics Workbook. The main part of this book has 132 pages of problems with six problems per page, i.e. 792 problems. (This is just over twice the 388 problems in my issue of Bain, after excluding Bain’s exact duplicates and the two duds.) I divided the 792 problems into six batches of 132 problems, which I labelled A to F. Batch A was the first diagram on each page, batch B was the second diagram on each page, and so on. The problems on each page appeared to be in random order of difficulty. I did not have any good argument that each problem batch was of the same level of difficulty, but thought it unlikely that Woolum would have deliberately made the later problems on a page easier than the earlier ones. The early part of my schedule was:

Sat Mon Wed Fri Fri Wed Mon
Week 1: A1, A2, A3, A4 Days: 1-7
Week 2: B1, B2, B3, B4, A5 Days: 8-14
Week 3: C1, C2, C3, C4, B5 Days: 15-21
Week 4: D1, D2, D3, D4, C5, A6 Days: 22-28
Week 5: E1, E2, E3, E4, D5, B6 Days: 29-35
Week 6: F1, F2, F3, F4, E5, C6 Days: 36-42
Week 7: F5, D6 Days: 43-49
Week 8: E6, A7 Days: 50-56
Week 9: F6, B7 Days: 57-63

Where A1, A2, A3.… are passes 1, 2, 3... of batch A, and similarly for the other batches. I did my first four passes of each batch at two day intervals on a Saturday, Monday, Wednesday and Friday. I did the fifth pass of each batch on a Friday, and the sixth pass on a Wednesday. This schedule is a slightly more aggressive version of the one that I used for batches C+D and E+F in the Bain Experiment. For the first ten passes, the day on which each pass takes place is given by the table:

Pass: 1 2 3 4 5 6 7 8 9 10
Day: 1 3 5 7 14 26 50 96 185 355

From Pass = 4 onwards, the pass takes place on Day = 1.92 ^ (Pass-1), rounded to the nearest whole number. I used the Empirical Rabbit Timer to time my solutions and collect the results. As in the Bain Experiment, incorrect solution times were counted as more than 30 seconds irrespective of the actual time spent. In Woolum, the title at the top of each page often gives essential information, such as mate in 2 (i.e. mate in 3 is not acceptable). The title also usually gives a helpful clue for finding the solution, but is sometimes highly misleading. I decided not to hide the title. I was reasonably generous with the scoring, counting my solution as correct if I got the right idea and the right first move. Here is a comparison of my performance on my first pass through batches A+B, C+D and E+F in the Bain Experiment, and my first pass through batch A in the Woolum Experiment:

(0-5 denotes 0-4.999... seconds, and similarly for the other “buckets”.) I did worse on Woolum A1 than I did on Bain E1+F1, which is not surprising because Woolum is harder overall than Bain, and contains many problems which are unlike anything in Bain. I did better on Woolum A1 than on Bain A1+B1 and about the same as on Bain C1+D1, except for the increased number of solution times over 30 seconds. Woolum has a sizable proportion of easy problems, but its harder problems are a lot harder than anything in Bain, which accounts for the larger number of solution times over 30 seconds. Not surprisingly, I improved at the problems I was practicing, but what about problems that I had never seen before? Here is my performance on my first passes through batches A-F:

Clearly, the overall picture here is one of steady improvement. My best performance was just little behind that on Bain E1+F1, which was much easier. However, my performance on the last two batches had fallen a little. Possible explanations are:

(1). The last two problems on each page were harder, on average, than the problems earlier on the page.

(2). The fall off was due to chance variations in my performance.

(3). My rate of improvement had slowed.

The next diagram shows my performance on the fourth pass through each of the batches:

Even after four passes my relative performances on the last three batches were similar to those on the first pass. This suggests that the fall off in my performance on the last two batches occurred because they were harder. This is another very encouraging result for this training method.

Woolum Errata

2011-05-01T10:13:00.000-07:00

For my next experiment, I used Al Woolum’s Chess Tactics Workbook (Expanded 4th Edition). This proved to be an excellent book for my purposes, but it does contain a number of errors and omissions. The book includes a short list of corrections, and Steve Eddins has posted a supplementary list:

http://www.eddins.net/steve/chess/2006/11/07/51

I have consolidated these lists, fixed some errors, added my own corrections, and suggested changes to the diagrams that do not have valid solutions. W indicates that the source of the correction is Woolum. SE indicates that the source of the correction is Steve Eddins. Otherwise, I am to blame! I do not claim to have found all the alternative solutions, and have only included those where I think it is helpful. If there is mate in 1 and a win of the Q in 2, for example, I have not noted that the latter has been omitted. I have tried to keep the errata as concise as possible on the basis that people are likely to copy the corrections into the book. Please comment on this post if I have made any errors or significant omissions, and I will keep it up to date. [A few typos corrected and sources added - 3 June 2011. Corrected p79, #5 now duplicates p119, #2 - 16 June 2011. p31, #4 corrected - 2 June 2011.]

p17, #4, add pawns to a6 and b7 (SE)
p18, #6, or 1.Bxe5+ Bxe5 2.Nxe5
p22, #5, 2.Nb4 should be 2.Nxb7 (W)
p28, #3, better 1.Bxe5 Bxe5 2.Re1
p30, #1, 2.Qh6# should be 2.Qh7# (W)
p31, #4, 2.Qxc4 fails to 2...Qxc4 3.Rxf8+ Qg8
p32, #5, or 1.Bd3 with the same idea
p42, #5, 1.Qe7+ Kxf5 2.g4+ wins Q
p43, #6, or 1.Ne6+ fxe6 2.Qxg7# or 1.Qh8+ Bxh8 2.Nh7# (SE)
p49, #5, or 1.Rxg7+ Kxg7 (or 1 … Kh8) 2.Qxh7# (SE)
p50, #1, or 1.Qd3+ Kxa4 2.Bc2# (SE corrected)
p55, #4, 1.Qf3+ Kxh4 2.Qg3# (W)
p55, #5. 1.Rxc6+ Kd7 2.Qe6# (SE)
p56, #1, or 1.Qg6+ Kh8 2.Qxg7# (SE)
p57, #1, or 1.Qf7+ Bxf7 2.Ne8# (W)
p64, #2, 1.Bxd7# (SE)
p64, #5, 1.Qe7# (W)
p65, #2, or 1.Ra1, 1.Rb1, 1.Rc1 or 1.Re1 followed by a back rank # (W/SE)
p65, #3, or 1.Rf5+ any 2.Nf4# (W/SE)
p69, #3, 1.Nxe6 Qf8 just wins the Q (SE)
p70, #4, 1.Qf5# (W)
p74, #1, 1.Qe6 Qe8 2.Rh8# (SE)
p75, #6, Black K on g8
p76, #3, mate in 3: 1.Rxg6+ Kh7 2.Qf7+ Rg7 3.Qxg7# or 2.Kh8 3.Qxg8#
p79, #5, White N on c3
p82, #2, mate in 3: 1.Nf6 Be4 2.Rxh7+ Bxh7 3.Qxh7#, 1...Rh5 2.Qxh7+ Rxh7 3.Qxh7#
p85, #4, 1.Ng6+ Kh7 2.Bg8# (SE)
p89, #3, Black N on d4
p94, #29, or 1.Rxf8+ Qg8 2.Nf7# or 1...Kh7 3.Nf5#
p98, #6, e.g. 1.Kg5+ Kg8 2.Kg6 Kf8 3.Qh8#
p102, #6, or 1.Be2+ Ng4 2.Ne5# or 1...Bg4 2.Ne5+ Nxf7 3.Bxg5#
p104, #1, or 1.Ba3 any 2.Kf8 any 3.Bb3# (SE)
p104, #5, or 1.Nb6+ with the same idea
p108, #2, move Black P from f7 to h7
p111, #1, White N on e5 and White P on h2.
p115, #4, Black K on g5
p115, #5, White N on e7 and White P on h5, 1.Ng1+ or 1.Nf5+ Qxh5 2.Rh2# (SE)
p116, #3, or 1.Bxg6 Re8 2.Qxh7+ Kf8 3.Qxf7#, 1...fxg6 2.Rxg6+ Kf7 3.Rg7#, 1...hxg6 2.Qh8# (SE)
p118, #2, or 1.Qf5+ Nxf5 2.e6#, 1...Ne6 2.Qxe6#
p123, #4, Black N on d8
p131, #3, or 1.Bb1 Nd2 2.Rc3#

Woolum's Chess Tactics Workbook

2011-05-01T10:08:00.000-07:00

I used Al Woolum’s Chess Tactics Workbook (Expanded 4th Edition) for my next tactics training experiment, so I will give it a brief review. The book has 792 problems in the main section, plus 24 triple Lloyds, 24 chess mazes, and 30 regular chess problems in the “Canadian Corner“, which originates from Jeff Coakley.

The book is intended to be used as instructional material for kids, and begins with the moves of the game. A majority of the problems are easy, but a significant proportion are not at all easy! John Bain’s Chess Tactics for Students is much easier, and is more suitable as a first tactics problem book, or as first speed training book for more advanced players.

The book is well presented with large and exceptionally clear diagrams. A large majority of the problems look like ordinary chess positions, but there are also some contrived compositions. There are a significant number of exact duplicate problems or near duplicates (e.g. the same position with another move added at the beginning), but less as a proportion than in Bain. I also recognised a number of problems from Reinfeld and other books. All the problems are White to move - which is not ideal - it is necessary to be able to see tactics from both sides of the board! However, if you have other tactics books, this should not be too much of a problem. There are some errors in the diagrams, and many errors and omissions in the solutions, but my next post provides an errata, which I believe fixes the important errors and omissions.

None of these shortcomings are show stoppers. Overall, the book provides excellent training material at a good price per problem. It is well suited to training aimed at spotting simple tactics very quickly, as advocated by Dan Heisman. [See Dan Heisman's 7-10 Basic Tactics Books for my review of all Dan's recommendations for this purpose.] The book appears to be distributed only via specialist chess suppliers - I got mine from the London Chess Centre.

Update on the Bain Experiment

2011-04-01T05:55:00.000-07:00

For my next few repetitions of Bain, I followed my own advice in the Scheduling Expanding Repetitions article for batches C+D and E+F, but stretched out a little the repetitions for A+B (which already had extra repetitions):

Day 1: A1+B1, A2+B2
Day 2: A3+B3, C1+D1
Day 4: A4+B4, C2+D2
Day 6: C3+D3
Day 8: A5+B5, C4+D4
Day 9: E1+F1
Day 11: E2+F2
Day 13: E3+F3
Day 16: A7+B7, C5+D5, E4+F4
Day 23: E5+F5
Day 30: C6+D6
Day 37: E6+F6, A8+B8, C7+D7

My performance on A8+B8 (i.e. the eighth pass of A+B) had fallen a little from A5+B5. My performance on C6+D6 had also fallen a little from repetition C4+D4. C+D and E+F both had the same repetition intervals up to the sixth pass, but my performance on E6+F6 was better than that on C6+E6. Its advantage had more than halved since the first repetitions of these batches, but I expected the gap to have narrowed more. This raises the suspicion that E+F might have been a little easy relative to C+D. However, the chart below shows the number of problems that were solved in under 5 seconds on my first pass through each of the batches:

The overall picture is clearly one of steady improvement. [Diagram corrected 28 May 2011.]

The results suggest that increasing successive repetition intervals by a factor of 1.9 rather than a factor of 2.0 might give better results for these simple problems. The precise optimum intervals are likely to be player dependent, however, and have to fit your schedule.I did the seventh pass of C+D on day 37, to even up the performance over the batches, so that I could revise the whole of Bain in one day for future repetitions. I also moved the next repetition forwards (repeating the two week repletion interval) to boost my performance. By this stage, I could easily carry out a pass through the whole of Bain in a couple of hours, even when tired.

Empirical Rabbit Timer

2011-04-01T05:46:00.000-07:00

Measuring my solution times with a stopwatch, entering them into a spreadsheet and constructing a histogram from the results was a laborious task, so I decided to write a Java program to do the work for me. This program brings up a window giving me the number of the next problem to be solved and asks whether I am ready:

Hitting any key (except “s”, which I will address shortly) starts the clock and takes me to the next screen:

Hitting “t” displays the time that I have spent so far (shown as on the next screen). Hitting any other key takes me to the next screen:

Hitting 0 or 1 (to indicate the result) records the problem number, the time spent solving the problem and the result - the program then increments the problem number and takes me back to the first screen. Hitting “r” (for redo) takes me back to the first screen without incrementing the problem number, and resets and restarts the clock (e.g. because I have just realised that it is Black and not White to move). Hitting “b” takes me back to the previous screen, without having stopped the clock. Hitting any other key results in an error beep.

I usually find it most convenient to use the Enter key on the numeric key pad to navigate between the first two screens. The Enter key then functions like the start/stop button on a stopwatch (except that I can use the “b” key to undo the action of hitting Enter). In the normal course of events, I simply press Enter twice to start and stop the clock, and type 0 or 1 to indicate whether or not I got the right answer. (I do not usually see the second screen, because I am looking at the problem.) The results are written as comma separated values, so that I can easily import them into a spreadsheet. Hitting “s” (for statistics) on the first screen adds the number of solution times falling within each histogram “bucket” to the results.

This was my first Java program. I learned sufficient Java to write it using Rogers Cadenhead’s Sams Teach Yourself Java in 24 Hours (which was cheap and had good reviews on Amazon). I used the free NetBeans and JDK development tools. The Empirical Rabbit Timer currently runs in NetBeans, and writes its output to the console. [See my later article Rabbit Timer Revamped for the improvements that I made to this program.]

Tactics Performance Measurement

2011-04-01T05:42:00.000-07:00

One method of measuring your performance at solving a set of chess problems is to measure the average time spent solving each problem and the percentage score achieved, but there are difficulties with this method:

* If you solve the problems faster, but your score goes down, you do not know whether you are doing better or worse.

* Similarly, if you solve the problems more slowly, but your score goes up, you do not know whether you are doing better or worse.

* You are rewarded for giving up quickly whenever you encounter a difficult problem.

(You might say that you are being rewarded for good time management, but the time management skill here is different from that in a game, and your skill at time management really ought to be measured separately. What we really want to measure here is just your ability to get the right answer quickly.)

These difficulties can be avoided by measuring the solution times for individual problems. The smallest time limit applied to each problem individually that would still have allowed you to score 50% (the median solution time) does not suffer from the difficulties identified above, provided that you are always able to get at least 50% right. The largest time limit applied to each problem individually that would still have allowed you to solve at least 85% would also work, provided that you scored at least 85%. Similarly for any other percentage. The number of problems that can be solved within a fixed time limit applied to each problem individually also does not suffer from the difficulties identified above. The cumulative distribution of the solution times gives us the percentage that were solved with any such time limit that we may choose, assuming that failures are counted as infinitely long solution times. Here is the cumulative distribution for my first pass through batches E+F in the Bain Experiment:

Measuring performance at solving a set of chess problems is all very well, but it does not necessarily have a direct relationship with your tactical ability in a chess game. The problem here is that the tactics in the chess problems may not necessarily be representative of what you are likely to meet in practice. You might do very well at solving the chess problems, but this skill might turn out to be of little practical value. Alternatively, you might do badly at solving the chess problems, but this might not matter much in practice. The solution to this problem is to construct a set of problems that is statistically representative of what you are likely meet in practice.

There are computer programs that will automatically extract tactics problems from the games in a chess database. The problems on the Chess Tempo tactics server were constructed in this way, see: http://chesstempo.com/faq.html#tactics. In principle, we could use one of these programs to construct random samples of chess tactics as they occur in practice, and use these to measure our tactical ability. (We need a collection of tactics exams to measure our progress, because we can only use each exam once, for this purpose.) The difficulty here is the same one that opinion pollsters face: you need a very large sample to achieve an acceptable level of accuracy. The solution to this difficulty is statistical profiling. We can reduce the sample size needed by ensuring that each sample of tactical problems has the same statistical profile as the whole population of chess tactics as they occur in practice.

We could, in principle, use problem sets that have the same statistical profile as in the whole population of chess tactics for training - but the sets would contain a very high proportion of trivial tactics, and tactics so difficult that they can only be found by a computer. It makes more sense to construct samples in which the level of difficulty is restricted to a narrow band, so that we can chose problem sets that are at an appropriate level of difficulty for us. The level of difficulty could be assessed by a computer program, which could tell us how many half moves deep the solution is - or better, perhaps, it could tell us the total number of half moves in all the variations of the solution. Alternatively, the difficulty for human players could be assessed by carrying out tests.

If we are going to statistically profile our problem sets, we also need to classify the problems by type, e.g. by primary and secondary motif, or something more sophisticated. We could, in principle, program a computer to carry out this task. Alternatively, we could use human assessment. We need to ensure that each set of sample problems has the same distribution of problem types as the whole population, and the same distribution of difficulty within each problem type. This not only ensures that each set of problems is representative, but also avoids the practical difficulty that some players might be better at some types of problems than others.

If we measure our performance at solving these problem sets, we are measuring our absolute performance at finding chess tactics, as it occurs in practice, rather than our relative performance against the competition. Clearly, we would also like to know how well the competition does for each problem set as a whole - and for each problem type / level of difficulty within each set - so that we can our identify relative strengths and weaknesses. We can relate our scores (e.g. median solution time) to those of other players by plotting them on a scatter graph against their ratings. A typical scatter graph might look like this:

Clearly, there is not going to be a one to one relationship between tactics performance and rating (even for players with reliable real world ratings), because some players will be better at tactics, and others at other aspects of the game.

The method of constructing the problem sets described above is essentially the same the one that I used to construct the batches of problems in the Bain Experiment. Bain is one of many problem books in which each chapter contains problems of a different type, and the problems within each chapter are sorted into ascending order of difficulty. If we want to divide the problems into two representative sets, which both have the same distribution of difficulty for each type of problem, we can take the first set to be the odd numbered problems, and the second set to be the even numbered problems. Alternatively, if we want six sets, and there are six diagrams per page, we can take the first set to be first diagram on each page, the second set to be the second diagram on each page, and so on.

If we carry out this process with a problem book, the statistical distribution of problems will not necessarily reflect that of tactics as they occur in real games. As we saw in the Bain Experiment, this makes it difficult to relate an improvement at solving those problems to an improvement at finding tactics in real games. However, this would be less of a concern with a larger problem set, and it is possible that problems that have been selected for their instructional value are more effective for training purposes than problems that have been randomly extracted from games.

The approach outlined here is different from that typically adopted by tactical training software, which usually gives tactical ratings to its users based on their performance at solving problems. The tactical ratings assigned by the online tactical servers usually use the Glicko rating system, with the problems given ratings and treated as opponents. Solving a problem in a timely fashion is counted as a win for the user, and a failure or a slow success is counted as a loss. For a correct solution, Chess Tactics Server assigns the user a result between 0 and 1, according to the time the spent solving the problem:

(The Chess Tactics Server website says that this graph was chosen to make their rating system work, and that the short time limits discourage cheating, which affects the rating of the problems, see: http://chess.emrald.net/time.php.)

With tactical training software, you are invariably allowed to solve the problems repeatedly - which will improve your performance at solving those problems - but this improvement will not be fully reflected in your ability to solve fresh problems. Consequently, you are given a false impression of progress. (A worse problem is that you often do not get the opportunity to repeat the same problems to your chosen schedule.) I did a Google search and found forum posts that said that one of the online tactical servers gives average players International Master ratings, whereas another gives International Masters the ratings of average players! [See my later article Rating Points Revisited for further comment on this topic.]

The Bain Experiment

2011-03-01T02:37:00.000-08:00

The purpose of this experiment was to test Dan Heisman’s basic tactics training ideas, using the problems from John Bain’s Chess Tactics for Students. In this experiment, I not only greatly improved at solving the problems that I was practicing, but also at solving problems that I had never seen before. Bain's book has 14 chapters:

Chapter Problems Motif
   1 2-31 Pins
   2 33-62 Back Rank
   3 64-93 Knight Forks
   4 95-124 Double Attack
   5 126-155 Discovered Checks
   6 157-186 Double Checks
   7 188-217 Discovered Attacks
   8 219-248 Skewers
   9 250-279 Double Threats
  10 281-310 Promoting Pawns
  11 312-341 Removing the Guard
  12 343-372 Perpetual Check
  13 374-403 Zugswang
  14 405-434 Identifying Tactics

(The first problem in each chapter is an illustration that duplicates the first problem of that chapter.) The book is formatted as a workbook. Beneath each diagram, the book says who it is to move, and gives hint for finding the solution. There are no solutions, as such, but the hints are so detailed that a solution is hardly necessary. The reverse side of each diagram is set aside for writing the solution, so I was able to cut out the diagrams along with their hints, doing only minor damage to some of the hints on the other side. I folded back the hints so that they were not visible, and marked each diagram with W+, W=, W#, B+, B=, or B#, according to the result required, taking care not to look at the diagrams. (This caused some problems. There are a few cases where the book says e.g. “win a rook”, and it is possible to win material in another way. I gave myself the benefit of the doubt in these cases.)

The book says that the problems within each chapter are in order of difficulty. I constructed six batches A-F of problems from Chapters 1 to 13. Batch A consisted of problems 2, 8, 14, 20, 26..., plus problems 33, 39, 45, 51, 57..., and so on. Batch B consisted of problems 3, 9, 15, 21, 27..., plus problems 34, 39, 46, 52, 58..., and so on. The remaining batches were constructed in the same way. This method ensured that, as nearly as possible, each batch had the same number of problems with the same level of difficulty from each chapter. If the book’s ordering by level of difficulty was perfect, there would be a very slight increase in difficulty from batch to batch. Any remaining variation in difficulty between the batches can reasonably be ascribed to random factors. I set Chapter 14 aside, because sorting the problems by motif would have given me prior knowledge, and I had a suspicion (which seems to be right) that they are duplicates of problems from previous chapters. I thoroughly shuffled each batch - which was almost entirely ineffective - so I scrambled the order by sorting them into two piles and then into three. I discarded problem 26, because it worked only if the opposing side blundered; and problem 95, because it was a complete dud.

The early part of my schedule was:

Day 1: A+B, A+B
Day 2: A+B, C+D
Day 4: A+B, C+D
Day 6: C+D
Day 8: A+B, C+D
Day 9: E+F

The first four repetition intervals for batches A+B were ½ day, 1 day, 2 days and 4 days, and the first three repetition intervals for batches C+D were 2 days, 2 days and 2 days. I measured the time taken to solve each problem with a stopwatch, rounding the times to a tenth of a second. In the diagrams below, I counted the times of any incorrect solutions as taking more than 30 seconds, whatever the actual time spent. The cumulative distributions of solution times for the first four passes through batches A+B were:

I was clearly faster at the outset than Heisman’s typical student, and improved more rapidly. However, a large proportion of Bain’s problems were either simple examples from Reinfeld, or simplified versions of more complicated ones, so I had an unfair advantage here! (N.B. You can click on the diagrams to enlarge them.) The histogram of solution times for the first five passes through A+B was:

Note the very rapid initial improvement from Pass 1 to Pass 2, which was carried out on the same day. I got 83% in under 5 seconds on the fifth pass. (N.B. For the diagrams in this section, 0-5 stands for 0-4.9 seconds, and similarly for the other “buckets”.) The histogram of solution times for the first four passes through C+D was:

Note the slow initial progress with the two day repetition intervals. Most of my difficulty was with D rather than C, so I did a quick untimed pass of D, before my first pass of E+F, to reduce my times for C+D closer to those of A+B. (I had hoped that the experiment would show an improvement on my first pass from A to B, from C to D, and from E to F. An improvement was observed from A to B, but I did worse on D than C and on F than E, so I do not believe that we can draw any conclusions here.)

Both the ½ day, 1 day , 2 days, 4 days, and the 2 days, 2 days. 2 days repetition intervals worked well here. The intervals of 1 day, 2 days, 4 days used in the Reinfeld Experiment should give much the same results as 2 days, 2 days, 2 days (see the earlier article on that experiment). It is not clear whether the additional repetition at ½ day would still have a significant benefit after many more repetitions at progressively doubling intervals.

Not surprisingly, I improved at the problems that I was practicing - but what about problems that I had never seen before? Here is a histogram of solution times for my first passes through A+B, C+D and E+F:

I was astonished! On my first pass through E+F, I got over 85% in 15 seconds, and very nearly 50% in 5 seconds. What was going on? One theory is that I simply got faster as a result of practice. Another is that I learned new patterns by repeatedly solving A+B that helped me with B+C, and that repeatedly solving B+C taught me still more patterns, that helped me further with E+F. What does the data have to say?

The simplest hypothesis is that the solution times were all reduced by a common factor. I found that dividing all the solution times for my first pass through A+B by 1.3 gave the closest match to the solution times for my first pass through C+D. Similarly, dividing all the solution times for my first pass through A+B by 2.6 gave the closest match to those on my first pass through E+F. (I used the method of least squares here.) The histogram below compares the counts in each “bucket” for my real passes through C+D and E+F with those simulated by dividing the solution times of A+B:

The fit is about as good as it could be, given the statistical variability. It is remarkable that my first pass of E+F was 2.6 times faster than my first pass of A+B.

What about the pattern matching theory? Imagine that x% of the problems in A+B are duplicated in C+D, and that any internal duplication within C+D is at the same level as that in A+B. On the first pass of C+D, I will have solved x% of the problems three times already, and the remainder will be new to me. I can approximate my performance on the x% by using the histogram for the third pass of A+B. My performance on the remaining problems within C+D can be approximated by the histogram for my first pass through A+B. I used the method of least squares to find the value of x% which made this approximation as close as possible to the histogram for my first pass through C+D. The best fit was with x% = 25%. I also approximated my first pass through E+F using the histograms for my first and fifth passes through A+B - the best fit was obtained with x% = 48% - which is almost exactly twice the value for C+D, as it should be if duplicates are equally distributed throughout the batches. The histogram below compares my real passes through C+D and E+F with the approximated ones:

Again, the fit is good. Bain has many problems that are the same as another problem within the book, but with one less move at the beginning. If I tackle the harder problem first, the easier one should show up as a duplicate, but if I tackle the easier one first, the harder one may not show up as a duplicate.

What conclusions can we draw? The data is consistent with my matching patterns that I already knew 2.6 times faster. Since Bain’s problems were either very simple - or examples from Reinfeld that I already knew - it is possible that I already knew all the patterns, and had just become faster at finding them. However, I am sure that I am not 2.6 times faster at solving all tactics problems at this level of complexity. I believe that most of my improvement was pattern specific. This interpretation is supported by the fact that my improvement in going from my first pass of A+B to my first pass of E+F was almost exactly twice my improvement in going from my first pass of A+B to my first pass of C+D. (N.B. If the experiment had been carried out on a set of problems that had the same statistical profile as simple tactics in real games, my improvement would be real, whether or not it resulted from pattern duplication.) It could be objected that, despite my best efforts, E+F might be easier than C+D, which might in turn be easier than A+B. It is not possible to completely eliminate possibilities like this from a single player experiment. Please feel free to repeat the experiment with the batches in the reverse order!

Dan Heisman’s Basic Tactics Training

2011-03-01T02:29:00.000-08:00

Dan Heisman discusses basic tactics training on his website:

http://danheisman.home.comcast.net/~danheisman/Events_Books/General_Book_Guide.htm

He recommends that:

* All of the problems should be easy enough to eventually be solved on recognition, within reason. They should also be basic enough to either be single motif, or very easy double motif. They should be building blocks for more difficult problems.

* Most of the problems should be to win material not checkmate. In chess, most games are won by attrition, not checkmates with equal material (what percentage of the games has the reader won with checkmate from a position of even material?). So a problem set that is 75% or more material wins ("X to play and win") and less than 25% checkmates seems about right.

* Most of the problems should be from normal looking positions that may occur frequently in games. No crazy positions; instead lots of problems featuring trapped pieces, removal of the guards, double attacks - normal stuff - not too many queen sacrifices, etc.

He thinks that there are about 2,000 basic tactics patterns, and recommends 7 tactics books that he says together may contain about 97% of these tactical patterns:

Chess Tactics for Students - John Bain
The Chess Tactics Workbook - Al Woolum
Winning Chess Strategy for Kids - Jeff Coakley
Back to Basics: Tactics - Dan Heisman
The Winning Way - Bruce Pandolfini
Winning Chess Traps - Irving Chernev
Bobby Fischer Teaches Chess - Bobby Fischer

(This begs many questions. How common and simple do tactics patterns have to be to be included? How different do two configurations of pieces have to be to count as two patterns rather than just one? Both the number of tactics patterns and the percentage of them in these books clearly depend on the answers to these questions. The answers may lie more in statistics than in geometry. Perhaps a tactics problem counts as both common and simple if most strong players can solve it very quickly. Perhaps problem B can be held to have a pattern that is not present in problem A if we can find players who can solve problem A very quickly, but who take significantly longer on problem B.)

[See my later article Dan Heisman's 7-10 Basic Tactics Books for further discussion.]

For Bain, Heisman recommends going through the book repeatedly (in any order) until you can solve 85%+ within 10-15 seconds. He says that typical time limits for each pass are 6 minutes, 3 minutes, 90 sec, 45 sec, 25 sec, 15 sec, and 10 sec for each problem on the 7th pass. The goal is to recognise the patterns, not just be able to solve the problems. He says: “You will be amazed how much this helps your chess - I am becoming more convinced that this homework is one of the most profitable you will ever do.” (This is effectively a mini 7 Circles programme for easy problems, and most of the criticisms that I made in the 7 Circles section apply here too. The main difficulty is that there is little value in becoming very fast at solving these problems, if that capability is short lived. It should be possible to fix this problem by using the methods of the Reinfeld Experiment, but there may be some refinements specific to solving simple problems very quickly.)

Heisman advances a very plausible conjecture in one of his early Novice Nook articles:

http://www.chesscafe.com/text/heisman04.pdf

“I would go so far as to conjecture that more basic the tactical problem, the more beneficial it is to do it multiple times until you can do it quickly, while the more difficult the problem, the relatively less benefit it is to do it over and over. The reason is that more complex combinations usually consist of many basic tactical motifs, but not vice versa. And secondly, you see the basic tactics in many combinations throughout most games, while difficult ideas are more complex, and so each one is more unique, and occurs more rarely - in fact, you may never have seen one just like it before - only somewhat similar.”

I had a go at Heisman’s Tactics Quiz:

http://www.chesscafe.com/text/heisman28.pdf

He says that the purpose of this quiz is to test recognition capability for easy, common tactical problems. It is a timed test of twelve problems. Your tactical rating is calculated using the formula:

Tactical Rating = 600 + 150 * Number of Problems Correct – 2 * (Total Time
– 90 seconds)

If you get all twelve right in 90 seconds (an average of 7.5 seconds per problem), you get a Tactical Rating of 2400, which is the maximum for the test. My Tactical Rating came out at only 1454. Ouch! I was in too much of a hurry, and did not check my answers. I took 4 minutes 23 seconds (22 seconds per problem), but I only got 8 answers right. On that basis, I appear to be a candidate for simple tactics training. It makes sense to be really quick (and accurate) with this stuff!

The stated goal for Bain training is to solve 85%+ of the problems within 10-15 seconds. I take this to be the time limit for each problem individually. The average time per problem could be more or less, depending on how long you spend on the remaining 15%. What rating would I get in the Tactics Quiz if I had scored 85% in an average of 10-15 seconds per problem?

85% of 12 problems is 0.85 * 12 problems = 10.2 problems.

For 10 seconds * 12 = 120 seconds:
Tactical Rating = 600 + 150 * 10.2 – 2 * (120 - 90) = 600 + 1530 - 60 = 2070

For 15 seconds x 12 = 180 seconds:
Tactical Rating = 600 + 150 * 10.2 – 2 * (180 -90) = 600 + 1530 - 180 = 2010

We have a simple and bold hypothesis here - that learning to solve the problems in Bain very quickly will improve my chess. I decided to put it to the test. Getting a copy of the book delivered at a reasonable price was not easy. It does not appear to have a British supplier, but I managed to get a second hand copy from a supplier on Amazon (.com) for $1.25 plus $12.49 USPS shipping. It is not the new 10th Anniversary Edition, but I do not expect that matters too much. The book arrived in 2 weeks, despite the estimate of 4-6 weeks. What the more frivolous amongst us would like to know is whether the author’s wife refers to him as "the Bain of my life," but I am afraid that I do not know the answer to that one.

Lessons from SuperMemo

2011-02-01T04:33:00.000-08:00

In the Reinfeld Experiment, the repetition intervals that worked best for me were 1 day, 2 days, 4 days, 8 days... This repetition schedule is similar to that underlying the SuperMemo learning system, which was developed by Piotr Wozniak in Poland. Most standard learning systems (e.g. the Leitner system) are based on the idea of presenting the questions that are causing difficulty more often than those that are not - but this is problematic for chess problems - where you will be spending more time on the difficult ones anyway. The SuperMemo software also presents the questions that are causing difficulty more often, but that is not fundamental to the method.

SuperMemo was designed to facilitate the memorisation of factual information in the form of simple questions and answers. Learning chess tactics differs from this task in many important ways. For reasons that will become apparent, I do not believe the SuperMemo software is suitable for learning chess tactics. Nonetheless, the research underlying SuperMemo provides very useful insights, and is potentially even more relevant to tasks that lie part way between simple memory tasks and learning chess tactics, such as learning chess endgames.

In 1985 and 1986, Wozniak conducted an illuminating experiment comparing repetitions at fixed and expanding intervals:

http://www.supermemo.com/english/ol/beginning.htm

In this experiment, 195 questions (English irregular verbs) and answers (simple present, simple past and past participle forms) were divided into three batches of equal difficulty: A, B and C. For each batch, the items were memorised in one session by repeating them until they were all answered correctly. Six further repetitions were then performed on each batch, after the repetition intervals given by the table below:

Repetition Batch A Batch B Batch C
   1 18 days 1 day 5 days
   2 18 days 5 days 5 days
   3 18 days 9 days 5 days
   4 18 days 24 days 5 days
   5 18 days 44 days 5 days
   6 18 days 70 days 5 days
Totals 108 days 153 days 30 days

(For each of these repetitions, any questions that were answered incorrectly were repeated until they were all answered correctly.) The error rate dropped most quickly with batch C, and most slowly with batch A. After 300 days, the error rates were approximately 10%, 15% and 45% for batches A, B and C respectively. The widely spaced equal repetitions gave the lowest error rate after 300 days, but the success rate for the earlier repetitions with the expanding intervals would have been higher, and the time taken for these repetitions correspondingly lower. Nonetheless, it is very interesting that the widely spaced repetitions of batch A did so well.

In 1985, Wozniak conducted an experiment to find the shortest repetition intervals that were consistent with a memory loss of less than 5%. For this experiment, he used questions and answers in the form of English word and Polish equivalent. He set his first repetition interval to 1 day, which his previous experience suggested was a good choice, and tried various values for the next three intervals. (Again, any questions that were answered incorrectly were repeated until they were all answered correctly, for each of the repetitions.) He chose the largest intervals which kept the memory loss below 5%, which turned out to be 7 days, 16 days and 35 days. He noticed that the intervals were approximately doubling from one repetition to the next, and decided to successively double the subsequent intervals between each repetition. After two years of using these intervals, he concluded that this assumption was reasonably accurate.

SuperMemo goes through all the questions that are scheduled for repetition, asking the user to answer each one in turn. The user tries to recall each answer before looking at it, and grades the quality of his recall on a scale of 0-5. The SuperMemo 2 algorithm is:

1. Split the knowledge into smallest possible items.
2. With all items associate an E-Factor equal to 2.5.
3. Repeat items using the following intervals:
   I(1):=1
   I(2):=6
   for n>2: I(n):=I(n-1)*EF
   where:
   I(n) - inter-repetition interval after the n th repetition (in days),
   EF - E-Factor of a given item
   If interval is a fraction, round it up to the nearest integer.
4. After each repetition assess the quality of repetition response in 0-5 grade scale:
   5 - perfect response
   4 - correct response after a hesitation
   3 - correct response recalled with serious difficulty
   2 - incorrect response; where the correct one seemed easy to recall
   1 - incorrect response; the correct one remembered
   0 - complete blackout.
5. After each repetition modify the E-Factor of the recently repeated item according to the formula:
   EF:=EF+(0.1-(5-q)*(0.08+(5-q)*0.02))
   where:
   q - quality of the recall in the 0-5 grade scale.
   If EF is less than 1.3 then let EF be 1.3.
6. If the quality recall was lower than 3 then start repetitions for the item from the beginning without changing the E-Factor (i.e. use intervals I(1), I(2) etc. as if the item was memorized anew).
7. After each repetition session of a given day repeat again all items that scored below four in the quality assessment. Continue the repetitions until all of these items score at least four.

In SuperMemo 2, the repetition intervals for the first two repetitions are fixed, but the ratio of subsequent repetition intervals depends on the quality of recall - questions that the user finds easy are repeated more often than those that he finds hard. The ratio of successive repetition intervals varies between 1.3 and 2.5. (Despite this statement, the algorithm above allows EF to grow without limit if q is always 5. I do not know whether a limit on EF was in fact imposed.) However, when solving chess problems, you will already be spending a lot more time on the problems that you get wrong, and repeating them more often aggravates this situation. In the Reinfeld Experiment, I tried doing one repetition less during the first week on the problems that had given me the least trouble, but this was not a success. Spending more time on the hardest problems will, in itself, make you remember these problems more clearly, so extra repetitions are not usually necessary. Furthermore, learning to solve easy problems quickly is at least as important as being able to solve difficult ones eventually. Dan Heasman has conjectured that repetition is of greater benefit for simple problems than for complicated ones, and he may be right. If you have a lot of difficulty solving a complex problem, crossing it off the list is often the best course of action. (I did this with a few problems in the Reinfeld Experiment.) However, if we are solving lots of very simple problems with a strict time limit for each problem, it might be worthwhile to make the intervals between subsequent repetitions depend on the time taken solving the problem, and whether you got answer right. (If you get it right first time in under 5 seconds, you may not want to repeat it all.)

If the quality of recall drops below 3 (i.e. the answer was incorrect), SuperMemo restarts the repetition process. This measure is clearly inappropriate for a chess problem that is too difficult and complex for you, but would make more sense if you failed on a simple problem.

At the end of each session, SuperMemo repeats the questions that the user got wrong (or got right with serious difficulty), and goes on repeating any that he continues to get wrong, until he gets them all right. I have tried revisiting the chess problems on which I have failed at the end of short sessions, while they are still fresh in my mind, and it seemed to help, but it is questionable whether it was worth the time spent. I did not consider using this technique in the Reinfeld Experiment, largely because I needed long sessions to get the work done, and I was usually tired by the end of a session. (Not something I would necessarily recommend!)

In SuperMemo 5, the repetition intervals are set using a more complicated method. The first two intervals are typically much larger than in SuperMemo 2. For a level of difficulty at which subsequent intervals successively double, the first two intervals are 6 and 18 days. (These intervals are very close to the one week and two weeks that I used for learning mathematical physics twenty years earlier.) The knowledge acquisition rate for SuperMemo 5 is said to be more than twice that for SuperMemo 2, as a result of the wider intervals. In the Reinfeld Experiment, I found that the learning process was much easier and faster if I spaced the early repetitions more closely even than in SuperMemo 2. More early practice is needed for the chess problems both because of their greater complexity, and because of the need to be able to solve them quickly. Later versions of SuperMemo have further refinements, but they seem to be specific to simple learning tasks.

Scheduling Expanding Repetitions

2011-02-01T04:25:00.000-08:00

In the Reinfeld Experiment, I started out with schedule written on a piece of paper, but this method soon became too cumbersome. What I needed was a more flexible method that enabled me to see potential schedule clashes well in advance. I decided to use a spreadsheet.

For the purpose of illustration, let us assume that we can study chess tactics every day, and that we wish to take on a new batch of problems every five days. Assume also that we will be repeatedly solving problems on days 1, 2, 4, 8, 16.… We open a new spreadsheet, and write r1, r2, r3, r4, r5.… into column A in rows 1, 2, 4, 8, 16.… to indicate the days on which the repetitions of batch A are to take place. We then copy and paste this column into column B with r1 on the day for the first repetition of batch B (day 5 in this case). Similarly, we copy and paste this column into the appropriate places for the other batches:

(I have also labelled the first day of each week in a column down the right hand side. We can move this column further to the right later on, if we want to add more batches of problems to the schedule.) In principle, the repetitions for each batch continue forever, with the repetition intervals doubling each time - but spreadsheets are finite, so we have to impose a cut off somewhere, but additional repetitions can be pasted in later.

The spreadsheet enables us to see the schedule very clearly, and we can very easily change it. In this case, the repetitions mesh together very well. The only problem is that we keep getting r5 for one batch and r1 for another on the same day. However, in all these cases, the previous day is free. Moving r5 to the previous day solves this problem, at the cost of reducing the interval from r4 to r5 from 8 days to 7, and increasing the interval from r5 to r6 from 16 days to 17. This change will make r5 a little easier and subsequent repetitions a little harder, but the difference probably will not be significant. It would have been better to move r5 forward a day, which would make r5 a little harder and perhaps r6 a little easier, but that does not fit. (I move the repetition itself, but not the repetition on the spreadsheet, because that would be confusing if I had to move it again. Note that moving a repetition does not change the dates of the future repetitions for that batch.) If we have a choice between moving two repetitions for different batches by the same number of days, it is better to move the later one. The later repetition will have larger intervals before and after it, so the percentage change in these intervals as a result of the move will be smaller. We should try to keep the percentage change in the repetition intervals as small as we can.

When we have completed a repetition, we can indicate this by emboldening the corresponding entry in the spreadsheet. I also change the background colour of the current week, so that I can easily see where I am in the schedule.

The schedule above does not have any free days, and r6 for batch A collides with batch G, off the bottom of the diagram. We could solve this problem by moving r6 forward 4 days, but we then run out of space. If we can do only one repetition per day, we would have to delay the take on of batch H. An alternative approach is to make the batches small enough to allow two repetition slots per day. We can then continue to take on a new batch every five days for a very long time indeed.

The Reinfeld Experiment

2011-01-05T07:04:00.000-08:00

Back in 1969, when I was a student, I devised a learning technique that I used when I had a particularly difficult piece of mathematical physics to learn. Mostly, I did not need this technique - I reserved it for use when I knew I was going to have serious trouble. The technique was very simple. I would thoroughly learn the material in question, wait a week, revise it, wait another two weeks and revise it again, wait another four weeks and revise it again, and so on doubling the interval between successive revisions. Since the intervals kept doubling, they soon became very large. In practice, I either passed the exam, and did not have to worry any more, or I was using the material so often that I no longer needed a revision programme.

Could I apply my old method to chess tactics? I thought so. For the learning phase, I could solve a batch of problems three times. For the revision phase, I could solve them again after one week, two weeks, four weeks etc. It all looked feasible, so I decided to give it a try. I divided Reinfeld’s 1,001 Winning Chess Sacrifices and Combinations into ten batches, which I labelled A to J. My original schedule was:

Week 1: 3A
Week 2: 3B + A
Week 3: 3C + B
Week 4: 3D + C + A
Week 5: 3E + D + B
Week 6: 3F + E + C
Week 7: 3G + F + D
Week 8: 3H + G + E + A
Week 9: 3I + H + F + B
Week 10: 3J + I + G + C

In Week 1, I would solve A three times (which I have written as 3A). In Week 2, I would solve B three times and revise A for the first time. In Week 3, I would solve C three times and revise B for the first time. In Week 4, I would solve D three times, revise C for the first time, and A for the second time. I planned to carry on in this way, with A due for its first revision in Week 10. (You will notice that taking on a new batch of problems every week results in an additional repetition each week every time the number of batches doubles.) I started solving a new batch of problems each Monday, and took Thursday and Sunday off. My daily schedule was:

Week 1: A1 A2 -- -- A3 -- --
   1 2 3 4 5 6 7
Week 2: B1 B2 A4 -- B3 -- --
   8 9 10 11 12 13 14
Week 3: C1 C2 B4 -- C3 -- --
   15 16 17 18 19 20 21
Week 4: D1 D2 C4 -- D3 A5 --
   22 23 24 25 26 27 28

(Where A1, A2, A3, A4 and A5 are the first, second, third, fourth and fifth repetitions of A, and similarly for B, C and D.) In reality, my schedule was much messier, because I conducted a number of experiments to find out which repetition intervals worked best.

In my very first attempt, I had A4 a week later, and was horrified to find that my performance had dropped sharply from A3, so I hurriedly amended my schedule to that above.

The cognitive psychology experiments quoted previously suggest that doing the first three repetitions on Monday, Tuesday and Wednesday should be less good than doing them on Monday, Tuesday and Friday, or doing them on Monday, Wednesday and Friday, but I did not notice an obvious difference.

I tried doing the third repetition for only those problems which had caused me trouble (i.e. those where I did not get the entire solution in the book, before looking at it, on one or both of the first two repetitions). This turned out to be a bad idea. The troublesome 20% of the problems took 50% of the time, and not doing the easier 80% slowed down subsequent repetitions and made them more difficult, so I probably was not saving any time. I did another repetition of the problems that I had missed to fix the problem.

When I got to the fifth repetition, it felt harder than the fourth repetition, which is not very surprising if you look at my daily schedule. The intervals between successive repetitions were 1, 3, 5, 17, so there was a big jump in going to the fifth repetition. Moving the fifth repetition forward a week would have avoided this problem, but it was too late for that, so I moved the sixth repletion forward a week as a quick fix. If I had done the fifth repetition a week earlier, the intervals would have been 1, 3, 5, 10. That is getting close to 1, 2, 4, 8, which I suspected would work better.

When I got to batch G, I decided to carry out a detailed test of the repetition intervals 1, 2, 4, 8... With this schedule, numbering from the day of the first repetition, the repetitions take place on days 1, 2, 4, 8, 16... (As noted in the earlier section: Introducing the Expanding Repetitions Method, this schedule is the reverse of the 7 Circles schedule, where the repetition intervals are 64, 32, 16 … 2.) The n th repetition takes place on day 2 ^ (n-1). I attempted a complete solution of each problem before looking at the solution in the book, and scored my success on a five point scale:

0 points: Wrong!
1 point: Right idea
2 points: Right first move
3 points: Proved the combination sound
4 points: Complete solution, as in the book

When I found a winning combination that was less good than the one in the book, I gave myself 2.5 points. I gave myself full marks when I found a better solution than the one in the book - but no bonus points for that - or for spotting other mistakes in the book! My progress so far is summarised below:

Repetition: 1 2 3 4 5 6 7 8 9
Percent score: 85% 93% 95% 95% 97% 97% 95% 92% 90%
Minutes/problem: 3.5 2.7 2.0 1.5 1.3 1.3 1.2 1.0 1.2

In this table, minutes/problem is the total number of minutes of the session (including checking the solution) divided by the number of problems tackled. This schedule was so successful that I decided to use for the remaining batches, and convert the other batches over to it. (When moving to the tighter schedule, the repetitions that have already been done will be a little late according to the new schedule. This makes the intervals between the repetitions more equal than they would have been if the new schedule had been used from the outset. We saw in the cognitive psychology section that making the repetition intervals more equal has little effect on long term memory retention - but performing a repetition too late makes that repetition harder - so it is better to use the tighter schedule from the outset.)

When I got to batch H, I found that it was difficult to squeeze all the repetitions into my schedule. I solved this problem by taking on a new batch every two weeks rather than every week. (I also had another week in which I did not take on a new batch, to move the previous batches over to the schedule of batch G.)

At the time of writing, I have completed the early repetitions on all the batches, and am I up to seven repetitions on the earlier batches (or eight if you count my failed attempt at the 7 Circles). I found this method to be much more effective than the 7 Circles at teaching me to solve these problems quickly and reliably (and to retain that capability) - but it is to early to assess the impact on my ability to solve fresh problems, or my chess generally. Watch this space!

[Repetition 8 added to the table on 15 March 2011. Repetition 9 added to the table on 19 July 2011.]

Reinfeld’s 1,001 Winning Chess Sacrifices and Combinations

2011-01-05T06:46:00.000-08:00

I used Fred Reinfeld’s 1,001 Winning Chess Sacrifices and Combinations for my first experiment with the Expanding Repetitions method, so I will give it a brief review. The book is an old classic that is still in print more than fifty years after it was first published. It consists almost entirely of diagrammed problems and their solutions, and wastes little time on idle talk.

The problems are in random order of difficulty, grouped by theme, e.g. pinning and knight forks. The positions are mostly well chosen, with the feel of real game positions. The level of difficulty ranges from very easy examples, to a small number of difficult ones. The book commendably includes a lot of crude tactics, like chasing pieces with pawns, that are practically important, but often do not find their way into other books. On the other hand, it has more than its fair share of queen sacrifices and brilliant but untypical combinations that the average player is not likely to be able to analyse completely from the diagram. Some of the combinations are also very similar or even the same. Most tactics books are like this, however!

The positions (but not the full solutions) are available free on the web:

http://www.chessville.com/downloads/downloads_tactical_exercises.htm

The book predates computer checking, and the solutions are often not the only way of winning, and sometimes do not work. It also is sometimes a bit of a lottery as to which defences Reinfeld includes. Some would say this adds to the realism and keeps you alert, but others would say that it is a real pain! Reinfeld's solutions are usually the best continuation, however, and as Lasker said, “if you find a good move look for a better one.” A significant disadvantage of this book is that its better problems are very widely copied by other books.

The book uses descriptive notation, which might be a problem to some. The quality of reproduction of the current Chess Lovers’ Library edition is poor, but that was not a serious problem to me. The pages of this edition also fall out, but the book is better as a series of problem and solution sheets.

The book clearly has its limitations, but I had already spent time working on it with my failed attempt at the 7 Circles, and the wide range of difficulty in the problems was ideal for my experiment. (I wanted to know how well the Expanding Repetitions Method worked both for easy problems and for harder ones.) For further tactics book reviews, see my later article: Dan Heisman's 7-10 Basic Tactics Books.

7 Circles

2010-12-02T08:03:00.000-08:00

Michael de la Maza famously improved by 620 rating points as an adult, using his 7 Circles method as part of his training. In his 7 Circles programme, he solved 1,000 tactical chess problems 7 times using the CT-ART 3.0 tactical trainer program. He worked through the problems in order of increasing difficulty, beginning with simple one-move mates and two-move combinations and progressing to 7-8 move mates and combinations. He said that the tactical trainer saved him hundreds of hours because he did not have to enter complicated positions manually into a chess program when he failed to understand the solutions. (Personally, I take the view that if you need a computer to understand the solution, the problem is too hard for you, and you should cross it off the list!)

His schedule was:

Circle 1: 64 days, 10 minutes per problem
Circle 2: 32 days, 5 minutes per problem
Circle 3: 16 days, 2.5 minutes per problem
Circle 4: 8 days, 1.25 minutes per problem
Circle 5: 4 days, 30 seconds per problem
Circle 6: 2 days, 30 seconds per problem
Circle 7: 1 day, 30 seconds per problem

In the 64 day circle, he spent up to five minutes trying to find the first move and up to an additional five minutes working out all of the variations. This is odd on the face of it. How can you possibly know that you have found the correct first move, without having worked out all the variations? I expect that he was guessing the solution a move at a time, and entering it a move at a time into the computer program, which told him whether or not each move was right, and probably also prompted him with each defensive move. Needless to say, this is not what you do in a game, but it worked for him. However, his articles said he had trouble carrying over the improvement he had made at the 7 Circles into chess games. I expect that he would have had less trouble carrying over this improvement if he had worked out all the variations before consulting the computer, but his 7 Circles would then have taken longer.

If you have read the cognitive psychology section, you will realise that the main disadvantages of this programme are that during your first few circles, your memory of the previous circles will be weak and your progress will be slow, and that your memory of the last few circles will soon fade away.

During the 64, 32, 16, and 8 days circles, de la Maza believed that he was improving his calculation ability - and during the remaining circles, he believed that he was improving his pattern recognition ability. That is probably accurate. The problem is that most of the improvement in his pattern recognition ability would have been short lived. It is perhaps noteworthy in this context that he took part in one tournament, achieved an outstanding result, and promptly retired from chess. What he lacked was a revision programme. He needed to continue to practice his seventh circle to maintain his pattern recognition ability.

Another objection here is that working on calculation ability and then on pattern recognition ability is putting the cart before the horse. Dan Heisman recommends doing a 7 Circles type programme on a large number of very simple tactical problems until you can do them very fast, and then moving on to more difficult problems, which looks more logical to me. Again, a revision programme would be needed.

Another issue here is that of what happens if you cannot match de la Maza’s achievement of halving the time taken for each circle - and it is going to be more difficult if you cannot afford as much study time - and have to stretch the programme out. 7 Circles is a catchy name, but I believe that the real end point of the learning phase for his programme is to be able to solve the 1,000 problems at under 30 seconds apiece, not the completion of seven circles. I expect that most players would need more than seven circles.

The following link gives to rating improvement for the 22 players who are known to have completed the 7 Circles programme (or some variation on it):

http://temposchlucker.blogspot.com/2005/05/ratingprogress-of-knights-errant.html

The link says that it is difficult to compare the ratings before and after the 7 Circles, because: some players modified the plan, trained tactics for much longer, had no official rating, had a rating from a different source, have not finished the programme, had to play more to get their new rating, did not have published ratings, abandoned the program, or their start ratings were unknown so a later (higher) rating had to be used or estimated.

More to the point, the 7 Circles has attracted a lot of attention (de la Maza’s articles the most visited on ChessCafe, according to Dan Heisman), and I expect that many more than 22 people have made a serious attempt at it. The players who completed the programme are likely be the most capable and highly motivated ones, who would have improved anyway, and those who did make an improvement are the ones most likely to report their results.

Another difficulty is that the players may have been doing other training, and would have to be playing chess to get a rating, so we have no way of knowing what caused the improvement. A third difficulty is that with a typical number of games played in a season, there is a large uncertainty in the ratings both before and after the 7 Circles, and the relatively small rating improvement typical of players rated over 1600 (the median improvement is currently 75 points) could easily have been due to chance.

The only way round these difficulties is to measure any improvement achieved in the ability to solve fresh problems (which could be measured much accurately and reliably than any change in rating). These problems could be in the style of Ray Cheng’s Practical Chess Exercises (tactical and positional moves, including tempting but unsound sacrifices, in random order) to approximate the conditions of a game.

Nonetheless, the achievements of de la Maza and a few others with the 7 Circles are remarkable. All the more so in de la Maza’s case, because he had a strident disregard for all forms of chess skill except tactics.

My own experience of the 7 Circles was that by the time I had tackled all of Reinfeld’s 1,001, and returned to the beginning, I found that I did not remember much! My accuracy improved a little (I just tackled the problems that I had got wrong the first time through), but I was not obviously faster. Nonetheless, this failure to learn was a good learning experience for me. There had to be a better way.

Once Through vs. Repetition

2010-11-30T09:56:00.000-08:00

The most obvious method of learning chess tactics is simply to solve a large number of problems, tackling each one once and only once. Solving a 1,000 problems 10 times is clearly going to be more effective than solving them once, but this is not a fair comparison. How do the two methods compare if you spend the same time on both? Is it better to solve 1,000 problems 10 times, or as many fresh problems as you solve in the same time? The relevant test here is the ability to solve fresh problems. The rate of improvement at solving a fixed set of problems will be greater than that for solving fresh problems - but this improvement will not be fully reflected in an improvement in solving fresh problems.

The motivation for the once through method is the belief that you get good at what you practice. In a real game, you do not have benefit of having seen the positions before, and perhaps replicating this in your training will increase its effectiveness. A disadvantage of the once through method is that there are likely to be long time intervals between occurrences of all but the most common tactical patterns, so learning to spot them is likely be a slow process - and may not happen at all - if you have completely forgotten the previous occurrences. Since memory cannot be used so efficiently in the learning process with this method, the average time spent tackling each problem will be greater than if problems are repeated. You are not going to be able to solve 10,000 problems in the time it takes to solve 1,000 problems 10 times.

The once through method is rather like learning French by watching French films with subtitles, watching each film once and only once. It is very similar to the way that small kids learn languages, but it is not going to be an efficient way for adults to learn. If I had to learn French this way, I would want the films on DVD, so that I could study each section repeatedly, and revise what I had learned.

Although learning chess tactics shares some important characteristics with learning a foreign language, there are also some important differences. The most obvious difference is that chess is visual, whereas language is verbal. There are many reasons why you may fail to solve a tactical problem, but the most relevant are:

* Failure to spot a tactical pattern (e.g. a double attack).

* Failure to study the position adequately before attempting a solution.

* Failure to work out the idea behind the combination.

* Failure to analyse efficiently (e.g. spending to much time analysing one of your less good moves, or failure to consider a defensive move for the opposition).

* Failure to visualise the position correctly during analysis (e.g. failure to spot that a file has been opened).

Repetition can potentially help you address all of these failings (and remember the lessons that you have learned). However, if you take this technique too far, you risk ingraining thought processes that are different from those that you need for solving fresh problems, and in a real game. Your optimum tactics training depends on the capabilities that you have at the start of the training, and one training method may not be enough.

To conduct a fully scientific comparison of the effectiveness of the two approaches, we would need a controlled trial with a large number of volunteers, in which they all abstain from all other chess activity during the trial (otherwise we would not know what had caused the improvement). We would also need a reliable method of measuring the volunteers problem solving capability (on fresh problems) at the beginning and end of the trial. To make sense of the results, we would need some way of profiling the capabilities of the volunteers. For example, if the volunteers all started off with an uncommonly poor knowledge of tactical patterns - that might skew the results in favour of repetition.

Last but not least there is the question of motivation. Repeatedly solving problems has the disadvantage that it is less interesting than solving fresh problems, which does not help motivation. On the other hand, the higher success rate when solving repeatedly aids motivation, so perhaps we have a draw on motivation.

Lessons from Cognitive Psychology

2010-11-30T09:55:00.000-08:00

The primary way of learning chess tactics is by repetition, either by repeatedly solving the same problems, or by solving different problems that contain the same or similar tactical patterns.

Cognitive psychologists have conducted many trials of learning by repetition for simple memory tasks, for example: Question: “What is the capital of Denmark?” - Answer: “Copenhagen." The trials show that long term memory retention is poor when the repetitions are closely spaced (often summarised as “cramming does not work”), but improves as the spacing is widened (until the subject completely forgets the previous repetitions). However, when the repetitions are widely spaced, the success rate improves only slowly with each successive repetition. Cognitive psychologists have found that a high success rate is important for maintaining student motivation. Successful repetitions are also typically faster than unsuccessful repetitions, so more successful repetitions can be performed within the same amount of study time.

To get the best of both worlds, it is usual to space the initial repetitions closely, and gradually increase the intervals between each successive repetition. This scheme has the advantage of ensuring that a high success rate is rapidly achieved, and is sustained or improved upon as the repetitions continue.

Cognitive psychologists have conducted trials comparing the long term memory recall achieved by repetitions at increasing intervals with the same number of equally spaced repetitions. The trials with equal intervals have often done slightly better - but equal intervals are possible only if the repetitions have a fixed end point - which will not be the case if the information has to be retained indefinitely. Nonetheless, this is still a useful result, as we shall see later.

These results form the basis of many practical learning systems, notably those for learning foreign languages. Some of these foreign language learning systems, e.g. the Pimsleur system, use modern methods in which the student infers the grammar.

Repeatedly solving simple tactics problems is partly a memory task. Although the exact position is unlikely to appear in one of your games, the same configuration of active pieces could, and is reasonably likely to do so provided that the problem is well chosen. This situation is analogous to that in learning a foreign language. A foreign language course does not just teach you to speak and understand the phrases in the language course, but also to speak and understand any phrase that can be constructed using the same words and underlying grammar (which the student usually has to work out for himself nowadays). In both learning chess tactics and learning a foreign language, we are trying to learn a skill (create a mental program) rather than just memorise data items. The underlying biological processes are the same for both memory and learning a skill: creating neural connections. There are, however, important differences between learning a foreign language and learning chess tactics as we shall see in the next section.

Another important cognitive psychology finding is that of the superiority of active learning over passive learning. For example, the example above, actively recalling the answer “Copenhagen” is more effective than just reading the answer. In chess, finding the solution to a chess problem is more effective at building memory and skill than just reading the solution. Similarly, when studying a game, laying a sheet of paper over the book and guessing the next move is more effective than just playing the game through.

Further information on the relevant findings of cognitive psychology can be found in the following article (and its references):

http://en.wikipedia.org/wiki/Spaced_repetition

This reference is particularly interesting:

http://www.psych.wustl.edu/coglab/publications/Balota+et+al+roddy+chapter.pdf

About the Author

2010-11-20T10:49:00.000-08:00

I currently live in Yorkshire, England. I am 61 years old and have played chess on and off throughout my life. My best performances were about FIDE 2,000, but I have not played for 15 years now. Goodness knows what my current playing strength is - but FIDE 2,000 (or its English equivalent) would be a very good achievement.

I injured my foot rock climbing earlier this year, and had to spend a lot of time lying on my back with my foot on a pile of cushions, and started to look at chess again. Since I could not sit up at a board, I solved a lot of problems. I did not do quite as well as I did 15 years ago, but what was most striking was that, for the most part, I got the same problems right and the same problems wrong!

I became interested in finding a more effective way to do tactics training, and this became a project in its own right. If I could find a method that worked on me, it should work on anyone!

Aside from chess, I studied physics at university (Imperial College and Cambridge) and have retired from a career designing and developing computer systems.

Introducing the Expanding Repetitions Method

2010-11-20T10:47:00.000-08:00

My current focus is on what I shall call the Expanding Repetitions (ER) method for improving your chess tactics. This method was inspired by Michael de la Maza’s 7 Circles, and Dan Heisman’s Novice Nook articles:

http://www.chesscafe.com/text/skittles148.pdf
http://www.chesscafe.com/text/skittles150.pdf
http://knightserrantfaq.blogspot.com/
http://danheisman.home.comcast.net/~danheisman/Articles/Novice_Nook_Links.htm

The 7 Circles and ER methods are similar in that with both, you solve tactics problems repeatedly, but there are some important differences:

* With the 7 Circles method, you tackle 1,000 problems all in one go, solving them 7 times, whereas with the ER method, you tackle problems in bite sized chunks that you can solve (or have a good attempt at it) within one day’s study time.

* With the 7 Circles method, you halve the time interval between each repetition (64 days, 32 days, 16 days, 8 days, 4 days, and 2 days), whereas in the ER method you roughly double it (e.g. 1 day, 2 days, 4 days, 8 days, 16 days, 32 days, 64 days…).

* The 7 Circles method is once and for all exercise, whereas the ER method is an incremental process that you can continue for your entire chess playing life, or for a long as you need it.

The main motivation for the ER method was my experience of studying tactics. My main insight was that although I learned a lot from my training, I soon forgot most of it! I decided to adapt a heavy duty learning system that I had devised as a student, and try it on Fred Reinfeld‘s 1,001 Winning Chess Sacrifices and Combinations. Shortly after starting my experiment, I discovered that cognitive psychologists had got there first, and that expanding the intervals between each repetition was the standard approach, and formed the basis of many successful systems for learning by repetition, notably those for learning foreign languages:

http://en.wikipedia.org/wiki/Spaced_repetition

(Cognitive psychologists use the term Spaced Repetition, which is defined to be learning by repetition with expanding intervals between each repetition. This is confusing in a chess context, in which the 7 Circles method uses spaced repetitions, but decreases the intervals between them.)

I subsequently found out that the repetition scheme that worked best in my tests was similar to that underlying the SuperMemo learning system:

http://www.supermemo.com/english/ol/beginning.htm

Most learning systems present the questions that are causing difficulty more often than those that are not, but this approach is problematic when applied to chess problems - where you will be spending proportionately more time on the difficult ones anyway - but this technique is not fundamental to SuperMemo.

The main benefits of the ER method are:

(1). You do not have to learn and forget. The closely spaced repetitions allow a high level of capability to be built up rapidly, and the more widely spaced repetitions ensure that this capability is retained.

(2). A large commitment of time is not required. A little effort often is all that is needed - but a lot of effort often will work quicker!

(3). Superhuman efforts are not required. You do not have to solve 1,000 problems in a day!

(4). Blind faith is not required. The method can measure your progress, not only from one repetition to the next, but also on problems that you have never seen before. This can be accomplished by presenting the problems as a series of “tactics exams”, and comparing your performances when each one is tackled for the first time.

I found many references to using standard learning systems for chess - particularly for memorising openings - but only one reference to applying these techniques to chess tactics:

http://patzerquest.blogspot.com/2005/03/supermemo.html

I have only been able to scratch the surface in this introduction, but will flesh it out in future posts.

Welcome

2010-11-20T10:44:00.000-08:00

Welcome to the Empirical Rabbit - the blog that seeks out hard evidence concerning chess training methods for the average player - particularly the not so young average player. (I considered calling it the Experimental Rabbit, but I thought I might be bothered by animal rights activists! Please be assured that the only rabbit being experimented on is me - but other rabbits are cordially invited to join in.)