Stockfish DD with the Shredder Interface

Having got Stockfish DD working in Arena 3.0, I decided to install it in the Shredder user interface, which is common to my Shredder Classic 4, and the more expensive Shredder UCI and Deep Shredder. Please see my previous article for the details of downloading Stockfish DD and setting up the necessary files. To install Stockfish DD, select Extras → Engines → Install Engine, select UCI Engine, click OK, set Engine to the path name of the Stockfish DD executable, and click OK again.

Shredder describes the newly installed engine as Stockfish DD 64 bit SSE4.2 (1024 MB), and recognises the two CPUs on my PC. I set Threads to 2 in Extras→ Engines → Engine Options, which can also be reached by Extras → Options → Engine Options. I ticked Use Engine Book in Extras → Options → Book Options. The hash table size was already set to 1024 MB in Extras → Engines → Hash Tables. The engine currently in use can be changed by right clicking in the analysis pane at the bottom of the window and selecting Change Engine.

When I restarted Shredder, the number of threads was set back to 1, and the CPU usage dropped correspondingly.  I sent an email requesting help to support at shredderchess. Stefan Meyer-Kahlen responded very quickly. The cause of my problem was that Stockfish was telling Shredder that it wants one thread rather than two, despite my setting in the polyglot file. I set the number of threads to 2, clicked the Create Engine... button at the bottom of Engine Options, and saved the new engine as Shredder DD 2 CPU. This new engine kept the two threads on shutdown.

The Shredder board is a little less attractive than the Arena board:

This screen shot shows the results of Analyse Game for a game that I imported from the clipboard. Shredder incorrectly calls the bar chart showing the engine's evaluation after each move a histogram, which confused me until I read the manual very carefully. When in the opening book, the moves pane shows ratings and rating performances in addition to the percentage scores for each choice of move:

Nonetheless, Chessbase Online on my iPad has access to a much larger database. The Shredder Classic 4 User Manual can be found at:

http://www.shredderchess.com/chess-program/shredder-classic-4-windows.html

This manual is much clearer than the Arena Help text, largely because the Shredder interface is more standard and less confusing. I find Shredder easier to use than Arena 3.0 and less troublesome. It also appears to have much fewer bugs. It has less features, but is likely to satisfy most player's needs. You get regular on-line updates, and the support is good. The Shredder engine has a reputation for having a more human playing style than most engines, and playing a better simulation of a weaker human player. Currently, even Deep Shredder is about 200 ELO points weaker than the strongest engines, and Shredder Classic 4 is about 200 points weaker still on a 2 CPU machine (which is about 60 ELO points weaker than a 4 CPU machine). (See CCRL and CCRL Shredder.) Nonetheless, it is still very strong by human standards, and you can easily add other engines and opening books. Shredder Classic 4 is well worth considering at EUR 29.99. I am still using it.

Stockfish DD in Arena 3.0

I have got Stockfish DD and its opening book working in Arena 3.0. Stockfish DD is currently one of the strongest chess engines on the PC, and could even be the strongest. It is a free open source program. Arena is a free graphical user interface for chess. Arena 3.0 was the latest version of Arena when I downloaded it. Arena 3.5 was released a few days later, but perhaps I am better off with the older version for now. I hit some problems, so I have decided to write an idiots' guide, both for my own future reference, and to help others.

Arena Basics

Arena can be downloaded from:

http://www.playwitharena.com/

I did not have any problems installing Arena 3.0, or customising the appearance, so I will not dwell on that. Arena was installed by default in the directory c:/ Program Files (x86) / Arena. The Arena interface looks very attractive:

(You can click on these diagrams to enlarge them.)  You can start a new game with Game → New, and move the pieces by dragging and dropping them. The buttons under the move pane are:

The first four buttons in green advance and take back moves. The cog button instructs the engine to calculate, make a move and continue playing that colour. The Analyse button starts continuous analysis, and the Edit button allows the user to make moves for both sides. (When either of the last two buttons is clicked, it gains gains colour and an outline to indicate that it has been selected. The Demo button causes the engine to play against itself. When it is clicked, the cog button turns into a red X. Clicking this X stops the engine playing against itself.) The options at the bottom of the Game and Position menus allow PGN and FEN to be imported and exported from the clipboard.

Levels → Adjust allows a fixed time per move to be set. Levels → "Limit Playing Strength to Elo" can be used to weaken the Engine. This works for the 32 bit and 64 bit versions of Rybka supplied with Arena, but is greyed out for the other supplied engines, and Stockfish DD. (The time allowed for any engine can be reduced by a percentage, by clicking on Engine → Manage, selecting the Details tab and the Special sub-tab.) The engine can be prevented from thinking on your time by right clicking the mouse in the engine analysis pane at the bottom of the Arena window, and ensuring that Ponder is not ticked.

Installing Stockfish DD

There are no detailed instructions for installing Stockfish, but I got some much needed help from:

http://support.stockfishchess.org/discussions

Stockfish DD and its opening book can be downloaded from:

http://stockfishchess.org/download/

The download stockfish-dd-win, contains the directories:

src
src_c11
Windows

and the files:

Copying
polyglot
Readme

The directory Windows contains the files:

stockfish-dd-32
stockfish-dd-64-modern

The opening book download stockfish-231-book contains the file Book.bin.

The file readme has a .md extension, and I could not read it, but I found this on the web:

https://github.com/mcostalba/Stockfish/blob/master/Readme.md

I set up a directory called Stockfish Chess in c:/ Program Files (x86) / Stockfish Chess, and copied polyglot, Book.bin and stockfish-dd-64-modern into this directory.

My laptop has an Intel Core i3 processor (which has 2 cores and 4 threads), 3 GB of main memory and runs 64 bit Windows 7 Home Premium. The polyglot file has the default settings Threads = 1, Hash = 128 and OwnBook = false. The advice is to set Threads to the number of processor cores rather than the number of threads. I set Threads = 2. The hash table size is in MB. I set Hash = 1024. I also set OwnBook = true.

To install Stockfish DD in Arena, select Engines → Install New Engine... Set File Name to the path name of the engine program, and click Open.

(N.B. With these settings, I find that, when I run Stockfish DD, the CPU usage is typically 50%, with memory usage at just over 2GB. Arena shows the hash table usage as 0%. However, it also shows the CPU utilisation of the engine as 0%, which is certainly not true.)

Arena Engine Configuration

Right clicking the mouse in the engine analysis pane at the bottom of the Arena window, and clicking Configure Stockfish DD, brings up the dialogue:

I set the number of threads to 2, the hash table size to 1024 and ticked Own Book, mirroring my selections in the polyglot file.

Opening Books with Arena

Arena opening books are stored by default in the directory c:/ Program Files (x86) / Arena / Books. You can select one of them as the current Arena Main Book using Book → Manage, which brings up the dialogue:

You can use the Load button to select the book, and tick the Use Arena Main Book box to activate it.

There is a well hidden tick box for selecting the Arena Main Book rather than Stockfish's own book. Select Engines → Manage. Select the Details tab, click on the line for Stockfish DD, and select the Books sub-tab. This reveals a tick box labelled "Use Arena Main Books with this Engine." When Own Book is ticked, as described above:

* When "Use Arena Main Books with this Engine" is ticked, Stockfish uses the Arena Main Book to make its moves, and Arena also displays it in the move pane when Book/TB is selected.

* When "Use Arena Main Books with this Engine is not ticked, Stockfish uses its own book to make moves. Nonetheless, when the Book/TB tab is selected, the Arena Main Book is displayed. This is confusing, but it is also helpful because the Stockfish book is invisible to the user.

When Own Book is not ticked, but "Use Arena Main Books with this Engine is ticked, Stockfish uses the Arena Main Book, as we would expect. When neither box is ticked, Stockfish does not use a book at all. (The Arena engine configuration appears to override that in the polyglot file.)

Analysing a Game with Arena

I had a problem getting Arena to analyse a game, and generate a graph of the engine's evaluation of the position throughout the game. The key ingredient is to click Engines → Automatic Analysis, select the Output tab, and tick the "Save calculated values in game" box. You can then click the Start button on the Source tab to populate the graph. (Clicking to the right of the Permanent Statistics graph toggles it on and off, and right clicking on the graph itself brings up some options. Left clicking on the graph selects a move.)

Verdict

Stockfish DD is much faster on my PC than it is on the iPad. I found that Arena 3.0 frequently crashed or otherwise misbehaved, but I did not encounter any show stoppers. Arena mostly worked well, but was not easy to learn. There is no user manual, and the help text is often unclear and difficult to follow. There are a huge number of options; and the interface is not very intuitive, and sometimes does not follow the usual conventions. I persevered with Arena because it is the only interface that will connect to my Novag Citrine. That is however, still a future project. I am still making good use of the Citrine's on-board engine as a sparring partner.

Chessimo on the iPhone and iPad

I previously reviewed the old Windows version of in my earlier article: Chesimo Endgames 01. This version of Chessimo provided:

>4,000 tactics exercises
191 commented endgames
1,440 endgame exercises
>700 commented strategy exercises

The exercises and comments were compiled by the Brazilian GM Gilberto Milos. The more recent iPhone and iPad applications additionally include 101 strategy tips, opening training and a collection of games. All versions of Chessimo attempt to make the user to repeat the exercises six times. See my critique of their approach in my earlier article. For the strategy exercises, my approach was to tackle the all exercises in all three modules, without repetition; and repeat this process as often as I thought necessary.

As a result of AoxomoxoA's comments to a previous post, I decided to investigate the iPhone and iPad versions of Chessimo, called Chessimo and Chessimo HD in the App store. The respective Apps are free, but activation of all the modules costs £5.49 ($8.97) in both cases. Individual modules can be activated at a lower cost. This is much cheaper than the Windows version, which cost me $49.90 two years ago. (N.B. There is no Windows version on sale currently, but a new version is promised.) I was horrified when I first saw the iPhone board:

The algebraic coordinates a-h and 1-8 were very distracting and unsightly. I looked in vain on the Settings screen for a button to turn them off. It was only later that I found that I could turn them off from Menu button on the screen above. The resulting screen was much more tidy:

Nonetheless, the board was rather small and indistinct on my iPhone 4S. I addressed this problem by deploying my rarely used Terms & Conditions glasses. The board was then OK. Nonetheless, the free open source Smallfish app displays a much clearer board on my iPhone. (I use  IOS 7 version with the brown colour scheme and piece set SFAlpha.) I do not need my Terms & Conditions glasses for this one. I tried loading the iPhone version of Chessiomo on my iPad, and got blown up pixelated images of the iPhone screens, which were not at all attractive. In contrast, the board for the iPad version was excellent:

I decided to buy all the modules for the iPad version. The iPad screens are not all perfect. Here is the screen for selecting a Unit from within a Module:

They really should have labelled the buttons with the unit numbers rather than just the word "Unit." Nonetheless, since the Units are in rows of ten, this is not a great problem. If you attempt to drag a piece, but fail to drag it out of its square, Chessimo has the irritating habit of telling you that your move is incorrect. If you try to move a piece of the wrong colour, it not only informs you of your mistake, but annoyingly highlights the piece to be moved.

The old Windows version has the advantage that the FEN for each exercise is written automatically to the clipboard, which is useful for analysis with an external engine. The old Windows version also has an option to play each exercise against the inbuilt Crafty engine. Neither of these options is available with the iPad version. I used Smallfish on my iPhone to carry out analysis, when I thought this was necessary.

The comments for the Strategy exercises are written in very broken English, with copious typos. The word "explore" is consistently used place of "exploit", for example, and "quality" is used in place of "material." I did not have any great difficulty in understanding what was meant in most cases, but found that there was often too little explanation. Chessimo provides useful exercises, but is not a substitute for a good strategy book.

When I bought the iPad modules, I was told that I had bought a "single activation of all the modules." When I tried Restore Purchases in the iPhone version, I was asked for my Apple password, but nothing happened. I did not even get an error message. The modules that I had bought for the iPad were not loaded into the iPhone version. A negative way of looking at this is that it violates the principle that content, once bought, should be available on all my Apple devices. Nonetheless, since the Chessimo Apps are free, this is fair enough. Shredder for the iPhone and Shredder for the iPad are both separate Apps that have to be paid for separately. Hopefully, if you buy modules for the iPhone version on the iPhone, you will be able to restore them for the iPhone version on the iPad. I am at all not confident, however. When I bought the old Windows version, I was not offered any instructions for moving my purchase to a new PC, and Chessimo has reputation for failing to respond to enquiries.

The are two methods of going through all the exercises in a Module sequentially without repetition. The first method is to select all the Units within the module in sequence, tackling the exercises until you get an alert that all the remaining exercises in the Unit are repetitions. (N.B. For the Modules that I have tackled, Units 13 to 20 and Units 33 to 51 consist entirely of duplicates.) The second method is to go sequentially through just the final Unit of a Module (Unit 51), which is a repetition of all the exercises in the Module. There are potential problems with both these methods.

The first method works fine for the first pass through a Unit. However, on the second pass, Chessimo visits the first problem in the Unit (which I expect is a bug), and then moves on to the first repetition in the Unit, rather than the second exercise. It is possible the select each exercise individually from a pull down menu, but that is pain. There is, however, a "Reset your training status" button which would fix this problem by wiping out you training history.

The second method hits difficulties if you have already tackled some on the exercises on another machine. Chessimo allows you to select only exercises that you have already solved within a Unit. A more minor problem is that Chessimo allows only one sixth of the selected time per exercise in the final Unit. This problem can be circumvented by setting the time allowed for each exercise to six times the value that you actually want. The maximum time that you can set per exercise with the iPhone and iPad versions is 30 minutes, which gives 5 minutes per exercise in the final Unit. This is thankfully much more generous than the maximum allowed in the old Windows version.

Overall, I was impressed with the iPad version. It was much more convenient to use than the Windows version, because the iPad itself is so much more convenient to use than my Windows laptop. Nonetheless, it is not so convenient to analyse using an engine, which I previously found to be necessary for many of the endgame exercises. All the versions of Chessimo put obstacles in the way of you implementing your own repetition schedule. Nonetheless, the bottom line is that you get a lot of exercises for your money.

Smallfish

I have been experimenting with the free IOS 7 version Smallfish chess app on both the iPhone and the iPad, and it is time to give it a short review. Smallfish is a modified version of the Stockfish app, which uses the very strong Stockfish engine. The Smallfish gives a much more attractive and clearer board than the Stockfish. I use the SFAlpha pieces and the brown colour scheme:

Smallfish inherits the Stockfish position set up pane, which I find easier to use than that on the Shredder app. I find it very convenient to use Smallfish to analyse positions on my iPhone. This is partly because of the small size of the iPhone, and partly because I cannot lose my position by accidentally touching another part of the screen when setting up the position. I prefer the grey colour scheme on the iPad:

Smallfish provides a graph of the progress of the Stockfish engine's evaluation of the position throughout the game, which is very useful. It is fiddly to select a precise point on the graph by touching it, but the left and right arrows allow the graph to be traversed a move at a time. (The left and right arrows are not available on the iPhone, but left and right swipes can be used instead.) It is possible to enter your games in "Play Both" mode, and use the graph function to find your mistakes.

Smallfish does have one very serious bug. [This bug was fixed on 17/1/2014.] When the user's pawn reaches the eighth rank, it is automatically promoted to a queen. The original Stockfish app correctly allows the user to under promote. Smallfish does allow the engine to under promote, so analysis is not affected. A new version of Smallfish has recently been released, it uses the new Stockfish DD engine, but the under promotion bug has not been fixed.

The number of nodes per second given by Smallfish is implausibly high. The number given by the original Stockfish application looks about right.  The number given by Chessbase Online, which also uses Stockfish, is implausibly low.

Changing the mode (Play White / Play Black / Play Both) is well hidden in Game Mode in the Options menu. It is more visible in the original Stockfish app, but a separate Mode menu would be helpful.

The user interface of Smallfish is more attractive than that of Stockfish. It is particularly impressive on the iPhone. There are some possible improvements though. It would be better if the A-H and 1-8 on the iPad were a touch larger. The last move indication is a little too bold for my liking and cannot be switched off.

Smallfish provides a choice of rather frenetic time limits. I tried game in 30 minutes. I expected the computer to respond very quickly at the level that I had chosen, and was surprised to find two clocks set to 30 minutes. The computer used most of the time on its clock. This is fine if you want to practice thinking on your opponent's time, but was not what I wanted. I tried a five minute game and found that my time counted down to zero and then stopped. It did not go negative, so I had no way of knowing how much time I had actually used.

I played a game between my iPhone and the iPad, which both have the same processor. The iPhone had 30 minutes on the clock and the iPad 5 minutes. The iPhone won. Some attempt appears to have been made to adjust the difficulty according to the time on the clock. Nonetheless, I have not been able to find any convincing research into human playing strength versus time on the clock, so I do not have much confidence in the rating levels offered.

I tried a time limit of 30 seconds per move, moving at about 1 minute per move. The computer took 30 seconds per move and accumulated my time reliably. The best approach appears to be to set a small fixed time per move and find the appropriate level of difficulty by experiment.

Overall, this a useful app, but the under promotion bug needs to be fixed. There are some other improvements that could be made, and a user guide would be helpful.