New Release - Backgammon-NN

[chrisw]

New · a self-learning neural engine

Backgammon-NN

A backgammon engine whose neural network taught itself to play — from random weights, by playing millions of games against itself. It's a full toolkit, not just a board: extend the training to grow stronger nets, build new networks, run automatic matches against other engines or against itself, and play it on your PC — in a graphical desktop app or a text-only console app.

https://whittingtonchess.com/backgammon

[chrisw]

New · a self-learning neural engine

Backgammon-NN

A backgammon engine whose neural network taught itself to play — from random weights, by playing millions of games against itself. It's a full toolkit, not just a board: extend the training to grow stronger nets, build new networks, run automatic matches against other engines or against itself, and play it on your PC — in a graphical desktop app or a text-only console app.

https://whittingtonchess.com/backgammon

Backgammon-NN for Windows with GUI
Windows 64-bit · 62 MB · free · no installer

The whole thing is in one file — the board, the Rust engine and the trained network. There's no installer, no Python and no PyTorch to set up: download it, double-click it, and you're playing. The net runs natively inside the app, so it plays exactly as strongly as it does from source.

It opens on its strongest setting — Monte-Carlo rollouts, about 0.8 seconds of thinking per move. If that's too much for you, drop it to 2-, 1- or 0-ply from the Opponent box, or take on the hand-crafted evaluator instead.

Release at:
https://github.com/Chris-Whittington-Ch ... kgammon-NN

Dev report:
https://whittingtonchess.com/backgammon-report

Backgammon sure beats chess for fun development, results come in minutes, as opposed to many hours, if not days, for chess.

[Essay]

Very interesting - many thanks Chris.

[chrisw]

Deeper net gained some Elo.

Updated bot GUI-engine

https://whittingtonchess.com/backgammon

[chrisw]

New release Backgammon. The NN is now at ply 0 parity with gnubg, which is supposed to be gold standard. I’ve still got some levers to pull to improve it further. Game play and search not yet optimised , been too busy getting the NN up to or beyond gnu. It’s a good framework for NN development because everything is so fast compared to chess, and there is nothing better than fast feedback from experiments. Dev reports and latest engine (with GUI) on the webpage.

[chrisw]

A live 1,000-match backgammon strength test between Chris Whittington’s Backgammon-NN and GNU Backgammon 1.08.003.
MATCH FORMAT
• 1,000 separate matches
• Every match is first to 11 points
• 500 seat-swapped pairs
• Standard Crawford and post-Crawford rules
• Doubling cube enabled
• Gammons and backgammons count normally
MAXIMUM TEST SETTINGS
• Backgammon-NN: rollout checker play and rollout cube evaluation
• GNU Backgammon: 4-ply checker play and 4-ply cube decisions
• Both programs use the CPU for match calculations
• NVIDIA NVENC handles the YouTube broadcast
REAL DICE AND OFFICIATING
• Dice come from RANDOM.ORG atmospheric-noise randomness
• GNU Backgammon rolls the dice and acts as match director
• GNU Backgammon enforces legal moves, doubling-cube actions, Crawford rules and scoring
• Every completed match is saved as an SGF record
• Seat-swapped matches receive independent real-random dice streams
The live display includes animated checker movement, enlarged dice, doubling-cube value and ownership, pip counts, game score, match score and cumulative series standings.

Mark Young

Backgammon-NN vs GNU Backgammon — 679-Match Update

The 1,000-match test is now 679 matches complete.

Backgammon-NN leads GNU Backgammon 354–325.

That gives Backgammon-NN a 52.14% match-winning rate and a lead of 29 matches. In individual games, Backgammon-NN is also ahead 2,133–2,019.

Test conditions:

• 11-point matches with the Crawford rule
• Doubling cube enabled
• Random.org dice
• Seats swapped in paired matches
• Backgammon-NN using rollout play
• GNU Backgammon using 4-ply play with rollout cube evaluation

This is not a decisive result yet, and 321 matches remain. But Backgammon-NN continuing to lead a very strong GNU setup after 679 matches is impressive. Its checker play appears to be extremely strong and fast, and so far it is more than making up for its less sophisticated cube play.

Let’s see if the lead holds through the full 1,000 matches.

FINAL RESULT: BACKGAMMON-NN VS GNU BACKGAMMON

My 1,000-match test is now complete. These were 1,000 full matches to 11 points—not just 1,000 individual games.

TEST CONDITIONS

• Backgammon-NN using its rollout checker-play profile
• GNU Backgammon using strong practical 4-ply evaluation
• Rollout cube evaluation for Backgammon-NN
• Doubling cube enabled
• Crawford rule enabled
• 500 seat-swapped pairs
• Fresh RANDOM.ORG atmospheric-noise dice
• 6,188 individual games played

FINAL MATCH SCORE

Backgammon-NN: 514 match wins
GNU Backgammon: 486 match wins

Backgammon-NN win rate: 51.4%
GNU Backgammon win rate: 48.6%

INDIVIDUAL GAME WINS

Backgammon-NN: 3,138
GNU Backgammon: 3,050

TOTAL MATCH POINTS

Backgammon-NN: 8,927
GNU Backgammon: 8,495

Backgammon-NN swept both matches in 111 of the seat-swapped pairs. GNU swept 97 pairs, and the other 292 pairs were split.

This is an impressive result for Backgammon-NN. It finished 28 matches ahead of GNU, and it also led in individual games and total points.

Its speed was equally impressive. Backgammon-NN made 132,092 checker-play decisions, averaging about 0.81 seconds per decision on my computer. Its cube and take evaluations averaged about 0.40 seconds.

My impression remains that checker play is Backgammon-NN’s greatest strength. Its cube play may not be as sophisticated as its checker play, but the checker play was strong enough to more than hold its own against GNU at 4-ply.

I also want to be fair about the statistics. The 95% confidence range for Backgammon-NN was 48.3% to 54.5%. Because that range includes 50%, this test does not scientifically prove that Backgammon-NN is stronger than GNU. The most accurate conclusion is that they performed very closely, with Backgammon-NN holding the measured edge in this test.

This does not prove that Backgammon-NN is stronger than XG Roller++ or the strongest program in the world. But it does show that Backgammon-NN is a very serious engine: extremely fast, extremely strong in checker play, and competitive with a strong GNU Backgammon setting over a large controlled test.

Final result: Backgammon-NN 514, GNU Backgammon 486.

https://prodeo.actieforum.com/t1931-liv ... -11-points

[chrisw]

Updated the backgammon executable

Download here:
https://whittingtonchess.com/backgammon