Wasp 2.5 testing

[matejst]

I see that Frank's beta testing of Wasp 2.5 is finished.

The results are marginally better, but I noticed that the results are not consistent against individual engines. My first impression (since I don't have many of the engines used) is that Wasp played worse against engines with a more complex eval, but I am not sure.

I had a look at the first three pgn and also watched the games at the TCEC. Perhaps a switch to magic bitboards would give a few plies more and solve the problems Wasp has in certain kind of positions. It becomes clear that speed, and not evaluation, is Wasp's biggest problem right now. A more efficient approach to hash use could be useful too.

Finally, having analyzed most of the games, I am a bit disappointed by the results at the TCEC. Wasp managed often to get to very promising positions, just to draw, or even lose. It easily could have achieve a point more, although its games against Fruit and Hannibal were difficult to follow.

[Frank Quisinsky]

Hi Boban,

Wasp won 3x more short games. Tactical skills are clearly improved (to the test version from Mai 2017). John is working now. I saw here different new problems in endgames. Playing style with many pieces on board is wonderful.

Most important for the moment is the time management. It seems Wasp have here problems with more cores (TCEC games).

Best
Frank

[matejst]

Dear Frank,

I thought time management wasn't such a problem at the TCEC, although Wasp had less time to finish its games than most of its opponents: usually, they got more depth faster, so I believed it was ok.

Wasp played a beautiful game against Ginko, it was very aggressive against Chiron, playing to the bare end, had a better ending against Texel... One cannot not have the impression that what's missing is just a little bit of something to achieve much impressive results.

Then, I can't believe that an engine that has a clearly better evaluation in complex positions suddenly doesn't know how to play the transition to endgame. There are two problems there: speed and endgame tablebases. I would like to see some other engines draw without TBs the way Wasp did against Ginko.

That's why I think that achieving some 2-4 plies more without changing the eval would give a lot, perhaps the 100 ELO John was writing about.

[jstanback]

I see that Frank's beta testing of Wasp 2.5 is finished.

The results are marginally better, but I noticed that the results are not consistent against individual engines. My first impression (since I don't have many of the engines used) is that Wasp played worse against engines with a more complex eval, but I am not sure.

I had a look at the first three pgn and also watched the games at the TCEC. Perhaps a switch to magic bitboards would give a few plies more and solve the problems Wasp has in certain kind of positions. It becomes clear that speed, and not evaluation, is Wasp's biggest problem right now. A more efficient approach to hash use could be useful too.

Finally, having analyzed most of the games, I am a bit disappointed by the results at the TCEC. Wasp managed often to get to very promising positions, just to draw, or even lose. It easily could have achieve a point more, although its games against Fruit and Hannibal were difficult to follow.

Hi Boban,

Switching to magic bitboards would probably give only a minor speedup, only giving a fraction of a ply extra depth. It's true that Wasp often does reasonably well in the opening and early middlegame but then blows it when transitioning to the endgame. It seems that Wasp's search depth is lower during this transition relative to other engines of similar strength and that may be part of the problem. Perhaps I have some issues with my hash table implementation. Wasp likely lacks some critical endgame knowledge that the stronger engines possess. Endgame TB's would help, but probably add only maybe 25 Elo. Right now I'm working to correct the time management problems that Wasp has when using many threads.

John

[Frank Quisinsky]

Hi John,

I have some concerns about changes in endgame. I never will missed the great style Wasp have in the beginning of the game. I like that so much, today Wasp is my personal number 1 in combination with the older Hakkapelliita version v3.

Example:
Hakkapeliitta have a great aggressive / tactical style in v3. The endgame was not good and the programmer changed here a lot. The result is TCEC v2 with a stronger endgame but the engine lost a lot of tactical power in the beginning. The result is more ELO but lesser fun. Some other examples in the past programmers made bigger changes in endgames and programs lost the great style in the middlegame!?
No idea why!

In my opinion the style of a chess program is much more important as ELO, if an engine have a basic playing strength from 2.800 - 2.900 (best chess players the World have).

Wasp is your baby and you know that I like your engine a lot but never the engine should lost the great style in the beginning.

So wonderful ... what you do ... you have now in version 2.5!
Without losing Elo points ... Wasp won 3x more games up to move number 49 in the last 4.000 test-run games. Have a look in the games and I am sure the programmer of Wasp, yourself ... is the biggest fan from the own engine.



...

A Wasp fan is loud thinking ...
Hope this is OK for you!

Best
Frank

Furthermore: My biggest wish for computer chess in the future is to have Wasp in a chess computer in combination with FEOBOS. Ruud Martin is working on an update, a newer modul (Phoenix website). Wasp and Hakkapeliitta in the modul from Ruud and the chess world is fully OK for myself and my computer chess pension. Because both programs are optimal for humans in self play. If I have that ... my word of honor ... no more messages by myself in the next 10 years in chess fora and all the other chess people are happy ... I too.

[Frank Quisinsky]

That's what I am thinking about computer chess.
The Elo hunting is more and more boring.

What can I do with an 3.500 Elo engine? Never I understand the moves if such a program is 700 Elo stronger as the human world champion. I can't understand many people here.

100 messages if TOP engine x or y is 30 Elo stronger as the preview version.

CONTRA productive!

I think in the middlegame of chess we can search the improvements. Best programs in the World are playing the endgame near to perfection. But after the opening book the playing strength is 800 or more Elo lesser.

Today are such program particular if in middlegames nice ideas can be produced.

My humble opinion!

Best
Frank

Fizbo is also an engines, plays wonderful chess. OK, Wasp is the topic. But Fizbo and Hakkapeliitta are great also.

[matejst]

Dear John,

I am not a programmer, so I can just rely on what I read on CPW -- that magic bitboards give a considerable speed-up. You certainly know it better.

Having spent a bit more time watching and analyzing Wasp's games, I found myself useless to help you in any way: I don't think endgames are Wasp problem. It was very difficult to find something better when analyzing Wasp's TCEC games, although I used Komodo 9 and The Baron, known for good endgame play. I don't use TB's though.

I don't even think that middlegame to endgame transition is -- from the point of view of evaluation -- a problem. Depth, perhaps. Anyway, it's really difficult to find what to improve, except for speed and king safety, although "king safety" is perhaps just another false impression, created by two or three games I had a look at (Rybka, Fruit, Hakkapelita?).

And just like Frank, I really enjoy playing parts of game against Wasp (1.02, and, to be honnest, against Zarkov 6.55 even more) and analyzing openings and games with it. While I can't be of any help, I still do hope that you will soon release a new version, although I like old versions too.

[matejst]

Dear Frank,

Trying new engines lately I discovered that I really didn't like the one using automated tuning.

[Frank Quisinsky]

Dear Boban,

I lost more and more fun with newer modern engines and automatic tuning techniques.

Can understand that programmers try to find new ways in chess programming.

But have a look in the results:
The endgames of TOP Engines are on Highend-Level and after opening books moves ... 10-15% of positions = 0,00 with a 3-fold in lines. Automatic tuning scrap.

Nothing to do with chess!
I dislike such things!

Looking in DNA of engines with FEOBOS and can see things ... better is to unkown all that.

I hope of more interesting developments in computer chess. New chess computers for an example. More as 20 years after the chess computer age we should start a new offensive.

We should try to give the younger generation new motivation with all the that we developed in the last 20 years. We can produced the dream chess computer in team work.

Thats much more interesting as 30 Elo more for engines with 3.300 Elo on one core.

Best
Frank

Please start a 100 games match ... Hakkapeliitta 3.0 vs. Wasp (if John released the new version) ... over christmas ... you will have a lot of fun. I downloaded TheBaron too ... not the time for looking at the moment.

[carldaman]

That's what I am thinking about computer chess.
The Elo hunting is more and more boring.

What can I do with an 3.500 Elo engine? Never I understand the moves if such a program is 700 Elo stronger as the human world champion. I can't understand many people here.

100 messages if TOP engine x or y is 30 Elo stronger as the preview version.

CONTRA productive!

I think in the middlegame of chess we can search the improvements. Best programs in the World are playing the endgame near to perfection. But after the opening book the playing strength is 800 or more Elo lesser.

Today are such program particular if in middlegames nice ideas can be produced.

My humble opinion!

Best
Frank

Fizbo is also an engines, plays wonderful chess. OK, Wasp is the topic. But Fizbo and Hakkapeliitta are great also.

Hi Frank,

I quite thoroughly agree with your stance on Elo gains, Frank. Elo should not be all. I've made my point several times that most engines have gained enough strength to be in a position to TRADE Elo for style!

Unfortunately the biggest hangup developers have is that they hate changes that lose Elo, although the overall style of play may be greatly improved that way. At the very least, they can release the engines in two flavors, standard and stylish, or otherwise allow the user to tweak the parameters. That way they could have their cake and eat it, too. Relatively few programmers think along these lines.

Anyway, I do hope that John Stanback releases Wasp 2.5, despite not gaining much Elo, as it plays a very attractive form of chess and that counts for a lot, too.

Cheers,
CL