Bumblebee bug fix release

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

zd3nik
Posts: 193
Joined: Wed Mar 11, 2015 3:34 am
Location: United States

Bumblebee bug fix release

Post by zd3nik »

After taking a break for a while I came back to discover a couple bugs in bumblebee's evaluation routines. So I've created a point release:

https://github.com/zd3nik/BumblebeeRele ... .0.36898e1

The linux binaries in this release were built in a much more up-to-date environment than the last release. Built on linux mint 17.3 with gcc 4.8.4, libc version 2.19. The first release was built on fedora 22.

I've only run a couple short round robins to verify the patches. So I can't say exactly how much of an ELO increase this version has over the prior. But the increase is "noticeable" in the small tournaments I've run so far.

Coming soon (I hope): version 1.1
User avatar
Graham Banks
Posts: 41415
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: Bumblebee bug fix release

Post by Graham Banks »

zd3nik wrote:After taking a break for a while I came back to discover a couple bugs in bumblebee's evaluation routines. So I've created a point release:

https://github.com/zd3nik/BumblebeeRele ... .0.36898e1

The linux binaries in this release were built in a much more up-to-date environment than the last release. Built on linux mint 17.3 with gcc 4.8.4, libc version 2.19. The first release was built on fedora 22.

I've only run a couple short round robins to verify the patches. So I can't say exactly how much of an ELO increase this version has over the prior. But the increase is "noticeable" in the small tournaments I've run so far.

Coming soon (I hope): version 1.1
If the strength gain is noticeable, could I suggest naming this release as Bumblebee 1.01?
gbanksnz at gmail.com
zd3nik
Posts: 193
Joined: Wed Mar 11, 2015 3:34 am
Location: United States

Re: Bumblebee bug fix release

Post by zd3nik »

Graham Banks wrote: If the strength gain is noticeable, could I suggest naming this release as Bumblebee 1.01?
Not a fan of the git revision hash to distinguish versions? Can't say I blame you.

I'll consider changing the version scheme, but don't hold your breath. I would like to finish running it through the gauntlet first, to verify there's a significant change - I've been fooled by the first few games of testing before (almost every single time in fact). The round robin I'm currently running is set at a quick 1 min with half sec increment and only 3 participants, yet it'll take another day to finish. After that I'd like to run one with a longer time control.
User avatar
Graham Banks
Posts: 41415
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: Bumblebee bug fix release

Post by Graham Banks »

zd3nik wrote:
Graham Banks wrote: If the strength gain is noticeable, could I suggest naming this release as Bumblebee 1.01?
Not a fan of the git revision hash to distinguish versions? Can't say I blame you.

I'll consider changing the version scheme, but don't hold your breath. I would like to finish running it through the gauntlet first, to verify there's a significant change - I've been fooled by the first few games of testing before (almost every single time in fact). The round robin I'm currently running is set at a quick 1 min with half sec increment and only 3 participants, yet it'll take another day to finish. After that I'd like to run one with a longer time control.
Sounds good. :)
gbanksnz at gmail.com
zd3nik
Posts: 193
Joined: Wed Mar 11, 2015 3:34 am
Location: United States

Re: Bumblebee bug fix release

Post by zd3nik »

Graham Banks wrote:
zd3nik wrote:
Graham Banks wrote: If the strength gain is noticeable, could I suggest naming this release as Bumblebee 1.01?
Not a fan of the git revision hash to distinguish versions? Can't say I blame you.

I'll consider changing the version scheme, but don't hold your breath. I would like to finish running it through the gauntlet first, to verify there's a significant change - I've been fooled by the first few games of testing before (almost every single time in fact). The round robin I'm currently running is set at a quick 1 min with half sec increment and only 3 participants, yet it'll take another day to finish. After that I'd like to run one with a longer time control.
Sounds good. :)
I added a 4th contestant to the 1/0.5 bullet RR so it took a bit longer to finish. The result: the "buggy" version of bumblebee did better than the "fixed" version, suggesting 2 things:

1. my evaluation routine is so bad that it gets better by adding randomization to it.
2. the first few games of testing are always, ALWAYS, misleading.

I should have stuck with the original name I intended to use: bunglebee

I've started a 40 round tournament with the same 4 engines and 5 min +1 sec increment. I don't expect the result to be much different, if at all, but we'll see.

The good news (to me) is that one of the engines in this match up is bumblebee 1.1 (still in development) and it did better than both bumblebee 1.0 versions.
zd3nik
Posts: 193
Joined: Wed Mar 11, 2015 3:34 am
Location: United States

Re: Bumblebee bug fix release

Post by zd3nik »

Turns out I had screwed up the engine setup in Arena in the tournaments referenced above. I created the bug-fix engine config by duplicating one of my (failed) experimental versions of bumblebee and I only updated the name of the config - forgot to update which engine it actually loaded!

After correcting that error and re-running the bullet tournament the results are exactly what I would have expected. The bug fix version of bumblebee scores just a couple points higher than the buggy version. The blitz tournament will take some time to re-run, but I expect similar results.
Jamal Bubker
Posts: 326
Joined: Mon May 24, 2010 4:32 pm

Re: Bumblebee bug fix release

Post by Jamal Bubker »

Thanks Shawn for releasing this new Bumblebee version !
zd3nik
Posts: 193
Joined: Wed Mar 11, 2015 3:34 am
Location: United States

Re: Bumblebee bug fix release

Post by zd3nik »

Jamal Bubker wrote:Thanks Shawn for releasing this new Bumblebee version !
My pleasure. I hope the engine serves you well, whatever you're using it for. :)