Search found 63 matches

by jswaff
Mon Aug 24, 2020 4:25 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: interpreting cutechess-cli SPRT + LOS output
Replies: 3
Views: 1924

Re: interpreting cutechess-cli SPRT + LOS output

Hi Ajedrecista, First, thanks very much for the reply. Please see the below output from the cutechess-cli 'help'. Note that H1 uses the ELO0 value, where the null hypothesis H0 uses the ELO1 value. -sprt elo0=ELO0 elo1=ELO1 alpha=ALPHA beta=BETA Use a Sequential Probability Ratio Test as a terminati...
by jswaff
Sun Aug 23, 2020 9:44 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: interpreting cutechess-cli SPRT + LOS output
Replies: 3
Views: 1924

interpreting cutechess-cli SPRT + LOS output

I'm trying to introduce a little more rigor in my testing process but I'm having trouble interpreting the results of a match between two versions of my program. P1 = newer version , with small change expected to a nominal improvement at best. Perhaps a small loss. P2 = best known version A 20,000 ga...
by jswaff
Thu Aug 06, 2020 11:59 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: non overlapping error bars
Replies: 11
Views: 1687

Re: non overlapping error bars

In this case there is just a single opponent - it's version A vs version B. The "restart" issue you mentioned is interesting. This is my Java engine, and I do _not_restart it between games due to the "warm up overhead" of starting a JVM. My first thought was that there must be some sort of resource ...
by jswaff
Wed Aug 05, 2020 5:48 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: non overlapping error bars
Replies: 11
Views: 1687

Re: non overlapping error bars

That's a fair question, and I don't really have an answer other than I thought more positions == more variability == better? The point has already been made to remove the different positions as a variable, and I agree. I had made an assumption that the match score using the first 10k positions in th...
by jswaff
Wed Aug 05, 2020 11:39 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: non overlapping error bars
Replies: 11
Views: 1687

Re: non overlapping error bars

+15 ELO from one half of a set of positions to the other half seems like a lot, which is why I didn't consider it, but maybe it is that simple. As I replied to Brian, I'll just have to repeat the experiment on a single processor to see if the results hold.
by jswaff
Wed Aug 05, 2020 1:24 am
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: non overlapping error bars
Replies: 11
Views: 1687

Re: non overlapping error bars

Hi Brian! I've run fixed depth tests with results exactly .500 as expected. The tests I'm posting about are simply game in 5+0.25, so of course there will some variability - that's expected. I have recently run tests with the same version and got a result that was close to, but not exactly 0.500, wh...
by jswaff
Tue Aug 04, 2020 6:10 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: non overlapping error bars
Replies: 11
Views: 1687

non overlapping error bars

I have two tests running - version A of a program vs version B. Both tests are running the same builds of each version (the MD5 signatures of the binaries match), and are running with the same time control under identical conditions, with the exception that the set of starting positions are differen...
by jswaff
Tue Jul 21, 2020 5:24 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Cutechessing with Fruit/Gnuchess on a 16-core errors with "connection stalls"
Replies: 8
Views: 1868

Re: Cutechessing with Fruit/Gnuchess on a 16-core errors with "connection stalls"

My guess is the engine has crashed. (I've seen that error before and that was the case.) Try opening a window with the list of processes and see if one falls off when you see that message.

Running cutechess with the debug option may give some more info.
by jswaff
Sat Jul 11, 2020 11:47 am
Forum: Computer Chess Club: General Topics
Topic: In memory of Tony Hedlund
Replies: 8
Views: 789

Re: In memory of Tony Hedlund

I'm very sorry to hear that. Prayers for peace and comfort.
by jswaff
Wed Jun 24, 2020 4:19 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: Just made an account to say thank you!
Replies: 5
Views: 1931

Re: Just made an account to say thank you!

I agree 100%. I do think unit tests are an important tool but as you pointed out they alone aren't enough. I am on my third rewrite of Prophet. In the first version I didn't know much about programming, let alone the value of testing. By the time I started Prophet3 I did have an appreciation of it, ...