creating an opening book

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

mvk
Posts: 589
Joined: Tue Jun 04, 2013 10:15 pm

Re: creating an opening book

Post by mvk »

Ferdy wrote:
mvk wrote:
Ferdy wrote:Building a book based on free strong engine is not cheating because everybody can take the engine and use it - playing field is even
How about non-free hardware?
Creating an opening book using an engine that is not yours, say a strong engine like SF to check every positions in the book is not cheating. Are we going to discuss the kind of HW used in creating such opening book and/or used by the engine in the analysis of the positions in the book?
To clarify, the question is what differentiates free programs from non-free programs in your judgement on this fairness matter, and how does that translate to hardware if it does. Given that Komodo (for example) is just as available for everyone as SF. And given that hardware is never free, but still a necessity to use even SF to make a book. It is not obvious how to reconcile that apparent contraction, hence the question.
[Account deleted]
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: creating an opening book

Post by Ferdy »

mvk wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:Building a book based on free strong engine is not cheating because everybody can take the engine and use it - playing field is even
How about non-free hardware?
Creating an opening book using an engine that is not yours, say a strong engine like SF to check every positions in the book is not cheating. Are we going to discuss the kind of HW used in creating such opening book and/or used by the engine in the analysis of the positions in the book?
To clarify, the question is what differentiates free programs from non-free programs in your judgement on this fairness matter, and how does that translate to hardware if it does. Given that Komodo (for example) is just as available for everyone as SF. And given that hardware is never free, but still a necessity to use even SF to make a book. It is not obvious how to reconcile that apparent contraction, hence the question.
The issue really is the use of engine that is not yours in building the book. The use of a free strong program say SF should not be considered cheating because everyone has access to it.
In competition the skill of the programmer should be tested as much as possible. If the author is able to implement SMP in his program that is fine, it is his skill, but using a single threaded engine with a high end cpu because he can afford it, is not fair when his opponents are also using a single threaded program but uses only a low end cpu.
Dann Corbit
Posts: 12541
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: creating an opening book

Post by Dann Corbit »

Ferdy wrote:
mvk wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:Building a book based on free strong engine is not cheating because everybody can take the engine and use it - playing field is even
How about non-free hardware?
Creating an opening book using an engine that is not yours, say a strong engine like SF to check every positions in the book is not cheating. Are we going to discuss the kind of HW used in creating such opening book and/or used by the engine in the analysis of the positions in the book?
To clarify, the question is what differentiates free programs from non-free programs in your judgement on this fairness matter, and how does that translate to hardware if it does. Given that Komodo (for example) is just as available for everyone as SF. And given that hardware is never free, but still a necessity to use even SF to make a book. It is not obvious how to reconcile that apparent contraction, hence the question.
The issue really is the use of engine that is not yours in building the book. The use of a free strong program say SF should not be considered cheating because everyone has access to it.
In competition the skill of the programmer should be tested as much as possible. If the author is able to implement SMP in his program that is fine, it is his skill, but using a single threaded engine with a high end cpu because he can afford it, is not fair when his opponents are also using a single threaded program but uses only a low end cpu.
And if someone has bought and paid for a chess program they should not be able to use it to analyze data for their opening book?

Joe uses Stockfish to analyze book positions. He paid $20,000 for his 48 CPU box. The playing field is not level, but he purchased the tool that tilts the field.

It was Joe who decided what positions to analyze. Fred, with his $800 PC could also produce the same data with stockfish. It will just take longer.

Has competition driven us utterly mad?
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: creating an opening book

Post by Ferdy »

Dann Corbit wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:Building a book based on free strong engine is not cheating because everybody can take the engine and use it - playing field is even
How about non-free hardware?
Creating an opening book using an engine that is not yours, say a strong engine like SF to check every positions in the book is not cheating. Are we going to discuss the kind of HW used in creating such opening book and/or used by the engine in the analysis of the positions in the book?
To clarify, the question is what differentiates free programs from non-free programs in your judgement on this fairness matter, and how does that translate to hardware if it does. Given that Komodo (for example) is just as available for everyone as SF. And given that hardware is never free, but still a necessity to use even SF to make a book. It is not obvious how to reconcile that apparent contraction, hence the question.
The issue really is the use of engine that is not yours in building the book. The use of a free strong program say SF should not be considered cheating because everyone has access to it.
In competition the skill of the programmer should be tested as much as possible. If the author is able to implement SMP in his program that is fine, it is his skill, but using a single threaded engine with a high end cpu because he can afford it, is not fair when his opponents are also using a single threaded program but uses only a low end cpu.
And if someone has bought and paid for a chess program they should not be able to use it to analyse data for their opening book?
This is unfair if that program someone has purchased is stronger than the strongest free program available. The author is no longer competing at even level. You can still use that engine but the book created with the help of that engine is questionable when entered in the tournament or competition unless of course the other participants are also using that purchased engine then we are back in even level.
User avatar
mhull
Posts: 13447
Joined: Wed Mar 08, 2006 9:02 pm
Location: Dallas, Texas
Full name: Matthew Hull

Re: creating an opening book

Post by mhull »

Ferdy wrote:
Dann Corbit wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:Building a book based on free strong engine is not cheating because everybody can take the engine and use it - playing field is even
How about non-free hardware?
Creating an opening book using an engine that is not yours, say a strong engine like SF to check every positions in the book is not cheating. Are we going to discuss the kind of HW used in creating such opening book and/or used by the engine in the analysis of the positions in the book?
To clarify, the question is what differentiates free programs from non-free programs in your judgement on this fairness matter, and how does that translate to hardware if it does. Given that Komodo (for example) is just as available for everyone as SF. And given that hardware is never free, but still a necessity to use even SF to make a book. It is not obvious how to reconcile that apparent contraction, hence the question.
The issue really is the use of engine that is not yours in building the book. The use of a free strong program say SF should not be considered cheating because everyone has access to it.
In competition the skill of the programmer should be tested as much as possible. If the author is able to implement SMP in his program that is fine, it is his skill, but using a single threaded engine with a high end cpu because he can afford it, is not fair when his opponents are also using a single threaded program but uses only a low end cpu.
And if someone has bought and paid for a chess program they should not be able to use it to analyse data for their opening book?
This is unfair if that program someone has purchased is stronger than the strongest free program available. The author is no longer competing at even level....
How is that different from purchasing the assistance of a grandmaster? Is that also unfair?
Matthew Hull
Dann Corbit
Posts: 12541
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: creating an opening book

Post by Dann Corbit »

Ferdy wrote:
Dann Corbit wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:Building a book based on free strong engine is not cheating because everybody can take the engine and use it - playing field is even
How about non-free hardware?
Creating an opening book using an engine that is not yours, say a strong engine like SF to check every positions in the book is not cheating. Are we going to discuss the kind of HW used in creating such opening book and/or used by the engine in the analysis of the positions in the book?
To clarify, the question is what differentiates free programs from non-free programs in your judgement on this fairness matter, and how does that translate to hardware if it does. Given that Komodo (for example) is just as available for everyone as SF. And given that hardware is never free, but still a necessity to use even SF to make a book. It is not obvious how to reconcile that apparent contraction, hence the question.
The issue really is the use of engine that is not yours in building the book. The use of a free strong program say SF should not be considered cheating because everyone has access to it.
In competition the skill of the programmer should be tested as much as possible. If the author is able to implement SMP in his program that is fine, it is his skill, but using a single threaded engine with a high end cpu because he can afford it, is not fair when his opponents are also using a single threaded program but uses only a low end cpu.
And if someone has bought and paid for a chess program they should not be able to use it to analyse data for their opening book?
This is unfair if that program someone has purchased is stronger than the strongest free program available. The author is no longer competing at even level. You can still use that engine but the book created with the help of that engine is questionable when entered in the tournament or competition unless of course the other participants are also using that purchased engine then we are back in even level.
He can also purchase better compilers than you can. He can get better hardware for development. Perhaps he can even hire a team of programmers to help him on his project. I think that all of these are fully acceptable, are they not?

Joe was born with an IQ of 173, while Peter has an IQ of 120. How shall we handicap Joe so that the competition is fair?

Raymond was born into a family with billions. Sandra was born into a family that lived in a hut with no utilities. Is Sandra competing on an even level?

All of these are much more unfair than using a tool, purchased with the money earned by the consumer, to perform their own analysis.

IMO-YMMV
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: creating an opening book

Post by Ferdy »

mhull wrote:
Ferdy wrote:
Dann Corbit wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:Building a book based on free strong engine is not cheating because everybody can take the engine and use it - playing field is even
How about non-free hardware?
Creating an opening book using an engine that is not yours, say a strong engine like SF to check every positions in the book is not cheating. Are we going to discuss the kind of HW used in creating such opening book and/or used by the engine in the analysis of the positions in the book?
To clarify, the question is what differentiates free programs from non-free programs in your judgement on this fairness matter, and how does that translate to hardware if it does. Given that Komodo (for example) is just as available for everyone as SF. And given that hardware is never free, but still a necessity to use even SF to make a book. It is not obvious how to reconcile that apparent contraction, hence the question.
The issue really is the use of engine that is not yours in building the book. The use of a free strong program say SF should not be considered cheating because everyone has access to it.
In competition the skill of the programmer should be tested as much as possible. If the author is able to implement SMP in his program that is fine, it is his skill, but using a single threaded engine with a high end cpu because he can afford it, is not fair when his opponents are also using a single threaded program but uses only a low end cpu.
And if someone has bought and paid for a chess program they should not be able to use it to analyse data for their opening book?
This is unfair if that program someone has purchased is stronger than the strongest free program available. The author is no longer competing at even level. You can still use that engine but the book created with the help of that engine is questionable when entered in the tournament or competition unless of course the other participants are also using that purchased engine then we are back in even level.
How is that different from purchasing the assistance of a grandmaster? Is that also unfair?
It is fine if the GM is using SF 6 for example or it uses its knowledge and experience alone in creating the opening book.
Dann Corbit
Posts: 12541
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: creating an opening book

Post by Dann Corbit »

Ferdy wrote:
mhull wrote:
Ferdy wrote:
Dann Corbit wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:Building a book based on free strong engine is not cheating because everybody can take the engine and use it - playing field is even
How about non-free hardware?
Creating an opening book using an engine that is not yours, say a strong engine like SF to check every positions in the book is not cheating. Are we going to discuss the kind of HW used in creating such opening book and/or used by the engine in the analysis of the positions in the book?
To clarify, the question is what differentiates free programs from non-free programs in your judgement on this fairness matter, and how does that translate to hardware if it does. Given that Komodo (for example) is just as available for everyone as SF. And given that hardware is never free, but still a necessity to use even SF to make a book. It is not obvious how to reconcile that apparent contraction, hence the question.
The issue really is the use of engine that is not yours in building the book. The use of a free strong program say SF should not be considered cheating because everyone has access to it.
In competition the skill of the programmer should be tested as much as possible. If the author is able to implement SMP in his program that is fine, it is his skill, but using a single threaded engine with a high end cpu because he can afford it, is not fair when his opponents are also using a single threaded program but uses only a low end cpu.
And if someone has bought and paid for a chess program they should not be able to use it to analyse data for their opening book?
This is unfair if that program someone has purchased is stronger than the strongest free program available. The author is no longer competing at even level. You can still use that engine but the book created with the help of that engine is questionable when entered in the tournament or competition unless of course the other participants are also using that purchased engine then we are back in even level.
How is that different from purchasing the assistance of a grandmaster? Is that also unfair?
It is fine if the GM is using SF 6 for example or it uses its knowledge and experience alone in creating the opening book.
So GMs who use programs that they have purchased are doing something unfair?

If they buy an operating system instead of using a free variant of Linux is that also unfair?

If they use the super-expensive version of Visual Studio that comes with case tools instead of GCC and GDB is that also unfair?

If a person uses a tool, which they have bought with money that they have earned, it seems ludicrous to me to call that unfair.

Get a job.
Earn some money.
Buy the tool.
Voila, even playing field.
A high end chess program is $100 or less. (Komodo $99.97):
http://komodochess.com/Komodo9.1-46a.htm
Houdini is 60 euros and 21% VAT if you live in Europe.

These programs cost a lot less than the computer you use to program.
Besides which, there are free programs of about the same strength.
Bleeding edge Stockfish compiles are probably the strongest chess engine.
And if you want to analyze better (perhaps to examine strong alternatives), cough up a C-Note and buy a tool to do it.

IMO-YMMV
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: creating an opening book

Post by Ferdy »

Dann Corbit wrote:
Ferdy wrote:
Dann Corbit wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:Building a book based on free strong engine is not cheating because everybody can take the engine and use it - playing field is even
How about non-free hardware?
Creating an opening book using an engine that is not yours, say a strong engine like SF to check every positions in the book is not cheating. Are we going to discuss the kind of HW used in creating such opening book and/or used by the engine in the analysis of the positions in the book?
To clarify, the question is what differentiates free programs from non-free programs in your judgement on this fairness matter, and how does that translate to hardware if it does. Given that Komodo (for example) is just as available for everyone as SF. And given that hardware is never free, but still a necessity to use even SF to make a book. It is not obvious how to reconcile that apparent contraction, hence the question.
The issue really is the use of engine that is not yours in building the book. The use of a free strong program say SF should not be considered cheating because everyone has access to it.
In competition the skill of the programmer should be tested as much as possible. If the author is able to implement SMP in his program that is fine, it is his skill, but using a single threaded engine with a high end cpu because he can afford it, is not fair when his opponents are also using a single threaded program but uses only a low end cpu.
And if someone has bought and paid for a chess program they should not be able to use it to analyse data for their opening book?
This is unfair if that program someone has purchased is stronger than the strongest free program available. The author is no longer competing at even level. You can still use that engine but the book created with the help of that engine is questionable when entered in the tournament or competition unless of course the other participants are also using that purchased engine then we are back in even level.
He can also purchase better compilers than you can.
GCC is free and it is good.
Dann Corbit wrote:He can get better hardware for development.
There are programmers that performs well even with modest hardware for development.
Dann Corbit wrote:Perhaps he can even hire a team of programmers to help him on his project.
This one is fine because he will learn from the programmers that he hired. If he is not learning and just hired skillfull programmers to make his program strong then that is not good.
Dann Corbit wrote: Joe was born with an IQ of 173, while Peter has an IQ of 120. How shall we handicap Joe so that the competition is fair?
There is no need to handicap, let them create an engine and compete in a tournament with opening book created by them or their engine or SF6.
Dann Corbit wrote: Raymond was born into a family with billions. Sandra was born into a family that lived in a hut with no utilities. Is Sandra competing on an even level?
Not all rich programmers are good and there are poor programmers that are really good.
Raymond being rich did not study much because of the wealth he got from his parents. While Sandra trying to improve her situation tried harder to get herself in school, she got interested in programming because of automation, and she happened to browse the CCC and got interested in computer chess programming, she successfully created a program and ready to participate but she could not find Raymond. Is Raymond will be able to compete or not?
Dann Corbit
Posts: 12541
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: creating an opening book

Post by Dann Corbit »

Ferdy wrote:
Dann Corbit wrote:
Ferdy wrote:
Dann Corbit wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:
mvk wrote:
Ferdy wrote:Building a book based on free strong engine is not cheating because everybody can take the engine and use it - playing field is even
How about non-free hardware?
Creating an opening book using an engine that is not yours, say a strong engine like SF to check every positions in the book is not cheating. Are we going to discuss the kind of HW used in creating such opening book and/or used by the engine in the analysis of the positions in the book?
To clarify, the question is what differentiates free programs from non-free programs in your judgement on this fairness matter, and how does that translate to hardware if it does. Given that Komodo (for example) is just as available for everyone as SF. And given that hardware is never free, but still a necessity to use even SF to make a book. It is not obvious how to reconcile that apparent contraction, hence the question.
The issue really is the use of engine that is not yours in building the book. The use of a free strong program say SF should not be considered cheating because everyone has access to it.
In competition the skill of the programmer should be tested as much as possible. If the author is able to implement SMP in his program that is fine, it is his skill, but using a single threaded engine with a high end cpu because he can afford it, is not fair when his opponents are also using a single threaded program but uses only a low end cpu.
And if someone has bought and paid for a chess program they should not be able to use it to analyse data for their opening book?
This is unfair if that program someone has purchased is stronger than the strongest free program available. The author is no longer competing at even level. You can still use that engine but the book created with the help of that engine is questionable when entered in the tournament or competition unless of course the other participants are also using that purchased engine then we are back in even level.
He can also purchase better compilers than you can.
GCC is free and it is good.
Dann Corbit wrote:He can get better hardware for development.
There are programmers that performs well even with modest hardware for development.
Dann Corbit wrote:Perhaps he can even hire a team of programmers to help him on his project.
This one is fine because he will learn from the programmers that he hired. If he is not learning and just hired skillfull programmers to make his program strong then that is not good.
Dann Corbit wrote: Joe was born with an IQ of 173, while Peter has an IQ of 120. How shall we handicap Joe so that the competition is fair?
There is no need to handicap, let them create an engine and compete in a tournament with opening book created by them or their engine or SF6.
Dann Corbit wrote: Raymond was born into a family with billions. Sandra was born into a family that lived in a hut with no utilities. Is Sandra competing on an even level?
Not all rich programmers are good and there are poor programmers that are really good.
Raymond being rich did not study much because of the wealth he got from his parents. While Sandra trying to improve her situation tried harder to get herself in school, she got interested in programming because of automation, and she happened to browse the CCC and got interested in computer chess programming, she successfully created a program and ready to participate but she could not find Raymond. Is Raymond will be able to compete or not?
Sure, Raymond can compete. He simply hires a team of programmers and gives them all a million dollar budget.
He is able to leverage his given advantages.
Or he might go to Berkeley and get a PhD in computer science, since he can afford it, even if he is out of state.
Of course, Raymond might be stupid.
I don't know him personally.

There was a book with a story in it about "handicappers". The strongest and swiftest athletes would have springs attached so that they were equal to the weaker competitors.

Maybe we should do that. Anybody got a box of springs?
;-)