Gaviota TBs, compressor available (win32 for now)

[michiguel]

tbcp is available here
http://sites.google.com/site/gaviotache ... e/download
together with the engine that can read compressed TBs.
if you type

tbcp <folder>

it will compress all the TBs available. There are different compression schemes (http://sites.google.com/site/gaviotachessengine/schemes) but the recommended one is scheme 4.
By default, the compressor will use that. type tbcp -h and you will find

Code: Select all

Compression/decompression utility for Gaviota Table Bases.
It uses different free compression algorithms. More information at
http://sites.google.com/site/gaviotachessengine/

quick example: tbcp gtb
 compresses all the endgame table base files in the folder "gtb"
 with the default compression scheme 2, and no deletion of the source files

usage: tbcp [-OPTION]... [folder]
 -h        print this help
 -v        print version number and exit
 -L        display the license information
 -c        compress (default)
 -d        decompress
 -k        keep, do not delete input files (default)
 -r        remove input files after compression/decompression
 -V        verify encoding/decoding on the fly
 -q        quiet (no output except error messages)
 -i <file> work on the given individual file, rather than a folder
 -s   #    set compression scheme (1-4, default=4)
 -S        compression scheme help
 -n   #    operate on up to 3-, 4- o 5-piece tablebases (default=5)

Copyright (c) 2009-2010 Miguel A. Ballicora
There is NO WARRANTY of any kind 

Please make sure you have backups before testing this. In Linux works very well, but you never know. You can compress/decompress any file if you want.
Not only the TBs. Use swith -d to decompress. Yo can test only 4-pc TBs with the switch -n4

After this is tested to make sure everything is fine, I will release the probing code under the MIT license (X11 license, to be more accurate).
For now, there is only a version for win32, but I will upload linux 32, 64, and Win 64 as soon as I get them.

Miguel

[beachknight]

tbcp is available here
http://sites.google.com/site/gaviotache ... e/download
together with the engine that can read compressed TBs.
if you type

tbcp <folder>

it will compress all the TBs available. There are different compression schemes (http://sites.google.com/site/gaviotachessengine/schemes) but the recommended one is scheme 4.
By default, the compressor will use that. type tbcp -h and you will find

Code: Select all

Compression/decompression utility for Gaviota Table Bases.
It uses different free compression algorithms. More information at
http://sites.google.com/site/gaviotachessengine/

quick example: tbcp gtb
 compresses all the endgame table base files in the folder "gtb"
 with the default compression scheme 2, and no deletion of the source files

usage: tbcp [-OPTION]... [folder]
 -h        print this help
 -v        print version number and exit
 -L        display the license information
 -c        compress (default)
 -d        decompress
 -k        keep, do not delete input files (default)
 -r        remove input files after compression/decompression
 -V        verify encoding/decoding on the fly
 -q        quiet (no output except error messages)
 -i <file> work on the given individual file, rather than a folder
 -s   #    set compression scheme (1-4, default=4)
 -S        compression scheme help
 -n   #    operate on up to 3-, 4- o 5-piece tablebases (default=5)

Copyright (c) 2009-2010 Miguel A. Ballicora
There is NO WARRANTY of any kind 

Please make sure you have backups before testing this. In Linux works very well, but you never know. You can compress/decompress any file if you want.
Not only the TBs. Use swith -d to decompress. Yo can test only 4-pc TBs with the switch -n4

After this is tested to make sure everything is fine, I will release the probing code under the MIT license (X11 license, to be more accurate).
For now, there is only a version for win32, but I will upload linux 32, 64, and Win 64 as soon as I get them.

Miguel

Hi Miguel,

Finally I thought I'd have time to try tbcp. As I am on the cautious side, I ran it with -h.
First thing is it exits immediately after displaying the help text. No problem, I ran it with '-h >tbcphelp.txt. However, this is all I got:

Code: Select all

Compression/decompression utility for Gaviota Table Bases.
It uses different free compression algorithms. More information at
http://sites.google.com/site/gaviotachessengine/
Copyright (c) 2009-2010 Miguel A. Ballicora
There is NO WARRANTY of any kind

No problem again, I can use what you write here, still a minor flaw.

BTW, approximately how much time does it take to complete 5-man compression? And, can I have a different folder for generating the compressed files?

Best regards,

Hi Gabor,

Producing compressed G TBs took around 12 hours on my Q6600.

best,

[michiguel]

tbcp is available here
http://sites.google.com/site/gaviotache ... e/download
together with the engine that can read compressed TBs.
if you type

tbcp <folder>

it will compress all the TBs available. There are different compression schemes (http://sites.google.com/site/gaviotachessengine/schemes) but the recommended one is scheme 4.
By default, the compressor will use that. type tbcp -h and you will find

Code: Select all

Compression/decompression utility for Gaviota Table Bases.
It uses different free compression algorithms. More information at
http://sites.google.com/site/gaviotachessengine/

quick example: tbcp gtb
 compresses all the endgame table base files in the folder "gtb"
 with the default compression scheme 2, and no deletion of the source files

usage: tbcp [-OPTION]... [folder]
 -h        print this help
 -v        print version number and exit
 -L        display the license information
 -c        compress (default)
 -d        decompress
 -k        keep, do not delete input files (default)
 -r        remove input files after compression/decompression
 -V        verify encoding/decoding on the fly
 -q        quiet (no output except error messages)
 -i <file> work on the given individual file, rather than a folder
 -s   #    set compression scheme (1-4, default=4)
 -S        compression scheme help
 -n   #    operate on up to 3-, 4- o 5-piece tablebases (default=5)

Copyright (c) 2009-2010 Miguel A. Ballicora
There is NO WARRANTY of any kind 

Please make sure you have backups before testing this. In Linux works very well, but you never know. You can compress/decompress any file if you want.
Not only the TBs. Use swith -d to decompress. Yo can test only 4-pc TBs with the switch -n4

After this is tested to make sure everything is fine, I will release the probing code under the MIT license (X11 license, to be more accurate).
For now, there is only a version for win32, but I will upload linux 32, 64, and Win 64 as soon as I get them.

Miguel

Hi Miguel,

Finally I thought I'd have time to try tbcp. As I am on the cautious side, I ran it with -h.
First thing is it exits immediately after displaying the help text. No problem, I ran it with '-h >tbcphelp.txt. However, this is all I got:

Code: Select all

Compression/decompression utility for Gaviota Table Bases.
It uses different free compression algorithms. More information at
http://sites.google.com/site/gaviotachessengine/
Copyright (c) 2009-2010 Miguel A. Ballicora
There is NO WARRANTY of any kind

No problem again, I can use what you write here, still a minor flaw.

BTW, approximately how much time does it take to complete 5-man compression? And, can I have a different folder for generating the compressed files?

Best regards,

This is really weird. I just downloaded what it is in the web on a Windows PC and the output it is ok. Are you running it directly from the console?

Anyway, you can test it with only the 3-pc TBs (using flag -n3) to quickly make sure it is doing what it is supposed to do. Make sure you run it with the -s4 flag to force it to compress to the scheme that gives more compact size (I thought that uploaded a version that compressed to scheme 4 by default but I see now that I did not do it). You should see that files with the extension .cp4 are generated. So, to quickly test this type

tbcp -s4 -n3 gtb

and it will generate in the gtb folder (assuming the uncompressed TBs are there) kpk.gtb.cp4, kqk.gtb.cp4, krk.gtb.cp4 etc. almost immediately. Once you see that it does this, you can type.

tbcp -s4 -n5 gtb

Scheme 4 takes 3 hours in my AMD 2.4 GHz. The compression runs with only one core.
Other schemes are much faster but do not compress as much as #4.

Miguel
EDIT: Now that I read again your message, it looks like you are not running it from a terminal, but from some sort of 'RUN' menu. Yes, the program will quickly display and exit.

[michiguel]

tbcp is available here
http://sites.google.com/site/gaviotache ... e/download
together with the engine that can read compressed TBs.
if you type

tbcp <folder>

it will compress all the TBs available. There are different compression schemes (http://sites.google.com/site/gaviotachessengine/schemes) but the recommended one is scheme 4.
By default, the compressor will use that. type tbcp -h and you will find

Code: Select all

Compression/decompression utility for Gaviota Table Bases.
It uses different free compression algorithms. More information at
http://sites.google.com/site/gaviotachessengine/

quick example: tbcp gtb
 compresses all the endgame table base files in the folder "gtb"
 with the default compression scheme 2, and no deletion of the source files

usage: tbcp [-OPTION]... [folder]
 -h        print this help
 -v        print version number and exit
 -L        display the license information
 -c        compress (default)
 -d        decompress
 -k        keep, do not delete input files (default)
 -r        remove input files after compression/decompression
 -V        verify encoding/decoding on the fly
 -q        quiet (no output except error messages)
 -i <file> work on the given individual file, rather than a folder
 -s   #    set compression scheme (1-4, default=4)
 -S        compression scheme help
 -n   #    operate on up to 3-, 4- o 5-piece tablebases (default=5)

Copyright (c) 2009-2010 Miguel A. Ballicora
There is NO WARRANTY of any kind 

Please make sure you have backups before testing this. In Linux works very well, but you never know. You can compress/decompress any file if you want.
Not only the TBs. Use swith -d to decompress. Yo can test only 4-pc TBs with the switch -n4

After this is tested to make sure everything is fine, I will release the probing code under the MIT license (X11 license, to be more accurate).
For now, there is only a version for win32, but I will upload linux 32, 64, and Win 64 as soon as I get them.

Miguel

Hi Miguel,

Finally I thought I'd have time to try tbcp. As I am on the cautious side, I ran it with -h.
First thing is it exits immediately after displaying the help text. No problem, I ran it with '-h >tbcphelp.txt. However, this is all I got:

Code: Select all

Compression/decompression utility for Gaviota Table Bases.
It uses different free compression algorithms. More information at
http://sites.google.com/site/gaviotachessengine/
Copyright (c) 2009-2010 Miguel A. Ballicora
There is NO WARRANTY of any kind

No problem again, I can use what you write here, still a minor flaw.

BTW, approximately how much time does it take to complete 5-man compression? And, can I have a different folder for generating the compressed files?

Best regards,

Hi Gabor,

Producing compressed G TBs took around 12 hours on my Q6600.

best,

mmmhhh.... That is slower than my modest AMD 2.4 GHz, which takes ~3 hours. I wonder why that difference.

In any case, it is task to leave doing overnight.

Miguel

[michiguel]

Hi Miguel,

I had to abort and restart a compression. Unfortunately, it seems all files are compressed again. This with only 13 remaining, I used scheme 3, it used 4 hours up till now.

Also, if I compress using different methods I would like to separate results in different folders. Unfortunately Gaviota uses only the gtb folder, is there a way to circumvent this?

You can tell Gaviota about other folders in the ini file, this what I have (three different folders with three different compression schemes that I tested): Gaviota will use the one that is determined by "tb_decoding_scheme". Gaviota will search in each of the folders and find the corresponding files.

Miguel

Code: Select all

;---------------------------------
;
; Gaviota Tablebase parameters
;
;---------------------------------

use_tablebase       = on
tb_compression      = on
tb_decoding_scheme  = 4
tb_probing_depth    = 2
tb_pieces_used      = 5
tb_cache            = on    
tb_cache_size       = 64 M

tb_path_1           = "/media/bigdisk/gtb--1/"
tb_path_2           = "/media/bigdisk/gtb--4/"
tb_path_3           = "/media/bigdisk/gtb--3/"
tb_path_4           = ""

[michiguel]

Hi Miguel,

I have found no way to use uncompressed gtb's with version 0.74.41. I tried 0, 1, 9 and blank space for decompressing scheme, all resulted in an initialization error message. Version 0.74.12 does not recognize the new commands of the ini file and exits.

Try tb_compression = off in the ini file.
If that does not work, there is a bug I should fix.

If that works maybe I should set scheme 0 as "uncompressed" and remove the tb_compression option? That may be more straightforward?

]
Before you ask, I wanted to do some speed comparisons with the various schemes (I have now 2, 3 and 4 generated).

Good!

Miguel

[michiguel]

Try tb_compression = off in the ini file.
If that does not work, there is a bug I should fix.

If that works maybe I should set scheme 0 as "uncompressed" and remove the tb_compression option? That may be more straightforward?

Yes, the suggested method works.

As for the second part, I like the idea but in the past I often found that other people had different taste.

If you remain at the current method, maybe it would be cleare to suppress the decompression method when decompression is set to off. Currently the compression scheme is still displayed even if there is none, which is a bit confusing.

The problem is that I can't do that in a ini file that is edited by the user. I think I will remove tb_compression =on/off and let the user set tb_decoding_scheme = 0/1/2/3/4. As you say, I do not like to have options laying around that have no effect because they are overridden by another one.

Anyway, If you open winboard with gaviota and go to engine settings, you can set the options in a better way, with a GUI. However, the ini file is going to be ignored afterwards (saved.ini will be generated and will take over unless, of course, you delete it). I still think I can improve this.

Miguel