Houdini 1.5a Autotune

mhurd · Post by **mhurd** » Sat Jan 15, 2011 11:03 pm

Being a curious sort, before I installed 1.5a into the Fritz GUI I re-checked autotune. So it appears that on my quad split depth 12 is the best setting.

Summary: Houdini 1.5
=======================================================
Split_Depth 10, Position 1: 2092 kN/s
Split_Depth 10, Position 2: 5805 kN/s
Split_Depth 10, Position 3: 5789 kN/s
Split_Depth 10, Position 4: 5855 kN/s
Split_Depth 10 - Average speed: 4885 kN/s
=======================================================
Split_Depth 12, Position 1: 5819 kN/s
Split_Depth 12, Position 2: 5804 kN/s
Split_Depth 12, Position 3: 5793 kN/s
Split_Depth 12, Position 4: 5905 kN/s
Split_Depth 12 - Average speed: 5831 kN/s
=======================================================
Split_Depth 14, Position 1: 5805 kN/s
Split_Depth 14, Position 2: 5873 kN/s
Split_Depth 14, Position 3: 5744 kN/s
Split_Depth 14, Position 4: 5810 kN/s
Split_Depth 14 - Average speed: 5808 kN/s
=======================================================
Split_Depth 16, Position 1: 5882 kN/s
Split_Depth 16, Position 2: 5856 kN/s
Split_Depth 16, Position 3: 5605 kN/s
Split_Depth 16, Position 4: 5770 kN/s
Split_Depth 16 - Average speed: 5778 kN/s
=======================================================
Split_Depth 18, Position 1: 5645 kN/s
Split_Depth 18, Position 2: 5751 kN/s
Split_Depth 18, Position 3: 5644 kN/s
Split_Depth 18, Position 4: 5553 kN/s
Split_Depth 18 - Average speed: 5648 kN/s
=======================================================
Average CPU time per thread: 624.507 s
CPU-corrected average speed: 5371 kN/s
=======================================================

Summary: Houdini 1.5a
=======================================================
Split_Depth 10, Position 1: 2092 kN/s
Split_Depth 10, Position 2: 5805 kN/s
Split_Depth 10, Position 3: 5789 kN/s
Split_Depth 10, Position 4: 5855 kN/s
Split_Depth 10 - Average speed: 4885 kN/s
=======================================================
Split_Depth 12, Position 1: 5819 kN/s
Split_Depth 12, Position 2: 5804 kN/s
Split_Depth 12, Position 3: 5793 kN/s
Split_Depth 12, Position 4: 5905 kN/s
Split_Depth 12 - Average speed: 5831 kN/s
=======================================================
Split_Depth 14, Position 1: 5805 kN/s
Split_Depth 14, Position 2: 5873 kN/s
Split_Depth 14, Position 3: 5744 kN/s
Split_Depth 14, Position 4: 5810 kN/s
Split_Depth 14 - Average speed: 5808 kN/s
=======================================================
Split_Depth 16, Position 1: 5882 kN/s
Split_Depth 16, Position 2: 5856 kN/s
Split_Depth 16, Position 3: 5605 kN/s
Split_Depth 16, Position 4: 5770 kN/s
Split_Depth 16 - Average speed: 5778 kN/s
=======================================================
Split_Depth 18, Position 1: 5645 kN/s
Split_Depth 18, Position 2: 5751 kN/s
Split_Depth 18, Position 3: 5644 kN/s
Split_Depth 18, Position 4: 5553 kN/s
Split_Depth 18 - Average speed: 5648 kN/s
=======================================================
Average CPU time per thread: 624.507 s
CPU-corrected average speed: 5371 kN/s
=======================================================

Mike

ernst · Post by **ernst** » Sun Jan 16, 2011 2:08 am

I wonder what position 1 is....

Houdini · Post by **Houdini** » Sun Jan 16, 2011 2:24 am

The 4 test positions are:

[D] r2qk2r/1b1n1ppp/1P1p1b2/pBpP4/P5Q1/2N5/1P3PPP/R3K1NR b KQkq - 0 1

[D] 5rk1/p2qr1b1/1Pn3pp/2pn4/2Np1p2/1N1P2P1/1P1BPP1P/R2QR1K1 b - - 0 1

[D] rqb2rk1/p4p1p/2pb1np1/3pn3/NP1B1N2/P3P3/2Q1BPPP/R4RK1 w - - 0 1

[D] r4rk1/p1qbnppp/1pn1p3/2ppP3/P2P4/2PB1N2/2P2PPP/R1BQ1RK1 w - - 6 1

Robert

ernst · Post by **ernst** » Sun Jan 16, 2011 2:35 am

I was ignorant about the autotune command in Houdini but gave it a try.

Code: Select all

Summary&#58;
=======================================================
Split_Depth 10, Position 1&#58; 7573 kN/s
Split_Depth 10, Position 2&#58; 7458 kN/s
Split_Depth 10, Position 3&#58; 7518 kN/s
Split_Depth 10, Position 4&#58; 7513 kN/s
Split_Depth 10 - Average speed&#58; 7515 kN/s
=======================================================
Split_Depth 12, Position 1&#58; 7347 kN/s
Split_Depth 12, Position 2&#58; 7460 kN/s
Split_Depth 12, Position 3&#58; 7114 kN/s
Split_Depth 12, Position 4&#58; 7360 kN/s
Split_Depth 12 - Average speed&#58; 7320 kN/s
=======================================================
Split_Depth 14, Position 1&#58; 7066 kN/s
Split_Depth 14, Position 2&#58; 7240 kN/s
Split_Depth 14, Position 3&#58; 7163 kN/s
Split_Depth 14, Position 4&#58; 6998 kN/s
Split_Depth 14 - Average speed&#58; 7117 kN/s
=======================================================
Split_Depth 16, Position 1&#58; 6945 kN/s
Split_Depth 16, Position 2&#58; 7234 kN/s
Split_Depth 16, Position 3&#58; 7256 kN/s
Split_Depth 16, Position 4&#58; 7468 kN/s
Split_Depth 16 - Average speed&#58; 7225 kN/s
=======================================================
Split_Depth 18, Position 1&#58; 7090 kN/s
Split_Depth 18, Position 2&#58; 7284 kN/s
Split_Depth 18, Position 3&#58; 6939 kN/s
Split_Depth 18, Position 4&#58; 6767 kN/s
Split_Depth 18 - Average speed&#58; 7020 kN/s
=======================================================
Average CPU time per thread&#58; 566.904 s
CPU-corrected average speed&#58; 7662 kN/s
=======================================================

On my Q6600 default is best. I wonder why Mike got such a low nps on position 1 and SD=10. What CPU is that Mike?

Houdini · Post by **Houdini** » Sun Jan 16, 2011 3:05 am

ernst wrote:I wonder why Mike got such a low nps on position 1 and SD=10.

Probably another process stealing CPU time from the engine.
It's better to restart the test to make sure.

ernest · Post by **ernest** » Sun Jan 16, 2011 3:28 am

mhurd wrote:So it appears that on my quad split depth 12 is the best setting.

Actually, if you run it again, with your Position 1 bug (at Split_Depth 10) corrected, you will see that your performance at Split_Depth 10, 12, 14, (and maybe even 16) is practically the same!...

(run again => different numbers anyway, because of MP non-reproducibility)

bob · Post by **bob** » Sun Jan 16, 2011 3:50 am

mhurd wrote:Being a curious sort, before I installed 1.5a into the Fritz GUI I re-checked autotune. So it appears that on my quad split depth 12 is the best setting.

Summary: Houdini 1.5
=======================================================
Split_Depth 10, Position 1: 2092 kN/s
Split_Depth 10, Position 2: 5805 kN/s
Split_Depth 10, Position 3: 5789 kN/s
Split_Depth 10, Position 4: 5855 kN/s
Split_Depth 10 - Average speed: 4885 kN/s
=======================================================
Split_Depth 12, Position 1: 5819 kN/s
Split_Depth 12, Position 2: 5804 kN/s
Split_Depth 12, Position 3: 5793 kN/s
Split_Depth 12, Position 4: 5905 kN/s
Split_Depth 12 - Average speed: 5831 kN/s
=======================================================
Split_Depth 14, Position 1: 5805 kN/s
Split_Depth 14, Position 2: 5873 kN/s
Split_Depth 14, Position 3: 5744 kN/s
Split_Depth 14, Position 4: 5810 kN/s
Split_Depth 14 - Average speed: 5808 kN/s
=======================================================
Split_Depth 16, Position 1: 5882 kN/s
Split_Depth 16, Position 2: 5856 kN/s
Split_Depth 16, Position 3: 5605 kN/s
Split_Depth 16, Position 4: 5770 kN/s
Split_Depth 16 - Average speed: 5778 kN/s
=======================================================
Split_Depth 18, Position 1: 5645 kN/s
Split_Depth 18, Position 2: 5751 kN/s
Split_Depth 18, Position 3: 5644 kN/s
Split_Depth 18, Position 4: 5553 kN/s
Split_Depth 18 - Average speed: 5648 kN/s
=======================================================
Average CPU time per thread: 624.507 s
CPU-corrected average speed: 5371 kN/s
=======================================================

Summary: Houdini 1.5a
=======================================================
Split_Depth 10, Position 1: 2092 kN/s
Split_Depth 10, Position 2: 5805 kN/s
Split_Depth 10, Position 3: 5789 kN/s
Split_Depth 10, Position 4: 5855 kN/s
Split_Depth 10 - Average speed: 4885 kN/s
=======================================================
Split_Depth 12, Position 1: 5819 kN/s
Split_Depth 12, Position 2: 5804 kN/s
Split_Depth 12, Position 3: 5793 kN/s
Split_Depth 12, Position 4: 5905 kN/s
Split_Depth 12 - Average speed: 5831 kN/s
=======================================================
Split_Depth 14, Position 1: 5805 kN/s
Split_Depth 14, Position 2: 5873 kN/s
Split_Depth 14, Position 3: 5744 kN/s
Split_Depth 14, Position 4: 5810 kN/s
Split_Depth 14 - Average speed: 5808 kN/s
=======================================================
Split_Depth 16, Position 1: 5882 kN/s
Split_Depth 16, Position 2: 5856 kN/s
Split_Depth 16, Position 3: 5605 kN/s
Split_Depth 16, Position 4: 5770 kN/s
Split_Depth 16 - Average speed: 5778 kN/s
=======================================================
Split_Depth 18, Position 1: 5645 kN/s
Split_Depth 18, Position 2: 5751 kN/s
Split_Depth 18, Position 3: 5644 kN/s
Split_Depth 18, Position 4: 5553 kN/s
Split_Depth 18 - Average speed: 5648 kN/s
=======================================================
Average CPU time per thread: 624.507 s
CPU-corrected average speed: 5371 kN/s
=======================================================

Mike

Split depth is about more than just raw NPS. Time to depth is also important. You drive NPS up by splitting deeper in the tree. But you also drive up overhead since ordering is worse near the tips. There is a two-pronged test you have to do, measure both NPS _and_ time to depth...

mhurd · Post by **mhurd** » Sun Jan 16, 2011 11:46 am

Robert wrote

Probably another process stealing CPU time from the engine.
It's better to restart the test to make sure.

I ran G boost to stop all non-essential services then ran the test again with nothing else running. This time split depth 18 comes out best. Other runs have had 10 come out best. Maybe due to non reproducibility I had better leave at default.

Summary:
=======================================================
Split_Depth 10, Position 1: 4867 kN/s
Split_Depth 10, Position 2: 5113 kN/s
Split_Depth 10, Position 3: 4755 kN/s
Split_Depth 10, Position 4: 4785 kN/s
Split_Depth 10 - Average speed: 4880 kN/s
=======================================================
Split_Depth 12, Position 1: 4922 kN/s
Split_Depth 12, Position 2: 4942 kN/s
Split_Depth 12, Position 3: 4626 kN/s
Split_Depth 12, Position 4: 4610 kN/s
Split_Depth 12 - Average speed: 4775 kN/s
=======================================================
Split_Depth 14, Position 1: 4726 kN/s
Split_Depth 14, Position 2: 5289 kN/s
Split_Depth 14, Position 3: 4470 kN/s
Split_Depth 14, Position 4: 4748 kN/s
Split_Depth 14 - Average speed: 4808 kN/s
=======================================================
Split_Depth 16, Position 1: 4699 kN/s
Split_Depth 16, Position 2: 4682 kN/s
Split_Depth 16, Position 3: 4650 kN/s
Split_Depth 16, Position 4: 4968 kN/s
Split_Depth 16 - Average speed: 4750 kN/s
=======================================================
Split_Depth 18, Position 1: 4735 kN/s
Split_Depth 18, Position 2: 5289 kN/s
Split_Depth 18, Position 3: 5365 kN/s
Split_Depth 18, Position 4: 4650 kN/s
Split_Depth 18 - Average speed: 5010 kN/s
=======================================================
Average CPU time per thread: 551.440 s
CPU-corrected average speed: 5271 kN/s
=======================================================

Ernst wrote:

On my Q6600 default is best. I wonder why Mike got such a low nps on position 1 and SD=10. What CPU is that Mike?

My CPU is an Intel Quad Q8200 @2.33 GHZ with 4 MB ram.

Mike

IQ · Post by IQ » Sun Jan 16, 2011 12:17 pm

I second Bobs remark. Using NPS to tune split depth is dubious, because different trees are searched and hash table interactions take place. A better test is time to depth or - as i prefer - time to solution, over a reasonable number of positions and using a large number of runs per position. Imagine larger subtrees, leading most likely to higher nps but also to more tree overlap and therefore to slower time to depth/solution. As your autotune code is already in place, maybe you can test this easily and give us feedback whether a time to depth/solution gives different results.

bob wrote:
mhurd wrote:Being a curious sort, before I installed 1.5a into the Fritz GUI I re-checked autotune. So it appears that on my quad split depth 12 is the best setting.

Summary: Houdini 1.5
=======================================================
Split_Depth 10, Position 1: 2092 kN/s
Split_Depth 10, Position 2: 5805 kN/s
Split_Depth 10, Position 3: 5789 kN/s
Split_Depth 10, Position 4: 5855 kN/s
Split_Depth 10 - Average speed: 4885 kN/s
=======================================================
Split_Depth 12, Position 1: 5819 kN/s
Split_Depth 12, Position 2: 5804 kN/s
Split_Depth 12, Position 3: 5793 kN/s
Split_Depth 12, Position 4: 5905 kN/s
Split_Depth 12 - Average speed: 5831 kN/s
=======================================================
Split_Depth 14, Position 1: 5805 kN/s
Split_Depth 14, Position 2: 5873 kN/s
Split_Depth 14, Position 3: 5744 kN/s
Split_Depth 14, Position 4: 5810 kN/s
Split_Depth 14 - Average speed: 5808 kN/s
=======================================================
Split_Depth 16, Position 1: 5882 kN/s
Split_Depth 16, Position 2: 5856 kN/s
Split_Depth 16, Position 3: 5605 kN/s
Split_Depth 16, Position 4: 5770 kN/s
Split_Depth 16 - Average speed: 5778 kN/s
=======================================================
Split_Depth 18, Position 1: 5645 kN/s
Split_Depth 18, Position 2: 5751 kN/s
Split_Depth 18, Position 3: 5644 kN/s
Split_Depth 18, Position 4: 5553 kN/s
Split_Depth 18 - Average speed: 5648 kN/s
=======================================================
Average CPU time per thread: 624.507 s
CPU-corrected average speed: 5371 kN/s
=======================================================

Summary: Houdini 1.5a
=======================================================
Split_Depth 10, Position 1: 2092 kN/s
Split_Depth 10, Position 2: 5805 kN/s
Split_Depth 10, Position 3: 5789 kN/s
Split_Depth 10, Position 4: 5855 kN/s
Split_Depth 10 - Average speed: 4885 kN/s
=======================================================
Split_Depth 12, Position 1: 5819 kN/s
Split_Depth 12, Position 2: 5804 kN/s
Split_Depth 12, Position 3: 5793 kN/s
Split_Depth 12, Position 4: 5905 kN/s
Split_Depth 12 - Average speed: 5831 kN/s
=======================================================
Split_Depth 14, Position 1: 5805 kN/s
Split_Depth 14, Position 2: 5873 kN/s
Split_Depth 14, Position 3: 5744 kN/s
Split_Depth 14, Position 4: 5810 kN/s
Split_Depth 14 - Average speed: 5808 kN/s
=======================================================
Split_Depth 16, Position 1: 5882 kN/s
Split_Depth 16, Position 2: 5856 kN/s
Split_Depth 16, Position 3: 5605 kN/s
Split_Depth 16, Position 4: 5770 kN/s
Split_Depth 16 - Average speed: 5778 kN/s
=======================================================
Split_Depth 18, Position 1: 5645 kN/s
Split_Depth 18, Position 2: 5751 kN/s
Split_Depth 18, Position 3: 5644 kN/s
Split_Depth 18, Position 4: 5553 kN/s
Split_Depth 18 - Average speed: 5648 kN/s
=======================================================
Average CPU time per thread: 624.507 s
CPU-corrected average speed: 5371 kN/s
=======================================================

Mike
Split depth is about more than just raw NPS. Time to depth is also important. You drive NPS up by splitting deeper in the tree. But you also drive up overhead since ordering is worse near the tips. There is a two-pronged test you have to do, measure both NPS _and_ time to depth...

Houdini 1.5a Autotune

Houdini 1.5a Autotune

Re: Houdini 1.5a Autotune

Re: Houdini 1.5a Autotune

Re: Houdini 1.5a Autotune

Re: Houdini 1.5a Autotune

Re: Houdini 1.5a Autotune

Re: Houdini 1.5a Autotune

Re: Houdini 1.5a Autotune

Re: Houdini 1.5a Autotune