TalkChess.com

[Larry Kaufman]

We have recently tried rating our matches and tournaments with both Ordo and Bayeselo to compare, with astonishing results. When we run gauntlets comparing two Komodo versions against a set of foreign opponents, the elo differences come out somewhat larger (on the order of 10% or so) with Bayeselo. But when we run direct matches between two versions, the elo differences are often vastly larger (on the order of 50%) with Bayeselo (!). A difference of 50% in scale is ridiculous, it means the two systems are VERY different. Since Ordo seems to correspond perfectly with normal elo differences for the given result, this seems to cast grave doubt on Bayeselo, unless our settings are the problem.
Our settings are mm 1 1 and scale 1, everything else default.
One clue: the highly inflated rating differences with Bayeselo seem to occur when the draw percentage is high. When it is in the thirties, everything looks pretty normal, but when it is over 50% we see these absurd elo differences with Bayeselo.
So, what is causing this behavior? Are the settings the problem? Does this mean that we are better off to use Ordo, if we want to see rating differences that bear some resemblances to the Elo rating system?