Search or Evaluation?

You can discuss all aspects of programming and technical matters here.

Moderators: Harvey Williamson, Watchman

Uri Blass
Member
Posts: 82
Joined: Sun Aug 12, 2007 1:40 pm

Post by Uri Blass »

ricard60 wrote:I allways ask myself how a grandmaster find a solution to problems of chess in seconds that for me takes minutes or even do not find at all. I ask him the following question:

Master how many moves do you see in a position of a game of chess?
the master answer:

I allways see only one

me:but how can this be? how can you win a game of chess only seeing one move?

Master: is simple i always see the best move.

This answer got deep in my mind and from that moment until now i allways believe that chess is more a software problem than a hardware problem and that is why sometimes we see a machine with a big hardware lose to others with smaller ones.
So if we have the best evaluation function we will come up allways with the best move.

Now the problem will be how can we get that function?

Regards
Ricardo
These is simply nonsense.

GM's think in games and if a player see only one move he does not need to think.

The fact is that hardware is clearly important in chess.
You can win with slightly weaker hardware but even the best software
is going to lose against many amateurs if the hardware is 100 times slower.

Uri
User avatar
ricard60
Senior Member
Posts: 1285
Joined: Thu Aug 09, 2007 2:46 pm
Location: Puerto Ordaz

Post by ricard60 »

In chess computer tournaments i have seen the following:

1)A computer with an 8 bit processor beat a computer with a 16 bit processor (better software)

2)But i have never seen a computer with a better hardware and software lose to a computer with weaker hardware and software

Best
Ricardo
Uri Blass
Member
Posts: 82
Joined: Sun Aug 12, 2007 1:40 pm

Post by Uri Blass »

ricard60 wrote:In chess computer tournaments i have seen the following:

1)A computer with an 8 bit processor beat a computer with a 16 bit processor (better software)

2)But i have never seen a computer with a better hardware and software lose to a computer with weaker hardware and software

Best
Ricardo
Everything happens and even better hardware and better software can lose in a single game.

I remember that Junior lost in the world championship of 2003 against Ruy lopez in the first round.

Junior was better in hardware and software but had bad luck.

Later Junior won most games when Ruy Lopez lost almost all the games.

Uri
bob
Member
Posts: 29
Joined: Sat Oct 13, 2007 1:24 am

Post by bob »

ricard60 wrote:In chess computer tournaments i have seen the following:

1)A computer with an 8 bit processor beat a computer with a 16 bit processor (better software)

2)But i have never seen a computer with a better hardware and software lose to a computer with weaker hardware and software

Best
Ricardo
I've seen that. Fritz vs Deep blue prototype, Hong Kong, 1995 comes to mind. But there have been others. If you play a weak program/computer against a strong program/machine, the weak one will _still_ win some games...
User avatar
ricard60
Senior Member
Posts: 1285
Joined: Thu Aug 09, 2007 2:46 pm
Location: Puerto Ordaz

Post by ricard60 »

I saw in a match of 6 games between an 8 bit processor beat 4 times a 16 bit processor and the other 2 were a draw

I know for sure that 1) happens much more times than 2).


Ricardo
Uri Blass
Member
Posts: 82
Joined: Sun Aug 12, 2007 1:40 pm

Post by Uri Blass »

ricard60 wrote:I saw in a match of 6 games between an 8 bit processor beat 4 times a 16 bit processor and the other 2 were a draw

I know for sure that 1) happens much more times than 2).


Ricardo
In most tournaments there is no big difference between hardware and of course in these cases better software is important but if you play a tournament when there are big differences in the hardware you can see that hardware is very important.

Try to give rybka2.3.2a 32 bit 100:1 time handicapp against other programs and you are going to find that rybka is going to lose against many amateurs.

Note that it is possible to do it under arena and you only need to set rybka strength to 1% and give programs to play 120/40 time control with ponder off(rybka is going to use 1.2 minutes/40 moves against 120 minutes/40 moves of the opponents without pondering)

Uri
User avatar
ricard60
Senior Member
Posts: 1285
Joined: Thu Aug 09, 2007 2:46 pm
Location: Puerto Ordaz

Post by ricard60 »

As you said, mostly in tournaments there is no big difference in hardware so the winner is the best software, of course that does not mean that somebody can win a tournament with any hardware but when there are differences in hardware the software is what counts. I also face 10 games with hiarcs palm 9.6 to different engines in arena an it won 7 to 3. The engines of arena ran in an atholon 2.0 ghz xp the palm is a zire 72 running on a 312 mhz processor. This software also running on palm hiarcs win a match of 4 games to a grandmaster hitting a rating of 2809.If we go back some years there were no software runing on a processor of similar clock frecuency that could have this rating.So for me software had made giant steps to be each time better and that is why palm hiarcs made such an achievement. In a combination software hardware a give software 6 to 4.Maybe this combination would not be the world champion but will be in the top 5.
User avatar
Peter Grayson
Member
Posts: 208
Joined: Sat Aug 04, 2007 3:23 pm
Location: South Wales, UK

Re: Mexico - Zappa 5.5 v Rybka 4.5- Zappa wins $10,000

Post by Peter Grayson »

It must be in the positional area, it can't be otherwise.
So in the case of Rybka it seems to be the focus of chess knowledge that has changed where priority has been given to strong pawn positions, particularly advanced and supported pawns not only centrally but also on the wings.

In some games it becomes clear that while opponent goes off on some wild goose chase king attack, Rybka is only interested in advancing and supporting pawns. Sometimes that is its achilles heel but more often than not is the key to Rybka's superiority.

How many Grandmasters try to mate their opponent as initial goal? Most look for positional advantages that may accumulate to a pawn advantage or pawn position advantage that may be enough to win. What Rybka's inbuilt knowledge seems to have done is clarify the positional moves at some expense of tactical or combinative knowledge. However, in doing so it has improved the evaluation to dispense with less promising moves quicker and hence its displayed search depth usually shows deeper than opponents in same position on equal hardware.

The main question then is how has it harnessed this knowledge in a way that makes it so competitive compared to other engines?

There are very high evaluations for passed pawns and advanced pawns that are clearly in error when it comes to opposite colour bishop endings so there is strong evaluation emphasis on where a pawn is on the board and whether it is blocked by opponents pawn without fully analysing if the pawn can actually be promoted. This suggests a number of evaluation short cuts that speed up the calculations and work in most cases but as mentioned above can get caught out with grossly erroneous evaluations. The major and minor pieces also work to support the pawns, On many occasions Rybka can give the impression of strong tactical awareness but on closer analysis it came about because of the strength of or threat from pawns. In many games, Rybka will identify winning pawn advances several moves before opponent engine finds the danger, all too often too late.

In terms of Rybka's evaluation values there may be a misconception that the values should be interpreted by normal convention. However a while back, there was some very interesting analysis in Eric Hallsworth's Selective Search magazine that showed that Rybka's evaluation process seemed very different to other engines. There is an assunption too that the true evaluation is also not hidden from view!

PeterG
Uri Blass
Member
Posts: 82
Joined: Sun Aug 12, 2007 1:40 pm

Re: Mexico - Zappa 5.5 v Rybka 4.5- Zappa wins $10,000

Post by Uri Blass »

<snipped>
Peter Grayson wrote:
It must be in the positional area, it can't be otherwise.
So in the case of Rybka it seems to be the focus of chess knowledge that has changed where priority has been given to strong pawn positions, particularly advanced and supported pawns not only centrally but also on the wings.

PeterG
It is possible that better search is the secret of rybka and not better evaluation.

What is the evidence that rybka wins games thanks to better evaluation?

Uri
User avatar
Peter Grayson
Member
Posts: 208
Joined: Sat Aug 04, 2007 3:23 pm
Location: South Wales, UK

Re: Mexico - Zappa 5.5 v Rybka 4.5- Zappa wins $10,000

Post by Peter Grayson »

What is the evidence that rybka wins games thanks to better evaluation?
Well that's a tough one and unless someone who has seen the code can tell us I shouldn't think anyone could be absolutely sure. However if you observe how Rybka single engine works during a game between engines on two PC's some characteristics become apparent.

There is a thread in the games forum suggesting Hiarcs has a weakness with advanced and passed pawns but I suspect it may have more to do with Rybka's prioritisation for pawn strengths as I mentioned earlier.
It is possible that better search is the secret of rybka and not better evaluation
I'm not sure the two can be fully separated. It is possible to speed search by bad evaluation as well as good and here it boils down to the engine author's prioritisation and accuracy of evaluation for those positions.

Interesting too if there is additional information in evaluation that we do not see. For example, say the author sets up an evaluation in two parts.
1. Overall value based on evaluation criteria - the one we see and
2. A set of flags that indicate if the resultant position has isolated passed pawn, supported passed pawn, advanced pawn, backward pawn, etc. etc. and testing certain of those flags determines prioritisation. So at some stage a number of moves may have similar final numeric evaluation but selected move is based on set flags too.
Perhaps everone is using this method already?
regards,
PeterG
User avatar
Peter Grayson
Member
Posts: 208
Joined: Sat Aug 04, 2007 3:23 pm
Location: South Wales, UK

Re: Mexico - Zappa 5.5 v Rybka 4.5- Zappa wins $10,000

Post by Peter Grayson »

There is a thread in the games forum suggesting Hiarcs has a weakness with advanced and passed pawns
Sorry, the pawns weakness debate was in
http://hiarcs.net/forums/viewtopic.php?t=410

regards
PeterG
User avatar
xenophon
Member
Posts: 26
Joined: Tue Nov 13, 2007 10:36 pm

Post by xenophon »

bob wrote:
ricard60 wrote:In chess computer tournaments i have seen the following:

1)A computer with an 8 bit processor beat a computer with a 16 bit processor (better software)

2)But i have never seen a computer with a better hardware and software lose to a computer with weaker hardware and software

Best
Ricardo
I've seen that. Fritz vs Deep blue prototype
Bad example. IIRC, Deep Blue was having a severe hardware problem in that game.
bob wrote: , Hong Kong, 1995 comes to mind. But there have been others. If you play a weak program/computer against a strong program/machine, the weak one will _still_ win some games...
bob
Member
Posts: 29
Joined: Sat Oct 13, 2007 1:24 am

Post by bob »

xenophon wrote:
bob wrote:
ricard60 wrote:In chess computer tournaments i have seen the following:

1)A computer with an 8 bit processor beat a computer with a 16 bit processor (better software)

2)But i have never seen a computer with a better hardware and software lose to a computer with weaker hardware and software

Best
Ricardo
I've seen that. Fritz vs Deep blue prototype
Bad example. IIRC, Deep Blue was having a severe hardware problem in that game.
bob wrote: , Hong Kong, 1995 comes to mind. But there have been others. If you play a weak program/computer against a strong program/machine, the weak one will _still_ win some games...
That was the point. There are many different reasons why this will happen, from hardware/software failures. to just random luck. I would _never_ bet on a computer to beat another, no matter how big the rating difference, if the potential loss of bet is important.
Uri Blass
Member
Posts: 82
Joined: Sun Aug 12, 2007 1:40 pm

Post by Uri Blass »

bob wrote:
xenophon wrote:
bob wrote:
ricard60 wrote:In chess computer tournaments i have seen the following:

1)A computer with an 8 bit processor beat a computer with a 16 bit processor (better software)

2)But i have never seen a computer with a better hardware and software lose to a computer with weaker hardware and software

Best
Ricardo
I've seen that. Fritz vs Deep blue prototype
Bad example. IIRC, Deep Blue was having a severe hardware problem in that game.
bob wrote: , Hong Kong, 1995 comes to mind. But there have been others. If you play a weak program/computer against a strong program/machine, the weak one will _still_ win some games...
That was the point. There are many different reasons why this will happen, from hardware/software failures. to just random luck. I would _never_ bet on a computer to beat another, no matter how big the rating difference, if the potential loss of bet is important.
I think that the case of deep blue prototype against Fritz is a bad example because it is impossible to compare the software.

We only know that deep blue has better hardware but the programs cannot run on the same hardware.

If you want better examples the ssdf run games with hardware that is not equal and I remember for example that Junior5(p200) lost games against Fritz3(p90)
bob
Member
Posts: 29
Joined: Sat Oct 13, 2007 1:24 am

Post by bob »

Uri Blass wrote:
bob wrote:
xenophon wrote:
bob wrote:
ricard60 wrote:In chess computer tournaments i have seen the following:

1)A computer with an 8 bit processor beat a computer with a 16 bit processor (better software)

2)But i have never seen a computer with a better hardware and software lose to a computer with weaker hardware and software

Best
Ricardo
I've seen that. Fritz vs Deep blue prototype
Bad example. IIRC, Deep Blue was having a severe hardware problem in that game.
bob wrote: , Hong Kong, 1995 comes to mind. But there have been others. If you play a weak program/computer against a strong program/machine, the weak one will _still_ win some games...
That was the point. There are many different reasons why this will happen, from hardware/software failures. to just random luck. I would _never_ bet on a computer to beat another, no matter how big the rating difference, if the potential loss of bet is important.
I think that the case of deep blue prototype against Fritz is a bad example because it is impossible to compare the software.

We only know that deep blue has better hardware but the programs cannot run on the same hardware.

If you want better examples the ssdf run games with hardware that is not equal and I remember for example that Junior5(p200) lost games against Fritz3(p90)
You are missing the forest because of all the trees... :)

My point was that for a single game, _any_ outcome is possible and even likely. And that by playing enough games, where "enough" is not really that large a number, that any specific outcome for one game is almost guaranteed
Post Reply