On Feb 24, 10:16 am, "chipsc...@[EMAIL PROTECTED]
" <chipsc...@[EMAIL PROTECTED]
>
wrote:
> > Here we go again. You two seem to place
> > your own /convenience/ ahead of such things
> > as getting meaningful results. Try playing
> > the two programs against one another on
> > two separate machines, so there is **no
> > possibility** of what is discussed above.
>
> In fact, I place a high value on my convenience as I am not paid to
> evaluate GitClub, and anything I do is likely to be rough and ready.
> But despite your clever comments, a little thought shows that doing a
> true scientific evaluation in a home environment is extremely
> difficult. To really do things correctly, a laboratory environment,
> with a pair of identically configured machines, would be necessary.
> Additional knowledge about the workings of the two programs being
> competed would also be required --- answering the question, for
> instance, of whether one machine thinks on the other machine's time
> has nothing to do with whether the contest takes place on one physical
> box or two, though the effect on the results will definitely differ.
> Often, from the available user options and the available
> documentation, it is not at all clear what is going on, and detailed
> monitoring could be required to determine the answers.
>
> My goal has been simply to provide anecdotal information, not
> scientific test results for which my home environment is not well
> suited. Enough anecdotal information, however, may at least give some
> idea of the strength of GitClub. This is the difference between
> scientific method, which requires controlled conditions and sufficient
> data, and intelligence analysis, which recognizes that complete
> information can't be gathered and draws conclusions --- at times
> erroneous to be sure --- from what is available and what actually
> *can* be gathered.
>
> But constantly running into bugs has soured my enthusiasm. While I
> have been able to finish some games with GC, spending an hour of
> precious spare time only to find yet another frustrating bug is not
> what I wish to do any longer. However, if you are indeed willing, and
> are equipped for laboratory grade testing, and you will do it merely
> for a supply of Twinkies and Ding Dongs, I am sure something can be
> worked out.
Whenever a situation arises where, say, two
machines are unequal in speed, I would just
give Sanny's program the faster machine, as
it needs all the help it can get. ;>D
From Sanny's commentary it is unclear
which version of Rybka he uses, how many
times he went to get a drink of water vs.
Rybka "thinking" during the times he reports,
and so forth; in sum, for all we know he may
be giving Rybka beta 0.001 only one second
per move to think, and this could explain his
funky results.
I think someone else could do better---
MUCH better than this. Yes, there are big
problems, like the fact that he keeps on
modifying the program at random, weaker
and then stronger, back and forth like a
pendulum. With a "normal" tester, we
might at least be able to discern the
version of Rybka being tested, and maybe
the technical problems would tend to
balance out, and not all be reported as
GetClub "winning" because the opponent
"resigned".
Maybe my hopes are set to high; maybe
Sanny's style of reporting is up to the
standards which are acceptable to the
average Joe; maybe. But I don't think so!
Personally, I find lots of blunders when
going over such games with Fritz 5.32-- a
very outdated program which is far weaker
than Rybka. So it makes sense to give
the GetClub program a sizable handicap,
and see what (the real) Rybka can do.
Maybe I will do this myself-- eventually.
I have downloaded free versions of such
programs as "Rebel" and "Shredder", and
it might prove interesting if I can guess
the proper handicaps somehow. I have
the hardware... strewn about everywhere.
Seriously, I don't see how anyone can
play two engines on the same machine
and not realize there is a good chance
that this is mucking up the testing
process; if we wanted to muck it up, we
could just send Sanny the Twinkies-- he
is like Ace Ventura-- the best there is at
mucking up! Everybody will be "resigned"
and GetClub will win every test; the
Russians did not fix world chess; Sanny
did!
-- help bot
|
