[AGA Contest] comments from Karen

[Erik Olson <erik@thekrib.com>]

Not sure why this bounced, maybe because Karen's subscribed to the digest 
version.

---------- Forwarded message ----------
From: "Karen Randall" <krandall@rdrcpa.biz>
To: <aga-contest@thekrib.com>
Subject: Judging
Date: Mon, 24 May 2004 12:11:05 -0400

OK, I've been trying to stay out of this for a number of reasons, but...

Phil wrote:

>>   I agree that the "place" aquascapes need to be determined by the judges
rankings.  However, I feel that such rankings should be based on the raw
score of each tank, from each judge.  <<

Unless I am misunderstanding you, as far as I know, this is what ALL the
judges have done in the past.  You might argue how WELL they've done this,
but I believe they have all done it to the best of their ability.

>> Furthermore, the very nature of the scoring method creates bias and
allows
for a mediocre aquascape to rank higher than an "inspired" design.  Each
aquarium will have something different going in its favor.  Sometimes the
weight of the category within the judging criteria creates an imbalance in
the actual "value" of the design.<<

It would be hard for this to happen considering the current weighting.

"Overall impression"  has 35 points
"Composition" has 30 points
"Selection and use of materials" (which factors into the above two scores)
has 20 points
That leaves only the 15 points for "Viability" that could be awarded in full
to the "uninspired" tank which was an otherwise solid entry in terms of
general aquarium keeping.

>>     In my very opinionated opinion I feel that any judge that does not
utilize the entire reach of total scores is not doing his/her job.  If a
judge doesn't score even the worst aquascape at anything less than 80
something is wrong.  While the qualities of design are increasing it's
highly unlikely that every one of them will be of "B" or better.
Furthermore, any judge who doesn't make a serious effort to keep personal
taste from influencing his/her decisions is doing everyone a major
dis-service and shouldn't be allowed to judge. <<

I agree with you to some extent.  Certainly not all tanks entered deserve a
"B or better".  OTOH, when you get down into the range of the clearly
"also-ran" tanks, you do have to consider that scores and comments become av
ailable to the public.  There is no reason to hurt people's feelings with a
score of 10, even if, as a judge, that's what you'd like to give the tank.
I think I use as full a point range as any of the judges and more than some.
And I'm quite sure I've never awarded a total score less than 40.  I wasn't
able to find my score sheets from the past AGA contests, but I looked at my
ADA score sheets from the last couple of years. (ADA sends you a book of
photos to work from, so it is easy to keep track of the scores for future
reference)  The categories for judging are not quite the same, but it is
still a 100 point scale.  My scores range from a high of 97, and a low of
43.

But I can honestly say that I do not worry about "ranking" tanks that I put
lower than about 6th place.  There can easily be more than one tank with the
same raw score below that rank, and I'll leave it that way. (If you look at
the internal category scores, there could be quite a bit of difference, but
the total score comes out the same.  Rarely have tanks that I've placed
lower than 5th or 6th made it up into top contention, in either contest.
And if one has, it's a controversial tank, as we saw last year.  In these
instances, as Erik has already explained, we all talk over the decision.  I
could respectfully agree to disagree with the majority, while still
supporting the use of a specific tank for best of show based on the opinions
of the other judges.

>>    I'm sorry, I knew what I was saying but it comes out less than what I
actually meant.  I was actually speaking of ties within scores one judge has
given.  For example, five judges score twenty tanks individually (not trying
to rank them 1-5) there is a pretty good chance that there will be ties
within the scores given by each judge.

Judge A  100, 96, 96, 84, 37 etc..
Judge B   99,  96, 94, 87, 40 etc...
Judge C  100, 94, 97, 88, 38 etc...<<

Erik has consistently asked for us NOT to leave ties among the highest
placed tanks.  This is pretty easy to do.  When you make your first pass
while judging, you are not comparing the tanks with each other  there are
simply too many of them, and with numerous photos of each, sometimes of
tremendously varying quality (and remember, as much as possible, we do NOT
judge photography).  You are comparing each tank with a subjective "ideal"
in you head for each category of judging, and assigning a score within that
point range. (from 0-15 or 0-35 depending on the category)  Then I go back
and do a "reality check".  Do my scores reflect my real "feelings" of the
order in which tanks should be placed.  If not, (and that sometimes happens)
I go back and adjust.  It's hard to get through one of the bigger categories
in one sitting, so sometimes you are just judging a little differently when
you sit down to score the second half of the class.

When it gets down to 2 or 3 tanks tied for the same overall score, and you
look at them side by side, you can usually justify bumping one category up
or down by a point to break the tie.  Even then, sometimes it is SO close
that even this is hard.  Last year, I had two tanks tied for first place in
one category.  One was a good, solid, and also beautiful tank.  The other
was really striking, but I did not think that tank would "wear well". (I
later found out that was true<g>)  The overall scores were exactly the same,
but the subscores were quite different.  I flip-flopped those scores a
number of times before I could decide in my own mind, which one to place
first.  But the result was CLEARLY subjective.  It was my job to place one
first and one second.

Please keep that in mind, Phil, as much as you'd like this NOT to be
subjective, it is, and can never be anything else.

Have you ever done thing where you have four color samples that are fairly
close together, that are matched up with 4 other color samples?  Just like
having perfect pitch, the number of people who can accurately determine
whether two similar shades of a color are EXACTLY the same unless they are
touching is EXTREMELY small.  In fact, a person's ability to judge color
varies over the course of the day based on chemical changes in your brain!
(probably more than you wanted to know!<g>)  When I think about it, there is
likely to be variation in scoring just based on the gender of the judges.
MANY men, not just those who are color blind, actually perceive a smaller
range of color than most women do.  It's just the way their eyes are made.

>> By adding a ranking mechanism into the equation we're adding a greater
element of subjectivity
which could potentially be avoided.<<

I strongly disagree with the idea that there either can be or SHOULD be any
pretense of objectivity  in what amounts to a "beauty contest".  Judging an
aquascape IS subjective.  I think that is one reason why Erik's idea of
letting each judge award their own "honorable mention" is brilliant.  It
gives entrants an incentive to be bold and innovative... to try something
that some judges might not like, but others love. (like an all-red tank ;-)

If judging an aquascape becomes a totally objective exercise, we will only
have ONE standard criteria, and there will cease to be innovation or
creativity.  When the MOST points awarded are given for "overall
impression", there is certainly going to be variation among the judges as to
which tank "impresses" the most.  I can certainly appreciate that Picasso
was a great artist.  But if I had to choose between a Picasso and a Degas,
I'd choose the Degas.  I'm sure there are others who disagree with me
entirely.  They are entitled to their opinion; neither opinion is "wrong" or
"right.

The ONLY objectivity that I think judges are duty bound to keep is that fact
that once you've judged some of these contests, particularly if you've
actually seen a number of the tanks in person, as I have over the years, it
is very important to be able to completely divorce yourself from "who" did
the tank, and judge, to the best of your ability, as if you are seeing each
entry for the first time.  But I think I do this fairly, and I suspect that
the other judges who have done this a number of times are honorable enough
to do the same.  Let's face it, even those who haven't judged, are probably
going to recognize a "Jeff Senske" paludarium when they see it in the
contest.  I happen to think that's great when people are actually able to
develop their own, distinctive "style".  That's exactly what makes a Picasso
so different from a Degas!!

Karen


  ------------------
  To unsubscribe from this list, e-mail majordomo@thekrib.com
  with "unsubscribe aga-contest" in the body of the message.
  To subscribe to the digest version, add "subscribe aga-contest-digest"
  in the same message.
  Old messages are available at http://lists.thekrib.com/aga-contest
  When asked, log in as username is "aga-contest", and password "second".