Re: AGA CONTEST - Digest V1 #160

[krandall@world.std.com]

At 08:30 AM 8/11/99 -0700, James wrote:

Subject: Re: Mock Judging Results - prescreening of images

>This may sound like I'm being picky, but if the above scenario happens, we
>will look like a bunch of hacks who can't follow our own rules and
>guidelines (or couldn't agree on them). I realize that there may be altered
>images which will get by most of us (some graphic artists are _really_ good)
>and we may end up, in spite of our best intentions, with altered images on
>display. We may also not have _any_ problem with altered images. Right now
>we are all just guessing.

Why is that different from the fact that some of the judges in our "mock
contest" judged aspects of a printed background as if they were actual
parts of aquascape.  This is _going_ to happen, even if there is no intent
on the part of the entrant to pull the wool over anyone's eyes.

>But I suggest that if we are _not_ going to prescreen the images, we provide
>explicit instructions to the Judges that electronically altered images are
>NOT eligable for the Contest OR the Showcase.

First, who are "we" that we are more able than the people we choose as
"experts" to ascertain whether an image is altered or not.  Why do we think
that _we_ will be able to tell if the judges can't?  We've already stated
that obviously altered images will be disqualified.  If the judges ignore
that, then there's not much we can do except not ask them back a second
time.  I happen to believe that if they are willing to take on this task,
they will do it to the best of their ability.  Also, there is no reason
that they need to work in a vacuum.  If the question whether some aspect of
an entry is within the rules, they should be instructed to consult with the
show committee (or manager) for a ruling.  In horse shows, the person who
fulfills this function is known as the "TD" or technical delegate.  It is
not this persons job to judge the entries, but to know the rules inside and
out and be able to adjudicate on matters that come up during the judging.

I think this whole thing would be a lot simpler if you all just expected
the judges and show committee to COMMUNICATE with each other, and worked
from the assumption that every one will try to do the best they can.  Yes,
there are going to be glitches.  We are trying to create something
completely from scratch.  I just don't see that these glitches can't be
smoothed out by a show committee that is reasonable and responsive.

>Alternatively, if the judging is going to be done via the Internet and Erik
>is going to set up a form to receive the scores and comments, the simple
>inclusion of a field labelled "Suspected Altered Image" could take care of
>any possible problem. If any Judge suspects that an image has been altered
>or faked, all that they would have to do would be to check off the
>appropriate box on the form. 

Just INSTRUCT the judges what to do if they suspect a fake.  To put it on
the judging form makes the assumption that people are going to cheat.
Let's _try_ to approach this with a positive attitude!

>We (the contest organizers) could then contact
>the entrant via e-mail (I would imaging that anyone capable of
>electronically altering an image is going to also be a computer literate
>person, so they will have e-mail) and simply ask for proof that the image is
>genuine.
>
> If the proof provided passes "muster", we can include the image,
>scores and comments for that particular image. If the entrant cannot provide
>proof that the image is genuine (like two or three more shots of the same
>aquacape from different angles which ALL show the same feature), then its
>out.
>
>Now, it occurs to me that requiring multiple images of each aquascape will
>pretty much rule out all but the most ardent of cheaters. It would be pretty
>difficult for someone to make an alteration that wouldn't show up in at
>least ONE of them. so I may be worrying needlessly here, but the above
>little "check box" could satisfy everyone.

I have a _major_ problem with this one.  If _I_ entered a genuine photo and
was then asked to provide "proof" that it was real, I'll tell you that
would be the end of _my_ involvement in the contest.  I have no desire to
"prove" myself to a bunch of people that paranoid.

If the image is clearly not "natural", it is safe to disqualify it.  If
there are doubts whether or not it is real, and it is not possible to reach
a consensus, I think it is better to err on the side of caution, and not
eliminate the entry.  

------------------------------

Erik Olson wrote:

>If we can't trust the judges to do their jobs, we shouldn't be using those
>judges.  If we are so paranoid about "looking like fools" that we have to
>pre-judge the entries, then we should be judging them as well.  *I* think
>the judges will do a fine job at figuring this stuff out.  

I agree completely.

This aside from what the _judges_ will think of _us_ and our inability to
trust their judgement.  If I were asked to judge, then been told I am going
to be put in a straightjacket to do it, I'd probably beg off.

>OK, that's the last from me on this issue.  Too many words to just say
>"let's not worry about this."

Amen.

------------------------------

>> Re: Screening out Fakes - I agree that we are going to have to be very
>> careful about screening out something without first doing some
>> investigation. The first part of the investigation could be to require
>> people to give us a list of materials, plants and fish in the tank. If we
>> see something in the photo which is not on the list.....or vice versa.....
>> that is at least a start. Again, I don't know if this will even be a
>> problem, but it is a _potential_ problem and we should have a policy on how
>> we want it dealt with _should_ it arise.

First, who is going to input all that info onto the web site?  Do people
care about that minutia?  What do you do if someone forgets to list 1 Botia
striata.  What if they misidentify it?  What if it's hiding behind a plant
what they take the photo?  That's not even getting started on the plants.
Many people are totally clueless when it comes to the correct names of the
plants they are growing.  I happen to be growing several wild collected
plants that as far as I've been able to determine are not currently
identified.  How do we handle that?  Be careful of how tightly
_requirements_ are written.  At the same time, make as many _suggestions_
as possible pertaining to how best to present your tank.  Did entry #2
_have_ to include a shot of the cluttered surroundings and too large light
fixture?  I hope not.

>> Re: Submission Elements, number of Images per submission - I definately want
>> to see more than one image of a tank and less than 5 but that is more to be
>> fair to the aquascape and the aquarist who submitted it than anything else.
>> It is very difficult to judge a tank with only one picture and having too
>> many can lead to problems like we had with Entry 005.

The problems with entry 5 were not that there were too many shots but that
they covered too long a time frame, and were not labeled as such.  But I
agree that too many photos do not help in assessing the tank. (the photos
of the Oto and SAE with that entry were cute, but gave us no additional
info to judge from)  regarding some people's comments that we should
_require_ (that onerous word again!<g>) shots from both angles as well as
stright on, if someone can't remove their flash from their camera, or shoot
at very low speeds, a face on photo will have a huge flash back in it,
which will certainly not enhance judging.  What about a tank like my new
150, that sits in the corner of the room?  I can move to the right for an
angle shot, but I'd have to break a hole in the wall to move to the left.
What about a "built in" tank where the ONLY view is straight on?

We're trying to figure out all kinds of ways to ferret out amazingly good
faked images IF they show up, and at the same time are considering setting
requirements that I can see will not be possible for _many_ aquarists with
normal, every day set-ups to adhere to.

I suggest _requiring_ a minimum of 2 shots, maximum of 5. (three would be
fine for me, but in case we get people who are really gung-ho, I think we
could live with 5)  _Suggest_ that it would be advantageous in terms of
judging for the tank to be seen from different angles.  _Suggest_ that if
the tank surrounding add to the overall "look" (for instance a handsome
custom made cabinet, or a tank surrounded by beautiful plants) that a view
of the tank surrounds can be included.  _Suggest_ that if the tank is
beautiful, but crammed into a corner between the washer and dryer in the
basement, it might be best to frame the tank tightly.<g>

>> Submission  Elements - like a listing and description of how the aquascape
>> was put together, i.e. materials used, plant and fish list will not only
>> help screen out potential fakes, it will help us place tanks into Categories
>> for Judging and display. It isn't always obvious in a picture that that
>> beautiful piece of driftwood just right of centre is in actuallity printed
>> on a commercially made background, or even that those nice red plants are
>> 100% plastic. We can ask for those sorts of details in the Submission
>> elements.

Agreed.  I don't think water chemistry information is necessary however,
nor are lists of plants or fish that have died or have been removed from
tank prior to the point of photos for the contest.  If photos are submitted
that were not taken within a short period of time, the photos _must_ be
labeled with chronological sequence.  Otherwise the show committee will be
forced to choose a particular shot as the standard for judging.
 
>> Contest Entries which come in without the required elements should be placed
>> in the Showcase but not the Contest (this shouldn't cause concern - life has
>> certain rules, if we set a few and people choose to ignore them, they can't
>> complain about the consequences. I am tired of pandering.).

I think that if an effort has been made to comply with the rules in good
faith, and a single item has been left out, considering how complicated
we're making this, it would be good PR to contact the person and allow them
to send the missing components.  Otherwise, this is starting to seem more
of a hoop jumping contest for both entrants and judges.  Again, let's _try_
to give people the benefit of the doubt, and use COMMUNICATION to sort out
problems.  If we are going to end up with a lot of people feeling put out,
put upon, and with hurt feelings, I see no reason to be doing this at all.
 
>> I think we will have to tell them that the scale for each Criteria is 0 -
>> 100. Once they arrive at their decision, they multiply it by whatever
>> percentage value the particular Criteria point carries, and then report THAT
>> value. 

WAY too complicated.  If our total is 100 points, which seemed important to
at least some people, stick with it.  If there are only 35 points available
in a category (that is, I believe the highest points available in the
highest category) they _can't_ give out 50 points.  They are not available
in that category.  35 points is a perfect score in that category.  they can
move down from there accordingly.

Again, the point scores are only relevant to the other scores awarded by
_that particular judge_  Only the placings should have relevance to the
whole contest.

At dressage shows, which is the kind of horse showing I do, scores are
expressed as a percentage.  Each movement is scored from 1-10, where 10 is
"perfect".  The scores are added up and divided by the total possible score
to determine the percentage.  These judges have to undergo a rigorous
training and testing program, followed by literally years of apprentice
judging before they are qualified to judge at recognize horse shows.  Then
they must attend, and be tested at a judges training forum every two years
for the rest of their judging careers.  This is to "standardize" the
results as much as possible.  Still, it is common to go to a show, and look
at the scores being given out for the same group of riders in one ring by
one judge to be 10% higher than they are in another ring by another judge.
This is for comparable performances.  But when you look at the PLACINGS,
the best rides are still winning in both rings. We all know that some
judges "mark harder" than others.  We accept that fact and move on.  

We do not have the luxury of an established judges training program for
this contest, nor is there any established "standard" of what "perfect" is.
 Therefore the raw scores we are assigning _cannot_ be taken as more than a
guideline for placing.  They should not be manipulated so that they all
look like they are using the same scale, and they should not be combined to
determine winning "scores".  


If someone wants to start at 50, and mark upwards to 100 that's cool
>> with me - at least they will be consistent with themselves. But we had some
>> judges do this on the basis of 1-10. I was getting very confused, jumping
>> back and forth between their e-mail, an Excel spreadsheet, and the web-site
>> opened in Front Page. Sometimes, a 17" monitor is just NOT big enough! Once
>> I got them all sorted out and in place, they all made sense, but it was NO
>> FUN doing the sorting out and trying to figure out how they got their
>> numbers - in one case, I just had to leave out a judge's individual criteria
>> points because I was just too tired to worry about it anymore. This isn't a
>> crack at the judges - or how they did this, I'm just saying that without
>> specific guidelines provided up front which we ask each judge to adhere to,
>> I'm going to be bald before this is over.

As far as I'm concerned, it seemed pretty clear.  I _thought_ that the
group decided on the total points available in each category.  I know you
were trying to accomodate everyone's different styles, James, but I would
have sent these people a note saying, "I couldn't make heads or tails of
this.  Please assign 1-35 points for category 1, 1-30 points for category
2...etc.) Let them do the work.  That's what you paid them for ;-)

>> But some judges (most, actually) gave me scores which indicated
>> that they had understood my possibly sketchy instructions, while others
>> seemed to have substituted other marking schemes and their numbers did not
>> correspond to anything I was expecting.

Did you ask them what confused them?  Are you sure that even now they know
who they are?  I think all confusion can be eliminated by just giving them
a couple of examples, and reminding them to ASK if they have ANY questions.

>> In the actual Contest, if there were
>> say 200 entries and 10 Judges -

Good God I HOPE not!!! We were originally talking about a panel of 5.  Even
that seems a little heavy to me, but I could certainly live with it.  I'm
not sure where you are going to come up with 5 "experts" let alone 10.  As
I mentioned at some point in the past, I think the only person that
everyone sort of agrees is an expert is Amano, and I'd be very surprised if
you could talk him into this endeavor.


  ------------------
  To unsubscribe from this list, e-mail majordomo@aquatic-gardeners.org
  with "unsubscribe aga-contest" in the body of the message.
  To subscribe to the digest version, add "subscribe aga-contest-digest"
  in the same message.
  Old messages are available at http://lists.thekrib.com/aga-contest