THE PANEL: Evaluation by numbers

THE PANEL: Evaluation by numbers

At Early Bird Speakers we like to live on the edge. Our meeting time is dangerously early, our meeting venue is a bastion of locked suitcases and secret handshakes, and our meeting etiquette is ruthlessly professional.

Last Thursday we decided to take the standard evaluation format, arm it with a pair of scissors, force it to run a marathon over a field of Lego in bare feet, carrying a mug of molten hot Bovril in the other hand, dodging a series of tripwires and avoiding a team of very persistent survey volunteers with clipboards, whilst blindfolded.

Vice President Public Relations of Early Bird Speakers, Katherine Eyres, reports on the interesting findings of this risky experiment involving the cornerstone of Toastmasters feedback: The Evaluation.

How it worked

The players

Instead of having 3 individual evaluators evaluating 3 different speeches, each speech was evaluated by a panel of 3 evaluators and chaired by an Evaluation Panel Chair.

The set-up

The panel were seated in a semi-circle formation towards the rear of the stage, with the Evaluation Panel Chair sitting slightly off the left hand side.

The running order and timings

Speeches were evaluated in the order they were delivered, one after the other.  Specific timings were as follows:

Master_Agenda_EVAL-PANEL_2013-11-28

The Evaluation Panel Chair would direct questions to each of the panellists individually, encouraging commendations and recommendations about each speech, and asking evaluators to hone in on some specific aspects of each speech.  After one evaluator had given their comments, the Evaluation Panel Chair would typically ask if the other panellists shared that view or had a different one.

What worked well

Evaluation is a subjective beast 

In the standard evaluation format, the speaker receives a single perspective through the eyes of a single person who sees the world in a particular way.  In contrast, the panel evaluation format allows the speaker to be the beneficiary of multiple perspectives.  Evaluator A  might pick up on a subtle nuance or subtext that another does not, Evaluator B might have a particular talent for detecting linguistic devices or speech rhythms, and Evaluator C might be supersonically attuned to pace of delivery and breathing.

Prime example: At last Thursday’s meeting one of our evaluators gave a recommendation that he would have liked to have seen the speaker draw further on his personal experience in order to make his message even more compelling.  In contrast, another evaluator had said that actually one of her main commendations was how for her the speaker had really effectively struck a perfect balance between factual empirical information and emotion.

Same time. Same place. Same speech. Different evaluators. Different impressions. Interesting…

A view from the firing line

Here are some very encouraging words from Best Speaker Award winner, Robert Taylor:

“Congratulations on the success of the first outing of the evaluation panel. I’d feared the double jeopardy of a diffused but somehow still lethal response, delivered by a smiling firing squad. In the event, as a speaker, I was impressed by the thoughtful and efficient way it was chaired, and the value of having a variety of responses to particular aspects of what I’d offered up that morning. Having sampled the relative sophistication and richness of the new format I’d be quite reluctant to go back to having only single opinion evaluations”.

The Evaluation Panel Chair – an essential cog in the evaluation panel machine 

Firstly, she had an important timekeeping role. This is particularly crucial as if the panel gets too tied up on one particular aspect the speaker may receive a too narrowly focused and incomplete evaluation.

Our Evaluation Panel Chair was very adept at asking open-ended yet well-structured questions.  She also was charming but firm in moving panellists along to ensure each speaker received feedback on various elements of their speech.

At a couple of appropriate moments she also gave an evaluator a right of reply if another evaluator had disagreed with a comment they had made about the speech.  This technique should however be used sparingly to ensure there is time to evaluate the speech with a reasonable breadth as well as depth.

If we had our time again

There are always lessons to be learned when trying something new.

To sit or not to sit?

The consensus seemed to be that the seating arrangements of the panellists and Evaluation Panel Chair should be reviewed to ensure they are in full view of the audience, which was problematic from some corners of the room.  Any physical or logistical barriers to the audience’s ability to get fully involved should always be minimised.  A solution might be to change the positioning of the chairs or to have the panellists present standing up.

Also, a note to anyone who delivers a speech or presentation whilst seated: be aware of the need to keep your posture upright and push your breath deep into your diaphragm to ensure your stage presence and vocal projection are not engulfed by the chair. This is a useful tip to take to the corporate environment if you are doing boardroom, panel or seated meeting presentations.

Striking the right balance

One of the speakers felt that overall the feedback had been weighted too heavily towards recommendations and should have been tempered with more positive feedback.  This is obviously an approach that evaluators should follow in standard evaluations anyway.

However, where there are multiple evaluators concurring on a specific recommendation, be alive to the fact that this may serve to compound this feedback in the speaker’s mind, whereas in a single evaluation this same feedback would have received a more cursory treatment.

Here the role of the Evaluation Panel Chair becomes pivotal in keeping tabs on time and directing leading questions to the panel to steer them into specific areas which are prime commendation territory.

3 is the magic number

At Toastmasters we are big fans of the ‘Rule of Three’, whether it’s the trusty tri-layered ‘commend’, ‘recommend’, ‘commend’ sandwich or a trifecta of tongue-twisting titillation.  As the saying goes, three heads are better than one. For this reason, sometimes three evaluators can also be better than one.

Are we suggesting doing away with the standard evaluation format?  No. This is a necessary and important skill to learn in a fledgling Toastmaster’s public speaking journey.

However, once the basics are mastered, opportunities to push one’s boundaries as a speaker and to evolve as a club should be seized with both hands – after first carefully putting down the scissors and the hot Bovril.

View a video of the session here.

____________________________________________________________________________________________

Top 3 tips:

  • The benefit of experience – Select more experienced evaluators for this more challenging and unfamiliar format
  • Variety is the spice of evaluation – Try to ensure your panellists have a range of different evaluation styles and strengths
  • Get feedback about the feedback – Ask for feedback from your audience and the speakers themselves about how they felt the session went and what might be improved for next time
Our Club Mission
To provide a mutually supportive and positive learning environment to foster self-confidence and personal growth where every member can develop communication and leadership skills. Join Us!