Judge

Your Mission: Ensure short-listed Contestant Research Proposals receive a full and fair evaluation.

The Judges read and evaluate the Contestant Research Proposals for the External Grants Review and select Grant Holders. The Lead Research Director selects the Judges from social science student, teacher, staff and administrator applicants.

The Judges must be:

  1. Enrolled as a student in Québec’s CEGEP Social Science Program or be a current or recently retired teacher, staff or administrator at a CEGEP
  2. Able to volunteer some time to review and evaluate the Contestant Research Proposals.
  3. Familiar with the Social Science Program and the Research Methods course material.

STEP-BY-STEP

Step 1 - SET CRITERIA AND DESIGN SCORING SHEETS

  • Review TRC website material, especially for this scenario.
  • Read the Suggested Evaluation Criteria.
  • Determine whether the Suggested Evaluation Criteria should be altered. Share your thoughts with your fellow Judges.
  • Meet with your fellow Judges to finalize the criteria and create a score sheet that all Judges agree to use when evaluating the proposals.

Step 2 - DESIGNATE ONE JUDGE AS COMMUNICATIONS OFFICER

  • One Judge has to serve as Communications Officer to manage files and lists and act as the communications hub. If no one steps forward, organize a random draw. Consider diminishing that Judge’s proposal review load in compensation.
  • The designated Communications Officer should quickly inform the Lead Research Director of this decision.

Step 3 - TAKE PART IN THE FIRST READING

  • The Lead Research Director will submit the Contestant proposals that meet the basic requirements.
  • To reduce the potential for favouritism and bias, names and other identifiable information will be removed from the proposals by the Lead Research Director. Be sure to include the code numbers assigned to each proposal on your score sheets.
  • Read and review the assigned proposals using the agreed score sheet with weighted values.
  • Ideally, each qualifying proposal should be read and evaluated by at least two Judges.
  • The Communications Officer finalizes the score sheet, assigns Research Proposals to the Judges and sends them their assigned proposals.
  • The score sheets are for internal use and will only be shown to the Lead Research Director.

Step 4 - GENERATE A SHORT-LIST

  • The Communications Officer determines how to organize the short-listing.
  • As a group, generate a short list of the better proposals based on the score sheets.
  • The Communications Officer communicates the short list to the Lead Research Director and asks for the contact information of the Ethics Board Reviewers for the next step.

Step 5 - SEND SHORT-LIST ITEMS TO ETHICS BOARD REVIEWERS

  • The Communication Officer sends the short-listed proposals to the Ethics Board Reviewers. It is preferable not to send the score sheets.
  • Be attentive to the timeline. Transfer the proposals as soon as possible and encourage the Reviewers to conduct their review in a timely fashion. The Lead Research Director may have specific dates for completion.
  • The Ethics Board returns the proposals indicating acceptance or rejection on ethical grounds. Accepted proposals are subjected to a second reading by the Judges.

Step 6 - TAKE PART IN THE SECOND READING

  • Determine together how to select a winner from the finalists that passed the ethics review.
  • If there are not too many, have every Judge read and review all the finalist proposals.
  • One option is to compare the scores from the first reading and devise a way to generate another score or some other distinguishable measure.

Step 7 - TALLY THE SCORES AND CHOOSE THE WINNER

  • From the finalists, determine which one is the winner.

Step 8 - TRANSMIT THE NEWS TO THE LEAD RESEARCH DIRECTOR

  • The Communications Officer informs the Lead Research Director of the winner.
  • The Judges’ decisions are final and no appeals will be considered.

Step 9 - TRANSFER THE PROPOSALS AND SCORING SHEETS TO THE COMMUNICATIONS OFFICER

  • The proposals and score sheets are all transferred to the Communications Officer who sends them to the Lead Research Director.