Scoring rubric template for a jury: criteria, weights and scales

What a scoring rubric contains

A scoring rubric is also called a scoresheet, judging criteria or a marking scheme. Anchors are its most important part: short descriptions of what a score means, so that a 4 from one juror means roughly the same as a 4 from another.

• Criteria: 3 to 6, each measuring one thing.
• Question: one sentence per criterion, such as "Can this team build it?"
• Weight: how much the criterion counts, as a percentage or a multiplier.
• Scale and anchors: for example 1 to 5, with a description for 1, 3 and 5. A 2 or a 4 sits between two descriptions.
• Comment: one to three sentences that explain the scores.

How to choose judging criteria

Start from the purpose of the call. If an award rewards measurable results, impact gets the most weight. If a grant pays for a plan, the quality of the plan and value for money matter most.

Each criterion should measure one thing. Innovation and originality, for example, usually measure the same quality twice. Every criterion should match questions in the application form, so applicants give jurors the evidence they need.

Keep to 3 to 6 criteria and publish them with their weights in the call, so applicants know how they will be judged.

Juryline has ready-made rubrics in its templates for awards, startup competitions, grants, scholarships, student competitions and hackathons. You can change any criterion, weight or scale before the round opens.

Weights and scales

A weight says how much a criterion counts in the total. Percentages that add up to 100 are the easiest to read. Multipliers such as 1, 1.5 and 2 work the same way: only the proportions matter.

Give the most weight to the criterion closest to the purpose of the call. Keep any single criterion below about half of the total, so one opinion on one criterion cannot decide the result alone.

A 1 to 5 scale is quick and enough for most juries. A 1 to 10 scale gives more room when many applications are close; then describe 1 to 2, 5 to 6 and 9 to 10. Avoid 0 to 100 per criterion: it suggests a precision jurors cannot apply consistently.

Example: weights of 30, 30, 20 and 20 per cent and scores of 4, 3, 5 and 2 on a 1 to 5 scale give 0.3 × 4 + 0.3 × 3 + 0.2 × 5 + 0.2 × 2 = 3.5 out of 5.

In Juryline, weights are relative: a weight of 2 counts twice as much as 1. Each score becomes a total out of 100 points, where the bottom of a scale counts as 0 per cent of that criterion and the top as 100 per cent. The example above gives 62.5 points.

Rubric template for startup competitions

For a shortlist, an accelerator intake or a pitch final. Team carries the most weight, because early products change and teams stay.

Scale 1 to 5 (2 and 4 sit between the descriptions).
Total = sum of weight × score, out of 5.

1. Problem and market (20%): Is the problem real and the market large enough?
   1 = unclear problem or a very small market
   3 = real problem, market not yet proven
   5 = painful problem in a large, growing market

2. Product and innovation (20%): How new and how good is the solution?
   1 = copies existing solutions
   3 = a clear improvement on what exists
   5 = a new approach with a lasting advantage

3. Traction (20%): Is there evidence that customers want it?
   1 = no users yet
   3 = pilots or first users
   5 = paying customers and steady growth

4. Team (25%): Can this team build it?
   1 = key skills missing
   3 = solid team with some gaps
   5 = complete team with a track record

5. Potential (15%): Can it grow beyond the home market?
   1 = local market only
   3 = can grow regionally
   5 = can grow internationally

Comment: [why you gave these scores]

Rubric template for grant calls

For project grants and calls for proposals. Relevance, plan and impact share most of the weight; value for money checks the budget.

Scale 1 to 5 (2 and 4 sit between the descriptions).
Total = sum of weight × score, out of 5.

1. Relevance (25%): How well does the project fit the aims of the call?
   1 = weak fit with the aims
   3 = fits some of the aims
   5 = fits the aims closely and meets a stated priority

2. Quality of the plan (25%): Are activities and timeline clear and realistic?
   1 = activities and timeline unclear
   3 = realistic plan with some gaps
   5 = clear activities, timeline and risks

3. Impact (25%): What changes for the target group and how is it measured?
   1 = benefit unclear and not measured
   3 = clear benefit, weak measures
   5 = clear benefit for a defined group, measurable results

4. Value for money (15%): Are the costs necessary and explained?
   1 = costs unexplained or out of proportion
   3 = mostly reasonable, some items unclear
   5 = every cost necessary and explained

5. Capacity (10%): Can the applicant deliver?
   1 = no relevant experience shown
   3 = relevant experience, some gaps
   5 = has delivered similar projects

Comment: [why you gave these scores]

Rubric template for awards and hackathons

For an award, use quality (30%), originality (20%), impact (35%) and presentation of the entry (15%). Anchor impact on evidence: 1 means no results shown, 3 means results shown but partly measured and 5 means measured results for clients, users or society.

Hackathon judges score during or right after the demos, so keep the rubric short and the anchors concrete.

Scale 1 to 5 (2 and 4 sit between the descriptions).
Total = sum of weight × score, out of 5.

1. Impact (25%): Does it solve a real problem?
   1 = no clear problem
   3 = a real problem, partly solved
   5 = a real problem solved in a way people would use

2. Technical difficulty (25%): How hard was it to build?
   1 = mostly existing components put together
   3 = some demanding work
   5 = hard problems solved in the time available

3. Design (20%): Is it usable and well made?
   1 = hard to use
   3 = usable with rough edges
   5 = easy to use and well made

4. Working demo (30%): Does it work?
   1 = no working demo
   3 = works with gaps or sample data
   5 = works from start to finish in the demo

Comment: [why you gave these scores]

Using the rubric with a jury

A rubric only works if jurors apply it the same way.

1. Calibrate first: let jurors score two or three sample applications and talk through any difference of 2 points or more.
2. Score every criterion and leave a short comment that explains the scores.
3. Agree a tie-break rule before scoring starts, for example the criterion with the highest weight.
4. Expect strict and lenient jurors even with good anchors. When each juror scores only a share of the applications, compare the plain average with a normalised ranking.

In Juryline, jurors see the descriptions for a low, middle and high score on the scoring screen and can compare their own scores side by side. A round can name a criterion that breaks ties.

Questions

How many judging criteria should a rubric have?

Three to six. Fewer gives jurors little to separate applications; more makes scoring slow and the weights hard to reason about.

Should I use a 1 to 5 or a 1 to 10 scale?

A 1 to 5 scale suits most juries. Use 1 to 10 when many applications are close and jurors need finer steps. Use the same scale for every juror.

What is the difference between a rubric and a scoresheet?

The rubric is the set of criteria, weights and anchors. The scoresheet is the form where a juror enters scores for one application. Many organisers use the two words for the same document.

Should applicants see the scoring rubric?

Yes. Publish the criteria and weights in the call. Applicants then answer what the jury will judge, and the decision is easier to explain.

What if two applications get the same total?

Decide the rule before scoring: the higher score on the most heavily weighted criterion, a vote of the jury or a shared place. Record which rule was used.