Evaluation Panels and Scoring
Selecting a cohort is the decision your centre is judged on. This page covers how to put people on a panel, what a panel member sees when they score, how the weighted total is worked out, and how to review an application before it ever reaches them.
What you will learn
- The two things that must be true before anyone can be assigned to a panel
- How to assign evaluators to a whole program or to one batch
- What a panel member sees and how scoring works
- How the weighted total score is calculated
- How to review an application and ask for changes or documents
Why panels are worth setting up properly#
Without a panel, selection comes down to whoever read the application and how they felt that day. With one, several people score the same application against the same criteria, and the result is a number you can defend - to a founder who asks why they were not selected, and to a funder who asks how you chose.
Good to know
The scoring sheet itself - the questions, their answer options and their weightings - is built when you create or edit the program. See Creating a Program. This page is about who does the scoring and what happens when they do.
Before you can assign anyone#
Two things must be in place, and if either is missing the assign dialog will simply show an empty list with no explanation.
- 1
The person must exist as a portal user
External evaluators are not a separate kind of account. Add them in Users & Permissions like anyone else. Give them a deliberately narrow role - an outside investor does not need to see your funding figures.
- 2
Their Panel Member switch must be on
On their user profile there is a Panel Member toggle, separate from their role. That switch is what makes someone appear in the list of people you can assign to a panel. Without it, they are invisible here no matter what their role says.
Tip: This is the single most common reason the assign list comes up empty. If you cannot find a colleague, go to Users & Permissions, open them, and check that switch.
Assigning people to a panel#
Open the program, stay on the Program Info tab, and scroll to the bottom. Panel Members is the last section on that screen, with a count of how many are currently assigned.
- 1
Press Assign Panel Members
Search by name or email. Only people with the Panel Member switch enabled will appear.
An empty list means nobody has the Panel Member switch turned on yet - not that something is broken. - 2
Decide the scope: whole program or one batch
The 'Assign to Batch' dropdown defaults to 'All batches (program-level)', which means this person evaluates for every intake of this program, now and in future. Choose a specific batch instead to limit them to that one cohort.
- Program-level suits your own staff, who will be involved every year.
- Batch-level suits a guest evaluator brought in for one intake - a sector specialist for a themed cohort, say.
- Batch-level scoping also keeps an external evaluator from seeing applications from cohorts they have nothing to do with.
Tip: Default to batch-level for anyone outside your organisation. It is easier to extend someone's access later than to explain why an outsider could see three years of applications.
What a panel member sees when they score#
A panel member signs into the same portal, but their menu is short - they see the applications assigned to them and little else. Opening one gives them the application and your scoring sheet side by side.
- Each question shows its answer options with the points attached to each, and its weight as a multiplier (for example 'Weight: 10x').
- Choosing an option immediately shows what that answer is worth once weighted - an option worth 5 on a question weighted 10 shows as 50.
- Every question has its own comment box, so a scorer can explain a low mark rather than leaving you guessing.
- A running Total Score sits at the bottom and updates as they work down the sheet.
- A General Remarks box captures an overall view of the application.
- The Submit button stays disabled until every question has been answered - nobody can submit half a scorecard.
Tip
If you wrote a scoring reference when you built the question - '1-2: needs improvement, 3: meets expectations, 4-5: exceeds' - it appears here. It is the difference between two evaluators marking alike and marking to their own private standard.
Screenshot not available
The scoring screen could not be captured for this guide because the demonstration portal has no panel members assigned and nothing awaiting evaluation. The description above is accurate; the illustration will be added once a live panel exists.
How the total score is worked out#
Each question contributes the points of the chosen answer multiplied by that question's weight. Those results are added together to give the total for the application.
| Question | Answer chosen | Points | Weight | Contributes |
|---|---|---|---|---|
| Strength of founding team | Exceptional | 5 | 10 | 50 |
| Size of the market | Adequate | 3 | 10 | 30 |
| Technical feasibility | Strong | 4 | 5 | 20 |
| Social impact | Limited | 1 | 5 | 5 |
| Total | 105 |
Careful
Weights are not percentages and do not need to add up to 100. This trips people up. If you want one criterion to matter twice as much as another, give it twice the weight - nothing else is required.
Reviewing an application yourself#
Before an application reaches your panel, someone on your team usually reads it. Open any application from the Registrations tab and you get the whole picture on one screen.
- Quick Actions down the left give you Accept Changes and Reject Application.
- Three buttons top right: Request Changes, Request Documents, and Change Status.
- 'Other Applications' is quietly useful - it tells you whether this startup is applying everywhere or has chosen you deliberately.
| Section | What it holds |
|---|---|
| Overview | Funding requested and its purpose, total funding received to date, how many other incubators they are with, how many other applications they have made, and team size. |
| Files & Docs | Everything they uploaded - pitch deck, certificates, financials. |
| Evaluations | Panel scores for this application once they come in. |
| Changes & Requests | The history of what you have asked for and what they sent back. |
Asking for changes before you score#
If an application is not good enough to judge fairly, send it back rather than marking it down. Request Changes lets you raise several specific points in one go rather than one vague note.
- Each request has a title and a description, and is tagged with its type - a general change, or a request tied to a specific field or file.
- 'Add Another Request' stacks up more; the counter on the right tracks how many you have written.
- The send button tells you exactly what you are about to do: 'Send 3 Change Requests'.
- The application moves to Changes Requested, and back to Changes Received when they respond.
Tip
Raise every point in one round. Three separate requests over three days reads as disorganised to a founder, and each round adds days to your own timeline.
Common questions#
Nobody at your centre has the Panel Member switch turned on. Go to Users & Permissions, open the person you want, and enable Panel Member on their profile. Their role is a separate thing and does not control this.
Yes, and it is common. Add them as a user with a very limited role, turn on Panel Member, and assign them to a specific batch. They will see only the applications you assign and nothing else.
No. Weight is a multiplier, not a percentage. A question weighted 20 simply counts twice as much as one weighted 10. Totals will differ between programs, which is fine - you compare applications within a program, not across them.
Submit stays disabled until every question has an answer selected. Ask them to scroll back up and look for a question they have skipped.
If you have the Panel Member switch on your own profile and you assign yourself to the program, yes. Many centre heads do exactly this.
Program-level means they evaluate every intake of that program, including future ones. Batch-level limits them to a single cohort. Use batch-level for guests and program-level for your own staff.
A panel member has not submitted. Check the program's Evaluations tab to see what is pending, then chase the person. Do not change the status by hand - you would lose the scoring.
