About Us

The Need for Scalable Assessment

Attitudes towards online education are shifting - more people than ever are going online for their education. Expositional content - lectures, videos, notes, etc. - scales well thanks to the ease of online distribution. However, assessment from subject experts has not kept up with the pace.

Assessment is viewed as a key difficulty in massive open online courses (MOOCs) by course directors. This is because the scalable assessment methods prevalent in MOOCs are fundamentally limited. They most commonly rely on automated assessment which restricts the depth of understanding that can be explored. Also, it makes cheating child’s play, which devalues the qualifications received. Next most popular is peer-assessment. As the assessors aren't subject experts, the quality of feedback and the reliability of the assessment are inhibited. Thus both peer and automated assessment are inadequate for effective education or reputable accreditation.

In contrast, human assessment allows course directors to create challenging, thought-provoking questions. Conceptual questions that require argumentation, interpretation, creativity and design reinforce students’ learning the most. Human assessment also enables personalised and informative feedback. However the number of students who can access human assessment is unnecessarily limited. Assessors are usually restricted to the course director’s colleagues and students. We think there is a better way.

Our Solution

Our solution is an online freelance marketplace that will facilitate scalable human assessment through crowd-sourcing. This would connect freelance, subject-specialist assessors with course directors through an online platform.

As well as earning generous rates, being an assessor provides opportunities for personal and professional growth. Assessing work and giving feedback is a powerful way of developing subject mastery [1]. It’s also a fantastic opportunity to build and demonstrate teaching experience. Our assessors can do all this while working flexibly - from wherever they like and on their schedule!

Anyone with a degree or currently enrolled in higher education can register as an assessor.

To ensure assessments can be trusted, we developed a robust statistical model to calibrate the marks, flag unreliable assessors, and adjust assessor rates. By double-marking some student scripts, assessors’ relative consistency and bias can be inferred and controlled for. This also allows the most exceptional assessors to be recognised and rewarded.