Benchmarking introductory programming exams: how and why

Ten selected questions have been included in 13 introductory programming exams at seven institutions in five countries. The students’ results on these questions, and on the exams as a whole, lead to the development of a benchmark against which the exams in other introductory programming courses can be assessed. We illustrate some potential benefits of comparing exam performance against this benchmark, and show other uses to which it can be put, for example to assess the size and the overall difficulty of an exam. We invite others to apply the benchmark to their own courses and to share the results with us.