This pilot study describes the assessment of interpreting through the application of scales originally devised by Carroll (1966) for machine translation. Study participants (interpreters, n=6; non-interpreters, n=6) used Carroll’s scales to grade interpreted renditions (n=9) in simultaneous mode by conference interpreters with three different levels of experience. Grading was conducted using transcripts of the interpreted renditions. Although the numbers of graders and graded renditions were small, the data indicates that interpreters and laypeople agree on the grading of intelligibility and informativeness in interpreted renditions.