What kind of assessment and evaluation will I use for the student-training final project? Authentic! This is the last installment in a four-part series concerning the ADDIE model. Here’s the scoop:
Mueller (2012) defines authentic assessment (AA) as a “form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills” (What is Authentic Assessment?, para. 1). This is in contrast to traditional assessment (TA), which takes the form of true/false, multiple choice, fill-in-the-blank and other standardized test formats that do not assess meaningful application (Mueller, 2012). Instead of taking a test that assesses knowledge recall, AA is a more holistic way of testing knowledge through application and skills, demonstrating knowledge of a concept. This is also a more active approach than passive TA, where students can guess at the answers and the educator can only infer that the concepts/ideas tested can be applied by students in the real world. Mueller (2012) sees AA and performance based assessment as synonymous, whereas Palm (2008) notes some nuances which differentiate the two. For example, Palm (2008) states that performance based assessment tasks “need not for example be real world applications or require much communication and high levels of cognitive complexity just because students’ activities are hands-on or because they have to construct an answer themselves” (p. 9). For Palm (2008), performance assessment defines the response format of a test and not the real-world applicability.
Regardless of the name used, AA and performance based assessments both use direct assessment. Wiggins (1990), sums up this concept by stating that “Assessment is authentic when we directly examine student performance on worthy intellectual tasks” as opposed to the “proxy” tasks that superficially assess knowledge (para. 1). For the instructional designer, the assessment task has to match the correct domain, be it the cognitive, psychomotor, affective or interpersonal domain, or a mix (p. 92). No matter the domain, AA shows through action that the learner has not only grasped the knowledge, but understands how to apply it in the real world.
Two examples of AA in the scope of the final training project follow two enabling objectives:
1. English major student-teachers should be able to define an English language-learning lesson objective given a real-life scenario and a specific topic with corresponding vocabulary until a concise, correct objective is achieved.
The above can be assessed directly by the facilitator through the completion of objectives by students. This would occur in a classroom setting because it is within the lesson planning section of the training and objectives need to be determined prior to application. Defining objectives is the foundation of lesson planning and the correct formation of such is paramount to teaching a foreign language and is a skill that will be used while creating every lesson.
2. English major student-teachers should be able to model correct English pronunciation during an English language-learning lesson given a real-life scenario, vocabulary words and practice sentences for 60% of the words and sentences attempted.
The above can be assessed through an oral test given by a native, English speaker. Because this objective lies within the psychomotor domain, the implications for real-world application are direct: if pronunciation is correct, ideas can be shared, the converse is also true.
How will I evaluate the training, you ask? Well, let’s take a look at Kirkpatrick’s levels of evaluation…
Kirkpatrick’s levels of evaluation are broken into four levels; Level 1: Reaction, Level 2: Learning, Level 3: Behavior and Level 4: Results. Level 1 evaluations are based on the reaction of those trained: how did they feel about the training? Hodell (2011) says that the most effective way to monitor learners’ reactions is to obtain evaluation directly after the training ends (pp. 66-67). Feedback forms, verbal interviews, post-training surveys/questionnaires and online evaluation are all examples of Level 1 evaluation (Chapman, 2012).
Level 2 evaluation concerns itself with learning. Hodell (2011) states that this is directly tied to the proposed learning objectives. In this way, Level 2 evaluation is easily obtainable by virtue of the fact that course objectives were detailed and well-written. Assessments before and after training, interviews and observations are all tools that can evaluate if the learning objectives were met (Chapman, 2012).
In terms of the student-teacher training, Level 1 evaluations can be achieved through feedback forms and selected verbal interviews directly after training. This would be most effective because a form could be handed out at the conclusion of the training to assess trainee reactions and required to be filled out before leaving the classroom, thereby giving immediate responses. Level 2 evaluation for the student-teacher training would have to include a performance-based assessment, where the student-teacher will teach an English language lesson and a facilitator will evaluate their performance. A verbal interview would also be included to ensure course content mastery and that the correct rationalization was present during the lesson-planning and execution process.
Chapman, A. (2012). Kirkpatrick’s learning and training evaluation theory. Retrieved from http://www.businessballs.com/kirkpatricklearningevaluationmodel.htm
Hodell, C. (2011). ISD From the ground up: A no-nonsense approach to instructional design. Alexandria, VA: American Society for Training & Development.
Mueller, J. (2012). Authentic Assessment Toolbox. Retrieved from http://jfmueller.faculty.noctrl.edu/toolbox/index.htm
Palm, T. (2008). Performance Assessment and Authentic Assessment: A Conceptual Analysis of the Literature. Practical Assessment Research & Evaluation, 13(4). Retrieved from http://pareonline.net/pdf/v13n4.pdf
Wiggins, G. (1990). The case for authentic assessment. Practical Assessment, Research & Evaluation, 2(2). Retrieved from http://PAREonline.net/getvn.asp?v=2&n=2