Case Study - INDAF: an App that produces Individual Assessment Feedback
Dr. Christof Leicht, School of Sport, Exercise and Health Sciences
Abstract
I developed an App (INDAF) that analyses the part marks that are awarded in a typical marking process. For each student, it then produces an individualised assessment feedback report, which highlights excellent areas and areas for development within their assessment. In the SSEHS-wide rollout in January 2022, 5,754 feedback reports across 42 modules were produced using INDAF, and over 4,000 were viewed by students. Students rated receiving such additional individualised feedback overwhelmingly positive, highlighting the value for their future learning. Staff highlighted the minimal extra time to set up INDAF, as well as the reduced student queries on marks release day.
1. Background
High quality feedback is regarded as the most powerful single influence on student achievement (Hattie & Timperley, 2007). Ideally, feedback should be a ‘dialogue’, and offer the option of ‘corrective advice’ (Wilson, 2012). However, this requires a significant time commitment from the person giving the feedback, especially under the backdrop of growing student numbers. ‘Ideal’ feedback can therefore heavily impact on staff workload, to an extent that may be untenable. On the other hand, the issue of the low NSS student feedback scores across the sector must be addressed.
Exam marking often consists of adding up part marks to produce a final overall mark. Within SSEHS (and across º¬Ðß²ÝÊÓƵ), as students’ exam performance feedback is often limited to them receiving their overall exam mark, a large untapped potential to provide students with individualised feedback lies in the evaluation of these already existing part marks. This allows large-scale provision of additional individualised feedback whilst minimising increases to staff workload.
2.Methodology
I developed an application (called Individual Assessment Feedback, INDAF), which extracts the information of existing part marks by an automated process and produces a feedback report. The following outlines the procedure for an example exam consisting of 8 questions:
- The module leader assigns exam questions to thematic subcategories (for example, questions 1-3: subcategory 1 (Physiology); questions 4-5: subcategory 2 (Anatomy); questions 6-8: subcategory 3 (Research methodology); questions 1, 3 and 7: subcategory 4 (Calculations).
- Upon marking, each student’s performance is rated overall (out of the 8 questions, e.g., 50%). In addition, INDAF calculates a mark for each subcategory (e.g., Physiology: 77%; Anatomy: 55%, Research methodology: 30%, Calculations: 55%). INDAF further highlights any particularly strong, but also particularly weak areas.
- An individualised assessment feedback pdf report is produced for each student.
In two pilot trials (Semester 1 and 2 exams, 20/21), SSEHS staff and student feedback on creating and receiving these reports was collected and used to further fine-tune INDAF. Students rated the provision of this extra feedback very positively, leading to the SSHES-wide INDAF roll-out in 21/22.
3. Issues
Staff had to adapt their approach to marking by using INDAF, which was met with resistance by some. To minimise staff resistance towards INDAF, staff feedback obtained during the pilot phases as well as informal consultation with colleagues informed the App development. I further aimed to reduce barriers by minimising the overall administrative workload (e.g., automating the process of uploading reports to the students’ profiles), and by minimising extra staff workload by training a dedicated INDAF Admin support person. I further created help videos based on screen capture recordings to allow independent use of INDAF. These were supplemented with written user guides.
4. Benefits
The main benefit lies in the extra individualised feedback given to students - in addition to an overall grade, they are provided with a detailed account of their exam performance. A main strategy to make the rollout successful was to involve staff throughout the process, allowing INDAF features to be applicable to the whole range of assessments within SSEHS. Finally, the positive student perception data gathered during piloting supported the case for a School-wide rollout.
5. Evidence of Success
All staff engaging with the process were able to produce feedback reports for their modules (overall: 5,754 individualised feedback reports across 42 modules). Student engagement with the feedback reports was high (>4,000 were viewed by students). Student comments included:
- “I really like this concept as I often wonder where to concentrate my focus for the next semester. Would be especially helpful for modules that semester 1 content can be included in semester 2 exams.”
- “I thought the feedback system was brilliant and really appreciate knowing where my weaknesses lie within the module.”
- “The mark breakdown and (especially) question subcategories are useful as this clearly shows the types of questions I felt most and least comfortable approaching. Thank you!”
Staff comments on using INDAF included:
- “Minimal extra time to set up INDAF”
- “Reduced student queries on marks release day.”
- “The system allows for feedback to be given with relative ease to the students, it will also allow internal and external moderation to be more transparent.”
6. How Can Other Academics Reproduce This?
INDAF can be shared across university (currently via the shared drive). This is tried and tested, the Wolfson School has successfully piloted INDAF in this year’s exams with plans of further implementation going forward. INDAF has a wide application range, as it produces individual feedback reports for any assessment that can be subdivided into subcategories. In SSEHS, it has mainly been used for short answer and essay-type assessments. INDAF is best suited for quantitative assessment analysis (i.e., analysis of part marks), but also offers the option to include individual qualitative comments (which may be particularly relevant in essay-type assessments). For access to INDAF, including help videos and user guides, please contact c.a.leicht@lboro.ac.uk.
7. Reflections
The SSEHS-wide INDAF roll-out was facilitated by the support of the SSEHS Associate Dean for Teaching and the Director of Undergraduate Studies. Central to the success was integration of staff and student feedback to fine-tune processes and to provide options to tailor INDAF to the different modules’ requirements. A solid administrative process was essential to the smooth running during the exam period. Next steps may entail inclusion of INDAF within University systems (e.g., Learn).
8. References
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–11.
Wilson, A. (2012). Student engagement and the role of feedback in learning. Journal of Pedagogic Development, 2(1), 15–19.