ATAIN presentation on Riding the rapids, finding the pools: how AUT is navigating the turbulent waters of AI and assessment design with Dr John Davies and Dr Nell Mann from the Office of Learning, Teaching and Educational Design at Auckland University of Technology.
Irna Elgort facilitated. John and Nell introduced themselves and started with karakia. The presentation is a collaborative effort including the whole LTED team, Felicity Reid and Heather Merrick and Andrea Grant.
Outlined AUT's approach to assessments - has principles (Mātāpono), Assessment Policies and Procedures (Aromatawai).
Shared the main assessment principles, assessment policies (using Channel 1 and Channel 2). Naming the channels in the policy is the easy part (assurance of learning, learning is values, measured and rewarded, codesigned with staff, support for students, academic integrity, programme level design).
Covered through the presentation - a reframing assurance of learning; evidencing learning in non-invigilated settings, supporting redesign of assessments; and tclarity of assessments for students.
Introduced the need to included manaaki into learning and assessments. Nell outlined the principles and philosophies underlying learning, teaching and assessment. What learning are we measuring? What learning are we valuing? Therefore need to reframe assurance of learning - what does it mean to assure? Should we broaden out the accepted norm of assurance of learning?
The AUT assessment policy 'the ongoing systematic process of gathering evidence of what a students knows, and can do. Assurance is about how the process can take place and must be present in both channel 1 and channel 2. Assessments in the quicksand require redesign to move into one of the channels.
Channel 1 is assured by invigilation, channel 2 assured by points of observation of learning (POOL).
In Channel 1, no AI is permitted. Channel 2 may include the use of AI. Channel 2 requires careful learning design of the assessment. Important to understand what happens through the assessment. POOLs take place across a range of settings in which the use of AI tools is permitted. Assessments are designed to provide insight into students' learning process beyond the final submission, ensuring that learning can be assured. POOLs can be evidence (in many forms) included in final submission. included in the rubric, encourage engagement and provide opportunities to students to demonstrate learning - notes, screen shots, etc. Quantity of POOLs contextualised to course outcomes.
Shared an examples of POOL from a software development practice/project management - evidence from GitHub logs and project management through a Trello board. In the past, this was just an 'end-product' assessment.
Neil continued with supporting the redesign of assessments. Staff are encouraged to evaluate an existing assessment and work out how it can work in an AI world. Can POOLs be created; can POOLs be applied to the assessment and what needs to be done to re-design the assessment tasks. Creation of POOLs help to anchor the re-design. in general, POOLs weighted higher (60%) than the final product.
Clarity of assessments for students is important so that are are aware of the importance of collecting and collating relevant evidence of learning. Challenges include ensuring the things we want to measure, the need to 'look around corners' i.e. establishing an evidence base to make progress when the future is hazy. There is also a need to change long-held beliefs with students and staff. There needs to be a programme level approach. There is a shift of quality assurance from being central to local.
Shared the impact of wearable technologies, especially its impact on Channel 1.
Q & A followed.
Questions revolved around the POOLs process, workload (which may increase), examples and how the process of learning can be followed through as it progresses.
No comments:
Post a Comment