Sacha Garrity, academic advisor in digital learning from Unitec, presents on how AI agents are used to support learning. The event is convened by Whitirea/Weltec. The work has received national recognition through the Tertiary ICT Association.
Lee Smith from Whitirea/Weltec welcomed everyone. A karakia was offered before introductions to Sacha and Sharnell Aumua. Their presentation is on the Ako AI agents project: Nursing.
Agents have been used for about a year. The Ako AI agents project involved supporting kaiako to customise mini ChatCPTs, hosted on Cogniti to be used by ākonga anytime when required, help to be responsive to ākonga needs and provide additional support to kaiako. Agents are embedded in Moodle so students do not have to move between two platforms.
An overview of Cogniti provided. Agent development informed by design-based research and Te Tiriti o Waitangi health principles. Agent developed with kaiako through co-design.
Quality assuring the agent is important. Testing undertaken through a broader group of kaiako. Follow a test protocol - does it do what is intended, promote learning, avoid unwanted responses. Then test with ākonga and gain feedback.
In short - interative design and test, extended test by kaiako, followed by student test and final Q & A.
Shared the nursing Ako AI. 3 nursing kaiako for each agent - (de-escalation agent, deteriorating patient and drug calculations. 4 developers and technical support. Stressed importance of drawing from and integrating Te Tiriti o Waitangi health principles.
Evaluation involved questionnaire, focus groups. Two iterations of testing and data collection for each agent.
Went through the drug calculation agent capabilities - providing fomulae for conversions/rations and calculations. Provide tutorial support. A demo ensued - basically a quiz, providing support when the student is unsure or unable to do the calculation.
ākonga feedback indicated that their learning confidence and competence was enhanced; user experience and design interactivity was appreciated; future AI tools and expanded learning support indicated - better integrations between LMS and Cogniti, videos to do revision, gamification / progress bar useful.
Demonstrated the deteriorating patient agent. A scenario agent. Two scenarios - hospital and community. Agent generates verbal and non-verbal behaviours, reactions based on student's response. New patient profiles generated at each use along with clinical/vitals. Feedback involved debrief, reflection and overall feedback - a challenge to build.
The agent has a 'toolbox' which is a link to 'help' in using the agent. Demo of agent in role as patient, perceptor and going through the debrief loop.
Overall, positive responses from students. Students trusted using the agents as they were developed by their tutors. Interest in having a range of agents to support learning across the nursing curriculum. Support for more clinical scenarios prior to going out to clinical.
Q & A
Oral responses - challenge with Te Reo Māori as pronunciation not the best.
24/7 support useful. Agents able to provide more in-depth feedback when they have time or need to undertake practice. A safe place to practice or clarify concepts etc. can be done.
Intentional learning is a key to developing how the AI agent is structured. Cogniti is user friendly and does not need IT degree to set up.
Volume of use brought up. Currently Cogniti coping but not sure of maximum number if a agent is open to entire institution e.g. for learning, academic or library support.
Toolbox discussed and how it can be used intentionally. Access to documents supporting the learning can also be accessed through the toolbox.
Security aspects discussed. Important to only provide resources which are relevant and Cogniti itself should not have access to internal organisational data.
Time from start to implementation averaged 3 - 4 weeks - mostly for SMEs to test, run staff test groups and gather feedback etc.
No comments:
Post a Comment