2025 review

 Well another busy year that has flown by rather quickly.

Next year, we are Te Pūkenga no more and return to being Ara Institute of Canterbury. An organisational restructure in mid- 2025 saw the Ara departments being dismantled and repackaged into three faculties. Each is helmed by a Dean of Faculty and supported by portfolio managers and a small number of faculty operations managers. My work team is now part of Ako (teaching and learning) Quality. We still do the same things, but the larger team is now made up of quality, learning design/educational development, kaiako (teacher) capability, and research.

This year, we have had a catch up on programme reviews. Many of our degrees, which are supposed to be reviewed every 5 years, were not reviewed through Te Pūkenga. The large institute was working through all the degrees under its wing, and unifying them i.e. based on one programme of learning. So far, only a handful of programmes were unified. So a large backlog of programmes now need to be reviewed, leading to my team's chief busyness this year as we work through many degree programmes and 'sub-degree' programmes for review and re-registration with the NZ Qualifications Authority.

AI and its implications on education has also been a focus. The AARIA 'Using Gen AI to support foundation/bridging ākonga (learners) was completed and launched in August. We are also piloting Cogniti - developed by the University of Sydney to provide support to teachers to build AI agents, and PebblePad, an eportfolio platform. 

Highlights include final publication of the book 'AI in Vocational Education' at the end of May. The book had a good number of downloads and a positive and supportive review.  

Three weeks  in Europe, culminating in the Journal of Vocational Education and Training Research (JVET) conference at Oxford, provided for R & R opportunities, a catch up with my cuzzie in Scotland and networking with VET researchers from Europe, South America and Africa. 

Presentations across the year have largely been on AI. I presented a plenary at the local NZ Vocational Education and Training Research Forum in November. There, the presentation of the lifetime award for the forum was extra special as NZ VET moves into another era and into post Reform of Vocational Education. 

The cessation of Ako Aotearoa, and the Centres for Vocational Excellence (CoVEs) will mean that funding for VET research will be very thin on the ground. Thankfully, my institute is supportive of research activities in the Scholarship of Teaching and Learning (SoTL) space, so we will continue with the various AI pilots/projects. 2026 will be very much a year of reconsolidation for the institute, so its a 'watch this space' process for the next year or so.

 

Cogniti mini symposium - link to video of presentations

Was unable to 'attend' the recent Cogniti mini symposium. Videos of the presentations are now available via this link. 

Almost all of the presentations are from universities, withe several from Chile.

Listened to the presentation from Associate Professor James Oldfield from Unitec on their collaborative project with Toi Ohomai and Manukau Institute of Technology on using agents in nursing. (12 minutes)

Detailed the various agents that were worked on - drug calculations, scenarios in nursing - de-escalation and dealing with difficult situations. Presented on student feedback on their perspectives on usefulness of the agents, usability and improvements that can be made to enhance the experience. 

,Uncheatable' assignments

As a continuation tothe blog post, on AI and assessments, here is a resource to think about how assessments/assignments can be framed in the age of AI written by Adam Pryor.

The blog begins overviews the ways education has viewed AI, viewing it has a threat to the assessment integrity. 

The solution proposes the use of 'specifications grading' providing clear competency statements and a formative process through teacher/AI dialog for learners to experience and evidence true learning.

Therefore I am now working with teaching teams to review their assessments. In short to put lower weightings on the usual summative assessments (100%) and instead transfer the weighting to a series of formative (but graded) touchpoints, which collate students' learning. For example, if the usual summative assessments is an essay, report or design, this is marked and weighted at 20% of the final summative mark. 20% is assigned to an oral presentation plus oral examination to establish authenticity of the learners' work. 60% is allocated to the 'portfolio' of how the student worked through to end up with the essay/report/design. In the 'portfolio' would be the record of the various steps undertaken, including the use of AI where appropriate and justified. The rubric assigns marks for reflective learning, accuracy of recording the learning process, and justifications/rationalisations and decisions taken. 

In doing, we shift the emphasis of assessments to the process of learning. Even if the final 'product' is not up to scratch, the portfolio of learning, recording the learning journey can still accrue sufficient marks for a good final grade. We can replicate more of what happens in the real world, that learning is a lifelong journey. We learn when we make mistakes as well as when things go well.

OPSITAra - day 2 morning - Notes

 Arrived yesterday evening to Dunedin after the Whitireia and Weltec research hui to join the OPSITAra research conference, an annual event to share the research that is undertaken across Otago Polytechnic, Southern Institute of Technology and Ara Institute of Canterbury. 

I made it to the book launch and caught up with things and people.

Friday morning dawned fine. Breakfast presented some time to touch base with several with similar research interests.

I then presented on our AI journey thus far, proposing that teachers using AI, shift from being 'guides on the side' to 'conductors of learning'.

Following on is Joe Taylor and Phil Osborne from OP on 'Making Feedback Matter: Exploring Student Engagement with Summative Assessment Feedback in the Bachelor of Applied Management Programme'. Shared the need to improve the summative feedback process. Provided background and rationale. We do not understand how students use (or not use) summative feedback. Summarised the literature on feedback. Focused on the work of Hattie (2015) on visible learning to higher education. Overall feedback is high, meaning feedback has a powerful influence on learning. However, feedback for formative assessment is more impactful than summative feedback. Summarised the work of 'delivering effective student feedback in higher ed. (Williams, 2024). Stressed the importance of feedback literacy. It is teacher responsibility to ensure students understand the importance of feedback to their learning. 

Then a presentation with Phil Osborne from OP on 'Exploring Neurodiverse Learners’ (NDL) Use of Artificial Intelligence in Tertiary Education' on work with Mairead Fountain and Rachel van Gorp. Sought to find out what are the best practice uses of AI by NDLs. Participants from a range of disciplines, length of time in study and range of neurodiversity. Qualitative through interviews (transcribed using Teams - and 24 turnaround back to interviewees for their comment), Use a copilot agent ' thematic analysis assistant' with reference to University of Auckland (Braun and Clarke) site on thematic analysis. Shared the prompt used. Also examples, key themes and process of cross checking / triangulation processes across the data. Preliminary insights include that NDLs use AI in nuanced and strategic ways; emotional support, accuracy concerns, ethical awareness and need for institutional clarity were other common perspectives. Shared whether the agent could then connect the data to the literature. However, as it was an analysis agent, this did not work well. 

Poster presentations then took place.

The second keynote at the conference is with Dr. Gianna Leoni, who presents on 'Kaitiakitanga: Protecting Mana Motuhake in the Age of AI'. Began with setting the context and the objectives of Te Hiku media, committed to revitalisation of Te Reo Māori. The maintain a large archive of the language and a digital transition of these began in 2013. In 2018 they began work to teach computers to speak in Māori. Today the emphasis is on indigenous language revitalisation. Access to the language is important for the Māori diaspora with a large population of Māori living outside of Aotearoa. Currently, AI is able to play a role but its used carefully. Presented on the importance of the transcription of video archives and the challenges with accents and dialects in Māori. AI can accelerate the process but needs careful training to deal with the variants of Te Reo. AI needs sufficient CPU/computing capacities; specialised / contextualised algorithms and 'clean data'. Important to understand how machine learning/AI brings together the key principles of computer science and linguistics. The contribution from Te Hiku is to advise on domain knowledge (needs to be accessible and authentic to stakeholders; investing in people; ); data (authentic data required; currently most of the data is crowdsourced and not necessarily accurate or trustworthy; unbiased data; secure; whakapapa-based; gathered through community events). Shared the mainstream automatic speech  recognition, OpenAIs whisper model and its lack of capability with Te Reo Māori therefore the need for a local focus. 

Data on bilingual data is especially difficult as for many Māori's main language, especially away from Aotearoa is not Māori. Shared how their work is being shared and used by the communities who need the tools, information, and knowledge. Through apps - whare korero, live streaming and Māori learning resources and apps based on traditional Māori pedagogy. Shared the work on a Māori parts of speech tagger, to help produce reliable translations (see Kaituhi, spell/grammar checkers, speech to text transcribers, life captioning, text to speech technology to generate synthetic voices, screen reader in Māori etc. Encouraged feedback on the apps and other tools so that they can be improved. There is a Kaitiakitanga licence for Kōrero Māori. Open to collaborative projects :) 

The conference closed with prizes best presentation and best presentation by a new/emerging researcher. Scott Klenner provided closing address, thanks, acknowledgements and karakia. 

I then join the Ara contingent to back to Christchurch by van. It was a warm NW summer day, so tops were made to top up on drinks, and snacks :) 

Whitireia and Weltec teaching and research hui - notes

 At the Teaching and Research hui in Wellington, convened at Whitirea and Weltec today. I was invited to provide a keynote on AI and presented on our AI projects, with an emphasis on the scholarship of teaching and learning. There was much to cover and I hope the general message got across. AI is here to stay, we need to apply it to supporting teaching and learning, teacher AI literacy is a pre-requisite and they then need to integrate AI literacies and use AI in a structured way to support teaching and learning.

I then stayed on for several of the sessions (across 6 streams) before leaving for the airport.

I kept to the technology, innovation and business stream as there were presentations across this that had relevance to our work on AI and programme development.

The conference began with a mihi whakatau (Māori welcome). Marie Merdan (research manager/ head of academic success) covered the usual housekeeping/safety briefings. and Leanne Ivil  Operations manager / director of teaching and learning welcomed the participants.  

I then presented the keynote on ‘AI in vocational education: The journey to personalised learning environments’ with an emphasis on the scholarship of teaching and learning and the potential for AI to support personal learning environments. I shared the journey thus far to build capability amongst our teachers, raise AI literacies with both students and teachers, and the challenges presented to integrate AI into various programmes of learning.

Then attended one stream of the presentations.

Akaike’s Information Criterion for Linearly Separable Clusters presented by Maria Eda Arado (presenter) with Roberto Padua. Technical topic on dataset management. Interesting from the point of view of how data is used/drawn on/organised etc.  across various industries and AI. Clustering of data is applied towards optimising data handling. Used the example of marine reserve ecosystems. Shared how clustering evolved and the most popular partitioning techniques, their challenges and some viable solutions.

Adopting AI Technology in Architecture Practices in New Zealand: A Practical Roadmap presented by Minh Nguyen. Discussed if there is future for architects in the age of AI. Overviewed the history of innovation in architectural design technology, From manual (pre-1980) to CAD (up to 2000s) to BIM (2000s), programmable design (2010s) and AI powered design (2020s). Discussed pros and cons of the parametric design process which is complex and takes a lot of learning and how AI supports and accelerates this process. Shared the ways AI can be applied to architecture. Presented the challenges of incorporating AI including costs, capability and the difficulties with reliability. A solution is to use Open Source AI architectural tools. To develop AI literacies with students, identify tasks into visual and text-based tools and help students coordinate these as and when required. Important to realise trust and relationships which are human traits.

Scaffolded Teaching Approach for Improved Engagement and Success in Engineering with Gopal Krishan (presenter), Christy Mathew, Maria Eda Arado & Glynis Valli. Shared a pedagogical intervention, used over the past few years, to increase student success and engagement with engineering mathematics. Traditional high-stakes exams cause high failure rates and stress, a more learner-centred approach is required, shared ways to help reduce stress and improve success. Introduced bi-weekly small tests, scaffolded the sequence of theory, practice and experiment. Placed a focus on feedback, consistency and designed for learning. Shared increased success rates from undertaking the shift with average marks increasing across the board. Student feedback concurred that stress was reduced and understanding improved. The approach is scalable and replicable through using frequent low stakes tests which reduce stress, builds confidence and increases success.

Poster presentations along with brunch followed where I caught up with two researchers (Dr. Brenda Saris and Alice Moore) conducting studies on bridging the learning/cultural divide when teaching Chinese students in the creative disciplines. 

Then a professional development session on ‘Plan smarter. Teach creatively. Hands-On with Copilot’ Alex Craven, Mike Laing, Angela Yates and Josh Clemen. A session for beginners using the standard Copilot365. A Kahoot ice breaker quiz starts the session with questions on level one AI literacy – what is AI? Summarised key points on what is Gen AI and how it can/cannot be used and the need to use Copilot for its corporate security. Went through the differences between the premium and vanilla Copilot365. Demonstrated standard ways to use Copilot to support teaching and admin work tasks.

I leave for the airport partway through the above to travel back down South to Dunedin for OPSITAra, which is the research hui for Otago Polytechnic, Southern Institute of Technology and Ara Institute of Canterbury.

 

 

Are the AI tutors from Open AI, Google and Anthropic effective?

 One of the major applications of Gen AI in education, is to support personalised education in the form of individualised learning support for each learner. However, due to the way in which AI is structured, its propensity to 'hallucinate' is still a challenge. Additionally in our work with chatbots (see project report for details) the chatbot has a propensity to 'drift' from its training after several students have used it. Therefore, there is still a challenge when using AI to underpin personalised learning.

This mashable.com article runs a test through the AI tutors provided by OpenAI, Google and Anthropic. The article provides for an interesting overview, the process for testing and then an summary of the performance of each. Links for each of the summaries bring you to a more detailed write up. 

The findings/discussion will not be new to teachers. Importantly, the writer decides that chatbots cannot replace great teachers; reinforcing that learning is social. 

Another post from AI toolkit undertakes a similar discussion. Although personalised learning is a 'holy grail' in education, the ways AI tutors go about provisioning answers, rather than supporting, nudging, encouraging learning is its disadvantage. When learners do not have to put effort into the learning process, learning does not take place. They end up with a 'cognitive debt' because the brain is not engaged in 'solving the puzzle' and 'making meaning'. In turn, learners develop 'metacognitive laziness'  and depend on AI to do the hard work of thinking required to increase learning and develop expertise.

Of note is the status of AI in education. This post in Education Futures warns of the ways education is losing control as commercial interests become the providers of learning platforms. Although written within the US of A context, many of the arguments laid out are universal and global. AI is not only a form of technology, but can dictate what is learnt, what is accessed (or not) and removes teacher and learner agency. 

Therefore, we need to be careful when thinking about AI-supported personalised learning. The design of learning into the AI agent/chatbot is essential to ensure that AI helps learning to take place and not replace learning.