,Uncheatable' assignments

As a continuation tothe blog post, on AI and assessments, here is a resource to think about how assessments/assignments can be framed in the age of AI written by Adam Pryor.

The blog begins overviews the ways education has viewed AI, viewing it has a threat to the assessment integrity. 

The solution proposes the use of 'specifications grading' providing clear competency statements and a formative process through teacher/AI dialog for learners to experience and evidence true learning.

Therefore I am now working with teaching teams to review their assessments. In short to put lower weightings on the usual summative assessments (100%) and instead transfer the weighting to a series of formative (but graded) touchpoints, which collate students' learning. For example, if the usual summative assessments is an essay, report or design, this is marked and weighted at 20% of the final summative mark. 20% is assigned to an oral presentation plus oral examination to establish authenticity of the learners' work. 60% is allocated to the 'portfolio' of how the student worked through to end up with the essay/report/design. In the 'portfolio' would be the record of the various steps undertaken, including the use of AI where appropriate and justified. The rubric assigns marks for reflective learning, accuracy of recording the learning process, and justifications/rationalisations and decisions taken. 

In doing, we shift the emphasis of assessments to the process of learning. Even if the final 'product' is not up to scratch, the portfolio of learning, recording the learning journey can still accrue sufficient marks for a good final grade. We can replicate more of what happens in the real world, that learning is a lifelong journey. We learn when we make mistakes as well as when things go well.

OPSITAra - day 2 morning - Notes

 Arrived yesterday evening to Dunedin after the Whitireia and Weltec research hui to join the OPSITAra research conference, an annual event to share the research that is undertaken across Otago Polytechnic, Southern Institute of Technology and Ara Institute of Canterbury. 

I made it to the book launch and caught up with things and people.

Friday morning dawned fine. Breakfast presented some time to touch base with several with similar research interests.

I then presented on our AI journey thus far, proposing that teachers using AI, shift from being 'guides on the side' to 'conductors of learning'.

Following on is Joe Taylor and Phil Osborne from OP on 'Making Feedback Matter: Exploring Student Engagement with Summative Assessment Feedback in the Bachelor of Applied Management Programme'. Shared the need to improve the summative feedback process. Provided background and rationale. We do not understand how students use (or not use) summative feedback. Summarised the literature on feedback. Focused on the work of Hattie (2015) on visible learning to higher education. Overall feedback is high, meaning feedback has a powerful influence on learning. However, feedback for formative assessment is more impactful than summative feedback. Summarised the work of 'delivering effective student feedback in higher ed. (Williams, 2024). Stressed the importance of feedback literacy. It is teacher responsibility to ensure students understand the importance of feedback to their learning. 

Then a presentation with Phil Osborne from OP on 'Exploring Neurodiverse Learners’ (NDL) Use of Artificial Intelligence in Tertiary Education' on work with Mairead Fountain and Rachel van Gorp. Sought to find out what are the best practice uses of AI by NDLs. Participants from a range of disciplines, length of time in study and range of neurodiversity. Qualitative through interviews (transcribed using Teams - and 24 turnaround back to interviewees for their comment), Use a copilot agent ' thematic analysis assistant' with reference to University of Auckland (Braun and Clarke) site on thematic analysis. Shared the prompt used. Also examples, key themes and process of cross checking / triangulation processes across the data. Preliminary insights include that NDLs use AI in nuanced and strategic ways; emotional support, accuracy concerns, ethical awareness and need for institutional clarity were other common perspectives. Shared whether the agent could then connect the data to the literature. However, as it was an analysis agent, this did not work well. 

Poster presentations then took place.

The second keynote at the conference is with Dr. Gianna Leoni, who presents on 'Kaitiakitanga: Protecting Mana Motuhake in the Age of AI'. Began with setting the context and the objectives of Te Hiku media, committed to revitalisation of Te Reo Māori. The maintain a large archive of the language and a digital transition of these began in 2013. In 2018 they began work to teach computers to speak in Māori. Today the emphasis is on indigenous language revitalisation. Access to the language is important for the Māori diaspora with a large population of Māori living outside of Aotearoa. Currently, AI is able to play a role but its used carefully. Presented on the importance of the transcription of video archives and the challenges with accents and dialects in Māori. AI can accelerate the process but needs careful training to deal with the variants of Te Reo. AI needs sufficient CPU/computing capacities; specialised / contextualised algorithms and 'clean data'. Important to understand how machine learning/AI brings together the key principles of computer science and linguistics. The contribution from Te Hiku is to advise on domain knowledge (needs to be accessible and authentic to stakeholders; investing in people; ); data (authentic data required; currently most of the data is crowdsourced and not necessarily accurate or trustworthy; unbiased data; secure; whakapapa-based; gathered through community events). Shared the mainstream automatic speech  recognition, OpenAIs whisper model and its lack of capability with Te Reo Māori therefore the need for a local focus. 

Data on bilingual data is especially difficult as for many Māori's main language, especially away from Aotearoa is not Māori. Shared how their work is being shared and used by the communities who need the tools, information, and knowledge. Through apps - whare korero, live streaming and Māori learning resources and apps based on traditional Māori pedagogy. Shared the work on a Māori parts of speech tagger, to help produce reliable translations (see Kaituhi, spell/grammar checkers, speech to text transcribers, life captioning, text to speech technology to generate synthetic voices, screen reader in Māori etc. Encouraged feedback on the apps and other tools so that they can be improved. There is a Kaitiakitanga licence for Kōrero Māori. Open to collaborative projects :) 

The conference closed with prizes best presentation and best presentation by a new/emerging researcher. Scott Klenner provided closing address, thanks, acknowledgements and karakia. 

I then join the Ara contingent to back to Christchurch by van. It was a warm NW summer day, so tops were made to top up on drinks, and snacks :) 

Whitireia and Weltec teaching and research hui - notes

 At the Teaching and Research hui in Wellington, convened at Whitirea and Weltec today. I was invited to provide a keynote on AI and presented on our AI projects, with an emphasis on the scholarship of teaching and learning. There was much to cover and I hope the general message got across. AI is here to stay, we need to apply it to supporting teaching and learning, teacher AI literacy is a pre-requisite and they then need to integrate AI literacies and use AI in a structured way to support teaching and learning.

I then stayed on for several of the sessions (across 6 streams) before leaving for the airport.

I kept to the technology, innovation and business stream as there were presentations across this that had relevance to our work on AI and programme development.

The conference began with a mihi whakatau (Māori welcome). Marie Merdan (research manager/ head of academic success) covered the usual housekeeping/safety briefings. and Leanne Ivil  Operations manager / director of teaching and learning welcomed the participants.  

I then presented the keynote on ‘AI in vocational education: The journey to personalised learning environments’ with an emphasis on the scholarship of teaching and learning and the potential for AI to support personal learning environments. I shared the journey thus far to build capability amongst our teachers, raise AI literacies with both students and teachers, and the challenges presented to integrate AI into various programmes of learning.

Then attended one stream of the presentations.

Akaike’s Information Criterion for Linearly Separable Clusters presented by Maria Eda Arado (presenter) with Roberto Padua. Technical topic on dataset management. Interesting from the point of view of how data is used/drawn on/organised etc.  across various industries and AI. Clustering of data is applied towards optimising data handling. Used the example of marine reserve ecosystems. Shared how clustering evolved and the most popular partitioning techniques, their challenges and some viable solutions.

Adopting AI Technology in Architecture Practices in New Zealand: A Practical Roadmap presented by Minh Nguyen. Discussed if there is future for architects in the age of AI. Overviewed the history of innovation in architectural design technology, From manual (pre-1980) to CAD (up to 2000s) to BIM (2000s), programmable design (2010s) and AI powered design (2020s). Discussed pros and cons of the parametric design process which is complex and takes a lot of learning and how AI supports and accelerates this process. Shared the ways AI can be applied to architecture. Presented the challenges of incorporating AI including costs, capability and the difficulties with reliability. A solution is to use Open Source AI architectural tools. To develop AI literacies with students, identify tasks into visual and text-based tools and help students coordinate these as and when required. Important to realise trust and relationships which are human traits.

Scaffolded Teaching Approach for Improved Engagement and Success in Engineering with Gopal Krishan (presenter), Christy Mathew, Maria Eda Arado & Glynis Valli. Shared a pedagogical intervention, used over the past few years, to increase student success and engagement with engineering mathematics. Traditional high-stakes exams cause high failure rates and stress, a more learner-centred approach is required, shared ways to help reduce stress and improve success. Introduced bi-weekly small tests, scaffolded the sequence of theory, practice and experiment. Placed a focus on feedback, consistency and designed for learning. Shared increased success rates from undertaking the shift with average marks increasing across the board. Student feedback concurred that stress was reduced and understanding improved. The approach is scalable and replicable through using frequent low stakes tests which reduce stress, builds confidence and increases success.

Poster presentations along with brunch followed where I caught up with two researchers (Dr. Brenda Saris and Alice Moore) conducting studies on bridging the learning/cultural divide when teaching Chinese students in the creative disciplines. 

Then a professional development session on ‘Plan smarter. Teach creatively. Hands-On with Copilot’ Alex Craven, Mike Laing, Angela Yates and Josh Clemen. A session for beginners using the standard Copilot365. A Kahoot ice breaker quiz starts the session with questions on level one AI literacy – what is AI? Summarised key points on what is Gen AI and how it can/cannot be used and the need to use Copilot for its corporate security. Went through the differences between the premium and vanilla Copilot365. Demonstrated standard ways to use Copilot to support teaching and admin work tasks.

I leave for the airport partway through the above to travel back down South to Dunedin for OPSITAra, which is the research hui for Otago Polytechnic, Southern Institute of Technology and Ara Institute of Canterbury.

 

 

Are the AI tutors from Open AI, Google and Anthropic effective?

 One of the major applications of Gen AI in education, is to support personalised education in the form of individualised learning support for each learner. However, due to the way in which AI is structured, its propensity to 'hallucinate' is still a challenge. Additionally in our work with chatbots (see project report for details) the chatbot has a propensity to 'drift' from its training after several students have used it. Therefore, there is still a challenge when using AI to underpin personalised learning.

This mashable.com article runs a test through the AI tutors provided by OpenAI, Google and Anthropic. The article provides for an interesting overview, the process for testing and then an summary of the performance of each. Links for each of the summaries bring you to a more detailed write up. 

The findings/discussion will not be new to teachers. Importantly, the writer decides that chatbots cannot replace great teachers; reinforcing that learning is social. 

Another post from AI toolkit undertakes a similar discussion. Although personalised learning is a 'holy grail' in education, the ways AI tutors go about provisioning answers, rather than supporting, nudging, encouraging learning is its disadvantage. When learners do not have to put effort into the learning process, learning does not take place. They end up with a 'cognitive debt' because the brain is not engaged in 'solving the puzzle' and 'making meaning'. In turn, learners develop 'metacognitive laziness'  and depend on AI to do the hard work of thinking required to increase learning and develop expertise.

Of note is the status of AI in education. This post in Education Futures warns of the ways education is losing control as commercial interests become the providers of learning platforms. Although written within the US of A context, many of the arguments laid out are universal and global. AI is not only a form of technology, but can dictate what is learnt, what is accessed (or not) and removes teacher and learner agency. 

Therefore, we need to be careful when thinking about AI-supported personalised learning. The design of learning into the AI agent/chatbot is essential to ensure that AI helps learning to take place and not replace learning.

Aotearoa AI Tertiary Network (ATAIN) - Student Perceptions of AI: Reshaping Pedagogical Approaches in NZ

 In this session of ATAIN we have a presentation from Dr. Francesca Benocci and Stella McIntosh from He Herenga Waka / Victoria University Wellington.

Their topic is 'Student Perceptions of AI: Reshaping Pedagogical Approaches in NZ'  Poll of participants indicate that AI provides both opportunities and challenges with AI.

Began with the challenges posed by AI - raising critical concerns around equity and learning and students are increasing using AI - not always to the betterment of their learning. Therefore important to find out how students use AI to move beyond reactive responses towards thoughtful engagement with AI.

Used anonymous survey and based on a Australian survey across 8 universities. Used Menti,com to run through some of the items in the survey. Questions cover perceptions on AI, its usefulness and harm.

Sample of just under 600 students. Mixed results with 83.4% were sceptical, 80.2% were worried, 49,8% optimistic and 51.6% excited. These are all lower than those from the Australian survey.

68% experienced stress and 63.8% felt shame :( 

Students used AI to 'make things faster', Helps me when I am stuck, makes things easier, reduces my stress. 

Shared things that discourage their use. 

Students report using AI to edit and improve writing, brainstorm ideas, summarise materials, step-step by teaching, finding information, code/formula check, generate written answers, transcribe lectures and complete assessments. 

Only small numbers of students use AI to help with part or all of the assessment. 

Discussion on one challenge/question and one opportunity or possibility. 

A quick wrap-up closed the session. In summary, students will use AI anyway, it is important to provide guidance as at the moment, it is very hit and miss. 

NZ Vocational Education Research Forum - Day 2

Another busy day at the conference :)

Greg Durkin, Director of the Building and Construction Industry Training Organisation (BCITO) opens the day. Greg began with an appropriate whakatuaki (Māori proverb). He summarised the contribution of VET to supporting and progressing the Aotearoa NZ economy and the need for parity of esteem and recognition of qualification levels for all disciplines. 

The day's first keynote is with Honourable Erica Stanford, Minister of Education. Detailed the work being undertaken to review the education system to increase equity, parity of outcomes, and lifting the standards of school-based education. Teachers are being provided with development to enact structured literacy and numeracy. By the time students complete their schooling, their literacy and numeracy will support their further educational journey along with their lifelong learning. National qualifications also need reform and require review as well. There is a lack of consistency and a lowering of the bar of the entry into tertiary education. Therefore, standards in literacy and numeracy require focus to ensure all NZers are set up for future success. Vocational and educational pathways need to be on par. A National qualification needs to have industry-led and educational that have parity. 

Encouraged the need for industry to be engaged to ensure that the curriculum has the knowledge and skills required to young people as they leave school. The Industry Skills Boards (ISBs) will need to play a key role in connecting the different workflows. Curriculum reform to a knowledge-based and industry-led will be a key. Equal value is assigned to these so that it increases the parity between vocational and academic. 

Then the second keynote is with Craig Robertson, CEO of the Victorian Skills Authority. He presented on 'The vocational renaissance (with some international reflections)'. Began with a overview of the VET landscape in Victoria / Australia and the history of the TAFE sector's ups and downs along with the ways VET is structured (stand-alone / independent, federation model and centralised). Then discussed the similarities and differences between Australia and Aotearoa NZ. 

VET in both countries inherited from the UK and still follows this model still. Reiterated the importance of policies, government roles on how education systems affect the country's economic and social aspirations. Used fireworks as an example of a linear process and digitisation which is much more flexible in the way the data can be reconfigured to create many variations, Summarised industry 4.0, examples and its impact on VET. The historical impact on jobs shows that males with lower education have had relative wage decline. Society has had a narrow, one-dimensional view of human talent and privileged cogntiive/white collar work. The cost is that there when AI arrives, we are unprepared for its competition. Work is more than cognitive. Summarised the changing nature of work with the rise of the new technician. Division of labour from the late 1990s is now shifting with there being less managerial and intermediate/coordinating jobs. Small businesses do not fit the larger organisational companies. The new worker needs to have a broad occupationally oriented skills, coupled with a sound understanding of the relevant theoretical principles. Check Lewis (2025)  innovation, tehnician and VET in JVET. 

Introduced the work of Jobs and Skills Australia's report of the impact of AI on work. Compared the % workforce potentially exposed to automation and those to augmentation by AI. AI will have most impact on cognitive work but less so on VET type work. VET workers will rise to the fore as a choice for sustainable careers. Shared a report from Vietnam of the effect of technology as an example, Critical thinking, written/verbal communication, numeracy, complex problem solving, etc are much higher in necessary than basic digital skills. AI is set to increase GDP due to its potential to augment human capabilities. It is important to think beyond a linear approach to training. Need to think about new skills with many 'programming/writing' jobs compared to physical labour. Focus needs to be put on science and thinking skills (i.e. the critical technical skills - VET). 

Summarised how various countries deal with the above. England has continued reviewed qualifications  - e.g. O, A, T, V levels in an attempt to cater to the large range of students leaving school. NZ moving from unit standards to skills standard. VET in schools moving to skills standards and equal volume of learning. focus on competence, provide flexibility and assessment and portability of learning. Australia moving into purpose-driven qualifications - type 1 competency to occupation; type 2 - competency to industry; type 3 - application of skills and knowledge driven by learning outcomes. Implementation of this concept has been a challenge thus far. Recommended to read Mark Carney's book 'and economist's to guide everything that matters'. 

 Sessions continue. 

First up, Greg Durkin and Phoebe Gill (Scarlatti) present on a ConCOVE supported project  'The $750 million question: who really pays for apprenticeships? Began by introduction of the rationale for the project. In the construction/infrastructure context, industry, government and taxpayers want a highly training workforce, better quality housing and buildings and who is doing the investing in training. Asked 'do you think employers spend more on wages or time spent on support? How much time does an employer spend on support per apprentices per month? What extra costs employers cover for their apprentices? 

The survey undertaken with 317 BCITO employers, industry specific pates online and modelling and extrapolation. Five categories of employer contribution - annual wages and block course fees; wages during block course; development support; time for pastoral care and engaging with provider. Interpretation complicated as there are many ways to define support, employers may not actively measure this, some could happen without formal training, no ROI, benefits and the study is selective pool of employers.

On average $28,000 per apprentice (range from 10000 to 70000). The vast majority is time put into pastoral care and provider engagement. These include one-on-one check-ins, career guidance, buddy system etc. Around 17 hours a month across supervisors, peers and apprentices. A bit of time saving if there are multiple apprentices. Time mostly in meetings with training advisors, reviewing workbooks, and completing evidence. Average of 10.5 hours from employers/supervisor and apprentices with BCITO. $$ support mostly for consumables, tool allowances, transport, meals/drinks and counselling services. Therefore, estimation of $100 per apprentices per month - which excludes consumables and tool allowances. 2/3 accessed subsidies to support block courses (fees and apprentice wages). Common to pay annual and block course fees, some split the costs, in general it was an all or nothing approach though. 90% paid apprentices for attending block course.

Most of the $$ is in the form of time ($637 million) but these are engaged employers and there will be wider variants across the construction industry. For BCITO/earnlearn apprentices (22,000apprentices) $659 million and Connexis/earnlearn (3,600) $104 million. What the employer gets back is more productive workers, return of 3x to 10x on the $$ /time put in. Well trained employees are crucial to a successful business and they can only import them, steal them or create them through apprenticeship. 

Then Dr. Mark Bradford, Claire Le Grice and Fiona Melrose from Skills Group/ Institute on ' ideas that RIPPL: a system level practical approach for disseminating research and innovation. Change is challenging, so how can people thrive through change? Fiona began with an introduction. The team work at Skills Group which exists within a wider sector context of reform, consolidation and disruption. RIPPL is a system level reproach that connects research, teaching and professional learning. Research belongs to everyone in the organisation. Uses a community of practice framework to bring together teams and to help socially construct solutions. The process needs to be psychologically safe so that people feel safe to experiment without fear of judgement, learning and creativity accelerate (Edmondson, 2019). 

Mark overviewed the background and ways RIPPL was developed. Used ZEME as a case study of the application to increase creativity and innovation. Ran 34 workshops and generated 90 pieces of work to disseminate what was learnt. 

Fiona continued with examples on how research can be empowering for many people. Research can be undertaken by everyone. Claire continued with details of the process. Especially how to engage people in the process - trust is important - opt-in freely, keep it playful, experiment with joy, honour the clock. 

Plenary from Dr. Michael Johnston from the NZ Initiative on 'vocational subjects for senior secondary school: What will it take to get it right? ' from his work on 'trade routes: charting new pathways from secondary school to industry training. Shared data of activity for 16-19 year olds and 20-24 years. In general, at 16 - 19 60.1% still at school, 10.9 in degree and 10% involved in workplace learning, dip study, certificates 1 -4 and workplace based training 11.3% are NEETs. By 20-24 most have left school, 23 % in university 55.7% in employment, 10% in via and 13% still as NEETs!

Schools primarily geared to teaching the NZ curriculum which morphs into a track to university at around year 10. We do not want to have a sieving mechanism earlier (as per countries in Europe/Asia). There is no equivalent of university entrance for VET. Limited basis for young people to make informed decisions (pathways and earning potential important). Culture seeing VET as a second rate option. A step in the right direction starting with the current change to the school curriculum. The NZ Certificate and Advanced Certificate of Education will replace NCEA in 2029 and 2030. Assessment for vocational subjects is likely to be through industry skills standards (replacing unit standards). Each subject can be split in two to total to 40 credits. These will either provide the NZCE with 40 credits in vocational learning or be used to provide students with an opportunity to try out a few industries.

Skill standards tend to have a workplace-based assessment, This may be difficult for schools to organise. Advantages of skill standards have potential to escape the stigma attached to unit standards (atomised, disintegrated); common requirements in skill standards as they need to be conducted in the workplace; 'indicate content' could facilitate curriculum development; and partially completed vocational subjects can be ported into tertiary institutions. 

Challenges of skills standards include common requirement to be conducted in the workplace, intention to report on 100-point scale with overall grades for each subject; and higher grades at the subject level will be needed to establish parity of esteem. Therefore, there will be higher grades for subjects (rather than standards); criteria for skill standards should represent as many levels of performance as assessors can reliability discriminate; graduate profiles for level 1 and 2 qualifications could be a good place to start; and purpose-built standards may be more coherent than using existing ones. Contextual considerations include that work-integrated learning opportunities will be essential; students should often be expected to transition during the school year; dual enrolments should be commonplace; VET must become business-as-usual for schools; and ISBs must be in it for the long haul. Schools need a better integrated system as currently they are independent entities and this has let to too many variations. 

After lunch, a session with Dee Earle, Principal research analyst from the Ministry of Education on 'from school to skilled work: monitoring the impact of Gateway and Trades academies'. This is based on a recent report. Gateway started 2001 and trade academies in 2010. Summarised the objectives of each. in 2024, 15,455 in 380 participated in Gateway and 11515 students in 24 trades academies across 402 schools. Both increase student retention, raise NCEA level 2 achievement and support transition into further education and employment. 17% of students participated in both.

Followed 2014 cohort - 16 then and 24 in 2023; and 2018 cohort who are 20 in 2023. Included students that took part and did not complete and students who did both.

Limitations were summarised - what it does and what it does not. Evaluated if they completed school/tertiary qualification, ever been an apprentice and in employment.

Findings are: similar outcomes for Gateway and Trades academies; most stayed in education at 17 and achieved level 2 or higher by 18; men likely to enrol in apprenticeships but low for women; men in employment, positive benefits, Covid 19 resilience identified, and trade academy effectiveness maintained even when numbers participating increased in 2019.

Both programmes improved outcomes for participants. Pathways into VET stronger for men, smaller effect for Māori and Pacific and little to no effects for women. Employment outcomes revealed strong effects for men and than women. Smaller effects for Māori and Pacific. Q & A followed.

A panel convenes to discuss the 'future of VET research'. 

On the panel are Arthur Graves and Katherine Hall with Josh Williams as moderator. Katherine shared a vision for centre of excellence Summarised the challenges laid out for VET, completions, funding, qualifications and skills. Large number of workers needed to support infrastructure but training plays an important way. Whether the changes to VET / school curriculum is able to deliver the skilled/trained workers required. There is already research out there and we need to draw on these to inform the way forward. 

Arthur shared the food and fibre centre CoVE story - there is value in applied research. F & F had beginnings through collaboration - government agencies, providers, industry. The CoVE was positioned to be industry-led, government funded and learner focused. The value proposition was to invest in transforming of the talent pool and growth of the food and fibre sector. 

Summarised some of the projects undertaken. Discussed the challenges of implementation. Funding in research has been funded through CoVEs, Ako Aotearoa which are external to where the challenges occur. Collaboration is still required to apply research towards solutions. 

What does CoVE mean? Katherine shared the vision of how to reimagine the centre of excellence. Innovation Lab (design) informs education (future proofing) and resources (to benefit industry). Also can be used to support advocacy plus feed back into think tank (research) to continue the process. 

Arthur discussed the need for an interface between school/tertiary education/industry rather than to think about the process as transition. Making progression and going on a journey are not transitions. They are inter-related and should be a progression. A structured interface (from 16 - 19) needs to be better organised. Dual enrolments are possible through legislation etc. but leadership is required. Dual enrolment needs to be explicit and be a standard rather than an option. 

Katherine was asked to select 3 ConCOVE projects. Difficult but degree apprenticeships, Tui Tuia, and the 'bystander series' were the ones that she selected.

What can we do as a sector that can be done to support future VET research? Use the opportunities for the secondary/tertiary system change. Slido pol followed to collect perspectives on the forum and to gauge appetite for another forum next year. 

NZ VET research forum - Day One AFTERNOON

The conference after lunch convenes for plenary address with Madeline Newman from AI Forum NZ on 'designing tomorrow: human futures in an AI world'. Introduced the AI forum, its goals and people, shared some of their work including working groups, summit, hackathons, key reports and white papers etc. Summarised the work of the AI blueprint for Aotearoa and the AI and architecture engineering and construction, AI in education. AI revolutionalise learning through personlising educational paths, efficient training and feedback and expanded global access. Also AI powered VR.

AI and creative industries summit brings global conversations to Aotearoa. Creatives partner with technology, with boosts in productivity. Human creativity along with emotional resonance important. AI can democratise creative work and raises the bar for professionals. Using robots as an example, 52 studies published since 2017 covering over 2500 individuals found that robots have a no effect on wages. Routine cognitive or physical tasks have diminished but those requiring creativity have increased. Highest ROI is where humans work in partnership with AI. Ai in Action report has latest stats on AI use and adoption, If AI frees up time, what do you do with it? 

AI governance - responsible use of AI just refreshed. Upskilling requires understanding how to use AI safely. Therefore, key items to do include: AI literacy, digital and AI access for all, responsible use - understand the risks and use AI safely, and ensure human centred design. Introduced advances in robotics and the implications on society and work. Uncertainties include the legal, cultural and public acceptance and readiness. Q & A followed. 

Then Dr. Dmitry Zavialov from the Skills group and Dr. Roman Mitch, on 'Aura farming: cultivating irreplaceable human leadership in the age of AI'. How can we bring across our strengths without replacing our humaness. Skills that can be replicated by AI means that we need to enhance who we are and what we are about. Educators need to create environments that help them increase the attitudinal aspects of humans so that these can be focused on. Resilience, working in teams, working across cultures etc. are examples. Roman shared example of how to teach technology when it is moving faster than we can keep up with it. The principle of what makes us essentially human, Shared work applying these principles. How artists use AI provide examples of how we can go forward. Therefore, programming can be thought of a an art form. 

Three plenary sessions follow.

First up with Emily Fabling Deputy Chief Executive at  NZQA on - Right touch, right size regulation: quality assurance with purpose. NZQA is about supporting learners to succeed and industry to be represented. Overvies of NZ education; what does right touch right size mean, NZQA framework and examples. Over 400 tertiary providers, 92% are PTEs. Universities have 44% of learners, Te Pūkenga 29%, Wananga 9% and PTEs 18% - 57% VET. Changes being made include: quality assurance across all (apart from universities), maintain framework, maintain international comarabiliy and effective liaison with overseas bodies to recognise overseas qualifications in NZ and to achieve overseas recognition of NZ qualifications. NZQA registers PTEs, approves programmes, qualifications, micro-credentials and standards, quality assuring provision of education and training, reviewing and updating NZQF and liasiing with overseas recognition bodies. Rules, norms and sanctions shapes lives of NZers and NZQA does the same for education businesses organisations, learners and communities. regulation can impose costs, limit freedoms, stifle innovation but these need to be considered.

Considered the importance of quality as poor quality affects all. Shared the recently reviewed NZQA regulatory framework principles - impact-led, anticipatory and adaptable. Regulations should support delivery and not be a barrier to collaboration or innovation. The evaluative quality assurance framework is now 16 years old. Based around entry controls and external evaluation and review. Additional activities have been added. However, VET has to been move more quickly and the EQAF needs to be reworked. Productivity Commission report in 2017 recommended NZQA should ensure all providers meet acceptable standards; risk-based monitoring of providers; and ensure providers have their own processes to assess and improve performance for their learners. 

Summarised the new integrated quality assurance framework (iQAF).  Diagram of the framework provides details, A good overview of the more collaborative approach. Quality assurance with a purpose shared - aviation and pilot training (case of recent regulatory intervention of provider); construction (statutory conditions imposed on two providers); and Pacific qualifications framework (endorse and to allow these to be referenced against NZQF).

I then present on 'Gen AI and trades/craft work: implications and effects. The overall theme is that AI in the context of trades/craft work is different in the way it will be used, and the interactions are less text dominant. There is a need to understand how AI changes the nature of trades work, as it will be used to augment both physical and cognitive work. 

The NZVET research forum awarded me for having presented at all of the forums thus far!!

Final plenary for the day is with Katherine Hall from ConCOVE Tūhura on 'reflections on a five-year experiment: the ConCoVE legacy. Encouraged to look at Sir Paul Callaghans presentation in 2007 on future for NZ. We do not seem to have got through many of the recommendations :(

Katherine summarised the genesis of ConCOVE and it's vision and mission. It sought to challenge the current ways things were done and to seek to shift it towards greater equity and inclusion. A systems change view was adopted, meant the need to find out what was occurring and to apply it to improving. Many projects were commissioned but they were all part of the greater objective of shining the light on what has happened, what could be and how to get these in place. Summarised some of the findings and how these had been used, reworked, or reimagined. 'the system is perfectly designed to get the results it gets', Shared recent stakeholder survey supporting the work undertaken and the ways ConCoVE has supported the construction/infrastructure industries. Effectiveness of ConCOVE supported. Call for the participants to keep the work visible, embed the work into delivery, preserve the kaupapa (work/knowledge). At a system level we need stronger advocacy, faster qualification design and delivery mechanisms, business skills.disrupt the narrative, challenge the system , take the evidence and apply it. 

The day ends with a networking event, taking everyone well into the evening. 

NZVET Research forum - Day 1 MORNING

 The Vocational Education and Training (VET) Research Forum has been reinstated and runs today and tomorrow at Te Papa (the national museum) in Wellington.

The Forum is opened with karakia and mihi from Building and Construction ITO (BCITO). 

Josh Williams (who MCed the event) welcomed the conference participants and ran through housekeeping and health-safety along with a ice-breaking session and a introduction to the conference, its purposes and the journey following, including that the system has always centred around people. 

Jane Kennelly, Director of People, marketing and stakeholder management at the Skills Group undertook the formal welcome and introduced the Minister. She celebrates the coming back of the forum after five years and summarises the many advantage afforded by undertaking research and its application to practice.

The first keynote is with the Honourable Penny Simmons, Minister of Vocational Education. Presented a positive approach to the future of VET in NZ. Reiterated the importance of strong pathways from school, both into VET and higher education. Supported that partnerships across the many players in the system and that these need to be well-connected. Regional challenges for youth unemployment needs to be focused on, apprenticeship completions need to be increased (from under 50%) and also strong alignment with industry important, Centralisation does not meet regional and industry needs. Therefore, the shift away from it will ensure better future progress. Reiterated the changes being undertaken to reinstitute polytechnics, and establish Industry Standards Boards (ISBs). This puts decision making back to the regions. Industry driven, locally led and regionally informed. $20 million in 2026 and 2027 to ensure that access to VET continues, especially in regions with high NEETs. Research needs to be relevant, application and informed by industry. Time for Q & A. Research funding possible through the new 'PBRF' (perhaps 9 million) and through ISBs (being worked through). Financial stability across the sector key to moving VET forward. 

Following on is a keynote with Katryn Rowan (Executive Director) for the Global Apprenticeship Network (GAN). Strengthening connections between youth and employer - Tomorrow's future-ready workforce. Summarised the vision and mission of GAN - to champion quality apprenticeships as a key driver for workforce development, social mobility and economic growth. Has been advancing apprenticeships for 10 years with 10 networks around the world along with key corporate members and partnerships with ILO, OECD, IOE etc. Presented on the rationale for GAN - too many young people leave school without clear paths, persistent bias towards university, employers struggle to find skilled talent and apprenticeships help increase inclusion, social justice and growth. NZ snapshot as someone coming in from the outside. NZ ranks 3rd in OECD in field of study mismatch. 30% of school leavers go to university. 60% lack a structured post-school pathways. Performance focused on course completions more than career outcomes. Upcoming NZ 'industry-led' secondary subject is a positive move. Then overviewed the international global challenge. 13% of youth employment (x 3.5 of adults); 261 million NEETs; 75% of employers report difficulty filling roles; 63% say skill gaps are a major barrier to growth; and 39% of skills will change by 2030. 

Shared that the consensus of the way forward to be the benefits to individuals and companies of work-based learning, clearer pathways and VET. ROI of apprenticeships proven across many case studies. Partnerships are important. Private sector/ businesses crucial. They see emerging skills first; provide real-world 'classrooms'; and bring structure and innovation. Advocated for a 'skills-first' focus, not just on qualifications. Shared the GAN corporate champions for apprentices (CCA) who are companies committed to supporting quality apprenticeships. Shared case studies from L'Oreal, Nestlē and Tassaroli S. A. in Mendoza. Key items include shared curriculum and involvement of key /crucial stakeholders for skills design and forecasting, shared training assets, flexible and targeted WBL, career guidance and early exposure to choices, SME engagement, data and governance. Vision, collaboration and results help provide direction going forward. 

After morning tea, a series of sessions organised into two streams.

Dr. Gemma Piercy from University of Waikato on 'where to next for lifelong learning in Aotearoa NZ'. Asked what has happened to the term and practice of lifelong learning? Is the policy focus on front-end education investment a barrier to lifelong learning? Lifelong learning - orientation to learn over the lifetime, acknowledging portfolio careers requiring continued learning.

NZers have increased completion of qualifications, with small sector of population with no qualifications (2023 - 12%). Lifelong learning became important through 1990s with 1/3 of workforce with no qualifications. Industry training increased. Tertiary sector - student loans, skills training tied to completions etc. Do we still depend on the market model - as this may no longer be fit for purpose. 2000's pivot towards 'front-end' loading with state support for short qualifications. 2010's tightened eligibility/ incentives. E.g. no student allowances/ loans to older NZers, late-life transitions harder to fund. 

Lifelong learning still needed due to rapid tech change, second/third chance need rising, capacity gaps in current policy levers, literacy issues identified amongst school age students before Covid, increase in precarious work and portfolios careers remain. What is the right mix between 'front-end' and balanced systems. There is a need to design policy for continuous upskilling and transition. Proposed restoration of genuine lifelong access. 

Then Camilla Karenhana from Ringa Ora on their project ' Tirohia ki tua - the impact of Māori in the service sector. Told the story of growing up through various economic changes and how this connects with the work being undertaken. In the community services sector, important to understand the impacts economic consequences on communities and individuals. Deficit view needs to be flipped. The ecosystem of how they work is through building relationships. Historical issues of mistrust need to be worked through before progress can be made. Relationships need to be genuine, build on trusts and not just on outcomes. Wānanga take place which are to listen, be open, identify what can be done, and be about them. Tuia was to connect Māori across industries. 

Went through the resource  (Tuia 2025) developed and how this can be utilised. 

Lunch followed to recharge after a busy morning :) Good to be back with kindred spirits.