Tuesday, October 28, 2025

Overview of using various AIs - Professor Ethan Mollick

With so many AI choices out there, it is difficult to select one which will effectively support the task at hand. Professor Ethan Mollick who was a very early adopter of Gen AI in the higher education sector, provides an overview of AI as they stand near the end of 2025. 

The blog compares Gemini, Claude and ChatGPT and their uses in an academic context. The various categories of chat, agent and wizard models are compared. Inputs of text, voice and images along with outcome in images, video, code and documents are also discussed and evaluated.

As always, there is not one Gen AI platform which will 'do it all' or do all well. Therefore, it is still important to match the objective / end goal, with the Gen AI. As Gen AI develops rapidly, it is also important to keep up with the play. For example, we have been encouraged to use CoPilot at my institute. To start with, CoPilot did not really compete well with the other mainstays - ChatGPT, Claude and Gemini. However, CoPilot has improved markedly and is now integrated into the many Microsoft tools we use daily. The M365 version of CoPilot provides ability to create agents and also a range of agents which are useful for common everyday tasks. Paying extra includes the research and analyst agents. These two agents are extremely useful not only for research but for general tasks requiring inquiry, comparisons and deeper evaluation. Therefore, it is important to keep up with the play!! 




Friday, October 24, 2025

Aotearoa Tertiary AI network ATAIN - presentation from Dr. Simon McCallum

 Notes taken from presentation from Dr. Simon McCallum, Victorial University Wellington on 'Adapting to AI'.

The presentation is part of a fortnightly series organised through ATAIN which is a SIG of Flexible Learning Association of  NZ (FLANZ).

Simon began with an introduction. He has been teaching game development since 2004 but has also taught AI since 1991. Noted that Gen Ai is everywhere and we use it unintentionally, unconsciously, but also using it consciously and strategically. Productivity benefits depends on training of the AI. Ai agents work together to automate generic tasks.

Across industries, adoption is mixed, some fast, some very slow. The risks include programmers with AI automating other industries and the use of 'software / automation on demand'. 

Revised the two lane approach to assessments. Students are using AI - At UVW 66% admit using it.

Covered the following:

 AI literacy - Core / domain specific - compulsory for all students and staff, understand if and when to use AI and avoid the risk of thoughtless AI use.

Assessments need to move to testing understanding and learning rather then outputs. Test meta cognition, use oral assessments.

All non-invigilated (lane 2) work should be considered as group work, using group work assessment techniques - assess process, influence, delta, learning journey. 

Assessments can embrace AI assistance. AI selects questions, create bespoke questions, suggest oral assessment questions. Human markers determine the grade. Provided details of the process from his context. Use AI to generate questions from student submission for them to complete, to check understanding they have presented in their essay!

Increase the quantity of group/team work so students can create human connections and increase their experiences with working with others. Invrease the amount of work that is groupwork but not the groupmark.

Proposed that NZ universities fund a NZ based server to assure AI sovereignty. A more equitable approach as all students will have access to high quality models instead of having to pay for the upgraded models. 

Also the creation of a position to report to Te Hiwa (leadership group to organise and manage AI across the entire university - teaching, learning, research and professional. 

Summarised agentic AI. Moves AI from being a chatbot to actually being able to 'do things'. AI will sort out a plan, and work through it to meet the prompt objective. Swarm coding can be activated, to check the outputs from each agent. Therefore, AI is not a search engine. It is better to have the AI question us to work out what we want done! 'help me do ----' 

Summarised project on how AI is used in NZ secondary schools. Mixed across schools on strategy, current use, professional / student use and community involvement. Schools welcome clearer policies and guidelines. Challenges are similar to Universities, assessments, professional development, etc.

Shifted to a summary of the extrinsic and intrinsic purposes of learning. If the motivation is just to pass the exam, then AI is an impediment. However, if there is a less constricted 'assessment' e.g. develop a game, AI accelerated capability and leads to extended learning. 

Therefore important to engage learnings to focus on intrinsic motivation, self-reflection and etc. and to hold them to account for what they want to  learn. Teacher as NPC . Encouraged learner negotiated assessments and rubrics, giving them agency. For example, have learners establish the range of marks assigned to various aspects of an assessment. This encourages meta cognition for students to structure their learning, their strengths/ weaknesses etc. 

How do we measure metacognition - confidence is good when it is accurate, under / over-confidence is a problem and AI makes this worse. Important to operationalise and elicit accurate statements from learners.

Invitation to use the group's Discord to continue the conversation and share ideas. 






Thursday, October 23, 2025

AI-generated assessments for vocational education and training - webinar

 Here are notes from the webinar on the ConCove Tūhura project AI-generated assessments for VET

The report provides the literature scan and details of the process undertaken to identify appropriate AI to undertake the task, and the processes to ensure that the AI- generated assessments would meet moderation requirements (quality assurance) for use for assessing VET standards. 

The work was undertaken by Stuart Martin from George Angus Consulting and Karl Hartley from Epic Learning. Both present in the webinar which begins with an introduction by Katherine Hall (CE for ConCoVE Tūhura) and by Eve Price (project manager at ConCoVE).

In Katherine's introduction, the rationale for the project was shared along with some of the journey taken by the project to break new ground.

Eve Price provided the background of the project. Most projects focus on integrating AI into ako or the prevention of AI for assessment. This project wanted to help support the time consuming 'back room' processes including resource and assessment development.

Karl ran through the approaches to the product. The evaluation/review processes could not really keep up with the speed at which assessments can be developed when it is supported by AI. 

Stuart shared reflections on how the process evolved and the various processes put in place, were reflected on and were then reintroduced into the AI-generation project. Explained how various quality pointers were met to ensure the efficacy of the process.

Eve detailed the need to be specific with what needed to be achieved - assessment, feedback, etc. Selection the correct AI is also important. Prompts are detailed in the project report. Important to evaluate at each step.

The bigger picture with micro-credentials, skills standards and AI-generated assessments all add innovations to the VET ecosystem. Understanding the policies and processes used by WDCs and NZQA need to always be part of the process, so that various quality points are met.

Stuart summarised some of the challenges and how the project worked through these. 

Karl talked on the importance of people in the process when AI is generating the assessments. Firstly, important to understand some of the mechanics of AI - what is under the hood. Secondly, quality assurance must be focused on the concepts, not so much the grammar/spelling etc. Thirdly, need to make sure assessment purpose is clear. 

Next, academic integrity and ethics were discussed. Important to ensure that there is understanding the impact of AI on privacy and data sovereignty (including indigenous perspectives). Important to train the AI to understand tieh Aotearoa context. Claude AI was selected due to its stance on human rights, ethics etc. 

Findings included: assessments did not meet moderation but improved the opportunities for inclusiveness and personalisation of learning. Failing moderation added to the learnings from the project. The items involved too many questions, answers being at too long and at too high a level. 

Eve reiterated the need to 'define what good looks like' to the AI, so that human objectives/ perspectives are taken into account. Important to ensure principles of ethics etc are maintained as it is important to 'keep humans at the centre'.

Karl's learning include AI drawing in novel content through its hallucination. The AI included assessor approaches into its assessment and this caused him to consider the learner information that should be included to provide direction. The U S of A standardised approaches to writing assessments, seemed to permeate the assessments produced by AI. This had to be superseded through careful prompting.

Flexibility to allow for personalisation to industry (example safety unit standard customised to a range of work roles/ disciplines); and learners (for ESOL, neurodiverse learners etc.). 

 Q & A followed 

The webinar was recorded. 

Discussions revolved around practicalities, challenges and solutions.

All in, good sharing that adds to everyone's learning about the roles of AI to support teaching and learning, integration of practice/practical and cultural contexts, the need to be aware of the fish hooks' in using AI, how quickly AI is developing to meet user needs, and the need to continually learn to ensure that the understanding of AI / ethics etc. form the foundation for working with AI. 


Monday, October 20, 2025

AI forum productivity report for New Zealand

This report - AI in Action: Exploring the impact of AI on NZ's productivity, is produced by the AI Forum NZ in partnership with Victoria University Wellington and PR powered by heft.

It is the third biannual report and collates an overview of the impact AI is having on productivity across NZ. Since the first report in 2023, there has been growth in the use of AI with accompanying effects on work, the workforce and contributions to the economy. 

The 3 page executive summary provides the main points. Key findings are then extended and discusses followed on by case studies.

In the businesses surveyed, 91% reported productivity gains, 50% view AI contributes to cost savings with 77% saying that there have been actual cost savings. However, less than 25% had savings over $50,000. Therefore consistent productivity gains.

Workforce impacts include increased job losses which reflect the country being in recession; 55% reported that new roles have been created; cost of setting up AI have reduced, strategic policy investments have been attained; operationally, AI cost less. 

AI's introduction requires the building of trust across the workforce with AI literacy being a key and the need to ensure that there is inclusive engagement for all.

Overall, data that reports on growing adoption, settling in of organisations into understanding how AI can be leveraged to increase efficiencies, and acceptance of AI as inevitable part of current and future business activity. 




Monday, October 13, 2025

Assignments in the AI era

 In light of this article from Radio NZ, whereby some universities in Aotearoa are no longer checking assessments using AI tracking platforms, a summary of ways to think about assessments in the AI age is of importance. There has been much discussion on how assessments in higher education need to be evaluated and re-thought, given the infiltration of AI into our work and study. This article in Times Higher Education, distills many of the main discussion points in academia on how AI affects academic writing.

The work undertaken at my institute is focused around holistic / programme wide assessment design, rather than on individual courses. The term 'programmatic assessments' is sometimes used to describe this approach

Some of the other strategies we have used, are summarised in this blog - NavigateAI (Dr. Ryan Baltrip)  In summary, to place greater weighting on recording the evidence of learning, rather than the product of learning. Therefore, portfolios and similar assessments are more useful than one off invigilated exams, or assignments. 

In Aotearoa, Otago Polytechnic's Bruno Balducci, have introduced the concept of AI safe design, a framework for the design of assessments which take into account the influences of AI. These are useful as a way to help educators work through the many pitfalls involved in redesigning assessments that will be authentic and relevant, but will not tempt learners into using AI to complete them.

The other concept we have used to help our teachers work out how to structure assessments in an AI age is the 'two lanes' assessment structure.  Here, lane 1 assessments are used to as assessments OF learning - or summative, higher stakes assessments. Lane two are the assessments FOR learning, taking on formative approaches to inform learners as they progress to the course.

Therefore, it is important to not just assume that current assessments will be appropriate but to undertake a stock take to understand the purposes of each assessment, and to put in place relevant assessments that will meet the purposes of each assessment i.e. evidence that the learner has met learning outcomes.  








Monday, October 06, 2025

Guide to using AI - school context

 Here is a useful guide for AI (in schools / US of A context). 

The guide begins with a section on how to use and why to use the guide.

The second section focuses on ethical issues - ensuring this is at the very front of any consideration for the use /integration of AI into teaching and learning. 

Discussions on the impact on students', risk and benefits and teacher perspectives follow.

The guide towards determining AI policies is then introduced and discussed. The 'how to create an AI policy' section is useful, drawing on key principles and providing suggestions. The checklist for developing AI policies (page 18) sets out the many parameters that need to be thought through as AI is introduced into the school curriculum.

A series of case studies and discussion pieces follow, documenting the struggles, challenges and pragmatic approaches adopted along with detailing various strategies and approaches. Discussions revolve around why, how, when and implications for introducing and using AI in schools. Strategies for assessments in the AI age are summarised (pages 30 -31) including the need to design engaging assessments, using paper based materials, having in-class assignments and assessments, adding oral assessments, emphasising the learning process, helping students understand the implications of using AI, clearly spelling out what is and what is not acceptable when using AI, and more frequent low stakes assignments.

A range of curated resources are provided for follow up and reference.

All in, a realistic documentation of how AI impacts on day to day school systems and environments. The pros and cons are drawn from case studies. The teacher voice comes through well and their perspectives and experiences are valued. Principles derived are relevant across educational sectors. 




Monday, September 29, 2025

Horizon Report - 2025

The 2025 Horizon report on Teaching and Learning was published earlier this year. 

As with previous reports, the horizon scan included summarisations of the social, technological, economic, enviromental and political trends which impinge on the the future of education.

The topics covered included :

AI tools for T & L; 'faculty' development for Gen AI; AI governance; cybersecurity; evolving teaching practice; and critical digital literacy. Therefore, this report is 'AI' dominated with the call for 'critical digital literacy' as a requirement for all, to ensure that AI utilisation and integration is grounded in ethical principles.

The case studies provided are mostly North American but there are a few case studies from Oz as well.

The report closed with discussion on the scenarios for growth (AI and VR shift education into the virtual but equitable access is still a challenge), constraint (caused by government regulation to draw on learning analytics for decision making but limits access to many), collapse (unregulated AI leads to collapse of truth) and transformation ( emphasis on workforce readiness at the expense of liberal arts education), although a mostly positive/optimistic stance is maintained. 




Friday, September 26, 2025

Enacting assessment reform in a time of AI - Tertiary and Standards Quality Agency (Australia)

 The Australian Tertiary and Standards Quality Agency (TESQA) has published a report to provide guidance on assessment reform in the age of AI. 

The main approaches are:

- taking a programme wide approach to assessment reform i.e. across the entire degree programme

- assuring learning in every "unit/subject"- i.e. across a course

- implementing a combination of the above - by ensuring assessment mapping is constructively aligned between learning outcomes and assessments. 

The strengths and challenges of each of the above are presented and discussed. 

Monday, September 22, 2025

How people use ChatGPT - report from Open AI, Duke University and Harvard University

A report on how people use ChatGPT provides for interesting reading. The authors are from OpenAI (the company that owns ChatGPT), Duke University and Harvard University. The report was published in mid-September 2025 and draws on data of usage between May 2024 to June 2025.

Section 5.2 has a table that details the topics people use ChatGPT for - writing, practical guidance, technical help, multimedia, seeking information, self-expression and others.

Section 5.4 summarises work activities undertaken using ChatGPT. These were mapped to the American O*NET. Almost 1/2 were to get information, interpret the information from others, and documenting/recording information. 

Of interest is that users who are in highly paid professional and technical occupations (and also more educated) are more likely to use ChatGPT for work. Therefore, there is a pre-requisite level of literacy required to get the most out of ChatGPT. 



Monday, September 15, 2025

AI teaching assistants

 As we embark on the next phase of our AI project, building Gen AI agents to support our learners, a look at what is out there is important. This site by Educators Technology (published June 2025) has a lists of top AI teaching assistants. 

The site provides a definition of AI teaching assistant as a tool to help with lesson planning, grading, content creation, student feedback, and classroom management. Routine work including real-time support to learners, instruction through automation, personalisation of content and conversational interactions with learners can also be supported.

24 teaching assistants, some of which are able to do many of the items listed above, and some more specialised ones are listed. Their functions are summarised. Many require payment. Some tech companies are leveraging technology with the inclusion of AI into established models, others are new and many perhaps more suited to the school sector (and the U S of A) rather than higher or vocational education. Most have no links, so a search is required to find them. 

The discussion on AI teaching assistants needs to always consider their efficacy and whether they will replace human teachers. A June interview by Business Insider with Sal Khan, founder of Khan Academy; A radio NZ overview from late last year; and an August post from Education Week (US of A context), all indicate that teacher relationships and presence are crucial to good teaching. What AI TAs can do, is provide 24/7 support on FAQs, along with providing a place for students to undertake practice or continued learning. 

Therefore, using AI TA's needs to be carefully thought through. They should not replace teachers, but support teaching and learning. Identifying the components of the AI TAs role is a crucial step in making sure that learning objectives are met. Interconnecting the AI TAs role to that of the teacher, and making this visible to learners, is also important. Therefore, one aspect of AI literacy is to be able to understand how, when and why AI is used and what is the role of humans when AI takes on various roles.                                                                        


Monday, September 08, 2025

AI and the future of education - UNESCO report

This UNESCO report on AI and the future of education consists of a collection of short articles from a range of academics. The report has 8 sections, following a introductory chapter on ' reclaiming education's public purpose through dialogue'. This chapter sets the scene and focus of the report, bringing in the humanistic emphasis required to meet the challenges and disruptions caused by the rapid arrival and uptake of AI. Chapter summaries are also presented. The full text is available on the website with hard copies available on request from UNESCO.

Following on, the sections are:

- AI futures in education - Philosophical provocations - has 3 chapters. As the section title implies, these discuss the implications of AI on education.

The first chapter by B. Akomolafe stressed the need to go beyond using AI as a tool but as something that will shift both the ontological and the epistemological foundations of education. 

- Debating the powers and perils of AI - with 3 chapters.

Both chapters - the first by A. Horn and the second by E.M. Bender, focus on the importance of teachers work and the need to ensure that there is investment in teacher education / professional development and to support their work with adequate and clear policies to guide their practice. 

- AI pedagogies, assessment and emerging educational futures - 5 relevant chapters.

All the chapter in this section, are useful for reading for all educators. A. Birhane begins with laying out the limits and and inherent dangers underlying the integration of AI into education. 

C.Aerts and followed on by P. Prinsloo write on AI's promise for adaptive and personalised education. There are many advantages but also inherent challenges which all educators need to be wary of.

Then M. Perkins and J. Roe  and the following chapter by B. Cope, M. Kalantzis and A. K. Saini address the issues of Gen AI and assessments. 

- Revaluing and recentring human teachers - with 2 chapters. 

In this section, the discussions from the previous sections are reinforced. There is great importance in ensuring teachers attain AI literacies to guide their work in an informed approach. 

- Ethics and governance imperatives for AI futures in education - 2 chapters.

As always with AI ethics must be front and centre to inform evaluation, implementation and practice.

- Confronting coded inequalities in education has 5 chapters discussing this issue.

The chapters in this section, argue for the need to be wary and to ensure that AI does not need to furthering the divide between the haves and haves not. The role of education in ameliorating the current private/capitalists AI push is presented. AI can assist with increasing inclusivity but it needs to be used in a planned and informed manner.

- Reimagining AI in educational policy: Evidence and geopolitical realities with 2 chapters.

Two think pieces, one from G. Siemens and the other from I. Toumi, both respected researchers and leaders in the educational technology field. Both warn of the moves by nations to harness AI for political and economic power and the importance of educators, education policy makers etc. in ensuring that AI is equitably availed and used for the common good of humanity. Some sobering thoughts in these two chapters :( 

The report closes with a brief conclusion. 

The report provides a 'state of the play' and collates the perspectives and thoughts of many of the thinkers in education, especially those who research in the educational technology space. Some of the chapters are speculative but the discussions are required for educators to dwell deeply into the consequences of engaging or not engaging with AI. In all, a worthwhile read.

As a accompaniment, here is a blog post by Carlo Iacono, on 'Inside the University 2030? a 'design fiction' piece describing how AI, as a partner for learners, makes many far-reaching changes to how teaching and learning occurs in the universities of very near future.

How things play out, remains to be seen but the pace of change is rapid, and keeping up with potential and possibilities for AI to support learning (rather than to replace it) remains a pragmatic way forward. 


Tuesday, September 02, 2025

Using AI to streamline constructive alignment and design portfolio-ready tasks

 Notes from a webinar organised by Eportfolio Australia on the topic of using AI to streamline constructive alignment and design portfolio-ready tasks.

The workshop is facilitated by Marty Sison who is a learning designer at Charles Darwin University.

The workshop outcomes are to use Gen AI to draft learning objectives from learning outcomes and content - then learning activities and following on with portfolio-ready tasks.

Steps to engage learners are to create learning objectives, then ensure there are learning checkpoints and follow through with learning portfolios. 

Used a padlet activity to define learning design terms. Then shared how padlet can be used to check if students are meeting the learning objectives by putting their ratings into each week's learning objectives. 

When AI is used, unit learning outcomes and content topics + prompt = learning objectives. Prompts were stored in a spread sheet, which was also formatted to collate the various responses and tables created by CoPilot. Advocated for using spreadsheets as it puts everything in one place. 

Demonstrated how unit outcomes and content can be generated using Copilot running on ChatGPT5.0 including evaluating the AI responses backwards. Then learning objectives for each week can be generated. 

Discussed learning checkpoints for active learning. Important to have assessment, feedback, external, internal feedback and help learners understand these. Should have authentic relating questions and/or concept reflection questions. 

Demonstrated how the connected learning checkpoints and activities can be created using CoPilot. Questions and answers were provided.

The spreadsheet created can then be used in turn to create experiences that can be represented through a portfolio. The learning portfolio helps to make learning visible. It can capture both the knowledge attained and the paths taken, turning multiple ways of representing growth into persuasive evidence.

Learning portfolios can be used as a collection of evidence, triangulated to learning objectives and be am authentic narrative of learning /growth and a persuasive showcase.

Therefore, the learning outcomes + content topics + learning objectives + learning checkpoints + prompt = portfolio tasks to evidence learning.

Demonstrated how the previous items along with relevant prompt helps to generate a range of portfolio assessments. 

For advise on prompt engineering, generally by grounded in your work. Always ask AI to evaluate and rationalise its decisions so that you can track things back. Control the output so that what is generated meets your needs and you do not have to reformate the AI response (e.g. table, column/row contents etc.) . Keep a record of the prompts so that you can recycle them. 

Interesting presentation. Will need to have a good play with our own content to see how the workflow can be adapted to our needs. 

Monday, September 01, 2025

futures barometer - Aotearoa NZ

The Aotearoa Futures Barometer was introduced by Dr. Eruera Prendergast-Tarena, Kaiwhakatere of Tokona Te Raki at the Ako Aotearoa Southern Forum in August. It uses a methodology from the Copenhagen Institute of Futures Studies  and surveyed 1000 New Zealanders along with 270 youth and 270 Māori responders. 

The themes include NZers' perspectives on the future, including their outlook, and expectations. The study also reported on participants' identity as New Zealanders, and their future aspirations. It provides a snapshot of on how a small group of Kiwis, perceive what the future holds for them and their optimism (or pessimism) on what the future holds. Economic, societal attitudes are polled along with how social media influences ways in which various sectors of society react to current and future trends and political shifts.

A good resource and it will be important to extend this across to a wider number of participants to be a better gauge of how NZers' thinking. 


Monday, August 25, 2025

Australian Government report - Our Gen AI transition: implications for work and skills

 Job Skills Australia have published a report on the implications of Gen AI on work and skills. 

The report collates data on the adoption of Gen AI across industries. Importantly, it recommends several important ways forward for the country, including the need to ensure that tertiary educators are provided with professional development to integrate Gen AI into their curriculum as they are the vanguard for the preparation of the workforce to work ethically and critically with Gen AI. 

There is no comprehensive Aotearoa equivalent although the government has set up some of its intentions in a ministerial document from the Ministry of Business, Innovation and Enterprise.

Of note that New Zealand performs poorly on AI-preparedness indices relative to small economy comparators and is the only OECD country without an AI Strategy, negatively impacting global perceptions of New Zealand as a location for digital innovation and AI investment;

An opinion piece from RNZ indicates that perhaps New Zealand's advantage lies not in chasing abstract, easily automated work, but in deepening its strengths in sectors AI cannot yet touch - food production, care and infrastructure.

All in, AI and the implications on work and in turn skills development and vocational education curriculum, requires careful consideration. However, there is a need for some urgency, given how quickly the technology is developing, to ensure all citizens are prepared for and able to work critically with AI, in whatever form its evolves to. Without doing so, we are ill-equipped to push back, when AI is imposed on occupations, whereby the management may only think about the bottom line. 



Tuesday, August 19, 2025

Ako Aotearoa Southern forum

 Presenting and participating in the Ako Aotearoa Southern Forum today, held at the University of Canterbury Student Union building.

The mihi whakatau brings delegates into the meeting space.

A welcome is then presented by Tai Samaeli, ACE Aotearoa who MCs the event.

Then an official welcome is from Cheryl de Rey Tumu Whakarae Vice Chancellor of the University of Canterbury. Connected the theme of the forum - cultivating thought leadership and excellence in teaching - with the founding premises of the University of Canterbury. To provision education for the region and nation. Education has the influence and power to transform, individuals, whanua, communities and societies. Challenged the need for the many changes being imposed on the current education system. However, change is a fundamental part of education. Especially if changes improve access, and contribute to the greater good of society. Reflected on current developments and reminded us that change and difficulties have always been part of human history. Now, education is much more accessible at all levels. It might feel that the world is in turmoil but this uncertainiy has always existed. Universities are part of the fabric of change. Shared the three Rs of education. Resilience, responsiveness and relevance as being fundamental to higher education. Business as usual no longer sufficient as change accelerates, Change can be a source of optimism and hope. Graduates, to be usefully contributing need to understand the context, to have critical thinking and to be able to communicate complex concepts. 

The first keynote is from Dr. Eruera Prendergast-Tarena, Kaiwhakatere, Tokona Te Raki on 'Beyond 2040'. Following on from Professor Del Rey's presentation, Erurera recommended the need to think like a tūpuna - in a time of great uncertainty to remember the future and re-engaging with hope and possibility. Shared the story of his Ngai Tahu ancestor who survived the Kaiapo massacre, and instituted land claims even in times of great deprivation and disadvantage. It was seven generations later, before the claim came into fruition. Introduced Tokona Te Raki as a social innovation lab to address the complex social problems we are being challenged. At present, many focus on the present, but it is important to look into the future. Introduced the Aotearoa futures barometer - 60% see no other country but concerned about the future and where the country is heading. In developed countries, many (30% thereabouts) feel that their children will have a better life (opposite in the global South). Top concerns in NZ include lost of trust in government and religious institions but education and NGOs still high (60% upwards). Polarisation is seem to be driven by rise of social media. Amongst youth (73%) and Māori see Te Tiriti as important. Shared a 2022 report from Treasury that delivered the warning that the next generation may not be better off than the current generation, Three forces converging in 2040 - climate reality, demographic shifts (older population, strain of superannuation, 40% of labour market entrants will ne Māori, Pasifika or Asian), and technology (elimination of entire categories of work, transform education, employment, human purpose etc. ). Recommended 2019 book on NZ and the future of work. Summarised the implications for education. The old playbook will not work - systems are inherently human and we need to work out a way forward. We need collaboration, partnership, investment and imagination. 

2040 selected as it is the bicentenary of Te Tiriti. It may provide an opportunity to open a window for transformation - for example, communities came together post-quake in Christchurch - and the city has been shifted from being more English than England, to being a city in Aotearoa. The future will that wait, will we be ready?  Shared imaging Aotearoa 2040 - to make a better country for our mokopuna- from control/fragmented to stewardship. Everyone needs to pitch in - young people and Pacifica workers reaching their potential, skilled migrants welcomed and supported, the diapora given reasons to return, knowledge transfer across cultures and generations, and everyone contributing to shared prosperity. Education remains trusted and is a vehicle to contribute towards the transformation. Therefore, to shift to shared futures from rear to hope, division to unity, isolation to cohesion, short term thinking to long term, reactive to proactive, superficial to transformative, and hyper individulism to collective responsibility. Challenged the audience as education leaders to model partnerships in our instituions, prepare tauira for collaborative futures, create spaces for difficult conversations, and champion long-term thinking. Staring with Te Tiriti as our guide, building our shared story, bridging differences and embedding inter- generational stewardship. We have the opportunity to weave the future together. 

After morning tea, I provide an update and details of the 'using Gen AI to support foundation/bridging ākonga' (full report now published), funded by Ako Aotearoa. I summarised the rationale for the project, the project team, the caveats (AI is not benign and must be used carefully so that it does not replace learning but supports learning), findings, recommendations and future projects. 

A discussion session on 'sector collaboration in tertiary education' followed. We discussed collaboration across our organisations and what teaching excellence is.

Then Tākuta Phil Borrell, (University of Canterbury) who was awarded the 2004 Te Whatu Kairangi Kaupapap Māori Award, speaks on Ngākau pono - the importance of authencity in teaching practice. From all the previous speakers, critical thinking was a common theme. Through narratives, went through his main themes informing the importance of being an authentic educator. Good teachers inspire and model possibilities. Importance of bringing practice and theory, usually by bringing experiences into teaching. Allowing your ākonga to know more about you help them empathise and find connections. Used Claire Good's keys to teaching excellence to provide characteristics. Extended and presented examples of being an authentic teacher. We should have fun in teaching using humour and honesty. Share stories and use it as a pedagogical tool. Teachers need to want their ākonga to succeed. 

After lunch, Graeme Smith presents his mahi/work on 'designing intelligence: building tools for the education we need next'. Drew on the presentations in the morning. Partnerships, change, long term thinking, fun, narrative warfare, value/s, possibilities, making better long term decisions, weaving, transformation, systems, humaness ---. Shared experiences building customised AI agents. Encouraged those who are AI-reluctant to use tools that are designed to achieve specific purposes. What needs to be done to build the AI infrastructure - we need to be involved in designing, otherwise it will be done to us. Emerging patterns include braided funding partnerships, values-led, and memory organisations. We need to stop managing delivery and start designing intelligence. Our role is not to preserve the system but to evolve it. Need to build - by adopting and adapting existing tools, think in braided partnerships and imbue values. 

A panel discussion follows with Tai Samaeli chairing and panel members including Josie Ogden Schroeder - CEO of the Kind Foundation, Dr. Cheryl Doig - Kai Ttiro Wāheke / futurist, Dr.Mahmah Timoteo - Māona Vā, and Sandra Fernandes Videira Gordon - Te Wānanga o Aotearoa. 

Introductions follow the panel discussing a few questions.

First question is around how we prepare learners for the future, for jobs and technology that do not now exist. Agility in thinking, doing. Prediction is linear but anticipation envisages many scenarios, some may be implausible, but there is a need to think through these. Creativity, curiosity, communication and critical thinking (4 Cs centre to access digital resources for those who do not have the opportunities). Help learners to build community. Questions then opened up. Where is the future for kids who now face disadvantage. We educators are the ones who need to take up the baton. It only takes one carer to bring a change. Relationships between teachers and students are important. Connection to community also effective. 

A celebration of Ako Aotearoa is led by Jennifer Leahy summarised the many endeavours and initiatives undertaken since 2007, when Ako Aotearoa was set up. The Southern Region has been productive across all spheres :) support the many sectors and objectives of tertiary education 

Followed by the close of the forum. 



Sunday, August 17, 2025

ChatGPT - study mode, Google guided learning, Claude learning mode and University of Sydney Cogniti - are they similar or different?

 There has been a flurry of activity in the Gen Au space of relevance to teaching and learning. First up was the launch of ChatGPT5.0 which allows for the use of it in 'study mode', This allowsa for a shift in the emphasis of using Gen AI to 'provide answers' towards using it as a 'study coach'. 

A few days later, Google also joined the move with its 'guided learning' in Gemini

Whilst, Claude has provided a learning mode for some months.

The above join University of Sydney's Cogniti as possibilities for teachers and learners to move towards personalised learning environments. There is also a recent start up - Wild Zebra - which provisions personalised tutors to students. 

However, as with all the 'vanilla' Gen AIs, each has things it does well and things it will struggle with. 

For example, here is a comparison of ChatGPT's study mode with Claude learning mode by Toms Guide. 

Using chatgpt to compare university of sydney cogniti with chatgpt study mode yields some differences.

Key Differences Summarized:
Feature
Cogniti
ChatGPT Study Mode
Focus
Educational context, feedback, integration
General problem-solving and learning
Integration
Canvas and other learning platforms
General use
Accessibility
Equitably available to all students
Requires access to ChatGPT
Feedback
Personalized and standardized
Interactive and conversational
Tracking
Tracks student-AI interaction
Does not specifically track
Bias
Potential for bias from training data
Potential for bias from training data


Therefore, each tool has pluses and minuses and as per all of our recent studies into  integration of Gen AI into VET, Gen AI tools need to be carefully selected, and learning planned and structured. 

A caveat with using Gen AI systems as 'tutors' is provided in a recent article by Flenady and Sparrow (2025). Their warning points to the often disregarded conceptualisation of Gen AI - in that it is NOT intelligent but build on algorithms for pattern recognition. They argue that Gen AI systems are 'epistemically irresponsible'. It is therefore important to always take heed of this warning and to ensure that all users have this at the topmost of their minds whenever they use Gen AI.




Thursday, August 14, 2025

AI in Oral assessment - Food and Fibre CoVE project presentation

 Updated - 26th August 

Attended part of a presentation organised through Scarlatti to present the work funded by the Food and Fibre CoVE. It is one of a range of projects on artificial intelligence.

The presentation shares work undertaken to use AI for oral assessments.A report is available, summarising the pilot and findings. The various outputs from the project can be found at this site.

The presentation showcases the AI agent for learner oral assessment and shares perspectives from users as to the AI agent's efficacy. The guide to using the oral assessment chatbot is found here.

Began with sharing of a 'case study' comparing conventional (paper-based requiring travelling to an assessment centre) to the piloted (mobile, oral, AI supported). Then a couple of warm up sessions followed by short overview of F & F CoVE. They have completed 109 projects over the past 5 years!! Also introduced Scarlatti, and the research team from Fruition, F & F Cove and the WDC. 

Then the sharing of using AI agents in Oceania followed by the demonstration of the AI agent.

AI agent oral assessments came about as written assessments were a barrier for many learners and an Ai agent needed to be tested. 

Began with sharing the AI in Education  articles - which was the way to find out what was being done. AI agents being used more by universities in Australia. Most agents created using Cogniti or custom GPTs. Most were in a text-based mode. Not as much use in vocation education or in NZ and most were focused on learning rather than assessments. 

Showcased the AI agent (VEVA) with proviso that it is still a prototype :) tested in a dairy training context. The agent does provide the answer if the assessee indicates they are unsure or unable to answer the question. Using this for formative assessment would be more pertinent. It works well if correct answers are provided but does fall over if the candidate is unprepared!

Transcripts are archived and can be used to grade the conversation. With the small number of candidates (14 + 11) the moderation of the grading, showed accuracy. 

Important finding included that an off the shelf product would not work. This was because a controlled conversation was required, precise outputs was also required, and funding/payment needed to keep the agent. 

Therefore, two agents were used. The 'examiner agent ran using Open AI Real time Voice API) and the Assessor ran on GPT 4o text mode.

A demonstration (akin to our own experiences with Ako AI) in that responses from AI can sometimes be difficult to predict. The controlled conversation was primed with standardised questions. Prompting content included course materials, assessment rubric and example answers which had been graded.  RAG (retrieval augmented generation) was used to search course information as there is too much course information to put in. Ethical guard rails were included. 

Shared the various ways in which the oral assessment AI agent could be used. These include summative and formative assessments, authentication of context and candidate, employer verification, pre-screening for readiness to be assessed and for recognition of prior learning (RPL).

Closed with where the project will move towards next. 

A panel made up for Scarlatti staff (Adam Barker, Sam COrmack, Leater Hoare) and users - Jenny Sinclair from Dairy Training and Tiffany Andrews from Fruition the answered a series of questions.

Overall, a good overview of the possibilities for using AI to support multimodal forms of assessments. 

Monday, August 04, 2025

Supporting the digital transformation of Vocational Education and Training - JRC (European Commission) publication

The European Commission has published 'Supporting the digital transformation of Vocational Education and Training' report. 

It is one of several prepared by the Joint Research Centre (JRC) to provide an evidence-base to inform policy and practice across many aspects of education and industry.

The report begins with an introduction which provides context, rationale and brief overview of the two main chapters.

Two chapters follow to provide an overview of digital transformation of the VET sector. 

The first chapter is on 'enabling the digital transformation of the VET sector. The enablers include" career education and guidance; development and use of information systems; ecological approaches and collaborations; flexible accreditation processes; holistic development of learners' skills; inclusion and sustainable human development; pedagogical and transformative potential of digital technologies; and teacher training for changing roles.

The second chapter summarises 'emerging trends and technologies in VET'. It brings together a synthesis of secondary grey literature, VET Erasmus+ projects and Centre of Vocational Excellence initiatives, and consultation with experts. The report stressed the importance of VET when compared to other educational sectors, due to its close alignment with industry. Emerging technologies included AI, AR, VR and blockchain-based digital technologies. 

A section provides case studies (Austria, Spain, The Netherlands and Albania) and the report concludes with an short overview on the impact and implications of AI on VET. 

A good overview of ongoing work undertaken across the EU to ensure VET is kept proactive with regards to digital technologies. 


Saturday, July 26, 2025

JVET - DAY THREE

 Notes from day 3 of the conference.

I chair one of the first sessions of the day in the stream - social role of colleges. There are two papers, so each has more time to present and a longer Q & A session.

First up is Naomi Alphonsus (University of West Cape), who presents on 'a community for expertise development- South African vocational colleges and their internal and external social characteristics'. Covered the rational for the topic of the project. South Africa had undertaken a change in qualifications, and several community colleges had undertaken to begin using the new curriculum. The study sought to find out what colleges undertook to enact the new qualifications.

Some aspects were that teaching and learning processes are connected to social characteristics within the college community. Expertise helps to broaden the preparation of work beyond qualification, curriculum and assessments. Institutional capacity is also important so that the behaviours, attitudes and 'ways of being' that support institutes to shift practice, can be better understood.

Shared the occupational conceptual model - occupational capacity requires specialised knowledge (systematic, occupational practice, skill, autonomy) and social organisation of work (professional bodies, institutions, state regulation). The model is based on notions of occupational capacity and scholarship across different disciplines. There is a relationship between training and practice in occupations. 

Defined the social characteristics of colleges revolve around curriculum delivery on necessary expertise development. Also include deliberately build collaborative relationships internally through teamwork; resolve conflicts amongst their disciplinary experts and externally through partnerships; successfully mediate external company skill expectation of students and with translating the curriculum internally.

Provided a series of examples between two colleges, one which was able to interpret and translate the new qualifications into the curriculum and another which struggled (and where their first cohort of learners did not pass the new qualification. 

Second with Gavin Moodie on 'colleges' roles in reconstructing social construction of their communities' education, workforce, and economic development. Covered the context, community service, anchor, collective capacity, social institutions and colleges' capacity.

Discussion based on article 'Colleges as anchors of their communities: emergence and agglomeration' (special issue). Understanding of the roles of colleges is diverse, with many requirements placed on them through governmental, societal or community requirements along with an update on thinking on the topic.

Defined and discussed community service (as this is a focus of community colleges) to include teaching (service learning and citizenship education; disseminate ideas through consultancies, knowledge transfer (on legs), research co-production and service through open days and meeting community needs. Used the anchor as a metaphor to think of community colleges anchoring to compensate for the withdrawal of the government services, Some provide a network to bring the community services together, be a hub to support the agglomeration of industry or services and help their communities thrive. College also contribute to collective capacity through learning networks, networks of civic engagement helping to contribute to the knowledge ecosystem and social capital (synergy and emergence are part of these processes).  The role of colleges as social institutions which can be envisaged as learning cities/regions, with enlightened attitudes, (re)forming beliefs and social institutions, and transform social policies and institutions (Bovin and Laruffa, 2024 - organisations need to support individuals' capacity to act and transform themselves and the society they live in). College capacity - what do we do recruit to move towards colleges being transformational organisations - through their ability to localise and draw on their communities to bring about change. 

After morning tea, I attend the sessions in the stream for 'social, economic and technological change.

Dr. Anthony Perry (University of North Dakota) presents on 'creating new pathways into emerging technology occupations'. Need to define emerging technologies so that students are able to plan career pathways. Finding indicates that students interested in trying to solve future wicked problems and see learning about emerging technologies as one way to gain the skills.

Emerging technology includes advanced manufacturing, biotechnology, AI and machine learning, AR/VR/XR. It is transdisciplinary /multidisciplinary. Emerging career pathways - lifetime value of skills in the labour market is decreasing, high pay high demand STEM-related emerging tech occupations require some post secondary education but not a bachelor's degree. It is rapidly disrupting traditional STEM careers. 

The typical primary/seconday math/science to post secondary to STEM occupation pipeline is unlikely to be sustainable. The study asks STEM students about their academic and career plans. Based on theoretical framework (Situated expectancy value theory - SEVT) - expectancy of success + subjective task value = achievement-based task choice (Eccles & Wigfield, 2020). 

Shared the rationale and details of the study carried out using semi-structured interviews and thematic analysis. Then discussed the various data and themes. Selection of emerging tech pathways draws on identity, utility value, not intrinsic value and expectancy of success. Therefore students see emergent tech as an integrator for transdisciplinary learning; the SEVT framework works, emergent skills is best learnt through experiential learning. 

Followed by Professor Michael Gessler (University of Bremen) on 'national hydrogen workforce and 'just transition: an international comparative conceptual review'. Many countries have hydrogen national strategies, but few have connected these to workforce studies. Some countries have done these but not all. These studies generally try to find out what skills and occupations will a hydrogen economy need, in what volumes and timelines? What can existing education, training and labour market systems mobilise? There is a International Renewable Energy Agency (IRENA). Hydrogen is already used in cleaning petrol and in fertilisers. Hydrogen's newer function is a an energy carrier. Renewable electricity capacity is mainly through hydro and solar. Solar creates the highest employment possibilities. 

Summarised study methodology - descriptive data extraction and qualitative data analysis. All predict workforce increase. Shared the work on skills analysis depth - skills demand, skills need, quantified skills qualifications, pathways etc. The 'just transition' analysis carrier out to look into action https://www.sciencedirect.com/science/article/abs/pii/S0361476X20300242, resources, timelines, monitoring, risk assessments etc. and also for implementation. Rated the various categories that were identified. Sorted the various countries into 'maturity model' - most countries at levels 1 -2 of a 4 level - strategic policy framer, technical skill maximizer, regional transition facilitator, or integrated just transitional planner. Good explantion of an interesting methodology and discussion was around the validity of it.

Then Professor Presha Ramsarup (University of Witwatersrand) presents on the topic of 'occupations in transition and VET. Occupations is a complex link between education and work. Occupations are not merely affected by sustainability transitions - they are agents and within them. If we integrate 'green skills' into the curriculum then occupations will put into practice, the precepts of sustainability.

Thinking of occupations as a 'regime' helps to understand the shifts in occupations, required to transform work. Green skills is shaped by a range of complex drivers and 'or 'underlying mechanisms' - political, economic, social, enviromental, justice and long-term ethics. Understanding skills formation to support green transition is a complex set of moving parts.

Greening occupations are not homogenous concept - Greening occupations require field of knowledge, materials worked on or with, kind of goods and services produced and tools and machinery. Has a larger project that are being carried out on the macro/economic and social-ecological drivers of demands, sector/systems and landscape level, occupations, skills, status of occupation etc. 

Introduced and discussed  framework how occupations transition  - political economy/ecoloty of the transitioning occupation, scope and change in operational tasks, field of knowledge, materials/tools, good and services produces and occupational change. Used the occupational change for electricians in the solar sector. Instead of shifting people who lose their work (e.g. miners) to another occupation (e.g. agriculture) it is better to provide them with a more generalisable skills (e.g. as 'artisans' able to move into a range of trades). Need to drill down to the skills in the occupation at risks and see what skills need to be moved towards 'new' occupations.

Important to find out what work is actually done as 'occupational titles' and 'qualifications' can be misleading. Work tasks and how they are coordinated and organised have to be better understood. So occupations may not be the best approach, however, it is how we understand occupations and the transition between occupations is perhaps the way to go. Therefore if occupations are 'regimes' are not only technical but political - shaped by power struggles, lock in effects  and selective inclusion/exclusion of knowledge and actors. 

The conference closes after lunch.

This conference is perhaps the most well-attended on vocational education research. Many of the participations focus solely on VET research. This inspires me to continue with my work, especially given that there is so little done in Aotearoa NZ, The recent changes in structures for NZVET means external funding for VET research will be hard to come by. We will need to lobby for better recognition and funding. 

JVET conference - DAY TWO

 Notes taken on the second day of the conference, a full day of activities.

In the first session of the day, I chair the 'VET and sustainable development stream.

- Harald Hanke (University of Bielfeld) and Stella Heitzhausen present on 'didactic framework of a train-the-trainer concept for sustainability-related continuing education. The context is in the electrical industry. Began with the challenges and likey solution. Introduced the transformation competencies in the context of vocational action. Shared the cases participating in the project and the didactic framework that was developed.

Made connections between the economy, society and the biosphere with regard to the streams of sustainability that run through and interconnect these. Education is an important focus as learning how to conduct work in a sustainable way. Workplace learning on sustainability is mandatory in Germany (from 2020). 

Competences for work include professional, self-competence and social competencies. Transformational competencies are well connected to these and sustainability can be integrated across to ensure that they are addressed. 

Provided an overview on the project - sustainability at work. The sub-project is collaborative development of sustainability-oriented train the trainer concept for metal and electrical industry. Project offers workshops and develops resources so that trainers can guide their apprentices to attain the necessary sustainability practices. Design-based research is used - analysis and exploration (document analysis and scoping workshop - to derive topics from industry participants), design and construction and evaluation and reflection (trial and feedback/self-assessments). Through these, concepts were generated - creating new values, balancing tensions and dilemmas and assumption of responsibility. 

Described and connected the framework - the didactic-methodological design. 

- Followed on with Marilisa Ohlwein (Leibniez University) who presents on 'integrating sustainability into vocational training: exploring approaches and challenges in the construction sector (wood technology, interior decor (painters etc.) and construction (roofers, bricklayers). Stressed importance of focusing on sustainability in construction as it is one of the most impactful industries on sustainability. There is high potential for the industry to improve - for example through energy efficient buildings, construction life cycles etc. Some crafts have educational regulations requiring sustainability to be integrated.

Shared the vocational training profiles (learning outcomes) include elements of sustainability - for painters and varnishers. However, resources to support these are still emergent and insufficient. 

Her study involved a literature review to find examples and resources. There was a large amount of literature (2015-2015) and a flowchart was developed to evaluate these for relevance to the study. Identified small number of articles that corresponded to requirements. Summarised these articles through thematic analysis. Drew these into some guidelines for designing VET learning environments to support the integration of sustainability. Approaches to support these were also shared - teacher engagement and involvement and structure (in the curriculum, lessons, outcomes). 

- Then Annabell Albertz (PhD student at the University of Cologne)shares work on 'implementation of green skills at industrial training institutes in India. Provide a background for the project - bearing in mind that India has a high number of industries that are connected to the green-skill workforce. Overviewed the VET system that is central to training the workers at the intermediate level who are the ones who integrate and implement the processes. There is a research gap in the area.

Perspectives of teachers on how to best integrate and implement green skills is studied based on Bronfenbrenner (1979) ecological system theory (exo (external landscape), macro (social), meso (institute) and micro (classroom) systems). Used group interviews of teachers from New Delhi and Bangalore. 

Findings indicate that green-skills are rarely implemented in teaching, although there are some extra-curricular opportunities to introduce and practice. Teachers did not have the knowledge or skills to teach, there was limited time in the curriculum and some saw it as being outside their job scope. Teacher training on the topic is not adequate, little cooperation between local community and national scheme, local institutes and industry. There is no overarching government guideline or policies so they are not included in curriculum documents. Some teachers do not see relevance as industry is not demanding that the skills be included. 

The absence of exo system - i.e. government policies and its impact on curriculum is a key requirement. Without this, there is no direction for it to be integrated. Shared the implications for practice and for research. 

After morning tea, the second keynote with Professor Sharon Gewirtz (King's College, London) is on 'half our future: can we address cumulative injustices and improve support for young people taking non-university routes into work?'

Reported on the near end of a six year study (young lives, young futures study) the common and diverse perspectives of young people who do not go down the university route. Began with a series of vignettes to provide an overview of the participants - many from socio-economical backgrounds, with challenges through school, neurodiversity, difficulties to move into work, some with successful transitions into apprenticeships.

Overviewed the longitudinal project with quantitative and qualitative strands. Participants were young people 16-20 and followed across 3 surveys and semi-structured interviews. The presentation draws on the first two quantitative surveys with 10 thousand + in the first and 6000 + in the second. 

For many of the participants, school was something to be endured, feeling unheard and unseen, and skewed careers support. Therefore, large group of young people disadvantaged by the school curriculum, where their interests and aspirations are less valued and there is recognitional injustice, participatory parity, and injustices of distribution. 

Then shared the different experiences of differently positioned early school leavers - leaving school at 16. In the UK, NEETs = 28% (16-18 year olds - over 1/2 a million). Type A - high-need stalled transitions (high unmet needs due to working class, SES and adverse childhoods); Type B - non-linear precarious transitions (difficulties finding work - precarious, part-time, low paid work); Type C- relatively privileged transitions (secured apprenticeship, on track to career aspirations, at least one parent in a professional or intermediate occupation). Inequalities, SES, ethnicity, family backgrounds see to be significant barriers.

Barriers at school included unrecognised/unsupported SEND and difficult up-bringing. Experiences of bullying and poor relationships with teachers, prolonged periods of no education. Career guidance was unequal. Type A did not attend or had missed. Type B - some attended but found teachers dismissed their interest. Type C usually had social and cultural capital through families and could draw on their knowledge and social social networks, parents actively brokered and supported apprenticeship applications. 

Findings indicate the existence of a 'class ceiling' (Friedman & Laurison, 2019) preventing those who are most disadvantaged from accessing high quality manual and technical employment, apprenticeships and training aligned with their interests and passions. 

Proposed recommendations - a broader 11-16 curriculum, redressing funding disparities and raising the status of technical routes in career advice; redefining 'social mobility', changes to school and workplace cultures, erosion of occupational hierarchies, creating more opportunities and fairer recruitment, and more joined up thinking across policies.

After lunch, I attend the sessions in the 'TVET regulation and assessment stream' - presentations from the English OfQUal (the qualifications authority)> 

- Here, the first presentation is with Steven Holmes and Fiona Leahy who present on 'assessing behaviours in apprenticeship end-point assessment. Started with providing the background and assessment processes for assessments in apprenticeships in England. Apprentices standards are interprested into assessment plan and there is an 'end-point' assessment, taken at the end of an apprenticeship. It usually has 2-3 assessments which may include project, observations, product etc.

Since 2025, assessments are more streamlined and flexible - assessments now more continuous, not just end point, can take place on programme and training providers are able to complete some of these.

Defined behaviours as - mindsets, attitudes, approaches needed for competence, whilst innate or instinctive, they can also be learnt. 

The study wanted to understand what does competence look like and the evidence required to support this judgement. Worked with 3 programmes - customer service, data technician, operations or department manager with a range of assessment methods, 2 experienced assessors.

Each assessor watched recordings of 2 apprentices and retrospectively think aloud to express their assessment decisions/judgement etc. discussion undertaken and also a follow up interview.

Findings indicate high level of skill and experience required of the assessor, especially questioning techniques and learner-focused strategies. Examined how decisions made about candidates meeting assessment criteria - weighing up the evidence. Decisions were made through strength of evidence presented - high frequency of examples provided, evidence of meta-thinking and self-reflection., verification of the authenticity of the evidence and the format (multimodal/observed) of the evidence. 

Alignment between written and actual criteria used for jusdgement - holistic criteria, placing the behaviour in a single context-bound criteria and challenges when trying to narrow towards the behaviour as not always easy to distinguish between skills and behaviours, and overlaps which can lead to duplication. Challenges also with how the criteria was written, and consistency within awarding organisations.

Recommend attitudinal behaviours assessable through context-independent, holistic assessments which can be more subjective and harder to judge and standardise. These need to be observed across a longer term, not only through 'end-point' assessments. Suggest some items to inform assessment reform - reframe some behaviours as skills and incorporate into existing skills standards. 

- Then, Catherine Large shares work on 'qualifications reform as a policy instrument in English vocational education'. In England the assessment systems functions to measure competence, competition, content and control. Based on assumptions that skills, leads to ability to work, which in turn contribute to the economy of nation states. Therefore, if qualifications are changed, then the various other functions of qualifications are impacted on, therefore qualifications reform must be carefully considered. 

England has a 'market'-based economy, with qualifications being a form of 'currency'. Programmes of learning become the means of attaining the currency. This all leads to learner, user, stakeholders, industry etc. each considering qualifications in difference ways, but the commonality is that they are a way to further the marketisation of education.

Between 1990s to present, school and post school qualifications evolved. The technical qualifications shifted from NVQs to Tech Levels, T levels and technical occupational qualifications. Apprenticeships have also shifted to align with the qualifications - from modern, to frameworks, to end point assessments and apprenticeship assessments. 

Qualifications change process begins with identification demand, then qualification content and occupational standards are set. Qualifications design principles and design of assessment instruments follow and the approval process than occurs before it is rolled out. 

Qualifications reform is seem to be a appealing - it is inexpensive, can be externally mandated, rapidly implemented and the results are visible. Effective educational change can happen if pedagogy, assessment processes and curriculum are understood and the impacts of qualifications are realised.

- Followed by Paul Newton on 'what is an 'outcome-based' TVET qualification? Explained why OBQs matter in England. OBQs have learning outcomes, assessment criteria etc. However OBQs have been criticised for many reasons. Discusses why is the OBQ approach flawed? The previous NVQs were epecially stringent - this could be because NVQs revolved around assessment of occupational competence and the link between the training process and the qualification was severed. OBQs therefore removed control of the currilum from providers to employers; they defined learning outcomes separately from bodies of knowledge and skill, solely what the learner can do, usually narrowly conceived activities. 

Traced the historical evolution of OBQs. They replaced a complex multiple of qualifications in existence across to the 1960s which often had poor completions. The Tech awards OBQs broke assessments into smaller parts, made qualifications consistent so that they had the same framework etc. and reduced qualification numbers. Also they were more authentic to occupations, flexible, had a unitised structure and had standardised quality assurance systems (moderation etc.) Presentation of syllabi was clear and included the depth of topic which provided a better guide for teachers.

Therefore an OBQ has design principles - teaching, learning and assessment plans must all be based upon the same explicit statement of intended learning outcomes - with alignment of curriculum, pedagogy and assessment. 

Afternoon tea is followed by my presentation of 'reform in vocational education in Aotearoa NZ'. I cover the before, during and after reform and the implications at each stage.

This is in the 'TVET policy and practice' stream.

I attend the preceding two papers in this stream before my turn comes up.

- First up are Daniel Lascarez Smith (Univsidad Tecnica Ncaional - Costa Rica and presentation via TeamsVC) and Johannes Karl Schmees (University of Derby) who present on 'unions as burden? dual apprenticeship transfer to Costa Rica and the consequences of missing labour representation'. Started off the context, the Costa Rica political background. Costa Rica experiences many social problems - youth unemployment, poverty, informal labour) and apprenticeships have been seen as a away to combat youth unemployment. In 2015, dual apprenticeship model was introduced with support from the German government and in 2017 a tripartite roundtable discussion moderated by the ILO, launched the design of dual apprenticeship into law.

The presentation arises from studying the 2017 event (3 papers published). One on the business sector, another on the state actors and the third on unions. The first two papers briefly summarised before the presentation focused on the third - union perspectives. The roundtable discussions were anchored in rational rules of communication. 

The role of unions in Costa Rica was then explained. Unions are only in the public sector, not in the private sector. Unions also say apprenticeships as a means to address the growing social inequality in the country. The dual apprenticeship principles were shared - with its social, organisational and individual philosophical and educational principles to contribute to the underlying decent work principals. Union and employer/state perspectives were summarised. There were differences on various aspects. However, when the dual education law was passed, not all the differences were resolved and many items did not make it across from the roundtable discussions! Since the law was passed, only 100 apprentices have been enrolled. There have been no modifications to the law :( therefore, the tripartite governance model for dual apprenticeships in Costa Rica was not carried through,

- Secondly with Associate Professor Asa Broberg and Professor Ali Osman (both from Stockholm University) on 'competence demands and organisation of VET in times of rapid change: automotive education in Sweden 1950-1960. Presents part of a larger project to understand VET development in relation to society changes and immigration in a historical perspective. There are 3 sub-studies on historical developments and 3 on migrants experiences.

Went through the reasons for studying the automotive industry between 1950 to 1960. There was rapid economic expansion at this time with SAAB and Volvo and accompanying increase in private consumerism including car ownership. Educational expansion was also taking place. 

The question is 'how did VET respond and what enabled/restrained the responses?,' The study used document/desk analysis. 

The study is grounded in curriculum theory and conceptualised through notions of frames, steering and arenas. Provided overview and examples of each of these and how these applied to the study being presented. 

Findings qualitative and quantitative were detailed - whereby more classes and specialised programmes were developed. New pedagogy was instituted, exchange classes and resources /materials and teaching plans accompanied these. 

A connection between VET and migration closed the presentation. 

A 'Lorna Unwin remembered: transforming JVET and the field of VET' follows.

The day closes with the conference dinner.

A longish day but a good variety of  presentations.

Friday, July 25, 2025

Journal of Vocational Education and Training Research - JVET conference - DAY ONE

 Below are notes taken at the JVET conference today, held at St Catherine's College, Oxford. The conference runs over this afternoon, tomorrow and Saturday morning. It is a busy conference with 6 streams running during the presentation slots and key notes today and tomorrow.

Day one begins with a conference welcome,/briefing, update on JVET from Stephanie Allias (call for special issue abstracts, journal articles of the year etc.) and keynote. 

Keynote is with Cristian Lincovil Belmar who is Executive Secretary of Secondary Technical & Vocational Education in the Chile Ministry of Education. His presents on 'research, policy and politics in TVET: sidenotes from the South'.

Introduced the definition of 'sidenote' and how he has structured his presentation to bring focus on these, which are often placed on the margins of the main text (in this case policies/politics etc.)

First sidenote - can vocational education be considered an institution? for instance, universities have a set of rules, practices, relationships and norms across the world. Similar in school systems. However, it is not the same with vocational institutions, which are much more socially/economically/culturally constituted. Hence, vocational education is actually more complex.

Vocational education can be framed as a border. Perhaps it is in a liminal space between other more well defined institutions (i.e. universities/schools). Depending on systems, the intersections between vocational education and other parts of the educational system, morphs and is often paid attention by politicians, due to its connections to ensuring the betterment of a country through vocational education.

In many cases, vocational education is marginalised by its role, which moves as political needs shift and country's focuses change. There is debate on the purpose of vocational education internationally but each country takes its own route, leading to the complexities of understanding vocational education.

Traditionally, vocational education has been seen to be education for the lower classes, to prepare workers. This, along with a lack of parity of esteem for vocational education, means there is little research undertaken. The challenge for the TVET community to move closer to the 'centre'.

He then shifts to the role of TVET in the Global South. Begins with definitions and background - 'our north is the south'. Chile has cycled through various models, often from Europe (Swiss, German) and Australia. These have been developed in very different contexts and do not always fit the Chile context. The landscape of Chile VET was summarised. There is high participation in higher ed (63% of adults 25-65 have degree). There is wide participation in vocational education - 36% in upper secondary/45% in HE). Both VET and HE are highly privatised. Introduced a 2018 publication - Estrategia Nacional de Formacion Techic-Profesional - that informed the VET system. Advised a shift from 'human capital' to capabilities to live a more free and worthy life, through strengthening social and intellectual skills'. This shifted the focus of VET towards a human capabilites approach - as per Wheelahan, Buchanan & Yu, 2015 - a NCVER paper on linking skills and qualifications through capabilities and vocatioal streams). Currently the Chile National TVET strategy (2020) states the purpose of Chilean VET is to 'ensure the development of people's diverse talens and capabilities throughout their lives, in accordance with the country's economic, social and sustainability needs, and contributing to a more equitable Chile with greater social mobility and decent work'. However, in practice, VET is still at the border, as politics shifts across ideologies. Shared the ways he sees the capabilities approach in VET across social.economic and educational recovery post-COvid, education and democracy for citizenship, sustainability and diversity and equity. Shared the 'for a worthy life' report - 3000 students from secondary and tertiary VET, to find out what they saw as living a worthy life. Also the new curriculum for TVET transformation, shifting from competency to capability. Plus the progress on the a new national strategy for VET. A thought provoking keynote.

After lunch, I attend the sessions in the Teacher Education and Development Stream.

- First up, Professor Joy Papier (University of Western Cape) with ' quality teaching and learning: student engagement across the digital divide'. Began with an introduction and background to TVET in South Africa. VET still has negative stereotyping and much work has been undertaken to make colleges 'institutions of choice'. The project is part of a larger whole with a focus on improving VET students' success rates through increasing digital literacy and technology integration in VET. There is a gap between what is perceived to work and what actually occurs. Overviewed some of the quality teaching and learning literature. Findings indicate that modern teaching and learning technologies and learner engagement are two essentials. Instructors always a hurdle, especially teacher digital capability.

Summarised their study (across 5 years and 33 projects) to identify factors that support student engagement. Paper-based survey used and followed up with students and lecturers. 75% of respondents agreed strongly that they learnt from lectures; high proportion also valued practical sessions. Students also said they participated in self-directed learning but lecturers did not see this as being valid. Quality lecturers were both knowledgeable and caring/nurturing.

2/3 of students used smartphones, only 54% had access to college Wifi and computers and small proportion had home access. Instructor expertise, interactive learning and engaging learning environment were important to keep learners engaged. Integration of multimedia, and access to digital resources also helpful. 'Good teachers' who make the best use of resources were found to be most effective.

- Then, Brendan Kavanagh presents on 'exploring the motivations and identity (trans)formation of second-career teachers in Ireland's Further Education and Training sector. Began with the rationale for the study. Further Education for Training (FET) in Ireland is misunderstood and under researched. Study to identify where support is required for novice teachers as they transition into teaching. To inform policy making etc. Then summarised where FET (often viewed negatively- black sheep, backwater, cinderella etc.) within the Irish educational system.

Second-career teachers leave a non-teaching profession and move into teaching at FET. Bring credibility to and authentic to teaching and learning. FET has 500 learner locations in Ireland, central in provision of apprenticeship and traineeships, wide range of subjects, disciplines, occupations usually to level 5/6 (certificate/higher certificate). 226,000 learners with 32,000 employed in FET.

Research carried out through survey, focus group and semi-structured interviews. Shared the literature review on intrinsic/altruistic motivations, extrinsic motivations, and professional identity. Findings revealed personal networks as a significant influence to second-career teachers starting in FET; they seek teaching for stability and balance; employment precarity and poor work/life balance were also strong motivations to move into teaching. Positive experiences in the FET sector attracted second-career teachers. Novice teachers expect stability but face initial precarity and uncertainty. The career change involves a profound shift in how individuals perceive themselves. A brief overview of limitations closed the presentation.

The last presentation in this session was with Megan Turner PhD candidate, (University of Technology Sydney) on the topic 'stopping the VET brain drain: the power of education and support structures to attract and retain high performing VET teachers'. Larger PhD project is to look at how to retain VET teachers to ensure that the teaching of skills is sustainable.

In the last decade in Australia, there has been a decline of VET teachers leading to a VET teacher shortage. Several states now have 'free fees' or 'earn and learn' programmes to train VET teachers. VET privatisation along with under investment in VET educators has led to casualisation of VET roles, cost cutting, increased workloads, and decline in work conditions. The need for a teaching qualification to teach in VET has decreased in levels, from University diploma in teaching (or Bachelor/ graduate certificate/diploma) to a Certificate 4 in training and assessment.

Introduced the teacher lifecycle model - from becoming (with enablers) to being a teacher and being enabled to thrive. Supporting industry experts to transition into teaching is an important part of becoming VET teachers. Support across the first 12 to 18 months are crucial to ensure new VET teachers continue. When established, continued institutional and government support is required along with continued job security, fair pay structures, opportunities to maintain industry currency and upskill and fair administrative workloads are all factors help retain VET teachers. A stronger VET system with career pathways, supportive systems, fair pay, secure jobs and sound teacher training are key to retaining and valuing VET teachers.

Summarised research methodology. Founded on critical pragmatism, systems theory, literature review and documentary analysis, in-depth interviews and case studies. A comprehensive comparative analysis of the VET system undertaken between Sweden, Denmark, Finland and Australia.

After afternoon tea, I attend a couple of sessions in the 'Pathways to FE, HE and the labour market' stream.

- Beginning with Zihao Lu (Phd from Nottingham University) with Professor Volker Wedekind presenting the work on 'examining the shifts in employment expectations among Chinese higher VET graduates during post-college transitions through the lens of 'capacity to aspire'. Provided overview of China - which has a huge higher VE as China has put large investments into higher and vocational education. HVET requires more funds than general HE but is funded much lower, In general, most of the HVET are from more disadvantaged students, HVET students moving into HE transitions is a critical period for many individuals.

Shared the rationale for the PhD study - and the use of 'capacity to aspire' (Appadurai 2004). Personal affairs are inseparable from broader ethical and normal norms and aspirations are always formed in the deep interaction with society. There are person's capacity to aspire, called an aspiration window is often bordered by each individual's circumstances.

Summarised research methodology and selection of participants. Then summarised the findings. For most, the preference post-college was to transition to university. However, for many, this aspiration can not be met due to economic constraints and many move into work. In general, aspirations for occupations were congruent with their qualification. Higher qualifications were aspired to as it would lead to higher salaries but also have higher prestige and long term career prospects. However, aspirations and realities clash when they move across to HE. Identity and stigma from coming via the HVET path along with challenges with learning at a higher level are challenges they face.

School to work groups lowered their expectations for salary and faced competition for jobs. However, most had settled into the lower salary by the time a third interview was carried out. Acceptance came about due to the current job market and lack of alternatives. Most in the group felt that HVET was not worth it.

- Then Professor Bettina Siecke (University of Mannheim) on 'structures and developments in guidance on initial and further training in small craft enterprises - a qualitative study. Provided background and rationale for the study. Careers counselling on further education is seen to be important, given the volatility and rapid change characterising many occupations. For SMEs, it is more challenging to provide resources or counselling. The study looks into how to develop resources to support careers counselling for SMEs.

Shared the research questions and aims - around understanding SMEs structures and how these need to be taken into account for the development of careers counselling resources. Defined the many types of ways counselling for further training is structured/organised/delivered.

Small (6) interviews with personnel managers and data categorised and collated into case studies. Brought together information on counselling activities - which varied from very limited to more extensive. Focus of sessions mostly on planning for further training, personal development and work-life balance. Several case studies - from no career counselling to more comprehensive counselling.

A move then made across to the last presentation in the stream 'The social role of colleges' to catch up on Leesa Wheelahan's ongoing work. She presents on 'vocational education colleges: adaptive or transformative institutions'. Continues discussion on the topic from past JVET conferences. Runs through the origins, problem and arguments. The reviewer of the paper (Wheelahan & Moodie, 2005) on institutionalisation of vocational education and its use of the capabilities approach was challenged by and this forms the basis of the ongoing discussion, presented today. The problem seems to be about institutions, the nature of state and their responses to globalisation.

FE, TAFE, vocational colleges are being deinstitutionalised, more marketisation and privatisation of the sector, interchangeable providers in a market, governments devolve their responsibilities to the market. Argues that vocational education needs to be expansive to meet present and future challenges - but what does this look like?

Useful theories informing the project include globalisation and the state, neo-liberalism (Wolfgang Streeck - the neoliberal state and the problem of democracy -see overview article and concept of individualisation of risk -/ and book for detail. whereby the individual needs to take care of themselves and takes the blame if things do not work out).

To try to meet the challenges, the capabilities approach (Bovin & Laruffa, 2018) views humans as receivers, doers and judges. Human flourishing requirs capabilities to live of life that has reason to value (Sen, 1999). Schroer, 2015 work on capabilities for work, education and voice. Also the principles under Catholic social policy which developed into the principle of subsidiarity - which now have several variants.

The role of institutions is therefore important, as it helps provide direction. Could using concepts of capabilities and subsidiary help institutions think about how to be more adaptive and transformative. What is the role of institutions going into the future, given the continued neoliberalists ideology that pervades western economies.

I then attend the JVET editorial board meeting which is followed by dinner. A longish day. Good to meet up with kindred spirits and catch up with familiar compatriots and also engage in scholarly discussions.