After lunch, I attend a session with Steven Hodge and Reshman Tabassum on 'provider practices to enhance curriculum relevance in dynamic industries. Reshman set the scene with a scenario. The project was to explore RTO methods of interporeting and translating training packages. Conruses were in vet nurising, agriculture and rural operations. Did the RTOs approach qualifications with different intended outcomes and purposes? Posited that learning qualifications lead to a specific occupation, is for multiples occupations and have cross or multi-sectional interest. Process on interpretation goes from pre-translation, to translation and then mobilisation. Pre-translation requires understanding VET system, provider organisation, students, industry, training context. Translation involves interpreting, organising, elaborating, framing and structuring (through the quality assurance processes). Mobilisation is a continuum between educator and resource 'delivery'. Used a series of interview vignettes to unpack the processes of interpretation. What is important here is that no matter how detailed or prescriptive a standard is, currency of the standards and how they may align with diverse contexts are not guaranteed. Educators will use agency to work with, around and outside of standards, as required, to bring about learning.
Following is Karl Hartley from Epic Learning reporting on the ConCOVE funded project 'AI-generated assessments in vocational education: enhancing quality and accessibility through cognitive design'. Customisation of assessments is one way to meet individual learners' needs. Assessment writing is complex work and manual assessment development is slow, consistent and inequitable. How can AI be used with safety and integrity to assist. Use a case study of developing assessments for a 'trades essentials' micro-credentials. applied cognitive science, ethical AI and educator validation. Began by selection of GPT-based model. Prompt engineering informed the working memory theory. 3-rounds of subject matter expert review process, and stakeholder consultation. Gen AI can generate multiple-choice and short answer quesitons, adapt the tone and language complexity, contextualise questions by trade/experience and enable personalised based learner profile. Personalisation features include literacy-adjusted versions, industry-specific variants, support for neurodiverse learners (format, language). A robust AI workflow to develop generic base assessment, then use base assessment and use AI to apply customisation. Prompt engineering strategy included the use of chain of thought prompting, using variables around writing style. providing example questions, assessment guides and for Gen AI to produce in batches of 5 assessments. Development loop was taking too long and involved too much time. Now development 5 assessment in batches. Seek rapid initial feedback for a 'go or no go' call to do a full review. Flip the feedback from 'what is wrong' to 'what is good'. Provided example of how the process looked, prompts and outputs.
Challenges include AI constantly evolving, AI have been heavily trained with US of A assessments and are biased and AI will make mistakes. Risk and mitigations of Ai hallucinations by using structured prompt design and ensuring a human view is added. Cultural bias with ethics review and co-design with Māori and Pasifica advisors. Need to be wary of over-reliance on AI and issues of privacy. Benefits include the ability to have more inclusive and relevant assessments for learners; reduced workload to get to a quality product for teachers; and an opportunity for greater consistency and innovation. Plans for the next steps are to seek learner feedback and validation; finalise an ethical framework, expand to writing assessments for other microcredentials and disseminate findings and guidance.
The project demonstrates viable use of AI in assesment design; establishes moderation alignment proof of concept; builds foundation for personalised learning, and aligns to system transformation.
Then a session on 'Planning and Actioning VET Research Studies: issues to consider' with Llandis Barratt-Pugh. Shared 10 'critical incidents' that have informed his research practice, each supported with the narrative of how these came about. Used autoethnography, critical incident technique, reconstructive memory theory and the experiential learning cycle.
The pointers include: tell industry what you can do for them then listen to what they want doing; start with the end in mind and plan toward it with frequent monitoring; seek out lions in the field and engage them as mentors; gain top commitment or the project may be swept away; publicise research studies to gain collaborators; collaboration is the key to multiple publications; a research proposal should be developed by serendipity as we don not know what lies ahead; when confronted with data collection dilemmas the options are to change the plan, call for support and the wisdom to know which is best; the research plan should include resources and action to use the finding constructively; taking a framework from a close discipline may shed unique light on the data in a wat that has not be done before; and the only certainty about a research project is that it will not progress as planned!
After afternoon tea, Craig Butler presents on 'Uncovering the Complexity of VET Teacher Identity' which is part of his PhD study. Summarised his background and his interest in the topic. Provided a overview of the study, with a theoretic framework of 'funds of identity', the research methodology, participants (25 VET teachers) and analysis method. Shared some of the findings around practical funds of identity resources (experiences, across many specialised contexts within and without a field) of vocation focus, VET practitioner, learner (life-long learning), expert performer, mentor, non-vocation (hobby , community work etc.) and institutional funds of identity through organisation role, employment type, and employment status.
Shared framework to visualise how participants, across their life-course, had at a point of time (the interviews) had a particular role identity. Implications include a need to better understand the integration of vocational and VET identities; and recommendations for further study. Shared how study could inform VET support of teacher identity and limitations of the study.
I offer the last session of the day, providing an update on our Gen AI projects with ' AI in Vocational Education and Training: Progressive learnings on the integration of Gen AI into vocational programmes'. The primary purpose was to provide an overview of the projects to date, the challenges and learnings and propose a way forward as to how VET can use AI to support learning and teaching.
AVETRA AGM is conducted and the conference reception and awards round of a busy day.