Assessment in the age of AI - open access journal article

 An interesting and timely article published Computers and Education: Artificial Intelligence. A relatively new journal with articles across the year compiled into an annual volume.

This article in volume 3 (2002) discusses the challenges of assessment in the age of AI and proposes some ways to leverage assessment practices which draw on AI approaches. The article critiques current assessment practices. The ways AI can be useful include automating assessment construction, AI assisted peer assessments, and using AI generated text to conduct 'writing analytics'. The challenges and caveats for introducing and using AI to support assessments are also presented with a focus on how AI may disfranchise educators from assessment processes.

A good reference article to use for supporting discussions on the impact of AI on assessments.

Presentation on Integrating technology into Swiss Vocational education - Professor Alberto Cattaneo

Last evening, we had the privilege of attending a presentation from Professor Alberto Cattaneo from the Swiss Federation University of Vocational Education and Training on various technology-enhanced learning projects carried out over the last 15 years.

The presentation came through connections made with  Professor Antje Barabasch, who is head of research at SFUVET and is currently on sabbatical in Aotearoa NZ, hosted by the University of Canterbury. 

Professor Cattaneo provided a overview of the Swiss 'dual system' and then detailed some of the projects. More detailed information is available in the recently published book on 'Educational Technologies for Vocational Training: Experience as Digital Clay'. In the decade and a half since the SFUVET began their work, they have progressed from static to moving multimedia (videos) and now work extensively in MR including AR and VR. Of note is their use of multimedia to engage learners and to use the records to nudge learners into reflective practice. Examples shown and discussed across the presentation provide us with affirmation of the work we have undertaken so far and goals to aspire to into the future. 

National Institute of Digital Learning (Ireland) - top 10 good reads for 2022

 The National Institute of Digital Learning maintains a topical blog with congent updates on the introduction, application and evaluation of digital technologies into education.

One of their latest blogs, summarises the top 10 reads in 2022, prefaced as -from theory to practice and back again. All the articles featured are open access and there is one recommendation from the 'closed' literature.

The blog provides links and undertakes a good overall survey across the articles (journals published in , countries etc.) and summary of each of the articles. Articles include two on the topic of learning design; Stephen Downes update on constructivism; topical ones on the metaverse and AI; and papers on research theory and how they can inform practice.

ChatGPT - impact on vocational education

 Just before Christmas, the internet was alight with various pieces on ChatGPT, an artificial intelligence (AI) chatbot which is able to generate written pieces/articles, generate code and compose songs and poems in various styles. Users input a small list of items, and ChatGPT produces the goods. The AI learns continually through Reinforcement Learning with Human Feedback (ELHF).

The platform is free (for the moment) with over a million users signed up within hours of launch, leading to it being often unavailable due to the sheer number of people using it. A larger number of information summaries, blogs, opinion pieces (some written by ChatGPT) have flooded the internet. These users, help the AI learn through its interactions, helping to increase the authenticity and realism of the outputs.

Relevant articles include ones related to how AI disrupts industries, its effect on education / classrooms, teaching practice and learning, how it could transform learning, the higher education response, including a 'if your can't beat them, join them idea '. 

 As usual, there is no available response contextualised to vocational education (sigh).The types of jobs threatened by AI bots include copy/content writing, customer support, and software developers /programmers, although the general consensus is that these jobs are safe for the moment, given the germinal nature of ChatGPT. Given the AI tends to provide written responses, similar challenges to those brought up in the articles above within school and higher education contexts arise when ChatGPT is used to develop written material for vocational education based assessments. For example, project proposals and the ensuing report, can be generated when an outline is presented to ChatGPT. Thankfully, practice-based assessments are the mainstay of vocational education and the learner still has to perform the required skills to the expected standards. 

How can vocational learners be prepared for the use of increasingly powerful AI tools that generate images and text when provided with the right cues? To begin, it is important for learners to understand the strengths and weaknesses of these platforms. Learners also need to provide the chatbot with the right information and the right occupational jargon, to ensure the output generated is realistic. This requires learners to have a good grasp of the content/topic, before they feed in the cues that are used to generate the output. The mechanics of writing (grammar, spelling, sentence structure etc etc) may be taken care of but it is still up to the user, to check, evaluate and finalise the resulting piece of writing.

As someone who spent many years honing the craft of becoming an academic writer, I can see the usefulness of something like ChatGPT to generate starter paragraphs. These emergent pieces may be used as precursory summaries, to help overcome 'writers' block' or to provide some of the scaffolds for the article's argument. Having tried out the platform with several 'article skeletons' I have found the generated text to be repetitive and sometimes long-winded. However, this could be due to the cues I have provided! It also works from what the user provides, so if the cues are not factually correct, the platform will be none the wiser. 

ChatGPT represents a challenge to how young people are taught how to write when written texts can be generated. Schools could ban the platform as it reduces the writing of school reports and essays, to plugging in some items around a topic, generating the text and copy and pasting. The long journey of learning how to write through practice, trial and error, and continued improvement, becomes lost when the act of actually consolidating a string of items into a cohesive whole is lost :( Therefore assessments focused on essays will need rethinking. It is now more important than ever, to think of assessments as FOR learning rather than OF learning. ePortfolios are a means to record the learning / writing journey and assessments based on evidence of 'reflective learning' rather than the end result (essay, report etc.) are one way to work around the challenges of AI generated work. 

We also need to think through the implications, ethics and philosophical underpinnings, When AI is able to respond in ways so similar to humans, it becomes difficult to tell the difference. As humans, we might lose some of our humanity, when a machine 'takes over' thinking and creative writing from us. With it, goes the self-actualisation associated with positive feedback from putting effort into difficult tasks. Evaluating the work of others, is not a replacement from putting in the hard work required to bring a creative piece of work into being. It will be difficult to push against the tide of AI generated work. We need to think through the wide implications, one of which may be an increased valuing of artisanal work, the one off bespoke products. AI may design these objects, 3D printers may be able to produce by the millions, but 'one of a kind' human designed and crafted products may be one way to ameliorate the rise of 'non-human' designed and manufactured goods. Vocational education, especially in the trades, will need to therefore emphasis quality of skills, to meet the market for bespoke items.

Plans for 2023

 The major challenge presented across the year which will impact on my work, is the integration into Te Pūkenga organisational structures and objectives across the year.

Te Pūkenga (The NZ Institute of Skills and Technology NZIST) brought together the 16 institutes and polytechnics (ITPs) and 9 of the 11 industry training organisations, (ITOs) in Aotearoa NZ. The formation of Te Pūkenga is but one of the outcomes of the Reform of Vocational Education(RoVE). Workforce Development Councils (WDCs) tasked with setting standards through consultation with stakeholders across the industries they represent; Regional Skills Leadership Groups (RSLGs) which focus on skill needs for each region; Centre of Vocational Excellence (CoVEs) to drive innovation in VET teaching; changes in the funding structure of VET, and simplification and alignment of qualifications,  all begin to consolidate.

Te Pūkenga itself is large by international standards, led along in Aotearoa NZ, supporting over 200,000 learners, with a workforce of 10,000 ++ and a charter that sets out aspirations to be met with regards to meeting the precepts of Tirit o Waitangi, equitable access to VET and parity of outcomes for all (and especially for Māori and Pacific peoples).

At present, my team are no longer part of People, Culture and Well-being and await allocation to one of the academic divisions being formed. This year will see most of the organisational structure being defined and put in place. There will be much change as the regional structures are also confirmed and the ways we work will include collaboration across our region (presently most of Te Waipounamou – the South Island) and also across the Te Pūkenga network. Amidst all of this, day to day work continues to support our teachers to provide engaging and relevant learning to our learners.

My membership of Nga Ohu Whakahaere o TePoari Akoranga o Te Pūkenga (the teaching and learning advisory group to the academic board for Te Pukenga) provides a window into the workings of Te Pūkenga and a means to contribute as the institute organises itself. 2 days of work a month is tagged to this role. At the inaugural hui (meeting) in December, the Ohu began the mahi (work) of confirming its terms of reference with a focus on its roles and responsibilities.

On the research front, I hope to submit two journal articles, both written last year and have them published this year. Most conferences I would participate in, have shifted back to f2f interaction. This presents a challenge as Te Pūkenga is not funding any work-related travel. I plan to fall back on using the $$ attained from recent publications and the marking of theses, to fund at least one conference in Australia, likely AVETRA at the end of April in Melbourne. 

It looks like another busy year, albeit with some uncertainties with regards to where my team fits into the wider organisational structure. Things should come clearer as the year progresses :)