Promoting authentic learning experiences: redesigning assessments to minimise student misconduct

Webinar organised by TurnitIn and hosted by ASCILITE. 

Summary of the webinar's objectives as:

 1. Authentic assessment design: Dive into the principles of authentic assessment design, which emphasises real-world relevance, meaningful tasks and opportunities for creativity and critical thinking. Explore methodologies such as project-based assessments, case studies and simulations that provide students with authentic learning experiences while reducing the incentive to cheat. 

2. Formative feedback and self-assessment: Discuss the importance of formative feedback and self-assessment in promoting academic integrity. Learn the role of continuous assessment methods, peer review and reflective exercises in fostering student engagement, metacognitive skills and ownership of learning outcomes. 

3. Assessment security measures: Explore security measures and deterrents to minimise temptations for cheating in assessments. Discuss strategies such as randomised question banks, timed assessments and proctoring technologies that mitigate opportunities for academic misconduct while preserving the integrity of the assessment process.

Notes taken:

Kwong Nui Sim representing ASCILITE welcomes participants and runs through zoom functions. Includes a Welcome to country and an overview of ACSCILTE and the annual conference this year in Melbourne in December.

Chukwudi Ogoh from Turnitin chairs the panel. Diagnostic poll starts to provide information from participants to the panel. Academic integrity still an important focus for participants. Many use portfolios or similar to reduce academic integrity.

Speaker introduced. Joon Nak Choi from Hong Kong University of Science and Technology begins with the view that there is too much focus on academic integrity with slow integration of AI into teaching and learning although students are racing ahead with using AI. Formerly, UK promoted values based education, Germans then introduce skills based education but presently knowledge based education is the main focus. However, Ai is challenging knowledge based education therefore it is important to return to value and skills based education. Therefore critical thinking, adaptability and judgement should be the fundamental objectives of learning through active learning to raise metacognitive awareness. Formative assessments are important. Essays can remain relevant and useful as they help assess critical thinking, logic and argument and application. Metacognition, adaptability, initiative and self direction can also outputs from essays. In the post-AI classroom, essays still possible by putting them through turnitin, or writing essays in class, and integration of AI into essay writing reflects workplace use.  AI assistants can be used to grade in class essays to provide timely feedback, allowing the essays to be used as a formative tool. Professor or teaching assistants can then use their time to personalise the feedback. Teach students how to use AI to brainstorm, first draft from AI with student going through to correct/triangulate, presentation on the topic required, and the final draft incorporates human feedback backed by reflection memos.

Dr. Dianne Stratton-Maher from University of Southern Queensland on embracing AI to reimagine assessment in nursing education. Teaches first year Bachelor of Nursing course on literacies and communication. Overall perspective is that AI is not a replacement for personal knowledge, but rather they are tools that can strengthen and enhance the learning journey. Undertook a qualitative descriptive design to integrate AI into the course. Case study used and students were introduced to ChatGPT to support the completion of the case study assignment. AI generated video to introduce AI and provide some AI literacy on how to use AI to support learning. A reflection was also required as to how using AI went and this was handed in as an appendix to their case study. The reflection included prompts used, responses received, evidence of evaluation of the responses etc. Shared examples of student work. 409 students submitted, 18 reported for AI but 10 were unsubstantial. Collected f2f and focus group data, using thematic analysis supported by CoPilot (with UniAQ data protection). Themes included confusion and adaption; functionality and skill development; effectiveness and reliability; academic integrity and ethical use, perceived benefits and future use; and feedback and support. Recommendations included the need to clarify guidelines and expectations; promote responsible use; providing training and support; encouraging critical evaluation; highlighting the benefits; integrate practical examples; monitor and adapt and education for staff.

Associate Professor Benito Cao from the University of Adelaide on 'don't ne sorry, just declare it'. No assessment re-design but focused on working with students on how ChatGPT is not reliable but can be useful in some aspects of academic work, Therefore, is ChatGPT used, its use is declared in an appendix. ChatGPT is not a search engine but a language model. Asked ChatCPT to generate a profile of himself and shared with students. Almost all the items were wrong! but the profile was written in a convincing manner. Summarised his university's guidelines including Deputy Vice Chancellor Academic to all students to encourage them to embrace and integrate AI but to use it with integrity. Assessment guidelines define how AI can be used - for example AI assisted ideas or AI assisted editing. Discussed the benefits, limitations and risks and risk management for Gen AI as provided but the South Australian Government. Industry guidelines also introduced, for example guidelines from Taylor and Francis allowing idea generation and exploration; language improvement; interactive online search; literature classification and coding assistance. AI must not be listed as author and its use formally acknowledged. Stressed to students the need to keep their voice and appendix recording use of prompts etc. required. 

Jane Mair provided an overview to Turnitin's approach to minimise cheating. A challenge presented by paper-based handwritten assessments. Short answers, multiple choice, sketches or labelling diagram, mathematics problems. Paper assessments persist as they align with authentic design, formative feedback can be provided that engages students and supports assessment integrity by limiting access to AI writing and other digital tools. Challenges for instructors, marking can be time-consuming, coordination and consistency for grading can be difficult, feedback inefficient, heavy reliance on multiple choice which may not assess relevant learning outcome and challenges in applying assessments for best practice. For students receive delayed feedback, some have difficulty understanding the feedback and there are increased errors and there may be need for clarification. 

Turnitin feedback now has the 'paper to digital' overview. Grading is faster using online grading, feedback is more timely, secure workflow and digital storage, and integration directly into LMS and Turnitin feedback studio. Supports long-form writing, multiple choice, short answer, math formulas etc. Demonstrated how it worked. Horizontal questions possible allowing each question to be marked across the cohort, rather than one student at a time. Similar answers can be graded at once! Feedback studio allows for an efficient workflow.

A higher education focus in all the presentations. Some good ideas if essays are still being used. Webinar is recorded and presentations will be available. 

Resources for AI - course for higher education and assessment ideas

Two good resources to support the integration of AI into learning and teaching.

Firstly, a resource for higher education, hosted by Tilburg University (the Netherlands). The course introduces Gen AI and LLMs, discusses the opportunities and covers limitations and ethics, helps educators evaluate the influence of Gen AI on course design, evaluate the impact of Gen AI on assessments and encourages modification where necessary and identifies various ways educators and students can use AI in a responsible manner.

Participants are advised that the course will take around 2 hours and is user friendly. There are short video clips to explain some concepts, an overview statement at the end of each module, and a self-check quiz.

The second resource is from the UK it is a slide deck, which offers assessment ideas for an AI enabled world. Various ideas are offered with background and suggestions for how they may be incorporated in to higher education. The 'cards' are opportunities for further discussion amongst colleagues with encouragement to customise the cards for use by individual faculties to mirror disciplinary influences and frameworks.

Learning Design Voices -

 Learning Design Voices captures the perspecitves of learning designers post-pandemic. The book is open access and edited by South African learning designers/researchers T.Jaffer, S.C. Govender and L. Czerniewicz. 

After an introduction, the book has three sections (provocations).

The first provocation is 'what might learning design become in the post-COVID university? has five chapters. Each cover one of several themes -

1) increased visibility and value of learning designers due to the need to shift to flexible delivery models.

2) open learning design - on the margins

3) learning designer as pedagogical advisor

4) learning how to design learning through mimicry and mentoring

5) indigenous learning practices

The second focuses on compassionate learning design for unsettling times, with six chapters. The chapters focus on issues of equity, access, humanising learning, inclusivity and community.

The final provocation ' the challenge of designing experiences' has 13 chapters. Here, various examples, case studies and reflective studies are presented.

14) adapting ABC learning design

15) Using Laurillard's learning types

16) Rapid development prototype model

17) blended learning

18) elearning tools

19) knowledge-identity nexus

20) learner centred learning

21) authentic learning design

22) developing critique and argument

23) rethinking the textbook

24) digital divide and accessibility to print-based learning

25) ten principles of alternative assessment

26) reimagining authentic online assessments for large classes

27) inclusive online assessment

All in, may relevant chapters to inform the work of learning designers. 

OPSITAra - Day 2

 Day two dawns fine but cloudy, after a drizzly day yesterday.

The conference begins with presentations across three streams. I attend and present within the teaching and learning 2 stream. 

First up, Bruno Balducci from Otago Polytechnic presents on 'meeting the challenges of AI and assessment security'. Aims to develop strategies for assuring the integrity of assessments. Control measures include preventing AI misuse through well-designed tasks. Used a pragmatic research design to find out what aspects were more vulnerable to AI. Research questions included - could AI be used to 'cheat' in selected assessments now and into the future. If so, in what ways? What are the common features. Semi-structured interviews with tutors (25), reviewing assessments (30+), checking AI use by students and running trials with ChatGPT to see what could be achieved. Identified strategies of specificity, practicality, process, personalisation and information flow to design and manage assessment tasks. Provided examples and discussed pros and cons.  Good summary of how good assessment design is usefully deployed to ensure assessments are personalised, authentic and locally contextualised.

Then, Rehan Masood with Maria Grace, James Flanders and David Finnie also from Otago Polytechnic, on 'Are we prepared for assessing AI content?'. Presented an overview of AI including the differences between discriminative and generative AI. Summarised the ways in which AI could be used in education. In assessments, there is a need to move away from stop-test approach towards meaningful learning activities. Summarised the ways teachers can use prompts for AI content creation. Study investigated how to us AI to generate content for assessments (in a construction courses context). Summative, formative assessments and rubrics generated. Missing from AI is contextual understanding, creative innovation, common sense/intuition and ethical considerations. Therefore, assessing these is one way forward. When creating questions using Gen AI, prompt literacy important. Recommended framework to improve assessment and assessment submission design.

I then present interim findings from our AI@Ara project. Provide summary of the project including what each of the 6 sub-projects did to integrate AI into their learning activities as framed by Sharples, 2023 concepts of social Gen AI for education. Then, preliminary findings reported which include interim framework for embedding AI literacies and recommendations.

Lastly, presentation from Lesley Brooks with Michael Greaves, Megan Kitching and Leoni Schmitd (Otago Polytechnic) on 'alternative modalities of postgraduate research assesssment'. Questioned why there are limitations to written thesis. Māori and Pacific learners have strong oral cultures, learners with disabilities disadvantaged and professional practice uses a range of methodologies that do lend themselves to thesis format. Investigated alternatives, their issues and risks and to find and offer solutions wherever possible. Interviews with 29 stakeholders. Caveats included modality being grounded in existing research; documentation, timeline with gateways to manage risks, institutional supports and awareness of the equivalence across the different modalities. Delphi technique then launched into with survey to 49 stakeholders. A key concept was the definition of assessibility - including availability of suitable assessors, ability to assess to the criteria, ability of programme to support the process and approach - Schmidt, L., Greaves, M. Brook, L. abd Kitching, M. (2022). Modalities of assessment matrix workflow. 

Recorded teaching and learning presentations follow morning tea. There were 10 recorded presentations across a range of disciplines. 

After lunch, a series of short presentations in the communities and institutions stream. There are presentations on local food growers, cyber security, prompt engineering and ChatGPT, performance-based research funding on accounting academics, and virtual exchange programme in environmental management. Presenters are from SIT, OP, and Wintec)

Then attend the last group of Teaching and Learning 3 presentations. 

Beginning with Ana Terry from Otago Polytechnic presenting 'the big picture: visual literacy in teaching and learning'. Defined visual literacy in teaching and learning as ability to design robust visually-led tasks and assessments, using visual thinking tools and methods, visual design principles to clearly communicate, select and /or create relevant graphics, and ethical, social and cultural issues in using and sharing visual media. Visual content often an 'add-on' and often not used to support the text, just as an aesthetic. Shared findings of a survey to find out how ITP teachers understand visual literacy and to use the findings to improve visual literacy professional development for teachers. Teachers generally confident with basic creation of graphics etc. but less so when integrating into the teaching objectives - support other literacies and discussing ethical and cultural issues. Indicates a text based biased to teaching and learning. As people progress through education, visuals reduce becoming dominated by text. In vocational education and training, graphics are more acceptable. Especially useful for diverse learners, to provide better representation across various media, rather than just text. Proposed the formation of a Visual Literacy community of practice to continue work in this area.

Followed by James Mackay and Helen Jeffrey with Hana Cadzow, Ema Tokolai & Rita Robinson reporting on progress of degree apprenticeships with 'the tale of two degree-apprenticeships: embedding work integrated learning degrees'. Two degree apprentices - one in Bachelor of Engineering Technology on Infrastructure Asset management (now in its fourth year) and Bachelor in Occupational Therapy (OT) (first delivery in semester 2 2024). Each shared the challenges. Both had skill labour shortages, and with OT there was a high demand for Māori occupational therapists. Generally, effective school and community liaison important, marketing and employer relationship development, collaborative programme development, developing institutional readiness and ongoing evaluation. Also, addressing local, regional and national skills shortages, institution has capability, clear employer demand, career progression pathway and the institution must have competitive advantage. Differences in each criteria between each degree apprenticeship. see guidelines - Gorb, E. (2022). Guiding principles for Degree apprenticeship - Manchester Metropolitan University. 

Last up, Danny Friburg with Amy Benians and Kath Danaher presented on 'Communities in the mist: Exploring CoP model for professional development. Reports on phase 3 on -neurodiversity CoP outcomes, established in 2022 with teachers working in the context. Defined CoPs (see Wenger & Snyder, 2000 and Alison Viskovic's 2005 work).  Study to understand the value of CoPs to individual PD; what are the percieved benefits and how to better support CoPs. Check Kennedy, 200f on increasing capacity for professional autonomy from transmissive (training, award bearining, cascade), to transitional,  standard based, coaching, CoP) and transformative (action research and transformative). Surmised that 6 year threshold signals a shift from transmissive to more transitional teaching approaches. 98 participants in pre-survey and 96 in post-survey. Shared findings. In general, teachers saw value in partcipating in the CoP. Awareness, knowledge increased including ability to find solutions , develop community and profesisonal networks and contribute to equity across the institution and meeting institutional values. Correlation between years of teaching and confidence in teaching neudivergent ākonga in post-survey but not in pre-survey. 

Video played of Jamie Smile - who is the recently appointed Ragahau and Research Director for Te Pūkenga. 

Various conference awards close the conference. 

ChatGPT and assessments

 One of the main challenges facing educators when digital tools can be used for a host of writing tasks, is that of assessments. How can the integrity of assessments be safeguarded when students are able to use nefarious means to complete their assessments.

Over the last two months, there have been a large number of articles discussing the issue. 

- FENews proposes the use of a variety of question types, effective proctoring systems and different testing types e.g. using oral presentations or practice-based assessments instead of written exams.

- The conversation advices the need to rethink assessments. In particular, to ensure they are authentic i.e. connected to personal context or the course specific material. Project-based, group and inquiry/problem based learning are recommended.

- workhe.com recommends that educators trust their students and to take on the possibility of discussing with students, the pros and cons of using AI to augment, or scaffold their work.

- Pulse in the context of teacher education reminds us that learning is a process, not a product or artefact. Hence authentic assessments are the way to go.

- uts provides good ideas on how students and teachers can draw on AI to support their work. Good ideas include asking students to use ChatGPT to generate a response to the assessment and then set up the criteria to critique the response and provide feedback to improve on it. 

In all, it is important to include context into assignments and to ensure students take ownership and responsibility for their learning. AI in the form of Chatbots able to provide responses which mimic those of humans, are here to stay. They will also improve rapidly as they learn from the feedback to responses provide by users. Therefore, it is important to ensure educators understand the implications and work through the consequences. The opportunity is now provisioned to improve learning and assessment processes, so that assessments are for learning and not of learning. 

Visible learning a way to end exams

 Here is a good overview of the current focus on re-thinking the ways education 'measure' learning. The short blog also includes the video by Toby Morris 'on the plat' summarising the challenges for those with lower social capital to attain equity within the current educational system.

The above, reiterates the overall them of this year' s Ulearn coference - see the notes on the first keynote setting the tone of - success for all and how 'measurement' of success needs to be shifted. 

For over a decade, I have advocated removal of 'summative assessments' from our course descriptors, as a way to move emphasis to formative assessments. Due to the current NZ Qualification Authority guidelines, it is not possible to remove summative assessments as such. So we are still 'teaching to the assessment' in many courses, as student success relies to 'passing the course'. Regardless of where the learner starts on their learning, they still have to meet 'standards'.

When I was teaching, about 1/2 the class would 'pass' the assessment, given slight 'steering' to establish a contextualisation of what they already knew, to the topic to be 'taught' and assessed. These students, just needed a few deliberate practice sessions, to polish skills, review theory and had the 'right' dispositions to do well. For the 'other' 1/2, it would take much effort on their part, to step up and my job was to help them, often, to just get to the start point (i.e. to scrape through). Yet, if they were willing to put in the effort, they would succeed in work. So, in the long run, it is the formative and the attainment of individualised learning goals which are more important, than 'passing the exam'. Many, in the 'other' 1/2, gave up along the line, as they saw the 'exam' as too difficult :(  Yes, they were making some progress through formative assessments. Many, feared 'failure' as that had been their experience, throughout school. Yet, formative assessments were looked forward to, as the feedback, provided the guide to improving, little by little, their confidence and skill. As confidence increased, so did their attitudes to learning. Always a win-win, as far as I was concerned. 

Therefore, my long 'crusade' to remove summative assessments. They are next to meaningless for many learners, measuring performance within a very narrow bandwidth. A means to stratify people into the 'cans and the cannots' when society needs everyone to contribute towards solving the 'wicked' challenges of humanity. So, more work still on promoting a better understanding of the role of assessments, in a world which is not 'black/white', where there are no 'right/wrong' answers and where 'out of the box' thinking may be the only way to ensure our planet is still livable for my grandson's generation.

Kick-off session #12 - Powerful and authentic assessments

 Today's lunch time kick-off session was facilitated by Jane Bates, one of our team of educational developers at Ara and hosted by Marion Hale. The topic covered was on powerful and authentic assessments, in particular the transfer of learning to work. 

Jane provided an overview before several lecturers (Michelle Thompson from nursing, Dr. David Weir from computing and Dennis Winter from architecture) provided examples of their assessment practices. 

Jane began with key points towards ensuring assessments enhance students' learning and work readiness. Reviewed the concepts of 'work ready' vs 'work ready plus' graduates; and the Ara teaching and leanring framework. Work-ready plus draws from the work of Emeritus Professor Geoff Scott. Work readiness includes generic/transferable skills and role and discipline specific skills and knowledge required for effective practice. These could be thought of as competencies. Work ready plus also includes capabilties like personal (self awareness, decisiveness, commitment), interpersonal and cognitive skills and knowledge. 

Summarised the Ara teaching and learning framework which takes us to 2022. Key attributes or capabilties include graduates being critical thinkers, able to act sustainably, communicate effectively, innovate, be cuturally intelligent and responsible. Provided link to access the framework.

Emphasised the need to ensure assessments are embedded into teaching and learning processes. Referred to presentation by Arun Pradhan who referred to 21st century requirements including digital fluency, learning to learn skills, problem solving, working collaboratively and empathy and the T shaped graduates with both disciplinary and broad soft and cross-functional skills.

Defined power assessments as high quality assessment practices that address key employment competencies and personal, interpersonal, and cognitive capabilities.

Assessments have to be authentic and demonstrate what people can do, are problem based and directly related to what is being learnt. Graduates need to have industry skills and knowledge but also the personal, interpersonal and cognitive capabilities to adapt to the challenges of the future of work.

Michelle shares her example, first year nursing students reinforcing communications skills as practiced within the health context. Used scenarios and students practice with each other to hone these skills. Used videos to assist with formative feedback and eventual completion of the summative assessment. Nursing have been refining this process for several years.

Dennis shared the architecture year 3 project. The projects are supported with a comprehensive brief as per architectural practice. Students present their project as it progresses with 'crits' from tutors, peers and industry, again reflecting industry practice. An exhibition of students' work and presentation along with portfolio of work also required to evidence their learning. 

David shared the project-based year 3 capstone computing course objectives and results. Year 3 computing students work on an industry led project through a semester. Their results are reported through a report, poster and a presentation. Important to ensure computing students are provided the opportunity to learn key communication and presentation skills.

All the three examples use authentic assessments to elicit students' reflective learning on not only the core knowledge and skills of the discipline but also the occupationally specific dispositions required to be effective professionals. Lecturers model the industry ways of doing and thinking and provide important 'checks' as students progress through the curriculum.

Active discussions ensued after each example is presented.

Jane provided summary - a powerful asessment checklist and examples of powerful assessments- before thanking all presenters and closing the session. 

Ara Kick off day -

Ara’s ‘Kick-Off’ day brought together all the staff of the Academic, Innovation and Research (AIR) division to launch the creation of the new entity- which was formed late last year through combining all the teaching departments with the academic division.

The day begins with welcome and key messages. Formal karakia and the Ara waiata begin the proceedings with George Nelson – Deputy CE and the Ara management team leading the waiata. Hemi Hoskins, Head of Department explains the significance and meaning of the karakia.

George provides the welcome and details on the day’s activities. Plans for the event to continue into the future at the beginning of each semester. First activity based around the AIR theme from Araport with learners on a journey on the AIRplane. Vision shared on the way forward for the Ara education ecosystem. Overviewed the current and proposed the future, using airports as a metaphor. Need to move beyond what we currently offer, to be more flexible and agile – meeting needs for the working population to continually develop their skills and knowledge as the future of work evolves.  Ecosystem includes shifting to supporting enterprise / entrepreneurial opportunities, seminars and workshops from staff after professional development participation and as ‘thought leaders’. Detailed some of the current and planned activities going forward into the next few years.

Executive team updates follow through a series of ‘pods’ to be worked through. Tracey Berry our customer experiences DC provided an overview of her team’s work to support successful students. The involvement of students as co-creators of various processes and systems is the main approach. Then Tony Grey our CE provides summary of strategic direction through building and support of climate and culture, moving from good to great, forming and continuing community and belonging, increase responsiveness, innovation and sustainability. Belinda DCE for People and culture than summarises the stategy to empower our people for tomorrow’s world. Dean our Chief financial officer summarised his role in resourcing the many current and future initiatives. Te Marino Lenihan, Ara kairahi used the harakeke bush as the metaphor for the importance of supporting the improvement of success for Maori students.

Then a series of workshops 7 workshops convene. The workshops centre around AIR activities including assessment, programme evaluations, programme development, teaching and learning plans, and online teaching and learning.

I participate in the session on assessments, with Glynnis Brook, manager for portfolios and assurance and facilitated by fellow educational developer Jane Bates. The session was an opportunity to review the development of powerful and authentic assessments. Began with reiterating that assessment is for gathering information and also evaluation as the interpretation of the information gathered. A good discussion eventuated around the difference between summative and formative assessments and the need to explain the importance of formative assessments to students. Overviewed the principles of assessment and practice.

After lunch, two sessions of workshops of 45 minutes each follow.

In the first round, topics include developing cultural capability, getting started with research, AR/ VR, innovation, inquiry base teaching and the students’ portal MyAra App.

The second round of sessions include getting research published, e-assessments for learning, neuroscience of emotions, agile development of programmes, Microsoft tips.

Firstly, I joint facilitate the session on getting started with research with Dr. Isabel Jamieson from Nursing. We collect questions from the participants, sort them into ‘themes’ and workshop the answers, using the available institutional resources.

Then, facilitate the session on eassessments for learning  - as an output of the Ako Aotearoa and NZQA funded project. The session workshops using templates for designing assessments of learning and how to match the learning outcomes to the types of digital tools which are relevant to provide the most useful forms of feedback.

The event closes with a wrap up with George. Thank you to all the presenters and the planning team.

A good opportunity to network with colleagues from the different campuses and catch up progress on various projects completed over the last few years. Also important to reinforce the organisation messages and policies so that the institution is in a good position to connect with or critique the outcomes of the coming NZ Review of Vocational Education (ROVE) announcements.

Assessing learning conference - DAY 2

A full day starting a 9am.

First up, Keynote panel – on the student voice facilitated by  Dr. Alistair Shaw with 4 students. As always a very valuable session. In short, students did not know about learning outcomes and how they connect to the assessment. Students preferred authentic assessments  which reflected real-world practice. Each institution has culture of practice and differen priorities. Not all have ability to provide authentic learning but assessments may be a means to bring authenticity into courses.

Andrew Kear with a team from the BCITO with ‘assessment in the workplace: principles for on-job assessment’. Gave out copies of publications relevant to NZ context. Shared the BCITO guiding principles and how they connected to Karen Vaughan and Marie Cameron’s good principles of workplace learning and assessments. A clear purpose for assessment is crucial. Provided overview of BCITO to apprentice support, workplace learning and assessment processes. Philosophy and approach is key with all BCITO understanding the distinct culture of BCITO. Belief in each learner is an individual. Group session to discuss how organisations may be able to support individualised learning programme. Assessments need to gather progressive evidence of learning and also be contextualised to be relevant to the learner. Learners should not be put through ‘hoops’ but have authentic evidence of learning recognised – maximising the use of naturally-occurring evidence. Evidence does not have to be written, could be video, aural etc. important to allow annotation of evidence. Moderation has to contribute to the validity and reliability of assessment decisions. Although time consuming and expensive, still has to take place and ‘communities of practice’ amongst ‘assessors’ and moderators, both taking part in the assessment. Moderation is a second opinion. Entire process requires appropriately recruited, trained and professionally developed people.

Followed by support of Faye Wilson-Hill and Niki Hannan from Ara Institute of Canterbury on their work with OneNote as an assessment tool. Provided background of the programme on why the assessment portfolio tool is used. Not only to be an assessment for learning resource but also to model to other teachers, a platform to support learning. Shared assessment principles – integrated into learning process, draws on learners’ experiences, encourages reflection and allows for multiple points for formative feedback. Moodle did not allow for all of these principles to be deployed. One note classroom notebook was selected as it allowed principles to use. Detailed process – how to start – shifting a word document into Onenote. Begin with familiar and work in the online environment first. Reflective practice has to be scaffolded. Showed example of how the notebooks used and structure of the notebook. Feedback is progressive as the course goes on so student have formative assessment for learning every 2 weeks.Feedback from tutors can be written or oral. Used a video capture (Panopto) for students to share portfolios if there is no collaborative space. Concluded with reflection on the process. Still learning but holds promise.

Then a choice of two plenary sessions after lunch. I select Dr. Eleanor Hawe’s on assessment for learning: A catalyst for student self-regulation. Defined assessments for learning and the second generation conceptualisations. In general, formative assessment research in school sector, formative feedback could be more dialogic; and the need to have explicit focus of pedagogy on preparing students to be independent learners. All assessment should support the advancement of student learning (Carless, 2015); assessment does not stand outside teaching and learning, but stands in dynamic interaction with it (Gipps, 1994); Students are no longer objects of their teachers’ behaviour but animators of their own effective learning (James & Pedder, 2006).

Second generation definition – assessment for learning is part of everyday practice by students, teachers and peers, that seeks, reflects upon and responds to information from dialogue, demonstration and observation in ways that enhance ongoing learning (Klenowski, 2009); Therefore, should be part of pedagogy – should be the formative use of assessment (not formative assessments). Aim to develop students as self-regulating learners who can monitor, regulate and control their thinking, behaviour and motivation while engaged in ‘academic tasks’. Sadler advocates that students need to know what is expected  (quality); sufficient evaluative knowledge and expertise to be able to compare current thinking / learning / performance; and a range of strategies to enable to effect improvement and further their thinking / learning / performance.

**Recommended five strategies. Promote student understanding about goals of learning and what constitutes expected performance; engineering effective discussion and activities including assessment tasks to promote and elicit evidence of learning; generate feedback (external and internal) that moves learning forward Use, notice, recognises, respond – Cowie & Bell, 1999); activate students as learning resources for each other; and activation of student ownership over and responsibility for their learning (Hawe & Dixon, 2017; Wiliam, 2011).

Provided an exemplar to illustrate the way in which 5 strategies are operationalised. Need to ensure contextualised to own discipline and practice. However, all 5 need to occur.

Then I present a short session on the eassessment project. The focus this time around, to connect assessments for learning and feedback opportunities to assisting students ‘learning to become’ as they strive and learn to meet graduate outcomes. Summarised the role of assessment for learning in assisting the student journey towards getting to the graduate profile. Details of the sub-projects and some interim findings. In particular, how to assist student to learn the many ‘qualities’ which are often difficult to describe and to work out where they are at and what they need to do to attain.

Followed by a session with Alastair Emerson from OPAIC on ‘developments in assessments for experiential, student centred and partially self-directed pedagogies’. Otago Polytechnic international based in Auckland’s student cohort tend to have only experienced chalk and talk, paint by numbers assessments and discouraged from forming own opinions. Flip learning has not worked, essays and exams are off limited use to assess capability, current assessments make passing the end goal, designed for a didactic knowledge transfer paradigm to constructive student co-created approach. Using formative assessments are incremental with worksheets and templates. Summative involves problem solving or project completion in a real world context involving actual company. Introduced experiential learning using guided self-directed discovery techniques, with diminished on texts, use templates and worksheets which suggest outcomes but do not necessarily have a set process. Need to make learning outcomes visible. Therefore move into project / problem / inquiry based learning. Provided example of worksheets and projects to encourage personalised learning.

Afternoon tea is followed by Plenary with Emeritus ProfessorGeoff Scott from Western Sydney University on assessing work-ready plus capabilities. Presented on the website flipcurric used to support the work. Advocates – good ideas with no ideas on how to implement them are wasted ideas and change does not happen but must be led, and deftly. Rationalised why bother about assessments. Learning impact is when learning design, aligned support and infrastructure and delivery intersect effectively. Summarised the 6 key components of a comprehensive, integrated HE assessment framework – check on flipcurric website. Focused on correct outcomes and assessments.

Learning outcomes – capabilities and competencies students are expected to demonstrate they have developed to a required standard by the end of a program or unit of study. Include personal, interpersonal and cognitive capabilities and the key knowledge and skills necessary for effective early career performance and societal participation. Shared his professional capability framework. Explained the subscales for each of the competences/capabilities. The Plus refers to future focus – sustainability, change savvy, creative and inventive, and clear where one stands on tacit assumptions driving current society. When through principles of powerful assessments and examples of types including key quality checks for assessment of prior learning and learning from experience.

Then Shaima Al Ansari reports on ‘the impact of PBL on employability skills development: The Bahrain Polytechnic industry project assessment case. Knowledge isn’t power until it is applied – Dale Carnegie. Explained rationale and context (business management studies). Project requires student to take on accountability, work with others as part of team. Assist with application of employability skills to attain professional identity. Summarised details of the industry project. A capstone project in the fourth and final year. Students set up a consultancy firm and are the associate consultants. Work with a real client on an ill structured / complex problem. Full time commitment with weekly 2 hour meeting with the academic supervisor who takes on the role of the HR specialist for the consultancy. Students have an orientation week before beginning. Project process detailed along with examples of guide sheets, assessment plans / schedules etc. Shared positive feedback from 'employers / clients' and  students. 

Networking session closes a busy day. Lots of reinforcement of principles we apply at Ara, the concepts underpinning the e-assessment project and some new ideas and resources to support educational development work and production of e-assessment guidelines.

Educational and work settings forum -

Presentations are now on-line on the Ako Aotearoa website. The forum took place in mid-August in Wellington and focused on workplace-based education with a range of NZ based speakers and a keynote from Australia. Over the long weekend (show day was on Friday last week), I had time on the Saturday afternoon when it got too hot to get any gardening done, to look through the powerpoint presentations.

Of most interest are the two keynotes – one from Professor John Buchanan of University of Sydney and Professor Richard Coll from University of Waikato. Both covered the principles of education undertaken in work settings and the nexus between work and under/post-graduate degree curriculums.

Pertinent to present work is the work of Dr. Marion Sanders from Bethlehem Tertiary Institute and Liz Bowen-Clewey of Competency International.

Dr. Sanders presented on findings from the Ako Aotearoa National Project 'maximising dialogue in field based experiences' from work with social work and teacher trainee programmes.

Liz Bowen-Clewey  presented on a way forward with work-placed based competency assessments. Taking into account a more holistic structure to recognising professional competency through the use of 'professional conversations' and other whole-occupation evidence gathering approaches.

Postman, Sennett and Jenkins on technology – cautions and recommendations

Some toing and froing via email at the Centre of Educational Development through my sharing of an article by Neil Postman has led to a discussion on how technology may be used to support teaching and learning.


I read Postman’s 1995 article – ‘Five things we need to know about technological change’ - as a caution – to remember that the media may also become the message. However, even if there will be pros and cons and winners and losers (Postman’s 1 & 2 cautions) and a risk that technology becomes the focus (cautions 3 & 5), technology is also ecological (caution 4) and WE as the makers and users of technology, need to understand the pros/cons and use technology ‘wisely’.

As per previous blog, Sennett’s book (the craftsman) also cautions on a reliance on technology. His example is the use of computer aided design (CAD) in architecture and how designs can become the be all and end all, mesmerising the architect with 3D rendered views of his/her creation and missing the important factor that buildings have to be designed to be lived in, be sympathetic to the environment (physical and social) they cover and comfortable and useful to occupiers/inhabitants.

The 2006 Jenkin’s et al. article – ‘Confronting the challenges of a participatory culture:Media education for the 21st century’ - explores the challenges posed by digital technology to education - with a slant towards formal school education. The examples provided through the article, inform us that it is possible to use technology to help student come to grips with learning for a future world that will rely on digital technology (and perhaps has already?).

So if we are preparing our students to become plumbers, do we also still need to help them become critical learners and users in a world rapidly becoming technology reliant? Who decides?? What will be the most appropriate vernacular/form of communication required by students learning to become plumbers/chefs/nurses etc.? What ‘communication’/media literacy skills are required? – Jenkin’s article includes Play, Performance, Simulation, Appropriation, Multitasking, Distributed Cognition, Collective Intelligence, Transmedia Navigation, Networking and Negotiation (see page 4 for details of each).

Therefore, will a reliance on text based communication be sufficient for future occupational practice? For instance, will trades people use YouTube to advertise their services on TradeMe? If YES, should we be therefore helping students learn to present on visual media instead of writing reports?? Both text and visual presentations require student to represent and organise their thinking – which form of expression will now be more authentic? What appropriate ‘thinking and recording tools’ should be introduced to students to help them make sense of, collate and represent their thoughts and learning?