Posts Tagged ‘change management’

Review: SACWG seminar, ‘The efficiency and effectiveness of assessment in challenging times’

On Thursday 24 November 2011, the Student Assessment and Classification Working Group (SACWG) hosted a one-day seminar, ‘The efficiency and effectiveness of assessment in challenging times’ at Woburn House, Tavistock Square, London. 

To open the seminar, Dr Marie Stowell (University of Worcester) set out the context for the day in her presentation ‘Efficiency and effectiveness in assessment’. She identified that one of the aims of SACWG is to explore variations in practice across the sector and how they impact differently on students, retention, and learning success, and she observed the importance of placing students at the centre of the process given the fee structure proposed for 2012 entry coupled to the implications of assessment and feedback to student satisfaction. In light of the new funding model, one particularly pertinent observation she made concerned the cost of teaching in relation to the cost of assessment: the latter is resource-heavy, particularly once one factors in elements such as formative assessment (for which quality is less assured than its summative counterpart), moderation, external examining, reassessment of failed components, and the possibility that students may be over-assessed in the first instance. She also suggested that assessment criteria may not warrant the detailed attention they are typically accorded, as students tend to take the more direct approach towards assessment of endeavouring by less formal means to uncover exactly what it is that the lecturer is expecting them to produce. These arguments may indicate that both the efficiency and effectiveness of assessment could usefully be enhanced.

The next talk, by Professor Alison Halstead (Aston University), explored how institutions have responded to the challenges of recent years, specifically, the White Paper and its implications to students and to Higher Education. She noted that the potential increase in the students’ financial burden will inevitably lead to heightened expectations concerning teaching quality, learning, and employability, in which respect assessment and feedback are currently among the most important issues. She warned that student challenges to the regulatory framework for assessment may be on the rise in the future and identified that it was imperative, in these changing times, to nurture outstanding, innovative teachers and for staff to support student learning and e-learning (including assessment). Calling for the abandonment of the rigid distinction often drawn between ‘teachers’ and ‘researchers’, she suggested that promotions should award teaching excellence on a par with research. Later sections of her presentation outlined recent initiatives at Aston, for instance, standardizing the use of Virtual Learning Environment across the institution, and introducing learning technologies such as lecture capture and electronic voting systems. Her view was that teaching-enabled practice, while it took more time upfront to implement, was worth the investment in terms of teaching quality and learning success.

A structured group discussion and question-and-answer session with the morning’s speakers ensued. One point that emerged strongly was the importance of maintaining a variety of assessments, organized in a carefully considered schedule that takes a holistic overview at programme level. The latter becomes much more difficult in degree courses that incorporate elective modules, though there are both pedagogical and satisfaction-related reasons for offering choice to students and giving them ownership of their programme pathway. Another preoccupation amongst delegates was that assessments do not become too atomized, but relate to one another even beyond the confines of the module with which they are associated; one of the more innovative solutions proposed was the possibility of assessments straddling two or more modules. The need to develop sustainable structures was also discussed (for instance, moving towards group assessment to cope with rising student numbers), as was the importance of considering (as part of change management) what the benefits of effecting the change might be; if these cannot be persuasively articulated to staff and students, the change may not be worth implementing. A final warning concerned being too driven by regulations in designing efficient and effective curricula: it may be more useful in the long term to refer obstacles presented by the regulatory framework upwards so that they can be addressed.

The seminar resumed in the afternoon with a talk from Professor Chris Rust (Oxford Brookes University) on ‘Tensions in assessment practice’, which opened by reiterating the themes of the seminar in noting that current practices are neither efficient nor effective. He discussed that students have a tendency to focus on the mark they will obtain from the assessment rather than on the educational content of their studies, and that their approach often becomes increasingly surface-level as they progress through their programme. He defended modes such as formative, self-, and peer assessment as potentially yielding more ‘authentic’ assessment, arguing that graduates should be able to evaluate themselves and their peers as an outcome of their programme, and that making greater use of these options might also free up staff resources for summative assessment. Noting that students do not warm to the notion of being assessed, he suggested that perhaps the word ‘assessment’ should not be used for formative tasks. He further observed that feedback practices might be made more efficient by strengthening the relationship between modules, such that students are encouraged to learn from feedback received in one module and to carry what they have learnt over to others. Lessening the sense of compartmentalization of individual modules would, in his view, lead to more inclusive structures albeit less flexible ones, in that standardization (for instance, in terms of the same word limit for all assessments) does not always result in appropriate assessments.

Then followed a second group workshop session, on the theme of ‘What can institutions do to mitigate tensions?’. After a structured discussion of the issues, each group reported back to the seminar as to the problems that they had identified and the possibilities for efficient or effective solutions. It would be impossible to do justice here to the vast amount of ground covered between the several contributing groups. To cite just a few examples, key tensions that were raised included giving formative assessment a greater purpose (a proposed solution being to tie formative and summative assessments together in more meaningful ways), the problem of ensuring parity when using several examiners for the same assessment task (which may be solved by grading the assessment as pass/fail only), and the evergreen question of quality of feedback versus timeliness of feedback (for which there was some discussion about feedback becoming ‘quick and dirty’). On the question of standardization of process, I took the microphone to report back on the standardized feedback proforma that had been created in liaison with the students and implemented across one programme at City University London (see this post for details), and suggested, with much support from the floor, that students should be more involved in consultation regarding matters of assessment and feedback.

Prior to the close of the seminar a final speaker, Professor Paul Hyland (Bath Spa University), provided some reflections upon the day’s discussion. Noting that assessment was a large topic with which to deal, he categorized the day’s discussion as having crystallized around four main areas: external scrutiny (ranging from students’ parents to formal regulatory bodies); administration and management; the tutors’ perspective on assessment; and the students’ perspective. He argued that discussions of  effectiveness and efficiency should always be mindful of the purpose of assessment. In his view, assessment should be concerned with measuring students’ performance and nurturing learning, whereas there exists a danger of (to put it crudely) simply setting assessments in order to get the students to do some work. In this context, a greater level of student involvement and engagement with assessment would therefore be beneficial. He also observed the need to use technology to improve existing practice, for instance, to supplement traditional modes of feedback with video and screen casts. Finally, he commented upon the importance of tutors having access to students’ feedback on previous assessments in order to understand where they are coming from and to be able to support them in their ongoing studies.

SACWG has kindly made available the presentation slideshows used by the speakers, and the comprehensive notes distilled from the two very productive group discussions (as reported back to the seminar by nominees from the groups), at the following link:

SLE webinar for ELESIG: From Virtual to Strategic – the reinvention of an online learning environment

July 19, 2011 Leave a comment

I gave my first webinar on Wednesday – was pretty good fun actually – although quite weird to be talking and making jokes, well what I thought were jokes, and not getting any feedback.  I think though that because people used the comments and chat box more there were more questions and you could respond to these whilst you were going through, which was great. And we had some good discussion afterwards.  So, I would definitely do a webinar again 🙂 oh and just as a matter of pride I had the most people attending an ELESIG webinar  – 50 people. V proud.

You can access all the resources on the ELESIG web site, and I’ve posted in the blurb that tells you what I am going to talk about below

Speaker: Professor Susannah Quinsee, Director Learning Development and Chair of Learning and Teaching Development

Most UK Universities have invested substantially in developing virtual learning environments (VLEs) over the past five to ten years. However, the rather monolithic and often inflexibility of VLEs, the rise of web 2.0 technologies and supplier changes, have made many institutions revisit this investment and question whether they really need a VLE.  As Mark Stiles (2007) has identified, VLEs often fail to inspire staff to engage in more innovative learning and teaching techniques.

At City University London we have a new learning and teaching strategy and a major review of our undergraduate education provision (determined by our recent University strategy).  These strategic drivers led us to ask the question “do we need a VLE?” to kickstart an evaluation process of our current VLE with a view to taking a more strategic approach to technology enhanced learning and improving our educational offerings. Over the past five years we have had considerable success in rolling it out the VLE across the University and most staff now accept that “CitySpace” is a core part of all teaching activities. After a six-month evaluation process with staff and students we issued a competitive tender for a new strategic learning environment and started the move to Moodle. Since then our emphasis has been on the strategic as opposed to virtual  because we saw the tools of a VLE as core to our University business but as one piece in a jigsaw of solutions, all wrapped up in our new institutional portal.

This webinar outlines the process we undertook in terms of stakeholder engagement during the evaluation phase and what we have done since then in rolling out the new SLE. It discusses our collaborative approach with Schools where we work in partnership to identify needs and pump prime development projects. The webinar will also explore how we have tried to develop a “redesign for delivery” approach to implementation which ensures the learner experience is at the heart of what we do.

Innovation Creativity and Leadership – Research and Practice

Innovation Creativity and Leadership – Research and Practice                    13th June 2011

ICL cogs logo

Archived abstracts and slides:

This event showcased a diverse range of research and activity in this field.  It very successfully met its aim “to be a forum for exchanging ideas and sharing experience of innovation, creativity and leadership in both research and practice.”  Central to this was the vibrant attending crowd.  True to its interdisciplinary nature, the event enabled lively networking across the disciplines, between lecturers, services, students and researchers, between academics, consultancies, and companies, from City University London and beyond.  Among the many people I encountered were a MICL (Masters in Innovation, Creativity and Leadership) student glowing about his inspiring and rewarding study experience and a creativity consultant that provided me with a list of useful techniques for change management.

The performance space proved to be an excellent venue although the area used for catering was a little cramped.  It would have been exciting to use the space to perform and have had sessions that enabled physical movement, interactivity and engagement with the actual techniques discussed.

Highlights for me included Patrick Jordan’s opening keynote.  He is described as a “design, marketing and brand strategist” as well as an academic.  His presentation was eye-opening and highly visual and exploded a number of assumptions about human behaviour in his discussion of bringing about change.

Clive Holtham co-created his slides with the audience using a crayoned flip book and invitations to interpret the symbols.Co-created slide with Professor Clive Holtham

Kathy Molloy and Kathryn Waddington explored critical reflection as a tool for learning about leadership.  This introduced me to the term ‘toxic leadership’ and also demonstrated how medical staff were able to embrace and benefit from the seemingly uncomfortable practice of reflection.

Sara Jones gave an excellent overview of the technologically enhanced spaces for creative conversations that they have been evaluating and experimenting with.

Above all I took away this quote shared by Andres Roberts that “there is no scientific evidence that seriousness leads to greater growth..”.  I left with a stronger conviction that bringing about change requires play and creativity.  This was an enjoyable day and excellent networking opportunity.  It was a demonstration of the kind of event that City University London should do more of and I look forward to the next one.

Mark Brown

Mark Brown: A Story of Hype and Hope: Breaking Free from the Technology Bungy          5th July 2011

During a recent visit to the UK from Massey University, New Zealand, Mark kindly stopped off at City to share this talk about educational technology.  Mark’s career summary and list of achievement are overwhelming but match his enthusiastic, energetic personality and oodles of knowledge and ideas.  He is an academic, teacher, researcher, educational technologist, leader, strategist, project manager and is involved in a number of international projects.  Mark travels for 3-4 months of the year.

Massey University has three campuses and 18000 face-to-face students and 17000 distance students.  Mark came to steal our UK students and invite them to a degree involving lower debt and the NZ experience.  He also told us about educational technology jobs at Massey and I’m sure not one of us left without at least a short delightful moment imagining ourselves living and working over there.

Mark’s talk shared the Massey approach, a fellow Moodle institution, insight into the significant HE sector and global changes, and role of educational technology within that context.  Our awareness was raised of the rise of the involvement of private companies in HE and the techniques they are using.  Above all we need courage and vision.  We need to ask the question, what can university’s offer in 2011?

Massey has internal Moodle servers to take advantage of the high network speed and an externally hosted Moodle installation with the Moodle partner Catalyst.  This setup includes useful processes for development and testing.  All the educational technologies are delivered together through a system branded ‘Stream’.  The branding plan included fridge magnets and feeds into the ambition to be the ‘digital university for the future’.  The e-portfolio is delivered separately and although they use Mahara which could be integrated with Moodle it has a different access route to ensure that students feel that Stream is university owned and the e-portfolio is individually owned.  Other key tenets of this approach include the student workload calculator and a commitment to quality enhancement above quality assurance.  Peer review and online peer review is at the heart of this.  “Quality culture that engenders conversations around teaching and learning.”

Here are some of my favourite quotes and a model from the talk:

  • “E-learning is a digital lubricant for globalization”
  • “Most e-learning initiatives are based on great leaps of faith”
  • “5 years ago youtube didn’t exist”
  • “We can’t just keep asking our academics to do more.  It’s not sustainable”
  • “Learning is messy”
  • On E-Portfolio:  “Staff have got the wrong metaphors” and of Mahara “What’s the problem its trying to solve?”
  • On distributed leadership: “The light comes from the cracks and we need to expose the cracks not cover them up”
  • Kaplan University advert: “Its time university adapted to you rather than you adapting to university”

Here is Cuban’s Technology Expectation Cycle:

Technology Expectation Cycle by Cuban

Slides from Mark Brown’s talk:

Mark Brown’s blog: http://tinyurl/

Point B: A Short Guide to Leading a Big Change

Peter Bregman (2007) Point B: A Short Guide to Leading a Big Change, Space for Change, New York

Point B book coverChange is notoriously difficult to manage.  This guide won’t make change easy or problem free but it will give great insight, tools and strategies that increase chances of success.

This guide refers to a big change but is relevant to all change on whatever scale.  It is a positive and practical book that focuses on inspiring, engaging and coaching people through change.  Apparently, ‘70% of all corporate change efforts fail’ p.5 because people resist change.  The book recognises that, ‘Change is primarily an emotional issue’ p.50.

Point B was a perspective shifting resource for me in leading institution-wide changes to learning technology as well as smaller projects and managing a team.  It gave me clarity in what my role should be and what skills I need to do this work.  In the current environment, being able to lead, support and manage change are essential capabilities for working in Higher Education.

This vignette provides five things it’s useful to know about leading and managing change according to Point B and a very brief example of the Point B model in action at City University:

Peter Bregman Point B Vignette

%d bloggers like this: