Archive

Posts Tagged ‘Student Experience’

Bring Your Own Device (BYOD): How Staff are Responding to the Mobile Technologies Their Students Bring With Them to Class

December 23, 2012 2 comments

The end of the year yields an opportune time to reflect upon various teaching innovations that have been discussed, written about, and presented earlier in the year, but which have not yet been included on the Educational Vignettes website. In this, the first of a series of end-of-year posts, I offer some thumbnail sketches of initiatives implemented in my teaching during 2012 to embrace mobile technologies with which students have been engaging to support their in-class learning – but with which, crucially, I was myself comparatively unfamiliar.

It can certainly be offputting to a tutor for the students’ attentions to be apparently divided between the lecture and their mobile devices; but this year I have seen evidence that, far from being a distraction, even the unsolicited use of mobile technologies by students can actually lead to their being more engaged in class. For example, in one lecture, when discussing crossover between classical musicians and popular music, I alluded to a Los Angeles-based string collective, the Vitamin String Quartet, who have released a fascinating series of albums of arrangements of popular music. Moments later, the whole class heard the unmistakeable sounds of a string quartet emanating from one corner of the room – one of the students had looked up the group’s website on her laptop, but had forgotten to ensure that she had turned off the sound…

Another such instance seems rather appropriate to this time of year : I was chatting to a student after a lecture who rather impressed me by dropping into conversation that Queen’s ‘Bohemian Rhapsody’ was the only track ever to have reached the UK singles Christmas No. 1 spot twice, in 1975 and 1991. When I asked him how he knew this, he reminded me that during the lecture, I had mentioned that the Spice Girls were the only act to have attained three consecutive Christmas No.1s (1996-98). His interest had been sufficiently piqued by this piece of information that he had used his mobile device to call up a list of UK Christmas No.1s, and noticed the double appearance of Queen’s ‘Bohemian Rhapsody’ there.

Versions of the two case studies below were previously presented at the School of Arts and Social Sciences Teaching and Learning Fête on 20 March 2012, and I am indebted to several colleagues at the Learning Development Centre without whose input and advice many of my endeavours this year simply would not have been possible.

Impromptu lecture-capture

Punk rock lectureIn a lecture on punk rock earlier in the year, a few students and I re-enacted the infamous interview between the Sex Pistols and Bill Grundy broadcast live on primetime Thames Television back in 1976. Reconstructions based on archived footage are available, but we decided that it would be more fun and interactive for the students to come to the front and recreate the experience for themselves. Given the relatively large number of people involved in the task, one student remarked that at this rate there would be nobody left in the class – to which I responded that this was very much in keeping with the belief-system at the centre of punk rock: the idea that anybody can get up on stage and be a performer.

Perhaps most interesting was one student who declared that she would be playing a role I had not even considered – that of the cameraman. She filmed our entire reconstruction using her mobile device (from which the screenshot, above right, is taken), thereby taking the notion of student ownership of their teaching and learning to a whole new level. (More recently, another student has written to me that his participation in the role-play was one of the most enjoyable parts of his educational experience – which is particularly revealing in that although he was indeed a part of the scene, and appears in the screenshot above, he actually had no lines to say!) The footage is now being uploaded to Moodle as a helpful reminder of the endeavour, and of the wider points it raised about punk’s do-it-yourself aesthetic.

Video podcasting of lecture summaries

Earlier in the year I was loaned an iPad by the School, and set the intriguing challenge of finding innovative ways to incorporate it within my teaching. Personal research soon led me to the Wired Educator blog in which a compelling case is made, albeit in a different context, for using the iPad for podcasting (see here). I have been audio podcasting since 2009 but switched to video (not a medium with which I am particularly comfortable) this year. My rationale for the change was that, while the pedagogical function of podcasting may be largely fulfilled by audio-only resources, images are more engaging for the students, encouraging a greater level of concentration and enabling them to see and interpret the speaker’s gestures and body language. I was also mindful of recent experiences within the institution with lecture-capture, which I have been increasingly using as the year has progressed, and of wider innovations in education such as the implementation of flipped classes.

My mode of operation was to record a podcast of 8-10 minutes in advance of each lecture, providing a summary of the key material and concepts of the associated class as well as discussing the set reading, and to release the recording via Moodle. Each podcast was intended to give the students some grounding in the content of the lecture (as well as to act as a ‘trailer’) and to provide some context on the preparatory reading, but they had an unexpected secondary function as a resource for the end-of-module examination. In the module evaluation, completed one week before the examination took place, one student wrote that “The weekly podcasts which were made were very helpful for revision” while another commented on the “Helpful podcasts on Moodle for revision purposes”.

As noted, many students already owned mobile devices upon which the podcasts could be played, downloaded, and re-watched at their convenience. For the others, I came to class every week with the podcasts pre-loaded onto my iPad (see screenshot, below) and students who did not have the opportunity to watch them in advance, or appreciated a second viewing to refresh their memories, were able to borrow my iPad for this purpose before the lecture or during the break.

Podcasts - screen capture

Moodle 2: The look and feel of it

December 21, 2012 Leave a comment

As the case with all SLE Working groups, the Moodle look and feel working group have also been busy paving the way to a new look and feel of Moodle with the emphasis on making it a good ‘user‘ experience. All of this work is being done taking into account schools and their requirements as well as making Moodle much more engaging and consistent to both staff and students. The group have also been working hard to ensure that staff and students don’t have a completely new system to start from scratch but are provided with some extras that ensure that there can be an integrated and intuitive learning environment for all.

Of course none of this can be done without obtaining staff and students’ views on how they find using Moodle as it stands at the moment. With the help of  Centre for Human Computer Interaction Design at City, user testing(both staff and students) took place in November and December 2012. The scenarios and tasks aimed to test the following areas across both Moodle 1.x and Moodle 2.x included:
• MyMoodle
• Module Details
• Assignment Submissions and
• Calendar

confusion1

A report has been collated which has been used to inform the recommendations passed on for the design brief. Some of the main problem areas identified were about:

-the ease with which students needed to use the system so for example it wasn’t always clear how My Moodle could be adapted to suit the person. Students tended not to notice the edit button for instance.

-students often wanted to have access to their timetables which were not available through Moodle. City Law School does provide access to timetables within Moodle so this is something that can be enhanced.

-Whilst users appeared to like the fact that each module had it’s own personality, there appeared to be a lack of consistency across different
module details pages.

In addition to these findings, the working group also set about seeking to identify what other institutions sites on Moodle looked like. Here’s a helpful Moodle theme exploration video devised by Matt Jenner at UCL. This was necessary to see if Moodle’s look could be future-proofed and ensure that we knew enough about what we are competing against.  Exciting to note is that there will be a fully mobile friendly page, which will make moodle much more usable on a mobile device. As well as that there will of course be a new improved MyMoodle page and module homepage (course format). We aim to keep you posted through weekly vignettes so have a lovely Christmas and watch out for new postings in 2013.

Use of the Personal Response System for Formative Assessment in Optometry: Dr Byki Huntjens and Dr Steve Gruppetta

With the recent founding of the University Personal Response System (PRS) Steering Group, co-chaired by Dr Siân Lindsay and Farzana Latif, this would seem to be an opportune time to profile one of the innovative approaches implemented within the University in using PRS technology for formative assessment.

Dr Byki Huntjens and Dr Steve Gruppetta are lecturers in the Division of Optometry and Visual Science who have introduced the PRS to undergraduate students in order that they may receive immediate classroom feedback during Clinical Skills and Optics lectures. A PRS handset is given to the students (against a small deposit) throughout their degree programme, and is registered to their name to enable responses to be matched to individuals. Each lecture features a succession of multiple choice questions (MCQ). Byki’s practice is to start later lectures with a set of MCQs covering the previous topic plus the background reading for the class, and test the students’ understanding of the new topic later on during the lecture. Steve includes material that potentially encompasses the previous lecture, the current lecture, or even paves the way for a new topic to be discussed. The end result is a series of technology-enabled formative assessments.

Although only the group scores are shown during lectures and the progress of individual students is not revealed, the results of the quizzes are uploaded to Moodle each week by topic and the students are thereby able to check their individual score. This enables them to track their progress over time, and doubles as a reminder of the topics to which they need to direct particular attention prior to the examinations. The Moodle grade book also shows the students’ ranking among the whole group, leading some of them to become slightly competitive. Indeed, the element of competition is actively nurtured – the top five students with the highest marks in the year are awarded a prize at the divisional Prize Giving event.

The students have shown excitement during the PRS quizzes and appreciate the immediacy of the feedback, the anonymity of the process, and the way that it articulates the lecture by providing an interlude. Steve has developed the practice of making the PRS quizzes, which he calls the ‘Optics Challenge’, distinct from the rest of the lecture by changing the background of the slide from white to black (see screenshot below). The students’ responses are also used by the tutors to adapt subsequent lectures to the level of understanding of the specific cohort; this has prompted a change of direction on several occasions. In addition, this information has enhanced the support that the tutors are able to offer when students have sought extra help.

The Optics Challenge Leaderboard

Byki delivered a presentation on the use of PRS technology for formative assessment at the Fourth Annual ‘Learning at City’ Conference on 13 June, 1.20-2.00pm (the video is available here).

Christopher Wiley, Byki Huntjens, and Steve Gruppetta
with thanks to Siân Lindsay and Farzana Latif

A Case Study of Interim Assessment in SEMS: Mary Aylmer

Mary Aylmer is a visiting lecturer in the School of Engineering and Mathematical Sciences (SEMS), teaching the CAD part of the module CV1407 IT skills, Communication, and CAD. She has developed an assessment pattern in which students produce five pieces of CAD coursework, each of which involves completing engineering drawings. There are two interim submissions each weighted at 2% of the final module mark, two larger submissions weighted at 16% and 40%, and an end-of-module test also weighted at 40%.

The 2% weighting for the interim submissions is intended to ensure that the students’ early work on the module is taken into account in the final module mark, which helps to focus them to the task. The exercises are carefully graded and enjoyable for the students to complete; they tend to take ownership of their own learning as the assessments are designed such that they are able to determine exactly what is required of them, so they can aspire to high marks.

SEMS CAD CV1407The obvious advantage of this assessment pattern is that it ensures that the students are definitely completing their initial work on the module. This means that they are well prepared for the larger submissions: they have already accrued plenty of experience of CAD in the first few weeks through the interim submissions, and are thereby placed in a strong position to tackle the difficult drawings. In other words, it ensures that they undertake the groundwork first.

The downside to this system for the tutor is that it generates a substantial amount of marking. Mary has also noted a tendency among students to query their marks, even in the case of the 2% submissions which are unlikely to have a significant impact on their overall degree average. It can become very time-consuming to justify marks deducted, particularly with 120 students each of whom submit 5 pieces of work.

Nonetheless, the outcomes speak for themselves. By the end of the module, the students can produce good CAD drawings fairly easily; and they have indicated through their feedback that they enjoy the course, which is very encouraging. While an assessment model such as this may be time-consuming for the tutor, it is evidently worth the investment if it results in robust learning and student satisfaction.

Christopher Wiley and Mary Aylmer

Using Debate as a Teaching Format

July 7, 2012 2 comments

Three years ago at a City University Creativity Workshop I met Kirsten Hardie who teaches Design at Arts University College of Bournemouth. She told me about a method she had invented called “On Trial.” By coincidence, in working with a group of City teachers recently, they quite unprompted suggested the use of a debate format as a method of increasing student engagement.

It was impressive how quickly they came up with a great number of creative ideas to widen the palette of teaching formats. The session focused on devising a learning activity that reflects often currently missing employability skills:

  • Critical thinking
  • Reflection
  • Persuasive communication
  • Self awareness

Devising fresh learning activities to promote employability skills

I took Chickering and Gamson’s 7 Principles for high engagement learning, as a benchmark.  We selected two of these: and participants were also encouraged to identify their own. The focus was on fresh learning activities; new ideas.  Here are some of the resulting creative learning outcomes from the participants.

Fresh  learning activities.

  • Reflection: in action and on action
  • Scenarios, role play and simulation
  • Debating
  • Combined learning with another school (interprofessional  learning)
  • A buddy system

The question is how can we enable great ideas like these to be put into practice? For example using debate in our teaching.

So returning to National Teaching Fellow Kirsten Hardie’s On Trial project that  explores the use of role play and debate in student centred learning. It promotes and facilitates creativity in and through learning. Students work with colleagues to explore and interrogate problematic issues relating to their specialism

“On Trial harnesses popular culture, and the seductive qualities of the courtroom, as experienced through television and film examples (both historical and contemporary), in a creative fashion to help students engage with tough academic issues and wider ethical concerns.”

In addition a fascinating article by Catherine Sanderson discusses and evaluates debate as an assessment and learning strategy to develop critical and reasoning skills and stimulate learning through assessment in first year Biomedical Science and Public Health Students.

Sanderson’s work with first year undergraduates indicates that although it may be tacitly understood that critical reasoning is an essential skill for all students, it is far too often left to the final year as a learning outcome or even reserved for post-graduate studies.

Innovation in Assessment and Feedback

April 20, 2012 2 comments

My dual role as University Learning Development Associate in Assessment & Feedback and Senior Lecturer in Music has led me to run several pilot projects in my teaching this academic year (2011-12), exemplifying innovative approaches to the practices surrounding assessment and feedback. Three case studies are given below.

(1) Using wikis in Moodle to track progress on undergraduate dissertations and deliver formative feedback

Last term I set up an wiki template in Moodle to provide each of my final-year undergraduate dissertation students with a resource that both of us could access and periodically update, for the purposes of tracking progress on their dissertations and offering formative feedback on draftwork submitted.

Major Project wikiThe wiki includes pages for the project’s working title, and a separate page for each of the meetings divided into sections for the date of the meeting, a summary of what was discussed, objectives agreed for next time, and the date of the next meeting (see screenshot, right). It was developed owing to the need to help undergraduate students keep on-track in their dissertation work at a critical time in their programme, and was inspired by the Moodle wiki previously set up for the purposes of recording undergraduate Personal Development Planning (PDP) as well as the University’s use of Research And Progress for postgraduate research students.

One student has engaged with this resource to the extent that he has created several new pages to record his ongoing progress in between supervisory meetings; the nature of the wiki is such that I can review his progress at any time and add suggestions or make revisions as needed. Another student always brings her Wi-Fi enabled laptop with her so that we can make updates to the wiki during our tutorials. Whenever one of us makes and saves a change, the other can instantly see it on their screen, which demonstrates the value of using mobile devices to support student learning – particularly as this student now takes the lead at the end of each supervision in ensuring that the wiki has been fully updated.

This would seem to be a helpful way of time-managing the task of researching and writing a dissertation, not least given that it is a challenging process that final-year undergraduates may be encountering for the first time. It also provides a concise and useful reminder (for supervisor as well as student) of discussions, progress, and objectives set at each meeting, while enabling them to take ownership of their learning. This pilot will be rolled out across the entire module next year and all final-year Music students will be expected to use it; there is also much potential for initiatives of this nature to be extended to other programmes and subject areas.

(2) Curriculum design developed in dialogue with the students: elective assessment components

One innovative assessment model that I have been developing for much of this academic year involves giving students some choice as to how they wish to be assessed. Consultation with senior academic staff within and beyond the University has identified that, while such practices are more logistically complex, it should not be supposed that there is only one way to assess students against a prescribed set of learning outcomes necessarily.

After considering several possible assessment patterns which were discussed with colleagues, I settled on the following model which essentially preserves the 30:70 ratio (standard across the institution) between the minor and major assessment points:EVS graph

  • 1 Written Examination (unseen): 30 marks
  • 1 Elective Assessment: 30 marks – the student chooses ONE of the following options:
    • Written Coursework
    • Oral Presentation
    • Musical Performance accompanied by Written Documentation
  • 1 Project developed from the above Elective Assessment: 40 marks

The Examination provides a common component for all students, irrespective of the pathway they choose for the Elective Assessment. The other assessments have been specified mindful of parity with existing module assessment patterns. The benefits to students are that the initiative enables them to play to their strengths, and to influence how they wish to be assessed and how they wish their marks to be apportioned. The Elective Assessment also permits an additional opportunity for interim feedback ahead of the final Project.

My consultation with the students as to whether such an innovation would be welcomed was revealing: the graphical result (above right) of a poll conducted anonymously using EVS handsets (clickers) speaks for itself.

The focus group that comprised 12 students in my class were also consulted on several other major points of curriculum design, including the content and schedule of the lectures as well as the manner in which they will be taught, assessed, and feedback delivered. They have decided upon all of the lecture topics themselves via a Doodle poll, and have been invited to write supplementary assessment criteria using a wiki; elements of self- and peer assessment will also be included in the module. Having discussed several different forms of feedback (written, dialogic, telephone, podcast, screencast) at the focus group, 33% of students said that they would prefer written reports, while fully 50% opted for dialogic feedback – an unexpected but welcome result.

(3) Student self-assessment of in-progress writing of a research dissertation

Earlier in the year, one of my senior postgraduate research students submitted a draft of a dissertation chapter to me in the knowledge that while some sections were complete, others would need revision either because she felt that they would benefit from further work or because she had yet to complete the research (largely ethnographic, for which she is entirely dependent on the availability of her study participants) that would enable her to finalize her writing.

Since I nonetheless wanted to give her feedback on her work in progress, I formulated the idea of suggesting to the student that after a couple of weeks she should return to the draft chapter herself to reflect upon her writing, and to embed comments electronically using Microsoft Word to identify sections where she felt that further revision would be necessary and to explain why. I would then overlay my own feedback in a similar manner.

In being able to review draftwork that the student had herself annotated, I found my attention being much more effectively directed towards the parts of the chapter upon which it was most fruitful to focus. I felt that I would have made many of the same comments as the student herself, and this means of reflection also enabled the student to ask further questions of her work that I was then able to respond to, and for us to engage in a form of written dialogic feedback (see screenshot below).

The student likewise reported that she found it very useful to return to her chapter in retrospect, and particularly to document the areas she believed required additional work. This is a model of self-reflective feedback that I am now seeking to adopt for future research students.

Dissertation feedback sample

Dr Christopher Wiley
c.m.wiley@city.ac.uk
20.04.12

Review: SACWG seminar, ‘The efficiency and effectiveness of assessment in challenging times’

On Thursday 24 November 2011, the Student Assessment and Classification Working Group (SACWG) hosted a one-day seminar, ‘The efficiency and effectiveness of assessment in challenging times’ at Woburn House, Tavistock Square, London. 

To open the seminar, Dr Marie Stowell (University of Worcester) set out the context for the day in her presentation ‘Efficiency and effectiveness in assessment’. She identified that one of the aims of SACWG is to explore variations in practice across the sector and how they impact differently on students, retention, and learning success, and she observed the importance of placing students at the centre of the process given the fee structure proposed for 2012 entry coupled to the implications of assessment and feedback to student satisfaction. In light of the new funding model, one particularly pertinent observation she made concerned the cost of teaching in relation to the cost of assessment: the latter is resource-heavy, particularly once one factors in elements such as formative assessment (for which quality is less assured than its summative counterpart), moderation, external examining, reassessment of failed components, and the possibility that students may be over-assessed in the first instance. She also suggested that assessment criteria may not warrant the detailed attention they are typically accorded, as students tend to take the more direct approach towards assessment of endeavouring by less formal means to uncover exactly what it is that the lecturer is expecting them to produce. These arguments may indicate that both the efficiency and effectiveness of assessment could usefully be enhanced.

The next talk, by Professor Alison Halstead (Aston University), explored how institutions have responded to the challenges of recent years, specifically, the White Paper and its implications to students and to Higher Education. She noted that the potential increase in the students’ financial burden will inevitably lead to heightened expectations concerning teaching quality, learning, and employability, in which respect assessment and feedback are currently among the most important issues. She warned that student challenges to the regulatory framework for assessment may be on the rise in the future and identified that it was imperative, in these changing times, to nurture outstanding, innovative teachers and for staff to support student learning and e-learning (including assessment). Calling for the abandonment of the rigid distinction often drawn between ‘teachers’ and ‘researchers’, she suggested that promotions should award teaching excellence on a par with research. Later sections of her presentation outlined recent initiatives at Aston, for instance, standardizing the use of Virtual Learning Environment across the institution, and introducing learning technologies such as lecture capture and electronic voting systems. Her view was that teaching-enabled practice, while it took more time upfront to implement, was worth the investment in terms of teaching quality and learning success.

A structured group discussion and question-and-answer session with the morning’s speakers ensued. One point that emerged strongly was the importance of maintaining a variety of assessments, organized in a carefully considered schedule that takes a holistic overview at programme level. The latter becomes much more difficult in degree courses that incorporate elective modules, though there are both pedagogical and satisfaction-related reasons for offering choice to students and giving them ownership of their programme pathway. Another preoccupation amongst delegates was that assessments do not become too atomized, but relate to one another even beyond the confines of the module with which they are associated; one of the more innovative solutions proposed was the possibility of assessments straddling two or more modules. The need to develop sustainable structures was also discussed (for instance, moving towards group assessment to cope with rising student numbers), as was the importance of considering (as part of change management) what the benefits of effecting the change might be; if these cannot be persuasively articulated to staff and students, the change may not be worth implementing. A final warning concerned being too driven by regulations in designing efficient and effective curricula: it may be more useful in the long term to refer obstacles presented by the regulatory framework upwards so that they can be addressed.

The seminar resumed in the afternoon with a talk from Professor Chris Rust (Oxford Brookes University) on ‘Tensions in assessment practice’, which opened by reiterating the themes of the seminar in noting that current practices are neither efficient nor effective. He discussed that students have a tendency to focus on the mark they will obtain from the assessment rather than on the educational content of their studies, and that their approach often becomes increasingly surface-level as they progress through their programme. He defended modes such as formative, self-, and peer assessment as potentially yielding more ‘authentic’ assessment, arguing that graduates should be able to evaluate themselves and their peers as an outcome of their programme, and that making greater use of these options might also free up staff resources for summative assessment. Noting that students do not warm to the notion of being assessed, he suggested that perhaps the word ‘assessment’ should not be used for formative tasks. He further observed that feedback practices might be made more efficient by strengthening the relationship between modules, such that students are encouraged to learn from feedback received in one module and to carry what they have learnt over to others. Lessening the sense of compartmentalization of individual modules would, in his view, lead to more inclusive structures albeit less flexible ones, in that standardization (for instance, in terms of the same word limit for all assessments) does not always result in appropriate assessments.

Then followed a second group workshop session, on the theme of ‘What can institutions do to mitigate tensions?’. After a structured discussion of the issues, each group reported back to the seminar as to the problems that they had identified and the possibilities for efficient or effective solutions. It would be impossible to do justice here to the vast amount of ground covered between the several contributing groups. To cite just a few examples, key tensions that were raised included giving formative assessment a greater purpose (a proposed solution being to tie formative and summative assessments together in more meaningful ways), the problem of ensuring parity when using several examiners for the same assessment task (which may be solved by grading the assessment as pass/fail only), and the evergreen question of quality of feedback versus timeliness of feedback (for which there was some discussion about feedback becoming ‘quick and dirty’). On the question of standardization of process, I took the microphone to report back on the standardized feedback proforma that had been created in liaison with the students and implemented across one programme at City University London (see this post for details), and suggested, with much support from the floor, that students should be more involved in consultation regarding matters of assessment and feedback.

Prior to the close of the seminar a final speaker, Professor Paul Hyland (Bath Spa University), provided some reflections upon the day’s discussion. Noting that assessment was a large topic with which to deal, he categorized the day’s discussion as having crystallized around four main areas: external scrutiny (ranging from students’ parents to formal regulatory bodies); administration and management; the tutors’ perspective on assessment; and the students’ perspective. He argued that discussions of  effectiveness and efficiency should always be mindful of the purpose of assessment. In his view, assessment should be concerned with measuring students’ performance and nurturing learning, whereas there exists a danger of (to put it crudely) simply setting assessments in order to get the students to do some work. In this context, a greater level of student involvement and engagement with assessment would therefore be beneficial. He also observed the need to use technology to improve existing practice, for instance, to supplement traditional modes of feedback with video and screen casts. Finally, he commented upon the importance of tutors having access to students’ feedback on previous assessments in order to understand where they are coming from and to be able to support them in their ongoing studies.

SACWG has kindly made available the presentation slideshows used by the speakers, and the comprehensive notes distilled from the two very productive group discussions (as reported back to the seminar by nominees from the groups), at the following link: http://web.anglia.ac.uk/anet/faculties/alss/sacwg.phtml.

%d bloggers like this: