Archive

Posts Tagged ‘Assessment’

Blogs for Learning and Teaching: More then just a passing phase

January 3, 2013 3 comments

blog-wordleLearning and Teaching blogs

Over the last few years, you may have found yourself subscribing to various blogs. These tend to provide bite sized information on areas that interest you. For instance our educational vignettes(created by the Learning Development Centre) enables the dissemination of case studies, reviews, and guidance on learning and teaching in general. However you may have also found that in some cases, there are other blogs that your local edtech team produces too. 

In this article, I will provide a brief overview as well as the noted benefits of using blogs for research and/or teaching purposes. I will also provide information on how blogs are being used at the University and how you might like to set up your own blog.

History of Blogging

The origins of the blog is the subject of some debate, but according to Blood (2000), the phrase ‘web log’ was first used by Barger (1997) and the shortened version by Merholz in 1999 (Merholz, 2002). Blogging as a phenomenon started to increase steadily after this time, and then there was an explosion in the number of blogs when the first free, do it yourself blogging tools became available in mid-1999, most notably Blogger.com.

Since 2003 there has been over 70 million blogs created, each with their own version of news. So the big deal about blogs is that it gives people like us the power of the media and has created a personal kind of news that appeals to a high number of small audiences. A simplified visual explanation of blogs can be viewed here.

The Benefits of Blogging

In relation to learning and teaching, blogging can be advantageous in a range of situations namely:

  • lecturers can provide feedback and monitor students performance more effectively;
  • it promotes self-assessment and continued assessment;
  • it promotes personal reflection and
  • it enables tracking of all the process (both by students and lecturers).

Priego a new lecturer at City London argues that blogging is the ultimate form of collegiality – if we understand collegiality as the relationship of professional colleagues united in a common purpose and respecting each other’s abilities to work toward that purpose. Priego suggests that blogging is already a multi-tool for today’s academic, whether early-career, established or somewhere in between. Useful for both researching and rehearsing ideas, it can even be an early form of publication.

How blogs are being used by students at City

At City, blogs are being used in various ways by both lecturers and students. Some examples of student blogs include:

  • recording Personal Development Planning (PDP) activities;
  • charting project progress;
  • managing group projects;
  • collaborating on the development of course resources;
  • commenting on lecturer-led blogs and
  • interacting with guest speakers.

The latest blog of blogs

Matt Lingard’s team at City have found a way to pull blogs that focus primarily on education and technology. There is now a new Education & Technology blog at the university. EdTech: Education & Technology is an innovative blog of blogs. It pulls together posts written by staff at the university from 8 different blogs (including this one!). So, rather than following 8 blogs, you can get them all in one place by visiting http://blogs.city.ac.uk/edtech, by signing up for email updates or subscribing to the RSS Feed.

As noted, the blogs are a mixture of team, individual & more general ones providing a wide variety of posts including case studies, conference reports, technology news, teaching ideas & much more. EdTech: Education & Technology will be of interest to a wide audience both inside and beyond the university.

How does it work?

The EdTech blog is powered by RSS feeds (RSS or News feeds are links to web pages that are read by computers and allow content to be moved around the web). It uses a ‘plugin’ to the main university Edublogs service called FeedWordPress. This imports blog posts from the 8 source blogs via their RSS Feeds. It’s an automated process requiring only minimal human intervention to classify the incoming posts.

Variations of a ‘blog of blogs’

This blog of blogs model could be used for combining any collection of blogs or other RSS/news feeds.  For example a cohort of students’ individual blogs could be combined into a class blog or a collection of news updates from mainstream media could be combined create a single rich contextual resource for students.

How do I start blogging?

To request a blog, log your request with the IT Service Desk. You can set up a private blog to support learning and teaching activities or you can request a public blog to publicise the work of your department.

So over to you, if you’d like to share your thoughts on blogging please do so under comments. If you’d like to find out more about blogs and how they can be used to support your research and/or teaching, please do contact your ed. tech team or the LDC.

Advertisements

HeLF meeting: Personalisation of Assessment and Feedback

November 6, 2012 Leave a comment

In the last ten years, higher education has changed beyond all recognition and Heads of E-Learning will be critical to the significant changes to come.  These were some of the opening words by Professor Rikki Morgan-Tamosunas, DVC, University of Westminster, in opening the Heads of E-Learning Forum (HeLF) Meeting held on 31st October.  The theme for this year’s meetings is personalisation and E-learning Heads from around the country came together to explore Personalisation of Assessment and Feedback.

Lisa Gray from JISC gave an overview of the JISC Assessment and Feedback programme supporting and sharing results from numerous projects now running in the UK: http://www.jisc.ac.uk/assessment

Slide outlining impact of EVS on teaching

Slide from Electronic Voting Systems Presentation

How do you avoid assessment bunching on courses? Catherine Naamani from University of Glamorgan shared their Assessment Diaries project designed to ensure assessments are fairly spaced and give students an overview of all assessments across their courses including type, submission date and feedback return date.  The tool linked in with BlackBoard.

Marija Cubric shared their uses of Electronic Voting Systems, known as clickers at City, for assessment at Hertfordshire.  This technology had on the whole been well received by staff and students.  The tool was deemed easy to use and made teaching and learning more enjoyable.

Gunter Saunders and Peter Chatterton finished the day with an exploration of their Making Assessment Count (MAC) project focused on feedback.  Their presentation highlighted a project at City within broadcast journalism enabling students to reflect on assessment feedback.  This project involves Kate Reader from the School of Arts and Social Sciences and here is a presentation about the work: http://estsass.co.uk/2012/07/23/presentations-from-the-learning-city-conference/

Slide from MAC project

Also discussed here was a change management curriculum design technique called Viewpoints that involved the use of cards with principles and examples that could be used to design modules.

 

Self- and Peer Assessment using Turnitin in SEMS: Cengiz Turkoglu

August 1, 2012 5 comments

Cengiz Turkoglu, a Senior Lecturer in the School of Engineering and Mathematical Sciences, principally teaches final-year undergraduate students and one of the MSc Aviation Management modules, with class sizes usually not exceeding 20 students. Each of his modules uses a similar assessment pattern comprising one coursework plus an examination. For the coursework component, he utilizes the self-review and peer review functions of Turnitin as part of the assessment.

The coursework has an initial deadline of a minimum of 6-8 weeks into the module to allow sufficient time for students to conduct research and write their essays. Once the students have submitted their paper, Turnitin’s PeerMark assignment function allows them to be either paired or randomly allocated another paper, which they are then required to peer-review. Given that there is always a range of standards represented by the students and their papers, one dilemma that Cengiz has faced concerns whether to pair the students randomly or to attempt to group them according to their standard. He never pairs them such that two students are asked to review one another’s papers.

The feedback provided by each student in peer review is subsequently made available to the original author – and the students are made aware at the time of writing that their comments will be released in this manner. At the same time, each author is asked to take a self-assessment exercise that follows exactly the same format as the peer review. As the process is conducted entirely online using Turnitin, it is completely paperless, which reduces the administrative workload and makes for a more sustainable structure.

For Cengiz, self- and peer review are only valuable if they lead somewhere in terms of the assessment process. With that in mind, once the feedback has been exchanged between students, Cengiz gives them a week to undertake further revisions to their original submission should they wish to do so. He asks that they do not rewrite their paper substantively, but confine themselves to minor amendments. Plagiarism of the peer-review feedback is not an issue because all the material is traceable and hence can be attributed. Only after the revised submission has been received does Cengiz mark the work summatively using GradeMark and provide his own feedback.

Detailed assessment criteria are provided, with the marking criteria broken down into six different categories each with their own weighting, of which one category is self- and peer review (worth 10% of the mark). The students are therefore aware from the outset that it is an integral part of the assessment, and its summative nature encourages them to engage fully with the process, since Cengiz’s experience is that students can be very assessment-driven. The questions they are asked for the self- and peer reviews correspond to the other assessment categories, so they judge each other’s paper, and their own, in exactly the same way as the examiner.

Cengiz has found this to be a very valuable exercise. It sets the students thinking about how to frame feedback, offering helpful advice to the author rather than simply giving praise or criticism. It also encourages them to consider issues such as whether the author understood the question and maintained focus, how well they researched the subject, and how coherent the arguments they presented were, based on their own reasoning or factual information they identified during their research. (The criteria matrix used by Cengiz is shown below; this is also entered as the rubric in Turnitin.) While students are variable in their engagement with the process, Cengiz notes that the best self-reviews and peer reviews recognize areas where the submission can be improved.

Turnitin screenshot - criteria matrix

Cengiz argues that the value of this assessment model is that it provides a simulation of real-life scenarios. In safety-critical industries such as aviation, for example, maintenance engineers are expected to inspect each others’ work on a regular basis, and the peer review process is widely used particularly by design engineers. In addition, all engineers should be expected to reflect upon, and to strive to improve, their own performance in order continually to develop themselves professionally. They may not necessarily always receive the most favourable advice from their own peers, so engineering students are prepared effectively for the profession through nurturing skills such as being able to evaluate the feedback they receive and to make their own judgement when taking decisions.

Cengiz justifies equalizing the weightings between the coursework and examination (originally weighted at 30% and 70% respectively) by citing the introduction of the requirements for self-assessment and peer review as a reason to give greater weighting to the coursework component. He strongly believes that examination is not the only suitable assessment method for his modules as the nature of the topics he teaches is such that they require understanding and the ability to apply this knowledge to real-life scenarios, rather than merely memorising content from text books or course notes. After studying on the Postgraduate Certificate in Academic Practice programme delivered by the Learning Development Centre at City University London, Cengiz has become an advocate of self-directed and reflective learning, and he encourages his students to become more critically self-reflexive so that they can learn from their own experiences.

If you would like to know more about this assessment model, Cengiz is happy to be contacted by e-mail: cengiz.turkoglu.1@city.ac.uk.

Christopher Wiley and Cengiz Turkoglu

Use of the Personal Response System for Formative Assessment in Optometry: Dr Byki Huntjens and Dr Steve Gruppetta

With the recent founding of the University Personal Response System (PRS) Steering Group, co-chaired by Dr Siân Lindsay and Farzana Latif, this would seem to be an opportune time to profile one of the innovative approaches implemented within the University in using PRS technology for formative assessment.

Dr Byki Huntjens and Dr Steve Gruppetta are lecturers in the Division of Optometry and Visual Science who have introduced the PRS to undergraduate students in order that they may receive immediate classroom feedback during Clinical Skills and Optics lectures. A PRS handset is given to the students (against a small deposit) throughout their degree programme, and is registered to their name to enable responses to be matched to individuals. Each lecture features a succession of multiple choice questions (MCQ). Byki’s practice is to start later lectures with a set of MCQs covering the previous topic plus the background reading for the class, and test the students’ understanding of the new topic later on during the lecture. Steve includes material that potentially encompasses the previous lecture, the current lecture, or even paves the way for a new topic to be discussed. The end result is a series of technology-enabled formative assessments.

Although only the group scores are shown during lectures and the progress of individual students is not revealed, the results of the quizzes are uploaded to Moodle each week by topic and the students are thereby able to check their individual score. This enables them to track their progress over time, and doubles as a reminder of the topics to which they need to direct particular attention prior to the examinations. The Moodle grade book also shows the students’ ranking among the whole group, leading some of them to become slightly competitive. Indeed, the element of competition is actively nurtured – the top five students with the highest marks in the year are awarded a prize at the divisional Prize Giving event.

The students have shown excitement during the PRS quizzes and appreciate the immediacy of the feedback, the anonymity of the process, and the way that it articulates the lecture by providing an interlude. Steve has developed the practice of making the PRS quizzes, which he calls the ‘Optics Challenge’, distinct from the rest of the lecture by changing the background of the slide from white to black (see screenshot below). The students’ responses are also used by the tutors to adapt subsequent lectures to the level of understanding of the specific cohort; this has prompted a change of direction on several occasions. In addition, this information has enhanced the support that the tutors are able to offer when students have sought extra help.

The Optics Challenge Leaderboard

Byki delivered a presentation on the use of PRS technology for formative assessment at the Fourth Annual ‘Learning at City’ Conference on 13 June, 1.20-2.00pm (the video is available here).

Christopher Wiley, Byki Huntjens, and Steve Gruppetta
with thanks to Siân Lindsay and Farzana Latif

A Case Study of Interim Assessment in SEMS: Mary Aylmer

Mary Aylmer is a visiting lecturer in the School of Engineering and Mathematical Sciences (SEMS), teaching the CAD part of the module CV1407 IT skills, Communication, and CAD. She has developed an assessment pattern in which students produce five pieces of CAD coursework, each of which involves completing engineering drawings. There are two interim submissions each weighted at 2% of the final module mark, two larger submissions weighted at 16% and 40%, and an end-of-module test also weighted at 40%.

The 2% weighting for the interim submissions is intended to ensure that the students’ early work on the module is taken into account in the final module mark, which helps to focus them to the task. The exercises are carefully graded and enjoyable for the students to complete; they tend to take ownership of their own learning as the assessments are designed such that they are able to determine exactly what is required of them, so they can aspire to high marks.

SEMS CAD CV1407The obvious advantage of this assessment pattern is that it ensures that the students are definitely completing their initial work on the module. This means that they are well prepared for the larger submissions: they have already accrued plenty of experience of CAD in the first few weeks through the interim submissions, and are thereby placed in a strong position to tackle the difficult drawings. In other words, it ensures that they undertake the groundwork first.

The downside to this system for the tutor is that it generates a substantial amount of marking. Mary has also noted a tendency among students to query their marks, even in the case of the 2% submissions which are unlikely to have a significant impact on their overall degree average. It can become very time-consuming to justify marks deducted, particularly with 120 students each of whom submit 5 pieces of work.

Nonetheless, the outcomes speak for themselves. By the end of the module, the students can produce good CAD drawings fairly easily; and they have indicated through their feedback that they enjoy the course, which is very encouraging. While an assessment model such as this may be time-consuming for the tutor, it is evidently worth the investment if it results in robust learning and student satisfaction.

Christopher Wiley and Mary Aylmer

Future Tense Learning and Teaching conference at Goldsmiths, University of London 18 May 2012: Lightbulbs, spoons and stigmas

May 25, 2012 2 comments
Low voltage light bulbs, Cjp24, Wikimedia Commons

Low voltage light bulbs, Cjp24, Wikimedia Commons

What do you like best at conferences? Catching up with friends, get away from the day-to-day routine? Some might say the canapés? For me, it’s definitely hearing a good speaker. Communicating your ideas effectively is an art in itself and I know when someone gives a good talk because it’s as if a little light bulb lights up in my head when I hear something that makes sense. Lindsay Jordan (University of the Arts), co-presenting with Mira Vogel (Goldsmiths) on the world of MOOCs and open education was one of those speakers. So many light bulbs lit up in my head that I must have looked like a Christmas tree (had you had the privilege to access my head, that is).

Leading on the PgCert course recommended to all staff teaching over 60 hours per year, she opted to introduce a range of digital-based learning and assessment components to the course. Her aims were to broaden teachers’ horizons through technology, use blogs as a truly key reflective tool and video as a means to assess.

Interestingly, the title of her presentation being ‘Engagement by stealth’ judiciously hinted at how there might be some resistance to use ‘new’ forms of assessment and learning.

On the one hand of the spectrum, some lecturers on the course spontaneously opted for video in their blogs to develop their point. Others could hardly bring themselves to publish content on the web and expose themselves. One student on the course (an experienced teacher) published a completely anonymous video assessment circumventing the exposure problem by creating a short animation and synthesising his voice to a female one.

So, clearly, the feeling of publishing one’s work ‘out there’ on the web for everyone to see is something not everyone is comfortable with.

MOOCs (Massive Open Online Courses) are about allowing information to be everywhere, just a click away, and some of us have an issue with that. As one member of the audience asked: “But who has control over these MOOCs?  Who monitors content?”

MOOCs are about participation, collaboration and sharing but there is something inherently scary about not knowing who will use your learning and whether it will be digested effectively. If it’s out there on the loose, how can you gauge its validity? Some answers came from Mira Vogel who showed us that MOOCs can be time-framed and tightly-structured. Another member of the audience asked whether MOOCs may be an answer to soaring university fees in England. How could universities possibly reconcile with the relatively free-paying world of MOOCs? Of course, no-one yet knows the answer to this.

A personal highlight was when Lindsay Jordan talked about how learning outcomes had been negotiated by her students (by setting up learning contracts as a starting point) and were used as assessment guidelines. Having explored ways to help my students understand assessment criteria better in the past, and how this was something they were really interested in, a light bulb in my head flashed: there is probably no better way than this to engage your students with assessment. When it turned out peer assessment (up to 10% of the final grade) was also used for blogs on that course, I thought: definitely the way to go.

Other words I will go away with are the ‘tyranny of participation’: it had never occurred to me that some people do strongly resent discussion forums. And video as being ‘disruptive’: you should really not forget that videotaping does take some people out of their comfort zone, to say the least.

Now, this was not all dark.  Lindsay Jordan shared with us student feedback which clearly indicated some students on the course felt that the process had helped them to get different perspectives on things, and that neatly takes me to another highlight of the day: the opening keynote entitled ‘the importance of stuff’, by Martin Conreen and Mark Miodownik, of Goldsmiths.

Spoons, Wolfgangus Mozart, Wikimedia Commons

Spoons, Wolfgangus Mozart, Wikimedia Commons

Two pleasant speakers, encouraging you to think outside the box, the keynote looked at how materials are a defining characteristic of society, and the related research projects carried out at Goldsmith. One such project titillated the audience no end, that of the spoons. Materials taste different, but why, was the starting point… The speakers took us through quiproquos and anecdotes, and the meanders of research processes and decisions. I won’t go into details but something will stay with me (maybe that was not the speakers’ point): if you stop to look at materials around you, you will see (well, taste) that they affect your perception of things.

And if you teach, or introduce new ways of teaching, then you need your participants to accept that there may be other ways to approach their learning or assessment.

Hibiscus_stigma, Ks.mini, Wikimedia Commons

Hibiscus Stigma, Ks.mini, Wikimedia Commons

A third and last favourite on the day for me was the 1 pm debate. The after-lunch timeslot is notoriously difficult as the audience digests its food, but the two well-experienced speakers, Melissa Benn and Fiona Millar rolled on with ‘Education and Equality?’ Melissa Benn started off with the remark that the question mark in the presentation’s title was highly relevant. The fact that we ask ourselves the question shows that it may not be the case.
In a nutshell, the debate pierced through raw facts:  that education is not equal and that the class system in England (the audience insisted this was not the case across the UK) still permeates families’ decisions as to whether they should send their children to ‘normal’ schools (as someone in the audience put it) or to public schools. The discussion, it seemed to me, all came down to perceptions (that one again!).

Two striking examples were given: that better-off families felt compelled to send their children to public schools and that young people from modest backgrounds themselves thought that state schools, generally speaking, were failing them. The panel seemed to hint that the vast majority of people were still imbued with class-related stigmas. The question was: do comprehensive schools prevent social mobility? And the answer, as I understood it, was: yes, all of the layers in society seem to carry this message, like a virus, which turns this into a self prophesy.
So… the end note was slightly depressing but, at the same time, promising. No-one knows what the future holds, but the message behind such a conference title might have been it’s for us to influence it.

REFERENCES

Engagement by stealth, PG Certs for a Digital Age, Lindsay Jordan, 21 May 2012, http://prezi.com/alsj8mbdjvbt/engagement-by-stealth/

Learning for free? MOOCs, Mira Vogel, 18 May 2012, http://prezi.com/telbnollfzx4/learning-for-free-moocs/

Future tense schedule: http://www.gold.ac.uk/gleu/futuretense/schedule/

Leaving audio feedback using GradeMark

May 22, 2012 4 comments
Microphones

Microphones Rusty Sheriff (2007): http://www.flickr.com/photos/rustysheriff/4880169398/ (CC BY-NC 2.0)

Are you using GradeMark in Turnitin to provide feedback to students? Did you know you can now record audio feedback on student assignments?

You can record up to three minutes of feedback on each student assignment allowing you to personalise your feedback. In the Sounds Good JISC project students remarked positively on receiving audio feedback commenting on the personal nature and level of detail provided. (Rotheram 2009a) Some students in this study did comment that they would like both audio and written feedback and you can still use the QuickMarks and general comments in GradeMark to leave written feedback if required.

How do I record audio feedback in GradeMark?

The attached guidance note provides step-by-step instructions on how to record audio feedback in GradeMark:  Providing audio feedback with GradeMark

So what do I need to get started?

  • You need to be using GradeMark in Turnitin to mark your students’ assignments
  • A microphone (An external microphone usually produces a better sound quality)
  • A quiet room to record the audio feedback. The MILL has two Podcast rooms that you can book to record your audio feedback – these provide a quiet space and the AV equipment that you need in order to record your audio feedback. Please send a calendar invite to video@city.ac.uk indicating the length of time that you would like to book a podcast room for and a member of the MILL team will respond to your request.

Tips on preparing audio feedback

  • Focus on the quality of the feedback as opposed to the quality of the recording. Don’t feel like you have to correct small speaking errors by re-recording. You can correct these as you would do in conversation. Do avoid poor quality audio as this can deter from the quality of your feedback.
  • Structure your feedback. Prepare a draft of the key points you would live to cover before you record.
  • Try to stay positive. Even when providing developmental feedback try to end on a positive note.
  • Speak clearly.
  • Make explicit how the feedback can contribute to the student development.  (JISC 2010; Rotheram 2009b)

References

JISC (2010) Audio Feedback [online] Available from: http://www.jiscdigitalmedia.ac.uk/audio/advice/audio-feedback (Accessed: 21.5.12)

Rotheram, B. (2009a) Sounds good: Quicker, better assessment using audio feedback [online] Available from: http://www.jisc.ac.uk/publications/reports/2009/soundsgoodfinalreport.aspx (Accessed 21.5.12)

Rotheram, B. (2009b) Practice tips on using digital audio for assessment feedback [online] Available from: http://www.kent.ac.uk/uelt/ced/conference/2009/Audio_feedback_tips_3_Rotheram.pdf (Accessed: 21.5.12)

%d bloggers like this: