Cranfield University L&T conference: realising the promise of e-assessment

Here at Cranfield University in deepest Bedfordshire, with no phone signal (3 network) or access to the campus wifi network; no Eduroam, so no live tweeting (lucky you). However there is a "graphic recorder" who will be sketching the event and posting the drawings on a large white board (pic to follow).

Professor David Stephenson opens the conference by asserting the entrepreneurial culture of the university. Cranfield is interesting to me for several reasons. Few universities (or Business Schools) have quite such close contacts with large, high technology manufacturing companies. Prof. Stephenson emphasises the high capabilities of the Cranfield academic community.

Dr David Walker from Dundee University is the keynote. He will be speaking about "Realising the promise of e-assessment". Which is interesting because despite their allegiance to technology in general, Cranfield is pedagogically conservative when it comes to learning technologies, preserving a techno-deterministic, content-centred view of e-learning, and an antipathy to adopting LT: just like everywhere, really. 

Walker asks a good question: where does assessment start? He will be putting forward a model of mature institutional approach to e-assessment based on the University of Dundee's experience, where Walker heads the e-assessment team, based in the Learning Resources department. E-assessment has been on the brink of transforming education for 30 years: "end-to-end electronic assessment processes where ICT is used for the presentation of assessment and the recording of responses" (JISC). Definition has broadened from MCQ. Walker emphasises the importance of curricular alignment in e-assessment. Quotes Ray Land. Walker takes a strong social constructive perspective: "Academics do not create learning, learners do."

Walker asserts that e-assessment is likely to start with summative assessment of learning, driven by efficiency needs. But, he warns, if we start here and there is no preceding formative e-assessment or e-learning activities in the curriculum there is lack of alignment. He gives the example of students typing all their essays and then having to hand-write exams. Summative assessment is high stakes activity requiring validity, reliability, credibility, fidelity, fairness and transparency. E-assessment can help in all these areas. But, high salience for the assessed and the assessors leads to high anxiety. Therefore U of Dundee encourages a lot of formative e-assessment including, reflective journals and scientific logs as well as "traditional" MCQs. He suggests a wide conceptualisation of e-assessment including the use of simple appropriate technology (VLE, Dropbox) to administer submission and MS Word insert comment to annotate scripts. If e-assessment is only thought of as MCQs, it is seen as both difficult and empoverishing.

Walker provides a typology of e-assessment tools:

  • Assessment management systems (QMP, Intelligent Assessment)
  • Personal Response systems (ask the audience, clickers)
  • ePortfolio (PebblePad, One File)
  • Originality tools (Turn-it-in)

Exemplifies informing strategy: top-down, middle-out and bottom-up. Reflect the wider HE environment and also cultivate local networks. Dundee eLearning policy aligned with BS7988. Recognises the "hidden team" and the "front of the house". Dundee policy is now sector template, adopted by Soton, Bedford and others.

Walker concludes with institutional approaches to embedding e-assessment:

  • Enlist small groups (eLearning Forum)
  • Use awards as a promotional channel
  • Offer appropriate staff development (mostly 1-2-1 consultancy)
  • Support a wider culture of professional development (PG Cert, CPD)
  • Have annual cycle (spiral)
  • Formal and informal mechanisms for receiving student input to pedagogical processes.
Later a break-out session on e-assessment was run by Venkat Sastry and Piers MacLean. 

Venkat follows Angelo (AAHE Bulletin 48.2 1995) in seeing Assessment generally as the systematic gathering, analysis and interpretation of evidence to determine performance standards. He is a mathematician who uses "traditional" (MCQ) e-assessment extensively to inform students of their own progress. He observes that question creation is the main bottleneck.

The breakout session was a good (covert) example of the use simple appropriate technologies for formative e-assessment. Piers set up a Google document to capture three sub groups who were asked to address three questions relating to e-assessment:

  • What do you know about e-assessment?
  • How do you, or might you, use e-assessment?
  • At Cranfield, what should the direction be?
Each group typed into the live document from laptops in the corners of the room. The document was projected on the screen. Piers then invited David Walker to lead a brief plenary drawing out the key themes.

Basically the groups thought e-assessment was suitable for compliance testing and basic factual knowledge. There was a view that "pure" e-assessment was about electronic marking, while other forms of e-assessment more or less added to the academics work load. There was limited use of discussion forums for peer mentoring and there was some innovative use of discussion forums for conducting summative assessment.

That said, the session ran very well and if it could be presented as an example of formative e-assessment in practice, might be worth pursuing in workshop/seminar settings. I am thinking of how I might incorporate this into workshop activity designs. It has possibilities in distributed collaborative settings as well in one room.

The day ended with a series of world cafes on "The modern academic", "Blended learning", and "Post graduates: students, clients, professionals".

A good day to provoke thinking about learning and teaching.

Posted via email from George’s posterous

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.