Sustainable assessment

Been asked to reread David Boud’s (2000), Sustainable Assessment: rethinking assessment for the learning society. For me the article dances around problems of performativity and supervision.

Implicit and explicit throughout is the assumption that individuals might become effective at self-assessment.

Assessment involves identifying appropriate standards and criteria and making judgements about quality. This is as necessary to lifelong learning as it is to any formal educational experience, although it may not be represented in formal ways outside the environment of certification. Assessment therefore needs to be seen as an indispensable accompaniment to lifelong learning. This means that it has to move from the exclusive domain of assessors into the hands of learners. A focus on methods and techniques needs to be replaced by a new conception of sustainable assessment required for lifelong learning.

Boud 2000, 151

On reflection I might see that I bring these problems. I often question supervised performance. Supervised performance is linked to real hunger through not lifelong learning but through lifelong employment. Real hunger is where will the next meal come from? For me? For my children? My neighbourhood, city, region. Can I provide my children the lifelong security I have had? There are plenty in the world who cannot, mostly through circumstances brought about by supervision and performance put at the service of, call it what you will: colonisation, surplus value, interest, rent, oligopoly; usurpation of a common wealth.

In abstracting many factors into the term “learning needs” we lose sight of the moral aspect of the question.

Sustainable assessment can similarly be defined as assessment that meets the needs of the present without compromising the ability of students to meet their own future learning needs.

Boud 2000, 151

Sustainability becomes an empty value unless there is consideration of some substance around what is sustained. Is learning a good in itself without regard to application or outcome? And, what is a “learning task”? Learners “… need also to be prepared to undertake assessment of the learning tasks they face throughout their lives” (152). Could daily activity become a learning task? How much activity should be framed as learning? Who frames the learning task? Who says: this-is-real-life; and: this-is-a-learning-task? When assessment is framed this way, questions of justice are raised.

Maybe this is easier when employed by a university or any educational institution. Boud reserves his comments for “…formal educational experience… within courses” (151). People like me and my colleagues are employed by institutions to assess formal learning activities in courses. If you remove “the domain of these assessors”, have you removed the need for assessment? For whom is the assessment performed, in the end? The assessors? Or, the assessed? Or another? Again I return to justice.

The idea of reciprocity appears to offer a means to retain justice within a framework of assessment for (and of) the other, but can a grand-narrative learning society serve reciprocal justice without doing to others? Boud draws on Frank Coffield’s excellent work. Coffield’s (2006) “Running ever faster down the wrong road”, summarises work undertaken throughout the 1980s and 90s, concluding, “… the government’s programme of reform in the public services, despite significant investments and successes, is now doing more harm than good.” I suggest this is as true today as it was then. There have been some successes in the first two decades of this century as there were in the last two of the previous century but, directionally, the inequities remain. Health and wealth inequalities are starkly illuminated by the coronavirus presence.

References

Boud, David. 2000. ‘Sustainable Assessment: Rethinking Assessment for the Learning Society’. Studies in Continuing Education 22 (2): 151–67. https://doi.org/10.1080/713695728.

Coffield, Frank. 2006. ‘Running Ever Faster Down the Wrong Road: An Alternative Future for Education and Skills. Inaugural Lecture’. Institute of Education. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.477.4397&rep=rep1&type=pdf.

How much of what kind of assessment?

Chris Rust and Mandy Jack initiated an exchange about assessment volume, kind and equivalence on the SEDA maillist.
Hours are most easily countable (and even that is not easy). There are also the number of assessment points and the “weight” given to each assessed point. Chris asserted that:
the only meaningful comparison is student hours and providing the total identified learning hours for the module (including the assessment activities) aren’t over 10 per unit credit there’s no problem.
I asked for clarification on the “total identified learning hours” per unit credit.
This is a slight diversion, but the question of equivalence is becoming increasingly important. So, too, is the question of “challenge”.
Toetenel and Rienties (2016) analysed 157 Open University course designs by all levels and activity types. Assessment (as designed or intended by those course teams) hovered around 25% of the time for all 3 UG “levels” and about 21% for PG. I do not know how much time the students actually did put in to each activity. Galvez-Bravo (2016), “… found that modules with more assessments had higher feedback (module appraisal) marks.” Simpson et al (2019) in a UK/US comparison found more assessment (both in number of assessment points and total volume) led to higher grades.
So a should a 15 credit course (150 hours) be expected (or designed) to have about 30-40 hours of assessment arranged over between one and as many as eight points in a semester? Two to four assessed tasks of 10-20 hours each? Where “exam revision” counts towards the total assessment load (two or three hours of exam and 10 hours of cramming the day before)?

Badges: Learning Gain or Just a Game, and what’s wrong with that?

Badges are Digital image files with text metadata stating criteria for which the badge has been earned. Badges are (presently) self-certified by Learner or Earner and  Self-certified by Provider or Issuer. Below are resources for a short session I ran for the Technology Experimentation Group (TEG). Continue reading “Badges: Learning Gain or Just a Game, and what’s wrong with that?”

A note on content, courses/curricula, and credentials

This note recounts a potted recent history of developments to do with online content and courses and speculates about the future of credentials in respect of the purpose of a university.

When learning management systems (LMS) or virtual learning environments (VLEs) were in their infancy around the turn of the century, faculty opposition to their introduction was sometimes expressed. Two reasons were often given (among others): if I put my lecture notes on the web no one will come to my class; and variations on a personal IPR theme: the students, the university or third party institutions will steal my content. Content was considered “king” and as long as universities and academics “owned” the content, their position was secure

MIT, with the Open Courseware initiative shattered the content-is-king myth. All the content from a leading university was made freely available: curricula, syllabus, reading lists, slide sets and exam papers.

The defensive focus shifted to courses. It wasn’t the content per se that was important. What the faculty and the institution did was select, organise, interpret, analyse and re-present content through curricula presented in courses (or modules, blocks, units, etc): sequenced events of limited duration (often a semester) presented in various modes (face to face and at distance)..

In 2007 groups of academics started to offer open online courses, hosted at universities but not requiring enrolment or a fee. This open online course movement became truly massive (MOOC) in 2011 when Stanford and MIT began to offer open online courses. From these beginnings spin out ventures (Coursera, Udacity, EdX, FutureLearn and others) started offering open online courses to the higher education “market”.

Now academic defensiveness has shifted. It is not the content and it is not the course or the curriculum that are offered uniquely by the university and faculty. It is the credential. Universities can award degrees and the degree, backed up by quality assurance processes is the guarantor of learning quality and the unique proposition which protects the value of the university.

Is it?

There is a movement towards micro-credentialling, or “badges” which I suggest is more important that many faculty and academics allow. I wrote briefly about badges here. I suggest this movement will continue the trend of opening up the university proposition and further challenge the role of the university in society.

A note on badges

On FSLT13 Badges were  awarded for completion of each of the four activities. Participants who wanted to collect the FSLT13 badgesl needed to register and enrol on the Moodle – AND needed to sign up for a Mozilla Backpack. Badges do not carry any academic credit but are a fun way to signal engagement with the course. Badges were be awarded using the WP Badger plug-in for WordPress, which implements the Mozilla open badges framework, Mozilla Backpack and Persona.

Why badges? We are doing this course to explore some of the developments on the cutting edge of contemporary learning and teaching practice. Badges for lifelong learning are on this rapidly approaching horizon: see Mozilla Open Badges Blog, HASTAC What’s on your badge list, and James Michie’s excellent and balanced presentation on badges on his Open Online Course #crit101.

First thoughts on the final (?) draft Strategy for Enhancing the Student Experience

Oxford Brookes Academic Enhancement and Standards Committee has made its final (?) modifications to the draft Strategy for Enhancing the Student Experience (SESE). The two objectives of this strategy are to:

  • [implement] approaches centred on critical reflection, impact evaluation and continuous enhancement of the student experience.
  • [maximise] student involvement in the development of policies and practices for teaching and learning and in extra-curricular, student led initiatives.

The strategy has principles and guidelines and (eventually) will have maps. Overall the tone is a bit directive: there is a lot of insistance and requirement, but the intentions are (I believe) worthy ones.

From a PCTHE perspective some of the key messages of this document are:

  • All staff who support learning will participate annually in high quality professional development (3.5.2)
  • All academic staff who support learning will engage with processes of evaluation, reflection and research into pedagogic practice (3.5.4)
  • The fundamental purpose of assessment will be to help students learn by providing formative feedback. (3.6.2) (so obvious but so often ignored in practice)
  • Students will be expected to take responsibility for their own learning, to actively engage with feedback and assessment (3.3.1) (this is perhaps one of the key messages of the Assessment Compact)
  • Students [will have the opportunity] to provide input and play a role from the outset in the development of new programmes. (4.3.4)

From a wider educational development perspective:

  • [With our] internationally recognised in-house expertise in educational development, we commit to routinely carrying out impact assessment, review and revision of all significant academic development initiatives and of measures taken forward in the SESE and the consequent strategy maps. (4.4.3.1)

Learning technology messages

  • [People will be] able to use technology to shape their own learning environment and interactions. (3.2.2)
  • [The] curriculum will be enriched by technologies that empower students’ development as self-regulating, digitally literate learners, able to shape their own learning interactions and author their own digital artefacts. (3.4.5)
  • The physical environment will be augmented by digital environments and technologies in ways which support a distinctive Brookes learning experience (3.7.2)
  • The functional access, skills and practices necessary to become a confident, agile adopter of a range of technologies for personal, academic and professional use. To be able to use appropriate technology to search for high-quality information; critically to evaluate and engage with the information obtained; reflect on and record learning, and professional and personal development; and engage productively in relevant online communities. (4.1. d)