Outcomes and Assessments: Point of Order

Some of you may remember that I am not antipathetic towards the outcomes and assessment movement. In fact, I'm rather a fan of the idea of faculty sitting down together and seeing how the students are doing and coming up with ways to make sure students graduate with a degree that means something. I like the idea of giving a common exam to all students on a course and having double-blind readings of the scripts. I like the idea of outside examiners. To me, those things are signals that there is some sort of conversation across a field about what is important, and it would certainly encourage me to work harder on content and on making sure I was better organized. I don't find such things threatening to my autonomy, as do many of my colleagues at SLAC and in other places. They are just reminders that I have to cover a certain body of knowledge, and not all students will answer all questions anyway.

I feel similarly about the sort of outcomes my department has, which are largely skills-based. Here I'm on very shaky ground and likely to really piss people off. Different historians have different sorts of skills expertise. We can't all be experts in all of them. I have a colleague who uses local archives all the time - I've never done local history, nor used an archive; at the same time, some of my colleagues have very different ideas of what it means to write a primary source analysis than I do. I think a student can write a perfectly good, and more importantly, rigorous, 5000-7000 word research essay that asks a question of a single source, perhaps a saga, or Orderic Vitalis' Ecclesiastical History; or perhaps of a few dispute charters or traditiones. Close reading is integral to what I do (so much so that sometimes I skimp on the literature more than I should). But if there is a single course that all students have to take to teach the skills we say we assess, it seems to me that faculty need to work together to define the skills and they need to give up some autonomy to make sure that, whoever teaches the course, the students can go on to all other classes in the department and be fairly successful without any one faculty member having to teach from scratch the stuff that students in their classes need.

In a way, outcomes and assessments done well and meaningfully are sort of a reflection of Rousseau's Social Contract. We all give up being able to teach only what we want,* or just our way of doing things, in order to make sure our students will be successful in other departmental courses, or if they transfer elsewhere. They have to have faculty buy-in and contribution at all levels, and faculty, even those who are willing, need to see the bigger picture.**

Lots of things can gum up the works. Lack of faculty buy-in, administrators who take far too dictatorial an approach (other than saying, "I don't care how you do it, but it needs to be done by X or we lose funding and accreditation."), Accreditation agencies and governments who are far too hung up on "accountability" when they have no bloody idea of what is important in a field (and everybody think they know what is important in history -- dog forbid that it requires anything like training to be an expert!).

But this week, I realized what the number one problem with having valid outcomes and assessments is. You could have the most cooperative faculty in the world and the most supportive administrators. It doesn't mean a damned thing if the students aren't prepared for university level work. If they can't manage, or won't do, the readings; if they don't come to class sessions, especially seminars and discussions, prepared; if they are ignorant of geography, how can we teach towards the outcomes we've set? How can our assessments and measurements of those outcomes be valid?

They can't.

If we teach at the university level, we have to have outcomes that reflect that level. But such outcomes are based on the idea that our students can work up to that level over the period they are at the university. How do we measure when we have to (and yes, this is true not just at my SLAC, but at colleges and universities all over the US) spend time on how to study, how to write an essay, how to read effectively,*** take notes, write an essay exam, become familiar with the most basic world maps of the present (let alone the past)...? Those things are just plain inappropriate as university level outcomes. Some of them belong in elementary school, for goodness' sake!

And how do we measure that the reasons students cannot achieve the outcomes we set, outcomes that should match up to those at other institutions like ours, are not necessarily because of our teaching, or even of student learning, are because the underlying assumptions of what it means to be ready to study for a university degree have not been met? Technically, it's not all that difficult to document, I suppose. We can ask students to self-report average time spent studying on a course (the are surprisingly honest!). We can scan and save student writing samples to demonstrate that students are not prepared, or that they have learnt something, but it may not be what they were supposed to learn for the course (mine usually write better at the end -- at least "write a well-argued analytical essay that answers a historical question, supported with specific detail," is one of our outcomes!

But I ask you -- how can assessment be meaningful if we spend as much time teaching students to be students as we do our subject? and how can the teaching in our subject not suffer if we are taking so much time away from it to give students the skills they need to succeed (to a point -- if students really are clueless and hopeless, I will ask them to drop). I have colleagues who simply fail such students, but there has to be a better resolution.



What do you all do, if you have to deal with such things? How will you, when you do?(which you might not if you are in the UK, since the Big Society mavens seem to be very happy with the idea of limiting uni education to the elite -- although they seem to have bugger all in the way of ideas as to what's going to happen to everybody else).




Update: Dave at The Long Eighteenth Century has pointed me to his very useful post on exactly this sort of thing! Thanks, Dave!








*Yeah, I know: most of us don't teach just what we want, but you'd be surprised at how many people think that academic freedom means complete control over the curriculum. There are things I don't teach, but it's not because I don't want to -- it's generally because I can't get to them because the most specialized class I teach is Ren-Ref!

**I will smack the first one of you that says this is a dean-ish comment. Just because I can think like an administrator doesn't mean I want to be one!

***this week I have had the fun of discovering that a student thought an incredibly common expression meaning "people were so focused on this thing that it became the driving force behind government and social policies," instead meant, "people discarded this thing and got as far away from it as possible." There was also the joy of having students tell me the only way they could learn was to bring their books to class and read along while we were discussing the information. No joke.