On the Outcomes and Assessments Borg
There has been much in the blogosphere lately about Outcomes and Assessments and (mostly) how we should Just Say No. It's all Clio Bluestocking's fault. She started it. Then Historiann picked it up, and Sisyphus riffed on that, and then Dance offered an alternative. The shortest post I could make right now is to say, "Wait, people? you know that idea of Dance's that you all seem to like? that's a lot of what OA is when done correctly!"
Somehow, I think people might expect me to say a little bit more, given my lengthy comments elsewhere. So here are some reasons I think OA is not necessarily the Borg:
- As I just said, OA done well can be as simple as sitting down and thinking about what you are doing, what you want to be doing, articulating those things, assessing for the evidence of those things, documenting how your expectations fit what the students are doing, then going back and tweaking your teaching and/or your expectations the next time around, documenting that, and so on. It's about documenting process and goals. Having said that, there are people who want to use 'bad numbers' to beat us over the head. Those are the people we need to fight, not the idea. Self-reflection is not a bad thing, whether on a personal, departmental, or institutional level.
- Outcomes at the course level are good pedagogy. Too often we find ourselves building new preps in a hurry, and many new faculty who have not taught before, as well as some of their more senior colleagues, are wont to build the class based on coverage -- Look! a textbook that has the same number of chapters as the semester has weeks! we'll do a chapter a week, and then ... This is not a good way to plan a class, unless your idea of history is to teach a narrative via lectures, and then get the students to paraphrase your narrative on essay exams. I don't think that's what a good history class should be, even at the survey level. A university history class, IMO, should be teaching the discipline of history, as well as a (or several) narratives. So I'm all for thinking about what I want students to get out of my classes, and it's not a difficult step for me to translate that to course outcomes.
But that's not really what we're talking about -- this is more department- and program-level outcomes. Well, honestly, I can't find any reason that my department shouldn't have a common set of ideas about what somebody graduating with a degree in history should know, and know how to do, when they finish. Since my department has a capstone course, something that, by the way, seems to make a lot of accrediting agencies very happy, if it's done well, it only makes sense that we sit down together and decide how to make sure that the students are given the chance to learn the skills for the capstone no matter whose classes they take. It also makes sense that people teaching the same course should agree on how they are going to measure the students' performance, rather than having one History 101 prof giving weekly quizzes, two midterms, a paper, and a final, and another giving a research paper and a final and a power-point presentation for the same course. And generally speaking, the students should be learning the same material, although obviously different people will focus more on their strengths and interests. But if there is such a thing as canon, then it should be taught in each section of a course taught by several people. It's only fair to the students. - Speaking of departments, one of the things that Historiann and others have said repeatedly is that we are all professionals, and we already examine our teaching, and do the kind of self-reflection that OA is supposed to encourage. In a perfect world, yes, that's true. In the one where I teach? Not so much. I've never taught at a place where there was not at least one truly dysfunctional department, and usually a couple of faculty colleagues who weren't professional, weren't at all self-reflective, and were the pedagogical equivalents of a night on bad tequila -- the students can't help but vomit, and they can't bear the thought of getting near the subject again for years. Now, these colleagues are not going to go away, and these departments are not going to get functional, but the process at least allows the faculty who do want to teach in a solid program, and do want to serve the students who want to learn to work together more efficiently, document when their unprofessional colleagues are not playing nicely with other children to the detriment of the program and to its students, and let the students know what is expected of them.
- Students. They are not customers. But they do pay money to be allowed to study with us. Is it a bad thing to work together to make it clear to the students why we do things the way we do? to give them clearly articulated expectations and then demonstrate that we grade according to the expectations we say we're going to grade by? Is there anything worse than the perception that a faculty member is unfair (well, other than the things that are really worse, obviously)? But seriously, is it a bad thing to have an outcome that says that students will learn to differentiate primary from secondary sources, and use them to create a well-supported narrative or analysis? And then to say, "this is what we mean by well-supported"? Let me tell you, I've been in meetings where my very professional colleagues cannot give examples of these things, or even the characteristics of them, instead relying on the, "I have a PhD and know it when I see it," argument.
This is the one that annoys me the most, I think. The idea that we are somehow above explaining what it is we do. There is an intrinsic beauty in history and its study, I think. And I don't really buy the idea that we should have to justify our disciplines to non-experts. But at the same time, we really don't live in a world where people think that way. And we are employees of universities and colleges. If they are public, well ... when does, "trust me, I'm the expert," work for us academics? Moreover, we have a glut on the market in the humanities these days. If you can't explain what it is you do, and how you grade to your own colleagues, then there are people who can do your job just as well who can and will. - That last bit kind of brings me to my second-to-last point for right now. I'm a little freaked out by how authoritarian I sound. Because honestly, I'm all about good faculty governance, good contracts, and the tenure system (with regular post-tenure reviews). But I also see myself as an employee of my institution. There are things I am required to do as part of my job -- I have to provide receipts for reimbursement, I have to turn in grades on an A-F scale no later than the semester deadline, I teach history, and not English or Religion, I teach the students the university admits, whether or not I think they should be there, and I go to godawful meetings when I am supposed to. Sometimes I even teach a class at a time I'd rather not, because that's what the Dean has asked me to do. In exchange, I get to teach my subjects pretty much in the way I like, I get to teach courses I like, more or less -- as long as they serve the students and the university, I get support to travel and research (to some degree -- and lots of moral support!), and a paycheck.
Having said that, I can't imagine the sort of top-down approach to OA that a lot of people are complaining about. The places where I've encountered something even in that realm are places where either the administration didn't take assessment seriously and are therefore cacking themselves because they have to try to come up with masses of data all at once, or where there was never a clear explanation to the faculty of why they needed to drive the faculty part of OA -- or where faculty are just so disaffected that they refused, and are now facing something that is not at all what they expected. No matter the reason, Clio N's experiences sound pretty abysmal, and I can see why she doesn't want to do it. - Last point ... institutional assessment. I'd like to think that it worked. I'd like to think that Student Services, Admissions, Computing, the Registrar, Campus Life, Facilities, etc., were really assessing themselves the way good faculty do. Even if they aren't? I think that OA can help to support some faculty concerns, because my guess is that this is one of the places where number crunching can give a fairly accurate idea of what's going on.
It's dinner time.
9 comments:
I actually agree with quite a lot of what you say here - I do think thinking, collectively, about what we're doing (or you guys are doing, now ;-D) is useful - but myself? I don't think there is a canon, and I don't think there has to be a lot of consistency in the way people teaching different sections of the same course teach it. (With 2 caveats: if one section requires a lot more/less work than another, students will complain, which is problematic; and sometimes certain topics are required for, say, teacher certification programs.) I personally think it's perfectly fine if two people teach Western Civ completely differently, assuming they don't also teach it badly.
(I mean, imagine my Doktormater and the early medievalist at my grad program each teaching the medieval quarter of a Western Civ survey. Doktormater got past Charlemagne by week 3; the early medievalist only made it to Charlemagne by about week 6 or 7, and spent at least a week on Charlemagne. Now, I don't entirely agree with the early medievalist's timeline, but I still don't think he and Doktormater had to teach the same thing.)
I don't think they have to teach the exactly the same thing, or in the same amounts, but I can't imagine *not* teaching Charlemagne at all :-)
Sort of a, 'here are all the things the students should have heard of, and we'll cover them in the ways best suited to our own styles and specialties,' thing. I heard more about Barbarossa and the Crusades as a freshman from my modern Russianist prof than I have ever taught myself -- or than I learned at Beachy U! But the readings and the professors still talked about them to some extent. OTOH, I heard WAY more about the Angevin counts at Beachy U than I ever did as a grad student :-)
Oh -- another exception -- if a course is a gen ed, for example, everybody has to make sure the course addresses *those* outcomes. And we came up with three core types of assignments that demonstrate the skills we all think are essential, that would be used in all of our surveys, and then people can do what they want on top of them.
What you are talking about here is, I think, assessment done right. That is, it assumes we are mostly all professionals doing our job; the structure is needed less at the course level and more at the program level. That's where you can realize -- oh, we need to make sure every student knows how to do X, Y and Z before they hit the capstone course.
Equally clearly, Clio B.'s outcomes assessment is not so grown up!
This is exactly what it feels like at our institution: "The places where I've encountered something even in that realm are places where either the administration didn't take assessment seriously and are therefore cacking themselves because they have to try to come up with masses of data all at once..."
As I wrote at Historiann's, we already have a review process in place, and it could probably be improved or adjusted and so forth to ensure that our student are getting exposed to a good education and that we can identify places where we can improve our own teaching. Our administration, however, implemented our plan to get numbers to feed the accrediting agency. So, it is an exercise in data production, not improved teaching and learning.
Ditto, esp to point 5. Except perhaps not for standardization of assignments--I'd be inclined to agree on a canon of skills and people can teach them however they want, and students can pick the section/assignments that suit best. Whether of not there is a canon of content should be up the department---but it should be discussed.
I've been trying to get my department to just talk about what our core courses are supposed to be doing FOR SIX YEARS. They hear "Standardize!" when what I mean is "Communicate!" It's not fair to the students to be *pretending* there's a master plan that we don't tell them or our visiting instructors about. I mean, I think I'm in tune with the unspoken master plan, but hell if I really know.
That's my major frustration with my department.
I also think the system as a whole might be stuck with the Borg-style, unfortunately---I'm a fan of preemptive regulation. E.g., we'll do it our own way (like my random idea) which will produce numbers and data the admin can show off, and fend off critique. But having fetishized professorial autonomy over the least notion of "teaching as a community working toward a shared goal", profs sacrificed the chance to do it themselves, their own way.
When we've talked about standard assignments, we've talked about things like "analysis of primary source"; or, perhaps, identifying an argument in an article. If we agree that these are things we want our students to be able to do, it stands to reason we have to make sure our students do them in the Lower division courses.
PS. Just going to vent, although actually it's only been *four* years of frustration, not six. This is what I want to scream at my dept:
"We have a core curriculum. We ask our students to put a massive amount of trust in us. They turn over tons of autonomy to their professors. And yet, we can't even agree to talk about anything that might result in the least wee bit of compromise to *our* autonomy? That's unethical, bordering on criminal."
I understand Clio Bluestocking's hesitation at feeding data to the accrediting agencies but given the realities of accountability that the outside world is expecting of not only taxpayer-funded institutions but also taxpayer-funded programs at private institutions, we have two options. We can determine our own outcomes that provide that data or we can have them "given" to us from the outside.
At our institution, our office of institutional research is bending over backwards to try not to intrude on us while still keeping the university in compliance with state and federal regulators. However, they can't do this forever as they need our discipline expertise to make the entire data system work. Just like us, they want relevant, reliable, and valid data.
Our refusing "on principle" to participate is quickly going the way of the dinosaur. Some of us have been trying to expound on this for over a decade so it is a not a "new" situation. It's been a long time coming.
It's possible to uphold an individual discipline's tenets and its academic rigor but it's up to us to work together to get from where we stand now to the translation phase required for funding not only our university but our own faculty positions. The current budget crisis everyone is now facing only makes this clearer as cutbacks will continue to be made and restoration of positions and funding will be different than the world we know today.
Hooray for ADM for engaging an enlightening discussion about a controversial topic that needs to be examined well beyond semantics and reach the world of common goals that benefit not only our students but our profession.
Post a Comment