Monday, December 17, 2007

Assessing Assessment

Assessing Assessment


The semester is officially over. All I can see is work piled up from here to the next apocalypse. But tonight I am filled with Chinese food and a little beer, and I'm tired. I've also been percolating some thoughts about this gig at SLAC, now that I've been there for over a year. And I'm thinking about some of the underlying assumptions for assessment and accreditation programs (and yes, for how they fit into the T&P portfolio) and how they leave me cold these days. I've also been thinking about World History again, but that's for another post.

At one point at the beginning of the semester, I thought I'd hit my SLAC groove. It lasted about ten minutes. This semester kicked my butt, and not in that good way that leaves you exhilarated. I taught 13 lecture hours this semester. Two courses were upper-division courses I'd never taught before, and for one of those I had never taken any formal coursework -- I just knew that the students seemed to want something other than had been offered in the course before. I'd have liked some input on the course, because it's a service course. As such, it makes sense to me that the course be fairly consistent no matter who teaches it. Not so much. So I made it up as I went along. Part of the course was on the history of the writing or history, the rest was on the doing of history. I think the best day in the course was when one of the students said s/he thought it was going to be one of those classes where you're not really sure how it all fits together at the time, but you find yourself drawing on in all your other courses. I really hope so. The other course was in a field I know, but the SLAC mentality seemed to have made it hard. Or maybe I just don't know how to teach Upper-division courses effectively yet. What I wanted was to divide the week between topical lectures/discussions and discussions of primary source readings. The topical part kind of went by the wayside when I saw students trying to find what I was talking about in the text. Most of what I was talking about couldn't be looked up. Except for the military history, students didn't really read the primary sources. There was, admittedly, a good bit of reading for SLAC -- all of about 100-200 pp. a week. The surveys should have been easy, but they weren't. Again, there was a problem with preparation. Students just didn't do the reading. They didn't bring notes. Several students turned in only one or two assignments out of four. Much of the time, the students hadn't prepared because they said they didn't understand the language. That is, there were unfamiliar words, and looking them up in a dictionary made no sense.

Well, that's sounding bitter. To be fair, I had some students who really did try hard, and did do all the work. They kept me sane. They also made me feel guilty, because the classes could have been so much better. And in terms of personality, sense of humor, and being a mensch? I'd say probably 75% of the students are just really there. Despite that, though, this is the first time in my teaching life where I really thought, "these people aren't me." Not one of my students showed an interest in being really good, let alone excelling. They all simply seemed to want to be good enough. Good enough means getting a D if it's a non-major class, and a C if it is. Minuses are fine. It's sort of a shock. When I taught community college, I had a lot of grade-grubbers. I had a lot of students who were there to save money and who wanted to transfer into selective universities for their last two years. My classes weren't required, and so only those with an interest took them. Here, I have great enrollment numbers. My classes are not required, but they do fulfill a requirement. And students take classes with me more than once, even when they don't have to. My evals are fine. Students say they like me and that my courses are challenging. But they don't seem to want the A. Or maybe those who do are, like me, so brought down by those who don't care that they've given up.

That was all in aid of getting me to assessment. We're doing lots of assessment things at SLAC. Everybody is, it seems. We review out courses, departments, programs, etc., and re-examine the instruments of assessment. We redefine and refine our outcomes. We are told we must measure not only our students' performance, but our own. How do we measure that? By whether or not our students are successful. We must survey them. We must find standardized tests in our fields to give exiting seniors. We cannot be objective, we are told, so we must let external criteria be our guide.

In principle, I think most of this is fine. I like collaborating and norming with my colleagues, to a point. I like it when I can bounce an idea off a colleague and have him or her tell me I'm being a hard-ass, or unreasonable, or too easy. LDW has served as an external examiner for other universities. I like the idea of having an expert not employed at SLAC looking at what I'm doing and telling me it's in line with what is being taught at other places. After all, there is no one else here who teaches what I teach. How do I know if I'm doing it right? How do I know if I'm letting my standards slip or expecting too much? Sometimes it happens. LDW told me today that he thought the essay question for the Civ final was really hard. Admittedly, he doesn't teach World History, so the idea of a very broad comparison seemed very difficult to him. I had to explain that I was looking for big-picture and not too detailed answers, rather than the kind of deep and detailed essay I'd have wanted in a Western Civ class.

On the other hand ... well, all of the assessment of us as faculty is predicated on the idea that students are here to learn. It's based on the idea that students do learn, and remember. Even though we like to cringe, because so much of assessment and accreditation seems to focus on whether students are getting value for money, it *is* also based on the idea we try to impress upon our students: what we teach is important, and a BA/BA is not just a piece of paper that means a better entry-level job. But what if that's all the students want? What if they aren't so worried about the experience, about savouring and remembering what we try to teach them? What if, as one of my students said to me, they just aren't that into it and really just didn't feel like getting the A they could have got, because all they needed to graduate was 70%? For most of my students at SLAC, that seems to be the rule. Going to SLAC is kind of like buying a Louis Vuitton purse: you pay for it so people can see you did, but really, it's not all that special (I was going to compare it to a Prada bag, but hey, at least they are really good quality leather and they are frighteningly overpriced. Schools like SLAC are more like LV -- you pay too much, but not an insane amount and can't always be sure it's not a knock-off)

All kidding aside, how do we assess places like SLAC in a way that is fair to the good faculty and to the good students? If a campus tries to push a reputation as 'selective', then how do we integrate the results for those students who came in on waivers? There are many days when I'm not so sure I taught my students history. Most days, at least I'm sure I taught them something, even if it's a few new words or a better way to write a sentence or something that will be useful in the long run, but may not be on the course description. And if they aren't, they don't get assessed. Unfortunately, I can't teach the things I'm assessed on unless I teach them the tools to survive in a college class. It's exhausting.

And soon, I will also be assessed on how well I've managed to be a scholar on top of the teaching and service. I got one article (pedagogical) submitted at the beginning of the year. I somehow got elected to an office in a professional organization. I applied for a Dream Job (search suspended) and a postdoc. I'm behind on reviews, and I still have to write a conference paper, a scholarly article, and finish negotiating a contract for a project that is scholarly, but in the way of an aid, rather than something monograph-ish. And when I get that negotiated, I've got to finish it. And then I can work on another paper and the book. In between teaching those 13 hours (only 9 next semester), plus a summer course, and a new prep every semester for the foreseeable future. I'll let you know how that goes.

12 comments:

Ahistoricality said...

all of the assessment of us as faculty is predicated on the idea that students are here to learn

That really is the crux of it, isn't it? They are supposed to come out of our courses with more information and better skills than they went in, no matter how hard they work at it, or else we're the failures.

To some extent you can avoid the trap by insisting on institution-wide comparisons rather than field-based comparisons; that controls for one of the critical variables. That introduces other issues of cross-disciplinary comparisons, but it avoids the kind of "speed-up" pressure administrators tend to apply once they get comparative data with the institutions they wish they were working at.

timna said...

I hope you're going to have a real break in between the semesters.

wow. back to my grade grubbers at the cc.

Anonymous said...

Would it be comforting to remember that students have been going off to universities in order to get a better job rather than for love of learning for at least 800 years? John of Salisbury was complaining about ambition and avarice as what motivated most students in the schools, even before they got formalised into universities and there were many complaints about the predominance of the 'scientiae lucrativae'.

As a temporary distraction, should you want one: I have
meme-tagged
you.

Belle said...

Don't know about you, Magistra's comment didn't make me feel better. I do the World surveys, and the departmental assessment and I am constantly trying to find some way(s) of doing both better. My SLAC has the full range of students, from those who will work hard for an A to those who simply want the piece of paper and are willing to settle for the C or C- or D, whatever.

It is really hard to have any meaningful criteria on assessment when the department cannot agree on learning objectives. Do we want factoids competence or mastery? Or do we want them to learn to think historically, critically, creatively? Or just to vomit back names and dates?

Thanks for the post. It makes my own musings more comfortable when I know there are others out there asking the same questions. Now, we just need to figure out what some answers might look like.

Steve Muhlberger said...

"Do we want factoids competence or mastery? Or do we want them to learn to think historically, critically, creatively? Or just to vomit back names and dates?"

Wow, are there really people who would speak out for 1 and 3?

What Now? said...

St. Martyr's had a very similar student population, with probably more students at the higher end who had chosen to go to a less selective school because of religious reasons (which of course brought about a whole slew of other issues). All that most of them wanted was the piece of paper so that they could go get a job that 2 years ago wouldn't have required a college degree and that shouldn't really require one now. They all wanted at least a C to get credit, and preferably at least a B- to balance out anything less than a C in another class, but otherwise they were mostly coasting and going to classes in between bouts of drinking. And, I agree with you, it was really, really frustrating.

At the same time, magistra's comment does make me feel better!

Anonymous said...

Every time I drop by and read your blog (since you went to SLAC) I think you're at MY SLAC--except we don't have a medievalist. All of it, the woodchucks, the students, it's so much like my place I'm stunned. At a Louis Vuitton college (a great metaphor!)lots of people really don't want the work and at my place student life is so hegemonic that everyone but the faculty thinks clubs and games are a good reason for students to be absent from class. And I too loved my CC students. One positive is that our folks don't seem to understand assessment so we can hold off some of the more onerous aspects of it (outside objective tests being one). Hang in there ADM! It really can get better...

Another Damned Medievalist said...

Argh! Magistra, it really doesn't make me feel better, because in John of Salisbury's day one's success and continued employment was at least based on the ability to teach a particular curriculum and be reasonably popular. But one gets the feeling that students were held much more responsible for their own success. Thanks for the tag, btw -- I must think about it, since you took Charlemagne! I think I'm goign to go with Boniface, if no one else has taken him ...

Belle -- yep. And Steve -- sad, isn't it, but yep! I don't think any historians feel that way, but our administrators sometimes do. You can prove mastery of factoids.

WN, yet another reason I'm happy you aren't at St. Martyr's any more!

Ahistoricality -- That's a good suggestion. I should think about it.

Timna -- I hope you are done now and enjoying your break, too.

Kelly in Kansas said...

ADM, you point out some very important issues in regard to assessment. We all assume all of our students want to learn and that simply isn't the case for all of them for as many reasons as there are students. And, each type of IHE faces different challenges with each of their diverse student groups. I think we are all searching for answers and, in the meantime, it's do as much "formative" and "summative" as you can and demonstrate it's a process.

We've been going through this in teacher education for over a decade and, as predicted, is has spilled over to the rest of the higher education world. We can choose to get involved and set the standards or let others do it for us. So pat yourself on the back for doing all you can and do take a break over the break.

And, of course, we'll get together at AHA!

Anonymous said...

Thanks, ADM, I enjoyed this a lot. I think one of the issues here is the fact that assessment requires input from those being assessed to measure the local circumstances (proportions of students on waivers, etc.), but institutions often ignore such input.

In general, the question to ask is whether the assessment is being done, and the information being collected, is to improve something, or out of fear of being scrutinized too closely.

DM

Anonymous said...

Whoops, that last sentence should have read:

In general, the question is whether the assessment is being done for purposes of self-imporvement, or simply from fears of being scrutinized too closely.

Best,

DM

xx said...

cheap goose jackets online sale Denmark Canada, UK,
goose trillium parka jackets online sale Denmark Canada, UK,
Canada goose freestyle vest online sale Denmark Canada, UK,
Canada Goose Chilliwack Bomber sale online sale Denmark Canada, UK,
Canada Goose Mens Citadel jackets online sale Denmark Canada, UK,
cheap Goose Expedition Parka coats online sale Denmark Canada, UK,
canada goose snow mantra parka online sale Denmark Canada, UK,
Canada Goose Yorkville Parka Jackets online sale Denmark Canada, UK,
womens Goose Chilliwack Parka online sale Denmark Canada, UK,
womens Goose Expedition Parka online sale Denmark Canada, UK,
womens Goose Kensington Parka Jackets online sale Denmark Canada, UK,
womens Goose Montebello Parka jackets online sale Denmark Canada, UK,
Canada Goose Womens Solaris Parka online sale Denmark Canada, UK.