Saturday, April 30, 2011

graduate seminars and assessment

by J J Cohen

Ask most any academic who has been teaching for the past five years or so, and you'll elicit a dyspeptic mutter about their institution's sudden craze for assessment and the clumsy way in which these measures have been implemented. Assessment is a mixed bag. On the one hand: yes of course we should have sturdy and transparent measures of how well we attain our goals. More than that, we should actually articulate our goals so that we and our students understand lucidly what a particular course and what a particular degree strives to accomplish. I was department chair when the assessment craze began at GW (triggered by ... an assessment of the university itself by the accrediting agency that assesses such things and, surprise, they said we don't assess as much as these assessors would like to see us assess, because after all assessment is so important that they have made their careers from it). I don't mean to mock assessment per se, but rather the way it sometimes is decreed from on high and ineptly implemented. One semester the syllabus does not have learning objectives, and the next it must, but no one is informed about why this matters, or how to formulate them effectively, and how one might be creative in articulating these goals. In the reading I did to start moving my own department along to mandated conformity (we needed to have 90% of our syllabi with these goals, or ... well, I'm not sure what. I told my faculty that one of them would be chosen at random and sacrificed in a festive bonfire outside the president's house). One thing that really stuck with me was how little thinking most of us do about the correlation between the goals of a course and the tests and papers we build into it. Often these are rituals inherited from tradition, but they do not necessarily test what we think we are training our students to do, or at least they are not necessarily the best way to determine if our students have attained the skills and knowledge we want them to possess.

That's a long winded way of saying that both undergraduate and graduate education in the humanities could be more creative when it comes to course requirements and pedagogical processes. It struck me, for example, that at a certain point within graduate training, multiple assessments via the same mechanism lead to diminishing returns. How many times does a graduate student need to compose a 25-30 page seminar paper during coursework? Three papers per semester times four semesters and we are at twelve such seminar papers. Wouldn't it be useful if at least one of these assessment exercises in a semester assisted with related skills (close reading, innovative method, cogent argument) performed differently? This semester I gave the students in my Objects seminar a choice for their final product: they could write a traditional seminar paper, or deliver a fifteen minute conference paper. If they chose the latter, they would be part of a mini conference open to all who wished to attend. They would be judged on overall presentation; handout; clarity; cogency; staying within time limits; performance; creativity. Four of the six students taking the class for credit chose the conference option.

I'm happy they did. Holding a conference gave the seminar a tremendous sense of closure, as well as an unalterably final deadline for work to be completed. We held the event Thursday, and it provided an excellent moment of community. The presentations were very good -- so good, in fact, that they could be brought to any "real" conference and be presented proudly. Plus the graduate students had the chance to obtain feedback on their style, their timing, their methods. I am hoping that nurturing these presentation skills will serve them well as they start to present papers professionally at conferences but also for some of them on eventual campus visits.

Having this moment together ended the course on a celebratory note. Also conference-like: we all went out for a drink afterwards, and spoke of the papers but also chatted about all kinds of other things, including conference banter and what you learn by socializing with people you don't know. I don't want to make it all seem drearily pedagogical; it wasn't. For me it was another reminder of the vibrancy I will miss now that I look at up to 18 months out of the classroom.


Another Damned Medievalist said...

Yes, to all of this. I really like the idea of giving the option of conference papers instead of the longer paper. Looking back, I realize that the Americanists in my program always wrote dissertation-chapter length papers. In *my* classes, we wrote different sorts of things: my advisor's papers were 20 pp historiographical essays, but his individual study papers (we each had to do three as part of coursework) were serious medievalist close reading or other skill-based 20-pagers. So I have a very detailed paper on 3rd C Roman coins to my name (that I really should probably think about doing something with, since the then-head of the Bayerisches Staatsmuseum said it would be publishable with some revision...). Some of my other courses required several book reviews and an exam or several short historiography essays (different to the ones in my advisor's class -- these were short and required one to identify particular schools and know who the movers and shakers in those schools were -- I would likely never have known anything about EP Thompson otherwise!).

I'm sure my grad department had no coherent plan, otherwise the Americanists would not have had such different experiences. And I'm not sure what would have happened if they had had to sit down and do any sort of assessment - probably the sort of thing you describe :-)

The problem is, we do have colleagues who are not self-reflective about their teaching. And when that happens, there is very little to be done (unless merit pay is involved!). I have a colleague whose regular response is, "we're all professionals; we know what a research paper looks like." But we don't. That's very clear from the work the students do. Or possibly, we all know what they look like, but we don't all think about how to teach students to make things that look like what we want. What I really hope is that grad students will be in classes like yours and see that there is a purpose behind teaching as we do, so that they are more likely to share with each other as faculty. One thing I think might be missing, though, even at a graduate level, is also responding to changing student populations and articulating the standards of the profession in ways that they get those things, too!

Rick Godden said...

Excellent post. I share your hesitance about assessment handed down from on high. It often seems like it's assessment for the sake of having it, to check off a box somewhere. So, it's easy to become antagonistic to the bureaucratic nature of how some of it is handled.

But, I couldn't agree more that we need to think more clearly, and perhaps creatively, about what our goals are when we teach. What skills are we teaching/reinforcing? What knowledge are we helping them attain? Why should they care?

I've been thinking a lot about this in terms of Freshman Writing, and the stakes are a bit different there in terms of needing to teach transferrable skills. But, I feel like this is such an important issue for English programs/majors. What are we teaching students to do? Why should they want to be a major, if they aren't planning to teach? I think we need to be clearer about this, and I have a lot more to say on this, but perhaps another time.

I also love the idea of giving different options from the standard research paper/seminar paper. I've given my lit students the opportunity this semester to substitute one paper with a creative piece. In the creative piece, they've been able to do things like adapt a text for a new setting (the tempest in gentrified Harlem) or add an interpolation. I then ask them to write a reflective piece to accompany it, discussing how they applied the terms/ideas of the course in their creative piece. This is also the class, Jeffrey, where my final paper is to write your own monster thesis, using your essay as some inspiration. I basically wanted to give my students a few different ways--creative, critical, something more expansive--to engage with the subject matter. I wanted them to think of the knowledge we were creating together as something other than static information.

Sarah Werner said...

I had a conversation on twitter in response to your post with Lisa Maruca, Karl Steel, and Holly Dugan, but the 140-character limit made it hard to get all my thoughts out. My point, then and now, is to agree that alternatives to the seminar paper format can be much more useful pedagogically for undergraduate students as well. When I was teaching Shakespeare survey courses, I used to offer two assignments, one a relatively standard paper using close-reading skills, the other a creative response to a play or plays of their choosing. I started doing this when I was teaching at a school where students hadn't always had good academic training up to that point, so I worried that standard seminar papers wouldn't let them reflect how much they actually knew about the subject: their writing problems would hinder them instead of giving them a means to express their ideas. So I developed the creative response assignment, which required them to respond to some aspect of the material both in creative terms and in a written rationale explaining their motivations and what they wished to express. I graded them primarily on their intent rather than on the execution of their idea. (I didn't want to replace judging their ability to write formal papers with judging their ability to write good poetry.)

I got wonderful responses, everything from screenplays and poems to paintings and songs. Some of the students talked back to the plays by writing alternate endings, or by telling the story from a character's point of view. Some wrote modern adaptations (in those cases, I generally advised students write out only part of the longer narrative and outline the rest). Some students produced pieces that were reflective, using the plays to talk about their lives. Regardless of the execution, it was usually clear to me which students were actively engaged with the problem of thinking about the plays, and which were just phoning it in. I loved grading them.

It was possible because these weren't writing courses and because the majority of my students weren't going to be going on to graduate study. Rather, my goal for the course was to get them to think about the plays both in terms of early modern culture and in terms of their modern usage. So I didn't see the point in training them to the level of strictly academic writing when what I wanted to know was whether they were able to understand and make use of the plays. This combination of assignments let me do that in a way that encouraged their creativity and gave them a way to show me what they could do rather than fretting about a writing format that they were never going to use again.

Alex Mueller said...

I think Rick's point about transferability is an important one, which may be one reason we rely on traditional seminar papers that engage in evidentiary reasoning, thesis-argument, etc. We believe this kind of written argument has value in contexts other than the English lit classroom. Yet, I'm not entirely convinced that other genres, especially the creative ones that Sarah discusses, can achieve the same ends, and often in more complicated and nuanced ways. For years this blog and the Geoffrey Hath blog have provided excellent examples, especially regarding role play. My students, like Sarah's, write from character/authors' points of view, but they do so in a larger class project that begins with blog writing. They select a character/author as their blog avatar at the beginning of the semester and write from that avatar's perspective throughout the course in response to all of the readings. I've discovered that students more readily engage in academic debate when they are freed from their "own" voice and assume that of another. You can imagine how liberating (and also limiting, by the way) it is for students to write as Margery Kempe, Chaucer's Wife, or even Milton's Satan. Most recently I developed this idea into a full blown "fan fiction" assignment in my Arthurian literature class. If you want to read more, I've shared my recent ideas about this here:

Alex Mueller said...

I just noticed my poorly worded sentence, which should say, "Yet, I'm not entirely convinced that other genres, especially the creative ones that Sarah discusses, CAN'T achieve the same ends - IN FACT I THINK THEY often DO SO in more complicated and nuanced ways."

Notorious Ph.D. said...

I'm still a believer in the seminar paper, but another thing that makes sense (especially for secondary fields) is to develop a syllabus. I could see one survey syllabus plus one specialized syllabus, each with at least a handful of fully developed lectures -- and having the students give one of these lectures as a part of their assessment.

Eileen Joy said...

Beginning this year, I decided to have my M.A. students do what I consider a medium-length seminar paper [12-15 pages], the idea being: there would be enough substance there to cut it down to 8-10 pages for presentation at a conference, or to expand it to 25 or so pages as a later thesis chapter. The bottom line is: make it practical. My university has also been going through some assessment upheavals recently, and while I sometimes cringe at "the culture of assessment," for the most part, I welcome the opportunity to re-visit how we teach our courses and what we think we are trying to accomplish. At the undergraduate level, we recently launched a senior seminar conference at the end of each semester, and it's been quite a success, where students make 8-10-minute presentations of their capstone papers, with professors moderating and responding.

At the undergraduate level especially, I have been having some misgivings lately about how, traditionally, we are teaching majors how to ultimately become grad. students in English studies, and then professors. We may not explicitly state it as such, but the kinds of papers we assign imply that what we are teaching is a certain kind of professional literary criticism [most suitable within the context of a conference or academic publishing], but is this really the direction we should be going in, as regards the future of literary studies, perhaps conceptualized in a variety of contexts [only one of which might be a future career in English studies]? This question troubles me a lot lately. I think we need to diversify the types of assignments we design, and I'm not even certain, some days, that inculcating my students in the nuances of postmodern literary theory and/or in scholarly research methods [traditionally defined: MLA/Project Muse database searches and the like] is necessarily the way to go. I have nothing more concrete to offer at present except a vague sense of unease over imparting to my students [speaking of *majors* here, undergraduate and graduate] what was imparted to me, as an act of regeneration, without radical modification.

Eileen Joy said...

I also want to share here that I've become increasingly concerned with imparting to students, over and over again, without modification, the standard format of *argumentative* papers, where there is a thesis + supporting evidence. As some have pointed out here, adding a creative writing option can be insightful and valuable [for both professors and students], but I wonder also if we can't think more broadly beyond the binary of the so-called scholarly paper versus the creative exercise? Is there a way to re-think the scholarly paper, article, seminar paper, whathaveyou, that moves beyond the thesis + supporting evidence model and allows for more creative analyses of texts/history that is not, nevertheless, "creative writing," more traditionally defined as: fiction, poetry, memoir, etc.? I've recently begun telling my students that I would prefer analyses of texts that are not so much arguments as they might be *unfoldings* of provisional, possibilistic meanings latent in the text, "what if?" scenarios, etc. What about a return to the "essai" as meditation, inquiry, exploration, wandering?

Mary Kate Hurley said...

I'd lost a really long and well-thought-out comment, but luckily Eileen posted almost precisely what I was thinking.

There has to be some way to learn that isn't founded solely on making a claim and proving it. That's a beginning -- an important one that I don't necessarily want to give up. But what kinds of evidence and argument might we find if we let go of some of the rigidity of our academic structures?

Alex Mueller said...

I want to enthusiastically support Eileen's call for " a return to the "essai" as meditation, inquiry, exploration, wandering." It's often become clear to me that many students aren't interested in an "argument" per se and that asking them to explore an interpretation can be more fruitful. Of course, many of these explorations become arguments anyway, but not just closed-minded arguments for one particular perspective. They more often than not embrace the multiplicity of meaning in the texts and situate their discussions against and within alternative points of view. We spend so much time asking to students to argue authoritatively, but we rarely ask them to stop, reflect, and disturb their newfound authority and voice.

Here's a quick example of how I try to suspend "mastery" over literary texts. For my intro to literature classes, I ask my students to write exploratory essays on common texts and then share their drafts within small groups of three or four. In response to this workshop, the students then must revise and respond to the papers in the group, quoting each other as they would secondary sources from JSTOR. It's compelling to witness students backing away or qualifying their previously held positions, which often results in revisions that are more precise and cogent in their thinking. This idea, by the way, was inspired by a practice described in Sheridan Blau's Literature Workshop, a book I heartily recommend for all literature teachers.

I'm not ready to give up on the thesis-argument paper, but I agree completely with Eileen that our writing assignments and assessments need to be reconceived. And who says a thesis-argument paper can't be creative anyway?

i said...

A few notes:

Alex, loved your article on Siege (OT, I know), and love the idea of the writing exercise you do. It suggests a solution to the problem I'm about to mention...

Eileen and MK -- as much as I love reading essais, and given my druthers would probably write that way very often, I find that this kind of pleasant meandering tends to be a sign of lack of work and thinking more often than not. And I say this not just of my students, but also of myself. As awful as it is to have someone pressing you to clarify your argument (and I would hope that most of my arguments are too complex to be shoved into one or two sentences), I've come to believe it's a deeply valuable exercise.

When my students hand me meandering essays -- even quite smart ones, with penetrating observations -- it's generally because they're giving me a first draft, and they only figured out at the end what it was they wanted to say. I'm not quite prepared to let them off the hook of thinking long and hard. This is not at all to say that I encourage them to give me simple arguments, and I tend to advise them (especially the more advanced students) that they can make small segues in the course of an interesting argument. I'm no fan of the essay in which every sentence strains toward the thesis. But it's one thing to break a form once you've mastered it, and another thing to break it before you even know what it is.

(Hence what I like about Alex's process -- that actually requires further work of drafting, thinking, etc. But an argument that is multifaceted, or aware of ambiguity and points of weakness or aporia, is not for that reason any less of an argument -- it's just an argument that is careful with evidence and sources.)

Mary Kate -- since I'm playing the gadfly here -- let me ask you to move beyond the suggestive rhetorical question. What kinds of evidence and argumentation do you imagine we might see in student essays if we gave up "rigid" academic structures? I assume none of us means we have to have the standards of proof required for, say, philological work in A-S studies. But what kind of evidence might come up in such an essay that wouldn't appear in print in PMLA or Modern Philology of JMEMS or Representations (or: pick your major journal). And in what sense would it be "evidence"?

I wouldn't like to suggest that I'm in the "trenches" -- most of my students are adept writers, and most have mastered the (horrible) five-paragraph essay. My task is often to push them beyond it. But many still have trouble making claims that are based on evidence -- and this was true of some Yale students as well. Sometimes I'll do debates, sometimes I'll ask them to take apart a point I made, but I consider it part of my duty to help them to understand the difference between a statement that's followed by some random words selected from the source text and a claim that uses the quote as evidence. And considering that we live in a society in which a lot of people have trouble questioning claims, thinking through problems of evidence, context, argumentation, etc., I see this pedagogical task as one of maximum importance. It has nothing to do with preparing students for graduate studies. It has everything to do with preparing them to parse through propaganda about immigration, health care, war, terrorism, national security, race, gender, human rights, global warming, theories of health, and so on, and so on...

Alex Mueller said...

(Thanks for the OT praise, Irina. I also adored your "Grammar of Pain" piece and am looking forward to reading your new one in Exemplaria!).

As for the discussion at hand, I agree that many exploratory essays tend to be digressive, unfocused, and lazy. I've read plenty of these - this is a real problem. Call me an idealist, but I don't think they have to be this way. The example I gave is designed to introduce students to academic dialogue, which tends to lead to the thesis-argument essay, appropriate use of evidence, etc., but I could also imagine the use of the exploratory essai (particularly in a grad course) that would emphasize possibilities based on the close reading of textual evidence.

I think the genre of the "commentary" could be a model for this purpose. We all know that many such textual commentaries we read in critical editions can be quite argumentative, close-minded, and heavy-handed, but the better ones (in my opinion) open up the field of interpretation rather than close it down. They are rarely focused on a thesis, they are only structured by line numbers, and they don't require introductions and conclusions. They do, however, require a careful attention to language, intertexts, and historical contexts. When I've asked students to write these (works especially well with Shakespeare), they often remain closer to the text than they normally would in a traditional literary analysis essay.

To get back to the original question about alternatives to the seminar paper, I think that (like most things) we need to achieve a better balance. That is, I believe we need to have students write seminar papers and thesis-argument essays for the very reasons that Irina mentions. I just think we need to diversify our assessments a bit, so that our students emerge from our courses with a variety of modes to draw from. Lately I've been disturbed by some of what I perceive to be an over-reliance on the argumentative mode, which leads many to draw lines in the sand and refuse to be persuaded by other compelling claims. In many ways this is the double bind of much of humanisitic studies. On the one hand, we want to assert our voices and positions against others. On the other, we want to engage in a fruitful dialogue that leads to new lines of inquiry, new arguments, new knowledge. If we don't provide our students with opportunities to reflect upon, analyze, qualify, and even retract their positions, I don't believe our literary debates can be as beneficial and transferable to other contexts.

i said...

Alex -- will you be going to Kzoo? We can continue the Society for Mutual Praise there... or the Society for People Too Interested In Medieval Violence. (Wait till I tell y'all what I got as an end-of-semester gift from my students.)

Your comment reminded me of one of my favorite assignments from undergrad. Jill Levenson, in her Shakespeare class, had us all do extreme close readings of Theseus' speech at the end of A Midsummer Night's Dream (the "lunatic, madman, poet" one). We had to OED every word, scan every line, and write a lengthy commentary. The kicker was that we also had to have an argument -- but it was understood that the form would make space for much more than the main argument. I had an absolute blast, and it taught me so much. (And I told her this.)

I think I am very conscious of the fact that my own tendency is toward commentary, towards wandering and musing and close reading with little point. I wish I could do just that sometimes. But in my brief years of teaching I've come to see that there's a real use to coming up with a clear "reading" of a text. For one thing, it helps other people teach, as your own Siege piece helped me in teaching that text the past few weeks.

That said, I do find that especially when it comes to scholarly monographs, I *much* prefer those that have a main theme but then meander and explore the differences and the details, than those with every chapter straining towards a forceful thesis. The former I tend to read all the way through -- with the latter I feel I've done enough just reading the intro. Obvs I would prefer to write the former, am almost incapable of doing otherwise -- but how to write a really punchy prospectus for it? (I'd be glad to hear anyone with more books to their name than I have chime in on this.)

Alex Mueller said...

Of course I will be at the Zoo. And of course my session conflicts directly with your pedagogy symposium. We will have to find some way to continue this conversation . . . we are too sympatico not to do so.