tag:blogger.com,1999:blog-21165575.post4140098733750963942..comments2024-03-10T20:46:19.274-04:00Comments on In the Middle: Peer-to-Peer Networks (A Guest Post by Martin Foys)Cord J. Whitakerhttp://www.blogger.com/profile/06224143153295429986noreply@blogger.comBlogger26125tag:blogger.com,1999:blog-21165575.post-35797690621410670192010-09-01T20:44:47.065-04:002010-09-01T20:44:47.065-04:00I don't want to turn (even more) negative, but...I don't want to turn (even more) negative, but unless there's something Richard said on Facebook that didn't come over here, when he says:<br /><br /><i>So here is what I conclude and why I conclude it. First, I think peer reviewing is fine as is.</i><br /><br />he doesn't actually explain why he concludes that the objections raised can be dismissed in a sentence. I'd like to see that explanation.<br /><br />As to Eileen's many (many) points, the one that immediately calls a response from me is her tale of the scholar with a hostile expert in his or her subfield. As many of you know, I work on early medieval Catalonia. In the English language, that makes me one of about ten, only half of whom are senior enough to be asked to review. I think I have now identified two of their styles well enough that I know when they have reviewed something of mine. Similarly, several of them must recognise me by now, not least because of where I differentiate myself from the others. To say this system is blind is ridiculous. It's a rare scholar, for example, who can avoid mentioning their own work, referenced or unreferenced, in a review.<br /><br />The other thing that now springs to mind is that doing this process online and openly enforces transparency. One doomed paper of mine was accepted for print in a certain journal on the basis of two reviews, one of which was in favour and the other uncertain of its author's qualification to comment. They had me make final revisions for press and then got cold feet and sent it to a third reviewer, who obviously came from a subfield I'd ignored and panned it. Somewhere in that, the system failed, whether the outcome was correct or not. (And, of course, I now have another place for the paper, so it's not as if the reviewing actually stopped it being published, just slowed it. The bomber will always get through.) Doing this sort of thing in public view means that rules of process will have to be clearly laid down and deviance from them will need accounting for.<br /><br />I'm only being anecdotal, of course, but none of us have more than our own perspective, even those who write thus:<br /><br /><i>The criticisms I have read of peer-editing are on based on utterly mistaken assumptions about how the process works. (None of my questions has received a satisfactory answer, from my point of view.)</i><br /><br />Surely the process works the way the people who are working it make it work... At PM the process is currently not working the way Richard expects. Alternatives are obviously possible. I thought we were talking about whether they were desirable, not defending a One True Way.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-21165575.post-27692526189802594122010-08-31T19:54:11.953-04:002010-08-31T19:54:11.953-04:00As a brief follow-up --and I think this a really p...As a brief follow-up --and I think this a really productive exchange -- I'm disheartened a bit by the fact that not everyone has re-joined the conversation at ITM. As Martin and Eileen have indicated, open peer review was something I thought would add to the issue. And I still do. I also think there is some healthy wrangling to go on over the idea. That's difficult to do when there is not a good faith interest in having a real conversation.<br /><br />it seems we have become more invested in controlled top-down criticism than critique.<br /><br />Thanks to those who have contributed here.jenboylehttps://www.blogger.com/profile/14562323018839311706noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-83176260957835457812010-08-31T18:56:09.237-04:002010-08-31T18:56:09.237-04:00And how, further, can we FOSTER new visions for mo...And how, further, can we FOSTER new visions for modes and production of scholarship and intellectual-creative work that, for lack of a better phrase at present, really go "outside the box"? Isn't that what the humanities should be about, anyway? As someone who studies the past, I've always felt, as I recently stated in a paper I presented in July [at the biennial meeting of the New Chaucer Society], that one of the tasks of medieval studies today would be to engage in a sort of subterfuge and resistance that draws on the resources of the deep past, hopefully formulated by us as, in the words of my friend and fellow medievalist Michael Moore, “*living* challenge[s], opening new perspectives which can lead us out of the impasse of our own times.” So, the past matters, but but not as static entity--more as matter always flowing, as we are always flowing, ever-changing. Holding on to "tradition" and an idea of "product" [over "process"] is just a kind of fearful way to work [and live]. If the past asks anything of us, it is to reinvent ourselves.Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-46216230970238625942010-08-31T18:48:27.680-04:002010-08-31T18:48:27.680-04:00[continuing]
In short, our vision isn't radic...[continuing]<br /><br />In short, our vision isn't radical *enough*. Much of the conversation here has been [perhaps unfortunately--and it's my fault, primarily] preoccupied with arguing with Richard Burt over whether or not the traditional system of double-blind, anonymous "expert" peer review is fine the way it is, or should be changed, and that's not really the primary issue to which the special issue of Shakespeare Quarterly or the forthcoming special issue of "postmedieval" are concerned. I think the larger concern [and I can only speak for myself here, not for Sarah Werner, or K. Rowe, or Whitney, or anyone involved in the SQ issue] has to do with, as Whitney puts it, creatively re-inventing what criticism is and can do [as an art form as well as a mode of so-called "scholarly" communication"]. It is also about, for me [and perhaps also for Jen Boyle], highlighting and/or representing our work as a type of collaborative "event-space" in which there are many con-fabulated intellectual [and other] comings and going as well as mediations and re-mediations of those comings and goings, and why shouldn't we work a little harder to better showcase this state of affairs? Also, why shouldn't we work a little harder, as Sarah indicates, to *enrich* and multiply the amount of feedback we all receive as we are working on/developing our projects? <br /><br />Whitney is right to point out that, in some respects, these so-called "experiments" [whether engineered by SQ or others] still return a "paucity" of information/feedback/products in comparison to what she has accomplished on her own with her own digital/online projects which, although they may not yet have academic prestige [whatever the hell that is], they are certainly helping her thought and development as an intellectual & creative artist. I've learned the same lesson through my association since 2006 with In The Middle and other online sites. The lesson here, then, is that the academy [and traditional outlets & arms for academic publishing] need to CATCH UP with what has already been happening and stop debating whether or not their precious systems for supposedly guaranteeing quality and "high prestige" can be protected in this "brave new world." In other words: hearts and minds have already been captured [and mentored!], so: what next? Some things, in other words, have *already* happened. Now, how can we better make sense of this, represent it, and harness its energies for the greatest amount of good for the greatest amount of persons?Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-82499389828963566882010-08-31T18:47:59.037-04:002010-08-31T18:47:59.037-04:00I'd like to turn now to this comment from Whit...I'd like to turn now to this comment from Whitney:<br /><br />"After the rich, mutually fulfilling exchanges I've had online, the traditional journal article is freeze-dried fruit. It's why I keep falling back on self-publishing digital projects or "creative criticism" published as art, rather than squeezing my research into a form to which its methodology is antithetical. Until we start talking about changing what we accept as "essay" and "argument" to meet our current media ecology -- i.e., *making* the dreams of Benjamin's trunk a reality -- structural amendments to the publishing process end up building libraries and writing loan policies for books that don't exist. The conversation will continue to flop lop-sidedly until we start talking more about, yes, what Darnton imagined twenty years ago (what Mallarmé imagined 120 years ago, what Georg Philipp Harsdörffer imagined 380 years ago) -- the book-to-come, digital Goethe's desks."<br /><br />I want to point out as well that I'm reading Whitney's comment here via a post she pointed me to via her Twitter feed, by Chris Kelty, on the ALLEGED closing of Rice University Press's, "How Not to Run a University Press," at the Savage Minds weblog:<br /><br />http://savageminds.org/2010/08/31/how-not-to-run-a-university-press-or-how-sausage-is-made/<br /><br />Key for me in Kelty's post was this:<br /><br />"Most scholars in the humanities seem to have their heads very firmly buried in the sand when it comes to the problem of scholarly communication -- even my fellow board members seemed more eager to crow about the necessity of maintaining high standards of rigorous peer review than to face either the challenges or opportunities in scholarly communication."<br /><br />[to be continued]Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-73403277335755451372010-08-31T16:48:13.576-04:002010-08-31T16:48:13.576-04:00I want to return to something that Whitney said, a...I want to return to something that Whitney said, and that Ayanna Thompson said in the Chronicle piece: one of the disconcerting things about traditional publishing is the relative lack of feedback that authors get (both Whitney and Ayanna seem to be talking specifically about the writing process, but I would add that there's silence after it's been published, too). Ayanna praised the sheer amount of comments she got to her SQ piece when it went up on MediaCommons, and Whitney talks beautifully about the paucity of feedback she's gotten elsewhere. I see one of the valuable things about this platform of publishing is the level of conversation it encourages. I've read lots of reader's reports to journal submissions, and I've written them, too, and I can attest to the fact that many of them are only a paragraph or two and often written in fairly vague terms. There are others that are not, of course; there are readers who provide beautifully detailed and generous comments that really respond to and help strengthen a submission. But I think many of us agree that's not the norm. So in terms of thinking about ongoing conversations, put me in Eileen's column: what a great thing these messy conversations are!<br /><br />And, again, should one wish to better understand what the process actually was at SQ, go to the MediaCommons site and read through the letter from Kathy Rowe and the many many layers of comments to the submissions.Sarah Wernerhttps://www.blogger.com/profile/06941029918210770136noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-75428824081953213652010-08-31T14:42:38.458-04:002010-08-31T14:42:38.458-04:00My response to Richard Burt on Facebook to his las...My response to Richard Burt on Facebook to his last comment: From EILEEN to Richard Burt: @Richard: I think you're partly mis-understanding the reasons behind this experiment--since the essays in "postmedieval" will have bee pre-solicited in advance, the aims for a hybrid/partial open review process are not necessarily pointed ...at changing something that is "wrong" the the typical double-blind peer review process that most journals have adopted, bur rather, is trying to highlight the development of scholarship as a certain sort of process, okay? My full comments are over at In The Middle, and I'll transfer yours there as well. I don't want to respond here at Martin's FB page to comments that are disconnected from the larger conversation over there, if that's okay with you.Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-18543232526147880512010-08-31T14:40:40.440-04:002010-08-31T14:40:40.440-04:00Just to keep this fun Facebook to ITM to Facebook ...Just to keep this fun Facebook to ITM to Facebook and back again dialogue rolling, here is another recent comment thread from Martin Foys' Facebook post pointing everyone to *this* ITM post:<br /><br />From EILEEN to Richard Burt: @Richard: I've posted a long set of comments at In The Middle, responding to your criticisms/questions. I'm the one who used the term "experiment," by the way, to describe the special issue of "postmedieval" that Jen Boyle and Martin will be co-editing together. I stand by that term.<br /><br />From Richard BURT: @Eileen. Great. Just be clear, I am not asking for a retraction or trying to stop the P-M experiment from going forward. Full steam ahead! I'm just asking questions based on my own peer editing experiences, experiences involving having ...my own work peer reviewed and serving on the editorial boards of ELR for 15 years and of the UMass Press for six years ( was chair for three) and for regularly reviewing manuscripts for University and trade presses. I think the current system works just fine. I have read blind submissions and non-blind submissions. No difference to me. No one I know has ever received preferential treatment (ELR was not blind submission, and to my astonishment, some excellent essays by SUPER FAMOUS were rejected). The criticisms I have read of peer-editing are on based on utterly mistaken assumptions about how the process works. (None of my questions has received a satisfactory answer, from my point of view.) So here is what I conclude and why I conclude it. First, I think peer reviewing is fine as is. Second, the SQ and P-M experimental special issues change nothing significant about peer-review editing. In both the SQ and P-M cases, the contributions are in fact peer-review edited (one could even say pre-peer-reviewed since all of the P-M essays for the special issue have already been accepted / solicited / contracted b y the editors, as was the case when I wrote an essay for a special issue of SQ on Shakespeare and film, and none will be rejected based on the experimental period; moreover, none of the contributors will have to take the comments into account the way a contributor would a reader's report). The only change is that more people without any authority over what gets published can comment on an early draft. Third, Adding the comments is not a good change, in my view. It just creates more work, extensd a process that is already quite extensive as is. (When there was a debate about an essay at ELR, the essay would be sent to 1 [ sometimes 2] more reader; the process could drag out for over a year.) Fourth, the experiment does not make peer-editing more transparent since all sorts of emails and phone discussions between the journals' editors as well as board meeting discussions about the issue and about specific essays in it will never be put online). (If you want to get an idea of what happens when "everything" gets published, check out the unusually heated responses to Sean Seasgreen essay on the Norton Brit Lit edition in Critical Inquiry; he quoted from emails.) Given all of the above reasons, I therefore have to conclude that the experiment is at best non-sensical, pointless, and a waste of everyone's time. That's just my personal review based on my experience and what I know of the experiment. I could be wrong, of course. I do not mean to offend you.Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-29029008446864832412010-08-31T14:12:22.791-04:002010-08-31T14:12:22.791-04:00"hear hear" also to Whitney's commen..."hear hear" also to Whitney's comments here; I've been following her blog for a while now and it's fantastic.Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-26264712025103021502010-08-31T13:36:09.802-04:002010-08-31T13:36:09.802-04:00Oh, and in case no one got the joke, the dream I d...Oh, and in case no one got the joke, the dream I detailed about the books printed online that authors would keep revising and crowds would come in to make marginal contents, etc., that is also a good description of "publishing" in the Middle Ages, and hence, the "system" we medievalist have to deal with all of the time.Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-63161652676612735692010-08-31T13:15:24.593-04:002010-08-31T13:15:24.593-04:00Did I say my last comment was it? I lied [haha]. B...Did I say my last comment was it? I lied [haha]. But seriously, just one codicil to all that precedes this comment:<br /><br />I think one of the most valuable and important functions of the humanities, which we often overlook at our peril, is to provide sites for experimentation and new creative *designs* and models for the way(s) in which we think and work, and that these sites and spaces should be as inclusive as possible. For one reason or another our discipline has become rigidly hierarchized over time, with all sorts of gatekeepers posted here and there as well as rituals and ceremonies of "orders" and "systems": how dis this happen? Our work needs to be more collective, more democratic, more inclusive, and frankly, there shouldn't be any rules at all. How else can we move knowledge forward? Sure: who will pay for it, employ it, publish it, etc.? [are the questions I hear coming]. We will.Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-90157248052875496402010-08-31T13:05:26.764-04:002010-08-31T13:05:26.764-04:00[continuing]
My final comments here, vis-a-vis RB...[continuing]<br /><br />My final comments here, vis-a-vis RB's provocative questions, have to do with the issue that I think is the most important issue being raised here, and also connects with Jen Boyle's interest in "event-spaces," and that has to do with "product" vs. "process." RB is happy to say that all he wants is "product" [never mind the "process" that went into it, which he will happily see "flushed"--it's only fodder, after all], to which, if he wants to, he will respond to with more "product." And so on and so forth.<br /><br />I actually think this gets to an issue that has been percolating as a series of discontents for a long while now in the academy, and which has also long been an important site for feminist work. In terms of prestige, getting tenure, etc., we have long valued the single-authored monograph over the supposedly more "messy" collaborative [perhaps never-ending, always ongoing] "project," and we prize the lone scholar who spends countless hours and years in his lonely study, emerging finally, like Nietzsche, with masterwork held aloft in one hand. The singular mind, the singular work, which will then be judged against all of the previous "important" singular works, and so on and so forth. No work *really* happens this way; it just looks that way, and also often gets rewarded as such.<br /><br />Although I know RB and others would balk in horror at the idea [partly because, again: who has the time?], but I have long thought how cool it would be to have books published digitally that would remain "open" online indefinitely in order to allow the author to keep revising it ad infinitum, and for others [the "crowds"] to come in an annotate its margins and send "messages" to the author. What a hell of a mess it would be to keep track of all of it, citation-wise, vis-a-vis other also-continually-evolving digital publications, but it would be a much more honest representation of the intellectual process, which is always--come on!--the process of a crowd. Speaking of which, I have several of them in my head all of the time, and so do all of you [thanks recent discoveries in neuroscience!]. Just as there is no singular author, there is also no "Central Meaning Maker" [as Daniel Dennett puts it] in your head, and Jen Boyle's emphasis on event-spaces nicely highlights the what I would call the crowded, always-flowing/changing, "evential" nature of life, intellectual and otherwise. It's difficult as hell to represent, but why not try?<br /><br />Speaking of trying: experiment, experiment, experiment. What else is there to do with this life we call creative and intellectual and imaginative? What a waste of time to always stay in place, just because we always have, and "if it ain't broke, don't fix it." Are you kidding me? I signed on to this life to fly, not to sit and wait.Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-78005406195967311862010-08-31T12:51:37.730-04:002010-08-31T12:51:37.730-04:00[continuing]
More importantly for this discussion...[continuing]<br /><br />More importantly for this discussion, however, vis-a-vis RB's comments, is the issue of what we might call "information overload." I think RB is right to bring up the question of whether or not we really need or *want* to wade through all of the comments that might come to us through an open or crowd-sourcing review processes [do we have the time? the ability to discriminate? etc.], and whether or not as scholars looking in from the outside we really want to, or have the time to, read all of the so-called "fodder" that goes into such an open review process. There is ample room for debate here, but again, I think RB makes some assumptions [stated as facts] that may or may not be true and/or desirable. As to the issue of "junk," trash/garbage, poop, etc. that could possibly be produced by an open review system, the world and its libraries and attics and studies already teem and flow over with thousands of small textual voices all vying for our attention, some of beautiful quality, some not. For every Proust there is a Hemingway, for every journal of a famous man the letters of a poor, unnoticed one, for every author an archivist, but not enough readers or visitors for everyone and everything [why else is Katherine Hayles so rightly excited by machine-reading?], and so on and so forth. <br /><br />Nevertheless, I personally stand on the side of "more" over "less." For a long while now, I have been an advocate of open-access, digital publishing initiatives, and of all varieties, whether it is strictly regulated along the lines of traditional publishing standards [to whit: Open Humanities Press, with its eminent Editorial Board overseen by humanities luminaries such as Alain Badiou and Katherine Hayles] or whether it is Nicola Masciandaro using online "creative commons" publishing-on-demand sites to publish the proceedings of a symposium on heavy metal and theory that he hosted in NYC last year [and which event was covered by the NY Times, I might add].<br /><br />Why do we cite the fact that we write for limited audiences as a reason to scale back what we do, or to limit who has a say in it? That makes no sense to me. If we really are read by no one, then stop writing altogether. Sheesh. I'm a card-carrying utopian idealist and actually think we should be working harder in the humanities to broaden our audiences, not narrowing them to willy-nilly and then being sarcastic/nonchalant about it. For all the jokes you can muster about books being used a toilet paper, I can tell stories of books that saved lives [including intellectual work]. Both stories are true, and I'll choose the latter as inspiration for my work, no matter how large, how small [if even one person] the audience. In the final analysis, for me, the traditional double-blind peer review process is a relic of a different age and I think we would all be lying a little bit if we did not admit that a lot of our professional anxieties and worries [especially for those of us more junior in the profession, or trying to get into the profession] stem from the non-transparent nature of this process. <br /><br />Is the idea of transparency in academic publishing a complete mirage/fantasy, as RB implies? Well, maybe in government, or in corporations: yes. But in academic publishing? Of course the system can be more transparent: is there anything really to gain from keeping it shrouded behind the veils of anonymous "experts"? Even the idea of what constitutes "expertise" needs to be re-visited: it is, after all, a concept/state of reality we have inherited in the wake of the hyper-techno-professionalization of the humanities, post-the 19th-century.<br /><br />[to be continued]Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-32192247730732084132010-08-31T12:24:52.905-04:002010-08-31T12:24:52.905-04:00My first article was published in the British Libr...My first article was published in the British Library Journal [which is quite a coup in studies in bibliography--my diss. was in intellectual history and one chapter was on the intellectual history of bibliography in the early modern period, vis-a-vis the contents of the Cotton Library], but first it went through the peer review wringers at Studies in Bibliography [Univ. of Virginia] and then at Libraries & Culture [Univ. of Texas]. I got some REALLY helpful comments from 3 readers, but 1 reader wrote, in a manual typewriter no less, 2 sentences that went something like,<br /><br />"She's got it all wrong. And: why did she even bother, the subject is not worth the time."<br /><br />Is there something wrong with an editor sending that comment on to me, and ONLY that, as an "expert" review? Uh, yes. Everyone has at least one story of getting comments that are just plain out of left field, wrong, and rude, while we all also have stories of getting incredibly tough-yet-incredibly-beneficial feedback as well. At the same time, so much time is often wasted when the reviewer and author have no acquaintance with each other or with each other's work and preferred methodological/theoretical approaches, and how many more letters do we all need to read where the reviewer assumes, for example, we haven't read something we have, and so on? Going back to my British Library Journal article, I'm in Old English literary studies, but the article, if you didn't know better, looks as if it were written by an expert in studies in librarianship/bibliography, maybe a historian of library science, etc. So one reviewer, noting that I had quoted Kevin Kiernan [the editor of "The Electronic Beowulf," among other things having to do with codicology and manuscript studies--his area of expertise] said something to the effect of, "boy oh boy, if this author only knew how controversial and *wrong* Kiernan is, like everyone in manuscript studies *knows*, she/he would never cite Kiernan." Let's see: the "Electronic Beowulf" took up one whole chapter of my dissertation, I'm well aware of the debates over his dating of the "Beowulf" manuscript, I happen to agree with him [as a fellow expert], AND he was a reader for my article and a supporter of my own work, so I *will* cite him, thanks. I also thanked him in print with the BLJ article appeared. Yeah, these sorts of misunderstandings happen all of the time and they don't necessarily interfere with the publication of an article [after all, my article was eventually published and I got some terrific comments, too], but do we really have to make all of these mistakes along the way? I don't think so.<br /><br />[to be continued]Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-76069717843990211342010-08-31T12:24:25.970-04:002010-08-31T12:24:25.970-04:00[continuing]
I'm not sure where to begin here...[continuing]<br /><br />I'm not sure where to begin here, but I'll try. Has peer review ever caused "scandals"? I honestly don't know [chime in if you do!], but is it a system that has no problems or capacity for error and abuse? My god, NO. Perhaps RB has been immeasurably blessed by the double-blind system as it stands now, but there are so many stories to relate about all of the ways in which it can fail miserably that I practically don't where to begin. For the reasons both Jonathan and JJC pointed out here, editors often have to scramble to find good so-called "expert" reviewers, and sometimes you really get the expert you need, and sometimes you don't. There is also the question of hostility toward certain methodological approaches, regardless of the ultimate quality of the piece in question. I have a friend, who shall remain nameless, who works in a sub-specialty of early medieval history that is so SMALL, expert-wise, that he has a total of about 3-4 readers for everything he sends out. He has been unable to be published in an north American journal in his field for his entire career [he is in his mid-fifties]--he publishes primarily in European journals--and it took him almost 15 years to finally get a book published [and he had all sorts of job woes & anxieties related to all of this] because it kept getting sent to ONE person who vehemently disagrees with everything my friend believes about their shared historical period. Yes, this person is my friend, so I may be biased, but I feel pretty strongly about the quality of his work: it's amazing, beautiful, and smart. Hardly anyone in historical studies writes as beautifully as he does, or with such conviction and high ethical standards. But career-wise, he's at the top of a certain expert-someone's blacklist, and that person has been able to hide behind his anonymous-but-not-really-anonymous-faux-anonymity. Is something wrong with that system? Uh . . . yeah! This may even qualify as "scandal" if someone's career is harmed, and I think we all know, deep down, that some careers *have* been harmed.<br /><br />But it isn't just the expert's fault, it's also the fault of the editors, at journals and presses, who sometimes don't provide enough managerial oversight. Editors *could* make the peer reviewing system as it stands now better by exercising stronger oversight of the reviews solicited and received [I and my co-editor Myra Seaman, for example, have made it a policy--time constraints, notwithstanding--to review ALL reviews, and if they don't meet our standards of politeness and fairness and expertise, we send them back for revisions, and in some cases, we revise them ourselves to be as helpful as possible to the authors]. But again, going back to JJC's comments, who has the time?<br /><br />[to be continued]Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-86379926926272438042010-08-31T12:23:20.313-04:002010-08-31T12:23:20.313-04:00Thanks to everyone for such a spirited conversatio...Thanks to everyone for such a spirited conversation here re: the Shakespeare Quarterly experiment. I want to respond to some of Richard Burt's questions but I also find them, to be honest, somehow "off" in tone. I can't really respond to questions that are framed with supposedly factual *statements* that are in no way factual at all, and are actually the opinions of Richard. I say this to Richard with the utmost respect for his scholarship, which I have long admired. So let me start with these assumptions, before I even get to the questions, which are premised on *assumptions* that may or may not prove to be true in the long run, and in any case, they deserve further rumination and debate, otherwise we're just foreclosing this whole conversation to begin with, and why would we do that? And my thanks to Jen Boyle as well for contributing comments here, since SHE is the one who initially conceptualized and will be co-editing with Martin Foys the forthcoming special issue of "postmedieval" on "Becoming Media" [Vol. 3, Issue 1: March 2012]--and that's "postmedieval" with a small "p," by the way--and I hope everyone will keep in mind that what Jen has in mind for this issue will go beyond the SQ "experiment" because she has some very novel ideas about the presentation of the issue that are not about the peer review aspect at all [some of her comments here cover that]:<br /><br />1. RB mentions the idea, put forward in 1996, of drafts of dissertations being put online, and then he says, "that never happened." NOT TRUE. Here at In The Middle, for example, Karl Steel shared more than several drafts of parts of his dissertation, and most importantly, his Conclusion. His brilliant book, forthcoming from Ohio State UP ["How to Make a Human: Animals and Violence in the Middle Ages"], graciously acknowledges the feedback he received along the way from blog readers and commenters. Karl can explain better than me how this process helped him, but I know intuitively that it did. There are countless examples of PhD students sharing portions of their dissertations via weblogs and the like and getting important feedback, from experts *and* from other student-peers, and I won't waste time providing more examples. Whether or not this process has "official" oversight from students' diss. committees is, frankly [in my opinion], beside the point.<br /><br />2. Regarding the NY Times article on the Shakespeare Quarterly's partial/hybrid open peer review, RB wrote,<br /><br />"The statements about peer-review being bad in the NY Times struck as being entirely fabricated. I have heard of no actual case where peer reviewing per se has been a scandal. I'm just sayin': it ain't broke, so why fix it?"<br /><br />Later on, in a further comment, RB notes the "non-existent problem in peer-reviewing as presently practiced."<br /><br />[to be continued]Eileen Joyhttps://www.blogger.com/profile/13756965845120441308noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-14057050206421703922010-08-31T12:14:30.272-04:002010-08-31T12:14:30.272-04:00I've been avoiding responding to any of these ...I've been avoiding responding to any of these discussions, because I'm a graduate student with limited experience of the "traditional" peer review process (and then only as an author, not as an editor or reviewer); but, for what it's worth:<br /><br />I participated in the SQ "experiment" (the word does deserve quotes, I think) as the author of a review of digital resources, which means my essay was not peer-reviewed so much as peer-commented. Although the stakes were low for me, I didn't think twice about participating: I already have a public academic persona that blogs and microblogs half-baked research on a daily, almost hourly basis. In fact, I posted scans of my notes for what will become my submission to the Postmedieval "becoming-media" issue over three months ago, and all my ongoing notes for the project are available on my public wiki. I regularly solicit and receive input on my reading lists, arguments, even prose style through twitter. Frankly, whatever the Postmedieval "open review" process throws at me will probably be less rigorous than what I've already put myself through.<br /><br />Which is, of course, exactly the problem. After the rich, mutually fulfilling exchanges I've had online, the traditional journal article is freeze-dried fruit. It's why I keep falling back on self-publishing digital projects or "creative criticism" published as art, rather than squeezing my research into a form to which its methodology is antithetical. Until we start talking about changing what we accept as "essay" and "argument" to meet our current media ecology -- i.e., *making* the dreams of Benjamin's trunk a reality -- structural amendments to the publishing process end up building libraries and writing loan policies for books that don't exist. The conversation will continue to flop lop-sidedly until we start talking more about, <a href="http://blog.whitneyannetrettien.com/2009/10/books-redundancy-getting-past-our.html" rel="nofollow">yes</a>, what Darnton imagined twenty years ago (what Mallarmé imagined 120 years ago, what Georg Philipp Harsdörffer imagined 380 years ago) -- the book-to-come, digital Goethe's desks.Whitneyhttps://www.blogger.com/profile/01064261761562860891noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-25799424859302920002010-08-31T00:26:57.275-04:002010-08-31T00:26:57.275-04:00Re Richard re Eileen, the publication as pooblicat...Re Richard re Eileen, the publication as pooblication topos predates Martial, actually. Catullus, in the first and last lines of carmen 36, calls the poetic output of voluminus Volusius "cacata carta", shitty sheets. <br /><br />Michele KennerlyAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-21165575.post-44707386083623917322010-08-30T15:05:02.986-04:002010-08-30T15:05:02.986-04:00Since Martin and I are editing this so-called expe...Since Martin and I are editing this so-called experiment, I wanted to chip a few cents.<br /><br />In part, I agree with the tenor of Richard's more general critique of all this: appeals to "radical" innovations and progressive dreams should draw skepticism.<br /><br />That said, I am not sure that such utopianism exhausts all that is in play here. Two follow-up points, one on a practical/professional note (which, interestingly, seems to be where many of these exchanges come to rest); and another less practical observation.<br /><br />"Open" peer review does not necessarily imply that everyone (upwards of 350 plus) can or will comment on material. The SQ instance is an example of a process where reviewers were asked to declare their interest and expertise before commenting. In other words, many open peer reviews function less like a crowd than a dinner party. The former implies shouting from the blind, the latter an event of interfacing that at least points to the possibility of taking “to” and “with” not “at.” While I don’t necessarily think that open peer review is more transparent (in the many ways we can imagine this term), it does create some opportunities for thinking about the “product” and the “process” in ways that we are not typically asked to consider. <br /><br />Many of us are re-imagining ways/forums/mediations that allow some of the theoretical in the performative, and the performative in the theoretical, to flow through and with our work and production a little more freely. Open peer review is, perhaps, another possible event-space. Speculative realism, OOO, animal studies, “thingness” revisited, media/mediation/ posthumography are all ventures that insist on re-imagining the affordances of object-environments-affects, of re-imagining what we mean we speak of the form in formulation, of the surfaces that augment the “thing itself,” and yet we often retract into very comfortable articulations of practice and product when we get down to the business of the tools we USE. <br /><br />Posthumography in the object list above refers to a special issue of Rhizomes , “Boundaries of Publication: Posthumography < http://www.rhizomes.net/issue20/ >. This is a tremendous issue – and Richard Burt has a fabulous essay in this collection on desks, files, machines, and death – and worth reading in full (Don’t flush). I mention this issue because I think that many of the points Richard makes in that article would enhance this conversation. How would we imagine the paratext of peer review? Is peer review its own kind of homage to the posthumous? While open peer review may not be more transparent, it is a paratext; it is a new/old kind of ghosting the text; it is a graph and trace that seems to me to deserve a try. <br /><br />Returning to a practical matter, the above issue was obviously thought through, and the contributions all speak to one another beautifully. I can’t help imagining, however, another surface to the images, texts, and performances of that collection. I like the idea of Benjamin’s trunk, Derrida’s papers, Goethe’s desk, and the apparatus of the publications publication all hovering together as paratextual morass.jenboylehttps://www.blogger.com/profile/14562323018839311706noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-31434919662872493462010-08-30T15:04:16.596-04:002010-08-30T15:04:16.596-04:00Martin Foys:
Hey Richard - thanks for those though...<b>Martin Foys</b>:<br />Hey Richard - thanks for those thoughts. A few things:<br /><br />Where do I call our P-M issue an "experiment"? I have no hypothesized expectations as I might in an experiment. Only curiosity and a desire to explore.<br /><br />I get the feeling you feel I'm charging up a hill waving some kind of battle standard trying to change academe. I'm not. It wasn't even my original idea to provide colleagues access to submitted essays for the issue - it was my co-editor Jen Boyle's.<br /><br />That said, I'm certainly not opposed to it - new technologies occasion new, untested methodologies, and I'm not adverse to seeing what happens when we implement them, since it is such relatively untested waters - if only (but not only, really) to let all contributors participate in a more immediate feedback loop with each other as they revise drafts, as I already noted in an earlier post - might be useful, might not.<br /><br />And now, I'd like to bring your questions and my answer over to the ITM blog, as others might have better answers for you - do you mind?<br /><br /><br /><b>Norman Hinton</b>:<br /> My feeling about peer review and about the absence of peer review are so complicated that I don't know HOW much room it would<br />take to detail them.<br /><br /><br /><b>Richard Burt</b>:<br />"And now, I'd like to bring your questions and my answer over to the ITM blog, as others might have better answers for you - do you mind?<br /><br />Not at all, Martin. do ahead, by all means. Thx.<br /><br />I can see the merits of expanding peer review to conclude contributors to an already peer-reviewed journal. I just don't understand why others see this process as not being within the purview of peer reviewed publishing.<br />I guess I hallucinated "experiment." Sorry.M.K. Foyshttps://www.blogger.com/profile/09753310457769493322noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-5742917047836951482010-08-30T15:02:31.309-04:002010-08-30T15:02:31.309-04:00More from my Facebook thread . . .
Richard Burt:
...More from my Facebook thread . . .<br /><br /><b>Richard Burt</b>:<br />Martin (cc'ing Eileen and Karl),<br />Here a few thoughts that might have some bearing on the special issue of Post-medieval you are co-editing.<br /><br />1. What would happen if you were to present the rationale in terms of what adding the pre-publication material does on its own terms to the publication rather than as a remedy to a non-existent problem in peer-reviewing as presently practiced / as a radically new way of getting comments (when it's really a variation on already existing electronic commentary practices)? What aspects of peer editing does it do better?<br /><br />As an analogue, I am thinking of the American Historical Review. When it first went online, the online edition had a number of features the print edition didn't. Online articles were linked in the endnotes. This was a great idea that never caught on since we still have to use online notes to get other online articles the way we do footnotes of articles printed on paper. The lead article article was written by Robert Darnton. His essay linked to several others he had published as well as to an interview with him. He had maps and some other stuff online. Historians seem very open to printing two versions of the journal. Every other journal I can think of makes the online and paper editions the same (in terms of content). But AHR was and is concerned with publication, not peer reviewing or pre-publication material.<br /><br />2. You called your special issue of Post-Medieval an "experiment," so I'm wondering what results you are expecting from the experiment. What decision or decisions about how peer editing works at Post-Medieval will follow from your experiment?<br /><br />3. If nothing follows from your experiment (and certainly the special issue of SQ will change nothing about peer-editing at SQ), what is the point of performing it? Restated: If it is just a one time experiment that will leave things at P-M as they are, what's the point of performing it? If what you are doing as an experiment in your special is clearly better, why not just change the peer-reviewing process at Post-Medieval now, especially if it is a remedy to an existing yet widely unrecognized problem?<br /><br />4. Finally, if P-M wants to do something radical, why not make it an open access journal? Your experiment will be performed, as far as I can tell, under the same constraints in place at all academic peer reviewed journals and all essays in the special issue you're doing will be peer-reviewed (as usual). So nothing really changes, no? Or does it?M.K. Foyshttps://www.blogger.com/profile/09753310457769493322noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-74501703664821122142010-08-30T13:16:51.034-04:002010-08-30T13:16:51.034-04:00the current system by which junior faculty are shi...<i>the current system by which junior faculty are shielded from committee and extracurricular work by their colleagues</i><br />What? Really? Where can I get a gig like that?Karl Steelhttps://www.blogger.com/profile/03353370018006849747noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-14750671980434077082010-08-30T08:41:53.888-04:002010-08-30T08:41:53.888-04:00And one more thing. A friend of mine who is a vete...And one more thing. A friend of mine who is a veteran journal editor is about to publish a piece that notes how difficult it often is to get scholars to agree to peer review each other's work: everyone complains that they do not have the time, and many people observe that since they are not being paid to do it they won't commit their labor. That second excuse seems especially unfortunate, and my friend blames the current system by which junior faculty are shielded from committee and extracurricular work by their colleagues, so that when demands to reach outside their own scholarship arrive they are reluctant to be so community minded: they just don't see it as an obligation, like earlier generations did, and resent not being compensated for it because it is "extra."<br /><br />I don't think that is quite true (even though I do often hear grumbling about the the unremunerated work scholars are asked to do, and do think many of us could be more generous in the public intellectual role we should be playing) -- BUT I also think it is wrong for the companies that publish journals to charge exorbitant subscription fees, make a huge profit from these fees, and still expect their for-profit endeavors to be underwritten by unpaid scholarly peer review. I have no problem with nonprofit and especially open access publishers not paying for this essential labor, but something rubs me the wrong way about journals being so expensive my library can't subscribe to them only because the CEO of the company that publishes the things likes his money wrapped in bags with dollar signs on them and stacked high in his vault.Jeffrey Cohenhttps://www.blogger.com/profile/17346504393740520542noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-18828726303217137432010-08-30T08:30:22.918-04:002010-08-30T08:30:22.918-04:00Great discussion. To me it seems clear that a hybr...Great discussion. To me it seems clear that a hybrid mode is best: some kinds of peer review are better than others (I prefer modes that encourage mutual responsibility, and am not a fan of blind review); peer review can't be replaced by a wiki or unpremeditated crowdsourcing. But as numerous academic blogs have demonstrated, gathering experts to maximize careful feedback leads to conversations that disseminate research in progress more widely (increasing its impact) AND leads to a better published product (journal article, book).<br /><br />Careful planning is the key: inviting a sufficient number of experts to weigh in, ensuring that the conversation doesn't get led off track by nonproductive or derailing comments, etc.<br /><br />An even better outcome: open access publication of the final products ...Jeffrey Cohenhttps://www.blogger.com/profile/17346504393740520542noreply@blogger.comtag:blogger.com,1999:blog-21165575.post-76912752029895847162010-08-29T20:57:34.536-04:002010-08-29T20:57:34.536-04:00I think, myself, that the obvious problem with pee...I think, myself, that the obvious problem with peer review is that articles are often not reviewed by people who know enough about a subject to say whether an article is good or not. In fact, academic peer review is too often confined to the editors' friends and acquaintances. The editor cannot ask someone he or she has never heard of to review, after all. The effect often looks like an old boy's club to the rejected.<br /><br />Open review on the internet, then, would have this advantage, that the scholar you need comments from, but whom you don't know exists, may find you. But it is very unlikely to occur in a useful time frame! And meanwhile there will be a lot of unhelpful comments from people who don't know much but, like me, can't resist a comments box.<br /><br />A way through this might be to allow only subscribers, who are presumably already interested in and at least slightly knowledgeable about their journal's field, to review. But this requires a well-established and large user base, and only really works for periodicals. I suppose some version for monographs might operate by having a 'reviewers' club' or similar, which cut you a reduced subscription to a publisher's whole range of subject catalogue if you reviewed a certain number of pieces each year or something, kind of like if Amazon paid for product reviews with coupons.Anonymousnoreply@blogger.com