This op-ed was written by Matt Dunn, PhD, former STEEP trainee. Dunn now works for RPS ASA as a Contaminant Modeler, but moonlights as a freelance writer and editor. We have been grateful to passively sample some of his time (that’s a PFAS joke). Any views, thoughts, and opinions expressed in this article are those of the author and do not reflect the position of STEEP.
There’s no way to talk about the stress of graduate school without being accused of 1. being “too sensitive”; or 2. trying to underscore to everyone how brilliant you are. How many memes have we seen from friends about that graduate student in the group chat? Or how many times have we been reminded by professors that graduate school isn’t supposed to be easy?
Both of those tropes can certainly be true, but it still doesn’t mean that being a graduate student is particularly easy. And while it will never be easy, nor is it supposed to be easy, that doesn’t mean we can’t identify ways to improve graduate studies and academia.
The thing I’ve chosen to complain about specifically is the culture of publishing. In my opinion, it needs the biggest overhaul. And the best part about being a journalist asked to write an op-ed? This will be all about my opinions.
While I’m grateful to say that I successfully defended my PhD, graduated, and moved onto a job, it certainly wasn’t easy. And that’s okay, as we’ve already established: a grad degree is not supposed to be easy. But the things that make it hard, such as researching novel topics, reviewing hundreds of papers, or studying for exams, shouldn’t need additional “help.” That “help” comes in the form of brutal mentors or particularly competitive peers. While those two things were certainly present in my PhD life, the bigger culprit was the miserable culture of publishing research.
On the surface, peer review makes total sense. I assure you: this is NOT a purely anti-peer review editorial. But peer review could certainly use a bit of a makeover.
What Is Peer Review?
For those who don’t know, peer review is the process by which scientific studies become published as research articles. Essentially, you write your paper with your co-authors (usually your advisor and whoever else was involved in your project), and then submit it to a journal that fits your research interests and is relatively well-regarded. The more elite the journal, the more competitive to get accepted. Your work will be glorified as more impactful, should you make the cut.
Anywhere from two to four reviewers will be assigned by the journal to look over your work. These are usually researchers from a similar field and might be anyone from a fellow grad student to a world-renown scientist. They will judge your work, offer feedback or suggestions, and ultimately tell the journal if your work should be published, or rejected. As someone who has gotten a few rejections in his day, it’s not fun.
So, What’s the Problem?
A lot is baked into the peer review process, to supposedly promote ethical and balanced review. For example, you most often do not know who reviewed your paper, nor do the reviewers always know whose paper they review. Commonly, however, the reviewer can figure it out simply based on general knowledge of who is doing what in the field. But it is nearly impossible for a submitting author, or reviewee, to do the same.
For example, there are few labs in the wide research world dabbling in passive sampling. So, odds are, a paper submission on passive sampling is likely from one of four or so groups of scientists.
The blind review process is supposed to limit bias, and I’m sure in many cases, it does. However, it seems to protect the reviewer more than the submitting author. In my young career, I’ve been lucky enough to be on the receiving end of some unprofessional and downright rude behavior from reviewers. Is my research always sterling and perfect? Of course not, we can all use feedback to improve. But it’s not horrific, and my advisor, my mentor, a leading expert in this field, would never allow me to submit horrific work.
One reviewer on a recent submission referred to writing so poor that the “number of issues that [they] ran into is so large that mentioning them all exceeds [their] task as a reviewer.” That’s science speak for “the writing sucked.” Rather comically, the second reviewer of the paper called our work “well written.”
This blindness brings forth a major question: who is it that is really reviewing our papers? Is reviewer #1, a brilliant wordsmith whose critique is not only valid, but should supersede all other comments? Or are they just another scientist with their own screwed up definition of “good scientific writing?” What if reviewer #2 is the leading expert in my field? Shouldn’t their positive reception outweigh the other? It’s also possible, reviewer #2, who said it was “well written,” has no sense of good writing, and that our work was indeed, poorly written.
This experience is not unique, it has likely happened to almost every MS or PhD student. But surely, in a field that tracks our accomplishments through publication, we must find a better way. Young scientists are told the mantra “publish or perish,” to instruct us that only through strong publications and high h-index (a measure of how often your work is cited), is success possible. Our careers can be reduced to a series of statistics, covering one highly uncertain aspect of science, and in this framework, left seemingly up to chance.
What Can Be Done?
Within the past year, I’ve marinated on this career question quite a bit. Peer review is not entirely flawed, and blind or double-blind reviews do have many positive attributes. However, I wish to propose a set of recommendations.
First, we should normalize submitting a list of direct competitors (if any) whom should be highly discouraged from reviewing your work by the journal.
I once had a competitor approach at a conference, after I submitted a paper. They informed me that someone from their group reviewed it. Eventually, I was confronted with strict comments from one reviewer (of three total). Gee, I wonder who it might have been?
I likely should have approached the editor myself and discussed this, but as a young scientist, you risk your reputation, doomed to be thought of as a sore loser or whiner. Based on what was communicated to me in this particular review, it seems that a reviewer later went on to write a rebuttal to that paper, which is highly inappropriate. You’ve had your turn to critique the work, and unless your rebuttal brings new information to the table (which it did not), it’s simply an attempt to squeeze some credit out of my work.
Should authors be able to ask for the identity of one of the reviewers after the final decision is made? I think so, in the case of hotly debated papers with as many as four reviewers and a wide approach to feedback. I once submitted a paper that ended up with four reviewers, a high number; two provided positive comments and approved publication, one asked for reasonable edits, but the final discarded the work as incomplete and poorly done. Again, who was that reviewer to exert such influence over the others’ more positive reception? I’d surely like to know. And seeing as the final decision was a rejection, not a resubmission, it won’t matter. That work will never be published in that journal based on the editor’s final decision.
While there is no perfect solution, I believe a more transparent system is almost always better. How are scientists allowed to leave disrespectful or rude comments when journals clearly outline that such practices are not tolerated? Well, let’s start seeing if it is the same people repeatedly.
How do they get to leave comments that show they misinterpreted your work, without providing the author a chance to refute, or discuss, or display that the reviewer is not an appropriate choice for the field of study? Theoretically, you can already do many of these things, but you risk the wrath of the editor, or journal, or your own advisor who really just wants you to move on. Ultimately, I believe that a more thorough pathway for refuting reviews is necessary. The current practice of emailing the editor assigned to your paper (and hundreds of others) will once again leave an imbalance of power in the hands of one individual.
There’s an imbalance of power in the current peer review process, where anonymous reviewers hold advantage over authors. This is especially true for grad student authors, who do not yet have the career or published reputation as foundation, regardless of skill, brilliance, or fantastic accomplishments already achieved.
If research is to continue with the “publish or perish” approach to career development, then the balance of power must be more equal. If every rejection of my work came from someone I could identify as a true expert, then they would be a more valid rejection. But recent trends show that journals are having a harder and harder time finding reviewers, which doesn’t inspire great confidence from me that the right person is always assigned the job.
The silver lining: at some point, we all get to the end of these tough research roads. Even after some of the setbacks that I detailed here, I finished my PhD and left as a published and cited author. The best advice for other students or folks who feel frustrated with the publication process is the least satisfying; just keep pushing forward. Take it in stride, don’t let it get to you, because then those tough reviewers and toxic folks in science win. Move forward, re-submit, and when it is your turn to be a mentor, or reviewer, or editor, you can be better, more fair, and be the change you want to see.
Moving forward, potential solutions are out there to improve these processes. Whether it is reviewers identifying themselves to authors after a rejection or sharing a CV to prove they have the expertise to back up their comments, there is a lot that can be done to validate the critiques against one’s paper. Especially at a time when journals are struggling to find reviewers to work in a timely manner, we should all have the chance to make sure the reviewer is qualified in our field.
I would also suggest that editors and mentors encourage students to re-submit to journals after they’ve been rejected. If you’re going to leave the fate of your science up to three random scientists, why not roll the dice again? If all of this seems so juvenile to you, that’s because the review process can be petty and juvenile! Surely, I’m not capable of fixing it to a perfect and equitable manner, but these are ideas that our community might discuss further.
But hey, maybe I am just a whiner. At least this whiner stuck it out long enough to get their PhD and make a pretty big impact in their community and field. But if you’re calling me a sore loser for writing this article, well, you might be part of the problem…?
