There’s a sort of Godwin’s Law for discussions on open peer review. Sooner or later, someone’s going to say, “We can’t expect early career researchers to sign peer reviews, because of fear of retaliation”. And that’s often a discussion-ender. What kind of monster could argue for carelessly letting early career researchers hurtle towards career suicide?
We shouldn’t let this be the end of a discussion. It should be the start of several more.
Firstly, just keeping peer review closed doesn’t do anything about the problem of abuse of power, does it? It’s not going to automagically disappear. It’s no good for science when people are able to intimidate others into silence or tiptoeing around the weaknesses in their work. This isn’t going to be behavior that appears only during peer review. Scientific communities are harmed by harboring bullies, especially in senior positions.
Secondly, since we know there are authors who would behave this way, and fear of it seems widespread, we should be discussing preventive and responsive mechanisms. There could be close to a hundred science journals with fully open identity peer review systems, including some funders’ journals, and some that have been doing this for nearly 20 years. (More on these journals below this post.)
Then there are journals that offer it as an option, like PeerJ and Nature Communications, and coming soon at Nature, too). Some encourage it, like the Royal Society’s Open Science. And people are having openly critical discussions about preprints, too. On top of that, some of us sign our reviews whenever we review, regardless of the journal’s policies. We don’t know how common that is.
Put all this together, and it’s enough for us to need concrete and productive responses to retaliation and the fear of it. After all, it’s possible that younger researchers are more enthusiastic about signing peer reviews, not less. So let’s talk about the vulnerability of being open — but let’s talk about the strength, too.
Critical thinking and analysis of others’ research are core scientific skills. The skills involved in critiquing, and then communicating your analysis well, are ones we all need to develop and keep honing. It’s not just during peer review either. You could be criticizing or “slighting” someone who could be vindictive at lots of points along the way — what you cite, what you say in the background or discussion section of a paper on your own research, and so on.
Being good at critique and articulation of it can build reputations. When we were discussing this at a reproducibility seminar in 2015, physicist Tony Weidberg said the culture in his field had changed on this over the years. There was a time when it was risky to openly criticize senior people. But these days, he says, it’s more likely to improve your career prospects.
Still, it’s going to be a risk sometimes. In the DOSPERT instrument for rating people’s relationship with taking risks, this makes the list:
Disagreeing with an authority figure on a major issue.
Confronting power is classic risk-taking behavior. It’s not a slam-dunk argument against open peer review at journals, though. You are not obliged to do a peer review if you have concerns. I’m openly criticizing people’s work — and even behavior — all the time, and have been for decades. Still, every now and then I pass on peer reviewing someone’s paper because I don’t want to face the burden of criticizing a person like that at that particular point.
You always have to pick your fights, don’t you? Faced with a prima donna or sheer bully author, I wouldn’t consider blinding a good enough safety mechanism, either. (I’ve discussed the studies on this issue before: from 10% to most authors could accurately guess reviewers’ identity.)
Just because you are asked to review a paper, doesn’t give you a responsibility for the paper unless you choose it. We can’t solve every problem ourselves. You can get practice and visibility for critical analyses in other ways, and for authors you feel confident about.
Saying you can’t take it on right now is a personal solution. But it’s not a solution for the problem. Very public peer review is potentially a solution, as I’ve argued here before. The more open all this is, the more chance we have to develop better norms in framing and dealing with criticism, and to make authors, reviewers, and editors more accountable. The evidence of your critique is there for others to see, if you are contending that your peer review is being held against you by someone. And the more open peer review is, the more it can be studied.
I don’t think any of this is enough, though. What else should we do? Consider this:
since becoming an asst prof, i’ve reviewed ~ 12 papers. signed one. received a phone call from the senior (tenured) author asking, “who do you think you are to make anything less than glowing comments?” so there are still dangers for young, non-tenured profs when reviewing.
— Tim Mosca (@drosophilosophy) March 2, 2018
Could we agree this is professional misconduct? Calling it that isn’t going to sweep the behavior away, of course. Acting on it would still be hard, too. It would usually be hard to prove, and if someone bore ill-will for a negative peer review, a complaint about it would likely make it worse. However, there are things you can do that can get your paper rejected, or get you banned from a journal. Should aggression to a peer reviewer be one of those things? I think so.
The only relevant thing I could find at COPE (Committee on Publication Ethics) was this: “Authors should respond to reviewers’ comments in a professional and timely manner”. [PDF] Well, I doubt that professionals acting unprofessionally generally recognize it.
Simply codifying retaliation against peer reviewers as misconduct in journal policy and elsewhere might not deter anyone — and the consequences of a person’s ill-will will usually be far more diffuse than Tim Mosca’s experience. Still, if journals and others named retribution against peer reviewers as misconduct with consequences, it could be an important signal. Culture is shaped by “rules”, as well as by repeated actions and habits, small as well as big.
Often, when you get invited to peer review, you get warned of the things you shouldn’t do: on pain of death just about sometimes, you’re told not to show anyone a copy of the manuscript, for example. But what about peer reviewers’ rights? Should there be a note at the journal about what to do if an author acts aggressively?
If authors giving peer reviewers grief is a thing that happens to plenty of people, should we discuss if contact at all is appropriate? It seems to me to be un-collegial and ill-advised at least. You get to respond to peer review within the process, and that should be the end of it. Even if you don’t intend to intimidate or distress someone, that could be the effect.
Discrimination and harassment are codified as professional misconduct for members of the American Bar Association. And this approach is starting to find its way into science’s professional organizations, too. Margherita Perillo recently pointed to this in her post about bullying in science, at the American Association of Cell Biology (ASCB) blog. She points to ASCB’s anti-harassment policy for its meetings, and to the American Geophysical Union (AGU) designating harassment and bullying as research misconduct.
We could get more explicit about our concerns for the effects on scientists and science of authors who intimidate peer reviewers. Journals and others should step up here. It could help shape culture. Clear signals like this make it easier for bystanders to act, and for people to seek support if it happens to them.
I think as a community we can and should do something more about ethics and standards for open peer review. But even if we do, fear of retaliation isn’t going to evaporate any more than impostor syndrome is. The things we fear can hurt us, that’s for sure. Facing fears, taking even little steps at not letting them inhibit us, and surviving and growing when things go wrong can all make us stronger though, too. Courage and character are built by it.
Life offers up an endless supply of opportunities for exercising the muscles we need as individuals to speak up to authority figures and to criticize, inside science and out! Confidence can be built in all sorts of ways. Somehow, too, we have to find the ways that we can take a stand against retaliation together.
~~~~
More posts on open peer review:
The Fractured Logic of Blinded Peer Review in Journals
Weighing Up Anonymity and Openness in Publication Peer Review
The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny and on Tumblr.)
Journals that have open (named) peer review:
- 70 of the BMC journals, some since 1999
- Atmospheric Chemistry and Physics, since 2001, followed by 11 more journals from the European Geosciences Union and the publisher, Copernicus (more on them here)
- BMJ Open since its launch in 2011, the BMJ since 2014, and BMJ Open Science since its launch in 2017
- f1000 Research (since 2013 I think) and the funders’ journals that have started with this model since 2016 (including Wellcome Trust and the Gates Foundation)
- European Journal of Neuroscience since the end of 2016
I would be pleased to know additions for this list in the comments section below, or via Twitter. [Many thanks to Trish Groves and Irene Hames for additions so far — see comments below.]
* The thoughts Hilda Bastian expresses at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.
