End of year review
No, this post is not a review of 2011 – rather it’s about the reviewing work I do…
I just finished one of my last reviews of 2011. The last of many: I probably do about 80-100 paper reviews per year.
There are some nice rewards to doing reviews: it helps with maintaining a healthy dose of reading – something many of my colleagues find challenging, and one of the reasons we developed TinyARM. You do get the occasional gem to review: a paper that really helps you to understand something, or look at it in a different way, or just a nice paper – and reading people who cite you is always nice too.
But there are many things that I find rather frustrating about reviewing papers… It is actually quite a bit of work. It is rather odd in my view how people sometimes expect you to just be happy to be asked to review. The message sometimes reads more like an order than a request. (Don’t do that if you want me to review for you!)
Also, there is something a bit dissatisfying about the process: you do the review and then the editor takes over. It can be weeks or months before you hear about the decision that was made regarding the paper. Rarely will the editor get back to you for further clarifications. Often, you don’t get to see what other reviewers thought of the paper. Or what was decided.
Probably 80% of my reviews are very negative (reject or requires major revisions and a new review round). Still, my main reasons for rejection are really simple to avoid:
- language issues: English is not my mother tongue either. It is not fair that those who master it well have less difficulty to get their results published. But that is not an excuse for sloppy writing. It keeps amazing me how often I read, re-read and re-re-read sections in papers and then still don’t understand the line of reasoning. Frankly, at a certain moment, I decide that it is not my problem but the author’s.
- evaluation: It’s wonderful that people think they’ve invented the best thing ever that solves all the problems in the world. But if you want me to be impressed, then your idea should be really, really, really convincing, or I’d rather see an evaluation that tries to honestly assess your results.
- related work: even if your idea is really great and you’ve evaluated it and it actually does seem to work, it is really important that you indicate how it advances the field and how it relates to what others have done – otherwise, it just reads like a nice solution rather than a scientific contribution.
- analysis: many papers I review just describe a piece of work – no analysis of pros and cons, of when the solution works well and when it doesn’t, of the trade-offs made in the design, of other approaches considered, etc. That leaves a bit too much work to the reader in my opinion…
I do worry a bit that I may be getting old and grumpy – an 80% rejection rate and maybe less than 5% accept-as-is rate is rather harsh. I’ve actually asked my post-docs to tell me when they think I’ve become so negative that I should stop doing any reviews at all!
But I also think that, if we all were a bit more strict about the reviews we do, our lives as readers would be so much more enjoyable – just think about it: there would be nothing published but well written interesting papers…