Reviewing papers for workshops, conferences, journals

Phew! I just finished reviewing 9 papers for the CCGrid 2005 conference. I got a good balance of interesting and “not-so-interesting” (to put it politely) papers. I spent a lot of hours over the last few days reviewing these papers (I am of the opinion that authors deserve thorough reviews because that’s what I expect from reviewers when I submit papers… especially when my papers get rejected).

This last review process got me thinking about the way in which papers get submitted to workshops/conferences/journals and then reviewed.

I think the current manner in which peer-review happens is inefficient as far as the reviewers’ time is concerned. Ideally, it’d be great to have a hierarchical system where conferences would consider papers that would have already been reviewed once and presented in an affiliated workshop. Of course, a conference could be affiliated with a number of workshops. The equivalent would apply for journals, which would be affiliated with some conferences or high-quality workshops.

This way, there would be a natural flow of papers that get better and better as they go up the hierarchy. Of course I am not suggesting that all the papers submitted to workshops would be accepted in conferences. Just that reviewers for conferences would not have to deal with all those papers that haven’t been previously peer-reviewed. The process doesn’t have to be strict since there are many academics/researchers out there who make sure that what gets out of their door is of good quality (at least as far as they are concerned).

But something needs to be done about the flood of papers submitted to conferences which are unreadable. I wonder whether this was always the case.