CCGrid 2007 reviews!

It’s 4.40am and I just finished my 10 (!!!) paper reviews for CCGrid 2007. Hmmm… I will think hard before I accept another invitation to join a Grid-related Program Committee.

I read some really poor papers, with no attention to presentation, no clear focus/contribution, nothing exciting, etc. Since I always try to give lots of feedback, I dedicated many hours of my life in this.

I think that I am seeing a pattern with papers in the Grid space. It seems that research work is presented as novel when in fact the real targeted area has been exhaustively investigated over the decades. I guess young researchers as they start their research work in the “Grid” identify issues related to fault tolerance, distributed computing, modeling, etc. and since they don’t find prior Grid-related work they think they have discovered something new. For example, I couldn’t believe that work on thread migration and sharing of variables in a distributed environment could ever be presented as new, with no references at all to distributed shared memory systems, tuple systems, parallel runtime systems, etc. Or that researchers would prefer to present their work on pro-active and re-active fault tolerance techniques to a Grid-related conference rather than the specialized and prestigious (in their field) fault tolerance conferences.

We definitely need to do something about the way in which scientific results are reported. I think people should be discouraged from submitting just anything. The review process should be completely transparent. It should be the entire community, the domain-specific community, that discovers filters out the good papers, the good research work. Why should fault-tolerance methodologies be reported in a Grid-related conference and the merit of the research judged by Grid, instead of fault-tolerance, experts? Also, why should the identities of the PC members and their views remain secret? Why should rejected papers be ‘forgotten’? The authors should be accountable to the community for submitting garbage and the reviewers for their comments. Once personal reputation is on the line, perhaps we’ll see some more self filtering and, who knows, the “publication factories” may start disappearing. The fact that careers are judged by the mere quantity of publications rather than their quality and their impact is a big problem. Government organizations seem to encourage the focus on publication numbers due to their project/university reviewing processes.

I think social networking can help us here. I have some ideas (already written!) and I am discussing them with Tony Hey and soon with Jim Gray. We all hope to make a change in this space. I am off to the Open Repositories 2007 conference in two weeks as part of my engagement with the Open Access movement. This is going to be one of my playgrounds for all the microformat-, social networking-, knowledge representation-related ideas.

Need to get a couple of hours of sleep. Tomorrow I will have to start reviewing the 10 (!!!) papers for WWW 2007. Oh man!