| 1 | Oct 13, 2011 5:25 PM | As an author or reviewer, merit, confidence, and expertise are the only things I care about. I believe that the "identify the champion" paradigm is most successful when merit and confidence are combined in a single score. For example, "meritorious, confident" == strong accept. "meritorious, not confident" == weak accept. I believe the expertise is important both to see what audiences the paper can reach and whether it has received attention from a domain expert. However, before designing a review form, I would want to consult with several other program chairs in order to understand how these other fields might help a chair manage the review process more effectively. I would also want to consult a social scientist who knows how to develop and validate questionnaires, to check the hypothesis that adding certain fields might cause a reviewer to think more deeply about the review. |
| 2 | Oct 8, 2011 8:10 PM | Elegance |
| 3 | Oct 7, 2011 9:28 AM | While I consider "Importance" to be a useful criterion, I found it was difficult to give a meaningful score for narrow but deep topics. |
| 4 | Oct 7, 2011 7:52 AM | I did not believe in the third field, but I learned otherwise. It should be stressed, though, that the grade expresses a rough tendency, not more, and that one has to read the review to get the real information. |
| 5 | Oct 6, 2011 10:21 PM | I'd have a single field for expertise+confidence |
| 6 | Oct 6, 2011 9:10 PM | Less fields are better. Maybe even clarity can go. |