| 1 | Oct 13, 2011 5:15 PM | I don't feel strongly about this issue. I think I prefer outside solicitations, but ERC with occasional outside solicitions would be OK, too. Using an ERC only would be a bad decision. |
| 2 | Oct 6, 2011 4:15 PM | Being on the ERC is quite a lot of work without much reward. It also makes it easier to guess your reviewers. Having said that it does maintain a high standard of reviews.
Being on the ERC, it was nice to have the chance to explain and discuss, but it took quite a lot of time, again, without much reward. If you would involve the ERC less in discussions, so they merely submit a review and that's it, that would save them quite a lot of effort. It would also save the problem of feeling second-rate to the PC in the discussions and arguments. |
| 3 | Oct 6, 2011 9:44 AM | I don't know exactly what was done at POPL last year, but what is wrong with the traditional practice of letting reviewers find subreviewers when that is appropriate? |
| 4 | Oct 6, 2011 9:04 AM | The flexibility is useful in some situations. |
| 5 | Oct 6, 2011 8:45 AM | The size of the ERC needs to be bigger to reliably get expert reviews. |
| 6 | Oct 6, 2011 6:50 AM | The advantage of an ERC is that there's a pre-built pool of reviewers. But as a PC member I want to be able to ask a colleague to help with my 30 reviews; the difficulty of doing so is a disadvantage of DBR |
| 7 | Oct 6, 2011 5:55 AM | I found some of the outside review very strange (giving only 4's, for example). |
| 8 | Oct 5, 2011 11:58 PM | Sometimes the ERC did not have the right expertise. And I think the issue of wanting the ERC to be able to 'calibrate' is less important when in recent years we are accepting as many "above the bar" papers as there are. |
| 9 | Oct 5, 2011 9:02 PM | It would be good to find a way to make it possible for one's grad students to help with the reviewing, so that they experience reviewing. To be clear, the PC member should still review the paper. Perhaps both the PC member and the grad students could submit separate reviews. Or we could allow grad students to read the paper and discuss with their advisor, if the advisor is reasonably confident of lack of conflict (e.g. if grad student's conflicts are the same or a subset of the advisor's). |
| 10 | Oct 5, 2011 7:32 PM | Again I don't like the question wording -- while having an ERC works well with DBR I would not be hesitant to seek external reviews. When I review a paper I often can think of a great external reviewer for it -- someone the PC Chair often doesn't know. This is a great benefit and a real loss when you just have an ERC. We should encourage it, and to avoid the COI issue, simply have people do it after they know the authors. Even if the external ad hoc review was SBR, I think it would be worth it. |
| 11 | Oct 5, 2011 3:36 PM | If the load is not too high, the additional investment ERC members have in the process seems to help the quality of reviews and discussion. |
| 12 | Oct 5, 2011 3:16 PM | I have mixed feelings about the ERC, but overall I thought it at least allowed external reviewers to have more impact on and participation in final decisions. |
| 13 | Oct 5, 2011 2:44 PM | The ERC seems a reasonable idea if using DBR.
On the other hand, if using SBR, I am not sure that an ERC is an improvement over the use of ad hoc reviewers.
Again, using an ERC means more work for the PC chairs and less work for PC members (who do not need to find ad hoc experts on their own). It's a matter of preference. |
| 14 | Oct 5, 2011 2:08 PM | Finding expert reviewers for each paper is the single most important issue under the current DBR system. I have seen quite a few instances this year where "knowledgeable" reviewers make bad decisions. I thus hope that the future POPL PC would more aggressively search for external *expert* reviewers. |
| 15 | Oct 5, 2011 1:59 PM | I would have the guardian for each paper solicit external review(s). |