| 1 | Oct 13, 2011 5:07 PM | Make it clearer to authors that they should go ahead and publicize their work in the normal way and through the normal channels. Suggest to PC members that they might want to avoid certain forums, like the Types mailing list, where new work is likely to be announced. |
| 2 | Oct 9, 2011 7:15 AM | I think it should either be implemented fully with no revealing of the authors until the PC meeting or it should not be used at all. |
| 3 | Oct 6, 2011 8:22 PM | I think author names of all papers should be revealed right after reviewing. This wasn't done this time by mistake, perhaps. |
| 4 | Oct 6, 2011 4:32 PM | Even the number of authors and of institutions should be hidden. This information makes it easier to guess the author list. |
| 5 | Oct 6, 2011 3:55 PM | I don't understand why author names should be revealed after the initial reviews. Revealing their identities could affect the rest of the process. I think it's better to keep anonymity all throughout the reviewing process. |
| 6 | Oct 6, 2011 11:09 AM | I felt there was at least an implicit suggestion that people shouldn't give talks on the paper they just wrote and, I think, an explicit suggestion that they shouldn't advertise their work globally in a forum like the TYPES list. I think any form of advertisement for an authors work should be ok and I don't think it should disqualify reviewers from reviewing the paper. |
| 7 | Oct 6, 2011 9:33 AM | I'd like to know that DBR makes a significant difference before seeing it employed again. |
| 8 | Oct 6, 2011 8:44 AM | I think DBR had a lot of overhead and not very much benefit. Because author names were revealed after the review is submitted, it still might influence the final discussion and decision. Overall, I think it is not very practical. As a reviewer I would have liked to have access to supplementary material while writing my review. As an author, I would also feel constraint, trying not to reveal to much information by not putting a submitted paper on my website, etc. |
| 9 | Oct 6, 2011 8:42 AM | If the ERC was much larger, then it might be possible to get the necessary breadth of expertise. |
| 10 | Oct 6, 2011 5:33 AM | I would prefer to continue DBR even in the PC discussion. I know of one case where the PC member explicitely expressed that he was scared to reject a big name and corrected his bad grade afterwards. |
| 11 | Oct 6, 2011 4:58 AM | See comment about anonymous supplements. |
| 12 | Oct 6, 2011 2:15 AM | The main problem is the amount of work for the PC chair. I have an IMMENSE respect for Mike's handling of this. I am not sure if I (or many other people) as a PC chair would still prefer DBR over SBR if you take this into account. |
| 13 | Oct 5, 2011 10:43 PM | no. |
| 14 | Oct 5, 2011 8:56 PM | I think we struck the right balance by revealing author identities after the first submission of a review. |
| 15 | Oct 5, 2011 7:22 PM | The rebuttal process must also have authors unknown to the reviewers, unless otherwise inevitable (e.g. for making technical report available). |
| 16 | Oct 5, 2011 6:08 PM | Not really, it was great. One hand, it would be nicer if it was easier to safely farm out reviews to say, grad students, on the other, the extra bar forced me to read the papers closely myself which is good. |
| 17 | Oct 5, 2011 4:59 PM | Why is DBR so complicated at POPL? I've reviewed for other DBR venues and there are very few formal rules, mostly common-sense judgments made by submitters, reviewers, and program chairs. Is there just not enough good will for that to work at POPL, or what? |
| 18 | Oct 5, 2011 4:11 PM | Process was fine, unclear how to improve it. |
| 19 | Oct 5, 2011 3:11 PM | Yes, but again this little textbox is insufficient. I will be glad to provide them directly to the PC chair. |
| 20 | Oct 5, 2011 2:12 PM | Overall, I liked the guardian model, but it does put substantial burden on the PC members and chair. An alternative would be a 2-phase review scheme like PLDI is doing this year, where quick multiple reviews are provided for each paper, where those with no positive assessment are not discussed or reviewed further. Those that do have positive initial reviews are reviewed in more depth, with possibly additional reviews solicited at the discretion of the chair and the guardian. |
| 21 | Oct 5, 2011 1:58 PM | Make additional materials anonymized and available immediately. In 2 out of 4 cases I would have liked to see the materials while writing my initial review, for convenience (since the review could of course be updated after viewing the materials). |
| 22 | Oct 5, 2011 1:57 PM | None so far. I think the procedure used this year (as documented in FAQ) is pretty good already. |
| 23 | Oct 5, 2011 1:53 PM | Keep the process double-blind all the way through -- that is, keep author identities anonymous until after the decision making process has been finalized. |
| 24 | Oct 5, 2011 1:49 PM | worked just fine |
| 25 | Oct 5, 2011 1:49 PM | Some submissions were not thrown out despite blatantly violating DBR guidelines. I suggest we abide by our own rules. |
| 26 | Oct 5, 2011 1:48 PM | See (1) above. |