| 1 | Oct 13, 2011 5:07 PM | When I am not forced to know who the authors are, I do a better job reviewing papers: I am more fair, and I give the most time to the most deserving papers, not the papers with the most famous authors. |
| 2 | Oct 6, 2011 8:22 PM | I have always preferred double-blind reviewing. My review of a paper should not be dependent on the authors. Knowing the authors prejudices me and I have to consciously fight against it. It's easier if I don't know the authors to begin with. I feel the process is a lot fairer, and especially fair for authors entering the area. As an author, I have liked submitting to conferences with DBR, like those in security. |
| 3 | Oct 6, 2011 5:41 PM | The POPL 12 version was the least painful version of DBR I've encountered. I think it worked fairly well and is no longer a deterrent to accepting reviewing assignments. I still checked the SBR box because I'm still not really convinced DBR is worht the trouble. But I no longer have a strong opinion either way. |
| 4 | Oct 6, 2011 4:14 PM | An anonymous first impression was helpful in reviewing. However there are more disadvantages than advantages: It meant that I felt unable to discuss my own submission with other ERC members during the review period, which was quite a busy conference season. It's also quite a hassle to organize. Finally, being in the ERC not the PC made me feel second-rate, despite having quite a lot of work to do; at least once a PC member made it clear to me that his opinion mattered more than mine. |
| 5 | Oct 6, 2011 10:28 AM | DBR with reveal after first review works very well. |
| 6 | Oct 6, 2011 9:33 AM | DBR is far too much complication for any (in my mind, marginal) advantage it may have. I'd like to see if there was any change in outcome from such a laborious process. |
| 7 | Oct 6, 2011 8:51 AM | I thought the guardians arrangement worked well to get reviews from experts. |
| 8 | Oct 6, 2011 8:44 AM | I liked the set-up with an extended review committee. It allowed me as an extended reviewer to allocate time in my schedule early and it helped me calibrate my scores and final evaluation. |
| 9 | Oct 6, 2011 8:42 AM | I do not think the quality of reviewing was as good this year as last year. |
| 10 | Oct 6, 2011 8:35 AM | IMO, guardians are important since they take care of quality. DBR vs. SBR is mostly attacking symptoms only. |
| 11 | Oct 6, 2011 6:47 AM | I think there are real advantages to DBR, but on balance I don't think the costs justify the benefits |
| 12 | Oct 6, 2011 5:33 AM | to not know the authors and their affiliation helps me to judge the paper objectively |
| 13 | Oct 6, 2011 5:26 AM | I enjoyed not knowing who the authors were when reading the paper. My main worry about DBR is whether it hinders the process of finding expert reviewers, so I would be reassured if there were some evaluation of that. |
| 14 | Oct 6, 2011 4:58 AM | Despite my paper being rejected, I think DBR was overall a good thing - in one case I found myself tempted to change my opinion of the paper when I knew the identities. I think maintaining blinding throughout the review process would be beneficial (but it is probably not feasible) |
| 15 | Oct 6, 2011 3:16 AM | I am a fan of the "guardian" process. So SBR is OK as far as I'm concerned so long as papers get guardians. |
| 16 | Oct 5, 2011 11:53 PM | DBR really helped to make the first look at the paper unbiased in either positive or negative ways. |
| 17 | Oct 5, 2011 9:42 PM | I was shocked a couple times to learn the identity of the authors. I might have been more forgiving in my review had I known who it was. |
| 18 | Oct 5, 2011 7:26 PM | I really don't have a strong opinion about SBR vs. DBR and think the whole topic has received more attention than necessary, but I slightly prefer DBR and think the 'compromise' of revealing authors after the first review is quite sensible. |
| 19 | Oct 5, 2011 6:08 PM | DBR allows me to focus solely on the content without dwelling consciously or subconsciously on the authors, their prior work, my opinion of their research etc. etc. Thus, in addition to making reviewing more FAIR, it makes it more PLEASANT. Finally, any questions that can only be addressed on knowing the authors' identity (e.g. supplemental material, prior research etc.) is made readily available after turning in the first review. In short: best of both worlds. In 10 years, we will look back and marvel that there was a time in the recent past, where we were actually debating SBR v. DBR. |
| 20 | Oct 5, 2011 4:11 PM | DBR imposes a huge overhead (especially for the chair), and I am not sure the global benefits compensate the overhead. However, this was the first time I reviewed anonymous papers, and I discovered that I find much more enjoyable to write a review if the paper is anonymous (no bias, no stress due to "I am reviewing a paper by XXX") (and probably I do a better job).
whenever I could not guess the authors (about 1 in 2), I could see |
| 21 | Oct 5, 2011 3:26 PM | I am unconvinced of the value of DBR, let alone that it justifies the overhead. I think that using an ERC with SBR would make a lot of sense. |
| 22 | Oct 5, 2011 3:13 PM | Overall, I think both options (DBR or SBR) are not very different, but since DBR complicates the process a little, I prefer SBR. Also, some papers like experience reports or tool papers would make less sense in a purely DBR review world, but this isn't much of an issue for POPL. |
| 23 | Oct 5, 2011 3:11 PM | There are a number of reasons. This little text box is not really long enough. Perhaps you should solicit thorough, reasoned arguments on both sides of the argument and discuss them in a special session at POPL'12? |
| 24 | Oct 5, 2011 2:39 PM | DBR worked well this year, no doubt. Still, I feel SBR is simpler and involves less work (especially for the program chair). The "guardian" idea was very good and seems independent of the choice of SBR versus DBR. |
| 25 | Oct 5, 2011 2:30 PM | I noticed the difference that DBR made in ensuring that reviews were not biased up front by positive or negative impressions about the authors. |
| 26 | Oct 5, 2011 2:12 PM | I believe it led to fairer more impartial reviews. |
| 27 | Oct 5, 2011 1:57 PM | The first impression of a paper is very important. Using DBR helps the reviewers direct their focus on the paper itself rather than the past track record of the authors. I think this is the right way to go. Even if you can sometimes correctly guess the authors, you can probably never guess the complete list of authors. |
| 28 | Oct 5, 2011 1:53 PM | Double-blind is the only way to prevent a significant, if unconscious, form of bias in the reviewing process. It should be adopted by all program committees across computer science. |
| 29 | Oct 5, 2011 1:49 PM | DBR is *critical*, there is too much information content in authors names that can bias reviewers before the read the paper |
| 30 | Oct 5, 2011 1:48 PM | Neither: I strongly advocate an intermediate point in which papers are anonymous at the bidding stage, and do not have the authors names explicitly on the first page, but are not otherwise anonymised. This would remove first-impression bias while not handicapping authors. |
| 31 | Oct 5, 2011 1:47 PM | Fairness |