| 1 | Oct 13, 2011 5:07 PM | This is my first time doing double-blind as a reviewer, as opposed to as an author or as program chair. It made a larger difference to my reviewing experience than I had expected. |
| 2 | Oct 6, 2011 8:22 PM | I had a good opinion of it to begin with. I saw the PC side of it now. |
| 3 | Oct 6, 2011 4:14 PM | On two submissions, if I had known the authors, I would have started with a higher opinion of the submissions than was justified. I would have realized before long, but the DBR saved me the time of realizing that people I respect could have done better. |
| 4 | Oct 6, 2011 3:55 PM | In the case of paper (number redacted), if I had known that (name redacted) was an author my overall merit would probably be higher, so it's a good thing I didn't know. |
| 5 | Oct 6, 2011 12:40 PM | I wish there was an option that said "My opinion is the same" as I generally think it's a good idea. |
| 6 | Oct 6, 2011 11:09 AM | As a reviewer, my opinion improved because there was one instance where I was truly surprised. The degree of surprise suggested to me that I might have held some unconscious bias for the authors had I known their identity in advance.
As a writer, I didn't like it because the paper I wrote built upon a previous paper I wrote. That made me feel awkward and nervous about how to obey the rules and properly hide my identity. Also, I wanted to look for post docs to work on the project and hence wanted to send email to the TYPES list about the position but that would also have broadcast my identity. I ended up waiting until near the end of the popl review time before sending the email to the TYPES list but I felt that this was suboptimal. Basically, I just didn't want to have to worry about protecting my anonymity during this review process. |
| 7 | Oct 6, 2011 10:28 AM | I was already happy with DBR before POPL'12. |
| 8 | Oct 6, 2011 8:51 AM | I am in favor of DBR in general.
I like how POPL'12 tried to address the usual concerns (hampered ability to get expert reviews, etc.).
On the other hand, I could still correctly guess authors for more number of papers than I imagined at the start of the review process. This is more to do with how people write (in particular, which papers they cite) than with the review process.
DBR is a wider issue than just reviewing, and I think it is important to engage the community. |
| 9 | Oct 6, 2011 8:47 AM | I was really surprised by authors in a handful of cases. My reviews might have been biased if I'd known authorship up front. |
| 10 | Oct 6, 2011 8:42 AM | I have no objection the aims of DBR, but it seems to require something like the ERC to work properly. Unfortunately, the ERC talent pool did not seem deep enough -- there were a number of papers that I did not think get the expert reviews they deserved. |
| 11 | Oct 6, 2011 5:33 AM | I have done DBR before, as a PC member (e.g., PLDI), and I have always liked the idea. I got confirmed. Since "I got confirmed of my positive opinion" is not an option, I took the first option. |
| 12 | Oct 5, 2011 10:43 PM | I have a lot of experience with DBR and consider it be much more fair than SBR. |
| 13 | Oct 5, 2011 5:59 PM | I thought I would be able to easily guess who the authors of various papers were. I turned out to be mistaken in many cases, and as a result, I stopped thinking about authors altogether. I think this is a good outcome. |
| 14 | Oct 5, 2011 3:26 PM | Nothing notable "went wrong", just another data point observed where the value was not apparent. |
| 15 | Oct 5, 2011 2:39 PM | My reviews were sometimes less biased as a result of not knowing the author's identity. |
| 16 | Oct 5, 2011 2:12 PM | I think papers were reviewed much more honestly and accurately. |
| 17 | Oct 5, 2011 1:56 PM | I think that it helped lesser-known authors get accepted. |
| 18 | Oct 5, 2011 1:53 PM | As before, not knowing the identities of authors kept my focus firmly on the contributions of any given paper. I like the fact that I know that my and other reviews are by definition unaffected by author affiliations or identities, positively or negatively. The main thing is the paper itself, and double-blind reviewing eliminates a potential source of bias. |
| 19 | Oct 5, 2011 1:48 PM | I already had a negative opinion of DBR, but my experience as an author, trying to conform to the guidelines while describing work that is part of a larger industrial and academic context, was extremely frustrating. It was not possible to do a really good job, and I believe this actively impeded the reviewers. Meanwhile, as an ERC reviewer it was clear which groups had produced all the papers I saw. |
| 20 | Oct 5, 2011 1:47 PM | I was worried that it would impose too much bureaucracy. The heavy discussion on the mailing lists, with dissent from many people had me worried too. But, the process turned out to be quite streamlined, revealing authors after the first round made it easy to access additional material, and the rebuttal process worked as it would in the SBR setting. So, overall I thought it imposed little or no hardship on authors and reviewers. As for whether or not it actually helped reduce bias: hard to say. I could guess at least some of the authors of all the six papers I reviewed. |