POPL 2012 DBR opinion (conducted October 2011)
2. What was the outcome of your submission(s) to POPL?
 answered question276
 
skipped question
0
 Response
Percent
Response
Count
Accepted
29.7%82
Some accepted and some rejected/withdrawn (only for multiple submissions)
6.2%17
Rejected or withdrawn
60.5%167
Prefer not to say
3.6%10
3. Were you on the POPL'12 program committee (PC) or extended review committee (ERC) ?
 answered question276
 
skipped question
0
 Response
Percent
Response
Count
I was on the PC
4.3%12
I was on the ERC
8.3%23
I was not on either the PC or the ERC
87.3%241
4. Please indicate your opinion of which choice (among only these two) is best: single-blind reviewing (SBR) as is typically employed by POPL, or double-blind reviewing (DBR) as we implemented it this year (as per the POPL'12 CFP, and the FAQ at http://www.cs.umd.edu/~mwh/dbr-faq.html). Your answer should reflect your perception of the best choice on balance, based on which process you think is overall the most fair, most accurate, most useful, etc.
 answered question246
 
skipped question
30
 Response
Percent
Response
Count
Traditional single-blind reviewing
29.7%73
Double-blind reviewing as per POPL'12 this year
70.3%173
Optionally provide reasons or qualifications for your choice; experiences from this year most welcome
view
68
5. How has your opinion about DBR changed as a result of your experience with POPL'12?
 answered question214
 
skipped question
62
 Response
Percent
Response
Count
My opinion of it improved a lot
11.2%24
My opinion of it improved a little
58.4%125
I think less of it than before
27.1%58
I think much less of it than before
3.3%7
Feel free to expand on your answer
view
61
6. How would you characterize the POPL'12 requirements for blinding your submission?
 answered question229
 
skipped question
47
 Response
Percent
Response
Count
They were easy to follow
76.0%174
They were mostly easy, but I was unsure about certain corner cases that applied to me
21.8%50
They were confusing; I often needed more elaboration (or fewer rules)
2.2%5
Elaboration or other comments/suggestions
view
17
7. Describe the changes you had to make to your submission, or extra work you performed, for this year's POPL compared to what you would have done if POPL had used SBR (check all that apply):
 answered question178
 
skipped question
98
 Response
Percent
Response
Count
changed self-references to the third person
89.3%159
anonymized citations for papers simultaneously under review
20.8%37
removed qualitative judgments that appeal to your past experience (please describe)
12.4%22
omitted text, examples, or references that you felt might have revealed your identity (please describe)
24.7%44
Other (or elaboration on the above)
view
45
8. How would you characterize the effect of the above changes on the quality of your manuscript?
 answered question226
 
skipped question
50
 Response
Percent
Response
Count
They improved it substantially
0.4%1
They improved it slightly
0.9%2
They had basically no effect
78.8%178
They hurt it slightly
18.1%41
They hurt it substantially
1.8%4
Please elaborate on your answer (esp. the two extremes)
view
13
9. Consider how you felt while your work was under review by the POPL'12 committee. Compared to how you would feel when using a typical SBR process, to what extent did the double-blind process make you feel uneasy or awkward about disseminating your work, discussing your work with colleagues, or giving talks at institutions where a PC or ERC member might learn your identity?
 answered question228
 
skipped question
48
 Response
Percent
Response
Count
I felt quite uneasy and/or awkward
6.1%14
I felt a little uneasy and/or awkward
28.1%64
I felt no differently than I would for a SBR process
65.8%150
10. How did you change your actions as a result of adhering to the requirements given on the FAQ (http://www.cs.umd.edu/~mwh/dbr-faq.html) ? Mark those activities that you specifically avoided but would have done if DBR had not been place.
 answered question101
 
skipped question
175
 Response
Percent
Response
Count
did not send paper directly to PC/ERC members for comment
52.5%53
did not announce draft of paper or related artifact on mailing list, forum, etc.
76.2%77
did not offer to give a talk at an institution at which a PC/ERC member is located
38.6%39
Other (or elaboration)
view
24
11. Please share any other thoughts you have about the DBR-style review process for POPL this year; constructive criticism is particularly welcome!
 answered question66
 
skipped question
210
 Response
Count
view66