Reducing Presentation Biases
The Positive Presentation Bias was discovered due to the statistical inconsistency in the jury scores of design submissions in the A’ Design Award & Competition: Normally, entries in A’ Design Award & Competition are voted against multiple evaluation criterion, one of the such criteria is the “Presentation Quality” with a relatively low weight on it. However, when a sample of 500 design submissions were analyzed in detail and re-voted analytically, it was discovered that jury members tend to give higher scores to designs whose visual representations were made better, regardless of the intrinsic properties of the design.
To remove the Positive Presentation Bias, A’ Design Award & Competition implemented several tools and approaches: 1. Uniform Submission System, 2. Uniform Jury Presentation System, 3. Preliminary Score and Feedbacks for Presentation, 4. Detailed Presentation Guidelines Book with Hints and Suggestions, 5. Submission Optimizer Software. 6. Showcasing Prior Winner Works, 7. Voting Guidelines for Jury Members.
The purpose of all the tools is to push the presentation quality of entries to a uniform, high level such that when jury members would vote on the submissions, the presentation of the entries would be less relevant because they would all be good, and therefore submissions could be judged more on their intrinsic qualities; thus the Positive Presentation Bias could be removed or reduced.