This is the juror's manual for A'Design Award and Competition, as a jury member, you are expected to read especially the section about "Surveys and Forms", which describes what these forms are, how to fill them and what they are intended for. When you finish reading the manual please click here to start voting.
Defining Custom Voting Criteria
Each design category has its own prospects, and therefore designs submitted into distinct categories should be approached and evaluated differently. We use standardized evaluation criteria for evaluating designs in a given category so that each design is evaluated in the same way in that category, and furthermore inter-temporal comparison could be possible within the years. On the other hand, each jury member could have different ideas about how to evaluate a given category. By conducting this survey we would like to understand how different jury members evaluate distinct categories. The results will be utilized to develop and standardize the evaluation criteria further.
Example
This is an example of a completed survey, please check the example to understand how it works, before starting voting on each category, each jury member is requested to complete this short survey.
CRITERIA EXAMPLE 1:this is the first and most important evaluation criteria for the specific competition category.
CRITERIA EXAMPLE 2:this is the second important evaluation criteria for the specific competition category.
CRITERIA EXAMPLE 3:this is the third important evaluation criteria for the specific competition category.
CRITERIA EXAMPLE 4:this is the fourth important evaluation criteria for the specific competition category.
CRITERIA EXAMPLE 5:this is the fifth important evaluation criteria for the specific competition category.
CRITERIA EXAMPLE 6: this is the sixth important evaluation criteria for the specific competition category.
CRITERIA EXAMPLE 7:this is the seventh important evaluation criteria for the specific competition category.
CRITERIA EXAMPLE 8:this is the eighth important evaluation criteria for the specific competition category.
CRITERIA EXAMPLE 9:this is the nineth important evaluation criteria for the specific competition category.
CRITERIA EXAMPLE 10: this is the tenth important evaluation criteria for the specific competition category.
Defining Custom Voting Criteria
In order to have inter-temporally comparable results for design competitions, we should focus on criteria based voting with weights assigned to each criteria. However, till now, predetermined weights for criteria have been used and these predetermined weights are not based on solid backgrounds; mostly the criteria weights are selected with simple reasoning during a council meeting and the same weights are used over the years. By running this survey, we collect juror preferences for possible weights (importance level) for each criterion and for each different competition category. By doing so, we could compare the criteria weights indicated by different jury members with the predetermined weights to see any discrepancies, and to adjust and improve the system. In addition by using reserve-engineering of preference orderings after the competition voting, we could derive the real criteria weight that jurors use when reflecting in action during the voting process of a design competition, we will also analyze if the “in-action” criteria weights are different from the “selected” weights in this survey. The results will be utilized to standardize the criteria weights and to understand and clarify the jury’s role on design competitions.
Example
This is an example of a completed survey, please check the example to understand how it works, before starting voting on each category, each jury member is requested to complete this short survey. By clicking plus (+) or minus (-) signs next to the criteria, distribute the total weights.
Total Weights to Distribute:
%
Criteria
Weight
Aesthetics This criteria is about the overall physical form of the design and its aesthetic attractiveness and beauty, the overall charm of the external appearance of the design. For example if the design is beautiful or not.
%
Functionality
This criteria is about how the design functions for the basic intended purpose of its creation. For example, if the design is a chair, it must be something you can sit on, if it is a lamp, it should create some light, etc.
%
Usability
This criteria is about how easy to use the design and how easy to learn how to use it. For example, if the design is an adjustable chair, it should be easy to adjust it, and also easy to learn how to adjust it.
%
Ergonomics This criteria is about how
comfortable to use the product in a physical way for the intended audience. For example if the design is an office chair, it should minimize physical effort and discomfort for office workers.
%
Engineering
This criteria is about how well the design is developed, for example for efficiency issues, material choices and realization possibilities. For example the inclusion of technical drawings, material efficiency analysis etc.
%
Presentation
This criteria is about how the demonstration of the design and its functionality is made in an efficient, effective and aesthetic way that helps you to understand it clearly. For example, if the design is clearly communicated or not.
%
Innovation
This criteria is about the quality and quanitity of positive unique aspects of the design that makes the design different from the designs in the same category or idea cluster, such as better form, new functionality etc, new artifact category etc.
%
Technology
This criteria is about the
quality and quanitity of positive technological aspects of the design that makes the design different from the designs in the same category or idea cluster, such as better performance, increased efficiency etc.
%
Fun Details
This criteria is about the
quality and quanitity of positive emotional aspects of the design that makes the design different from the designs in the same category or idea cluster, such as being surprising, funny, attractive, cute, emotional etc.
%
Market Value
This criteria is the potential market success that you conceive for the design which includes production efficiency, market demand, easy of communication and branding, the added value generation, new functionality etc.
Voting For Submissions
The voting process is composed of four parts, the first is the overall score voting (from 0-10), the second step is feedbacks, the third step is based on criteria voting and the last step is a survey. The overall score voting does not use criteria based voting or any pre-determined system, this voting although looks simple, requires the jury member to have his own voting methodology in mind. In the second step, the jury is asked to enter some feedbacks for the submitted designs; these feedbacks can be used by the designers to improve the submitted designs. The criteria based voting on the other hand, uses pre-determined criteria to vote for designs, we are going to compare if the overall score voting is consistent with the pre-determined criteria weight to see. The survey is conducted to understand several key points of the design, and is composed of ten questions, each of them are asked to enlighten fundamental questions about the design competitions such as the impact of presentation, innovation, category compliancy etc.
Example
This is an example of a completed voting, please check the example to understand how it works, before starting voting on the individual submissions. The system is configured in such a way that once you vote for a submission your vote will be saved and you will be redirected to the next submission in the category.
What you see below is a sample voting interface, thumbnails of the main image, and optional shots are shown, in addition the system indicates if a video presentation or additional documentation such as technical drawings etc are included or not. By clicking on the image thumbnails, real size versions can be seen, it is important to see at least the main image in full size to have a better understanding of the design, also if video or documents are presented, we suggest you to download and view them. On the bottom, design details are given, which could provide more insights to the design such as the core function, production technologies etc.
Production Technology, Dimension / Package / Technical Properties, Tags, Research Abstract, Challange
Here, you see the overall score voting for the given design. From the select bar, choose the appropriate vote.
Overall Score Voting Example
Vote
Overall Score Please select an your overall score for this design from the menu on the right. A higher score means better design.
Here, you see the feedbacks form for this design, the expert feedbacks are important for designers to improve their submissions in the future, this section is required as it also helps you evaluate the design in your mind before passing to criteria based voting.
Feedbacks Example
Feedbacks Please write any feedbacks regarding the design to the field below, the feedbacks are important for designers to improve their designs or to get expert ideas and consultancy, so it is highly encouraged and required to write comments.
Below, you see the criteria based voting for the given design. For each criterion, from the select bar, choose the appropriate vote for the criteria. Please note that because each criteria has different weights, the average of Critera Based Voting should not be necessarily equal to Overall Score. Please do not try to match the overall score with the criteria votes.
Criteria Based Voting Example
Criteria
Vote
Aesthetics This criteria is about the overall physical form of the design and its aesthetic attractiveness and beauty, the overall charm of the external appearance of the design. For example if the design is beautiful or not.
Functionality
This criteria is about how the design functions for the basic intended purpose of its creation. For example, if the design is a chair, it must be something you can sit on, if it is a lamp, it should create some light, etc.
Usability
This criteria is about how easy to use the design and how easy to learn how to use it. For example, if the design is an adjustable chair, it should be easy to adjust it, and also easy to learn how to adjust it.
Ergonomics This criteria is about how
comfortable to use the product in a physical way for the intended audience. For example if the design is an office chair, it should minimize physical effort and discomfort for office workers.
Engineering
This criteria is about how well the design is developed, for example for efficiency issues, material choices and realization possibilities. For example the inclusion of technical drawings, material efficiency analysis etc.
Presentation
This criteria is about how the demonstration of the design and its functionality is made in an efficient, effective and aesthetic way that helps you to understand it clearly. For example, if the design is clearly communicated or not.
Innovation
This criteria is about the quality and quanitity of positive unique aspects of the design that makes the design different from the designs in the same category or idea cluster, such as better form, new functionality etc, new artifact category etc.
Technology
This criteria is about the
quality and quanitity of positive technological aspects of the design that makes the design different from the designs in the same category or idea cluster, such as better performance, increased efficiency etc.
Fun Details
This criteria is about the
quality and quanitity of positive emotional aspects of the design that makes the design different from the designs in the same category or idea cluster, such as being surprising, funny, attractive, cute, emotional etc.
Market Value
This criteria is the potential market success that you conceive for the design which includes production efficiency, market demand, easy of communication and branding, the added value generation, new functionality etc.
What you see below is the survey example for a particular design, under the options below, you are expected to choose the most appropriate answer. Please note that questions can change over the years. The purpose of the survey is to understand the relationship between numeral and verbal voting, in addition the survey help us to improve the voting system. A higher number means better performance for the criteria; for example if you give 95 to Aesthetics , it means the design is very beautiful, and if you give 20 to aesthetics, it means the design is ugly. A score higher than 50 means beautiful, a score less than 50 means ugly, and a score of 50 would indicate neither beautiful nor ugly.
Survey Example
Question
Answer
Do you think this design deserves an award? If you think this design deserves an award, choose yes, if you are not sure whether if the design should receive an award or not, chose somewhat, if you are against this design receiving an award choose no.
Is the presentation of the design well done? If you think that the presentation of the design such as visuals, views and optional images and video were made good, choose yes, if you think the presentation is neither good or bad, choose somewhat, if you think that the presentation is not good, choose no.
Is this design easy to understand? If you have understood what this design is about, choose yes, if you did not understand what this design is about choose no, if you somewhat understand what the design is choose somewhat.
Is this design well developed? If the design is well developed or designed simple but in a good way choose yes, if the design is too simple or over complicated, please choose no or somewhat. The question is not about the complexity of the design, but how details were thought etc.
Is this design original? Do you remember similar designs? If the design is original choose yes. If the design is not original or if you remember similar designs choose no or somewhat.
Is this design realastic or easy to produce? If you think it is easy to realize or produce this design in a commercial way, choose yes. If you think this design is realizeable but not in a commercial way, choose somewhat, if this design impossible to realize or if it would cost too much to realize choose no.
Is this submission a “design” or “art” or “craft”? If you think this submission is "designed", choose Design, if you think this is more art then design, choose "Art", if you think that this design is more craft than art or design, choose Craft.
Is this design fun, surprising or interesting? If this design makes you surprised, or attracts your interest or looks fun, choose yes, if the design is not so interesting, fun or surprising choose no, if the design is somewhat interesting, surprising or fun, choose somewhat.
Do you think the design fits the submitted category? If you think this design fits the submitted category, choose yes. If you believe this design does not fit the submitted category, choose no, if you somewhat think that the design fits the submitted category or if you are not sure choose somewhat.
Do you think the voting criteria applies for this design? If you think that the voting criteria for this competition category can truly evaluate this design, choose yes. If you think that the voting criteria can somewhat evaluate this design choose somewhat, if you believe that the voting criteria cannot evaluate this design in a rightful way, choose no.
Abstract
In order to have intertemporally comparable results for design competitions, we should focus on criteria based voting with weights assigned to each criteria. However, till now, predetermined weights for criteria have been used and these predetermined weights are not based on solid backgrounds; they were selected with simple reasoning. On the other hand, by using reserve-engineering of preference orderings, we could derive the real criteria weight that jurors use when reflecting in action during the voting process of a design competition. To do so, we run a true design competition where the jury is asked to vote twice, first being the preference order of designs and second being the criteria based voting. We aim to gather following information: 1. What are the real criteria weights that are used by jury members when they are voting designs in a preference order. 2. What are other possible criteria that should be considered when voting for designs. 3. How can we use the real criteria weights to improve voting processes of design competitions. Finally, we would like to run a survey to collect further information about the fundamental rules that govern the voting mechanism in a design competition. This article explains how we could design an established, fair and founded voting system for a design competition.
Abstract
Design Competitions have become a common way to gain fame and authority in the last years, thus the demand for design competitions skyrocketed, with increased demand and high-rates of participation, number of competitions also increased exponentially and finally equilibrium was reached. However, in this case, we now have around 400+ International design competitions organized each year, plus thousands of local competitions are being organized throughout the world. With increased amount of choices, the designer now faces a decision to choose which competitions to participate and which not. This created a demand for a quick way to identify which competitions are worth designers’ time and which competitions should the designer stay away from. However, due to enormous amount of information available (submission criteria, terms and conditions, call fiche, brief etc) of each competition, designers usually read only the brief and continue with submissions. It happens in many cases that the designer might even lose rights to her own design by just participating in such competitions. In addition to this, there exists some competitions that are absolute loss of time, but on the other hand we also see that some very good design competitions such as with respected jury, tradition and organizers are lost in this ocean. This article explores how we could identify good design competitions by assigning a score to specific criteria.
Abstract
It is common knowledge that the presentation of a project is somewhat important to be successful in a design competition, but what is the extend of this hypothesis? If you were to check many design competitions in detail, you would figure out that only a trivial weight is assigned for the quality of presentation (usually 5% in the score sheet) when judging your design, and in some cases the presentation might even not be considered as a check point.
