Food Research International 48 (2012) 893–908

Contents lists available at SciVerse ScienceDirect

Food Research International

j ourna l homepage: www.e lsev ie r .com/ locate / foodres

Review

Sensory profiling, the blurred line between sensory and consumer science. A reviewof novel methods for product characterization

Paula Varela a,⁎, Gastón Ares b

a Instituto de Agroquímica y Tecnología de Alimentos (CSIC), Avda. Agustín Escardino, 7. 46980 Paterna (Valencia), Spainb Departamento de Ciencia y Tecnología de Alimentos, Facultad de Química, Universidad de la República, General Flores 2124, C.P. 11800, Montevideo, Uruguay

⁎ Corresponding author. Tel.: +34 963 900 022; fax:E-mail address: pvarela@iata.csic.es (P. Varela).

0963-9969/$ – see front matter © 2012 Elsevier Ltd. Alldoi:10.1016/j.foodres.2012.06.037

a b s t r a c t

a r t i c l e i n f o

Article history:Received 27 April 2012Accepted 29 June 2012

Keywords:Sensory descriptive analysisConsumer profilingQDA®Napping®SortingFlash profilingFree choice profilingRepertory gridCATAOpen-ended questions

Sensory descriptive analysis is one of the most powerful, sophisticated and most extensively used tools insensory science, which provides a complete description of the sensory characteristics of food products. Con-sidering the economic and time consuming aspects of training assessor panels for descriptive analysis, severalnovel methodologies for sensory characterization have been developed in the last ten years. These method-ologies are less time consuming, more flexible and can be used with semi trained assessors and even con-sumers, providing sensory maps very close to a classic descriptive analysis with highly trained panels.Novel techniques are based on different approaches: methods based on the evaluation of individual attri-butes (intensity scales, check-all-that-apply questions or CATA, flash profiling, paired comparisons); methodsbased on the evaluation of global differences (sorting, projective mapping or Napping®); methods based onthe comparison with product references (polarized sensory positioning), and based on a free, global evalua-tion of the individual products (Open-ended questions). This review aims at reviewing theory, implementa-tion, advantages and disadvantages of the novel product profiling techniques developed in the last ten years,discussing recommendations for their application.

© 2012 Elsevier Ltd. All rights reserved.

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8941.1. Sensory characterization, from classical descriptive analysis to the emergence of novel profiling techniques . . . . . . . . . . . . . 894

2. Novel methods for product characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8952.1. Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895

2.1.1. Theory and implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8952.1.2. Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8952.1.3. Application, advantages and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8962.1.4. Modifications to the original methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 896

2.2. Flash profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8962.2.1. Theory and implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8962.2.2. Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8972.2.3. Application, advantages and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 897

2.3. Projective mapping and Napping® . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8982.3.1. Theory and implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8982.3.2. Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8992.3.3. Application, advantages and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8992.3.4. Modifications to the original methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 899

2.4. Check-all-that-apply questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9002.4.1. Theory and implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9002.4.2. Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9002.4.3. Application, advantages and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 901

+34 963 636 301.

rights reserved.

894 P. Varela, G. Ares / Food Research International 48 (2012) 893–908

2.5. Other methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9022.5.1. Intensity scales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9022.5.2. Open-ended questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9022.5.3. Preferred attribute elicitation method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9032.5.4. Polarized sensory positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9032.5.5. Paired comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 903

2.6. Comparison of the methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9033. Conclusions and recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 905References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 906

1. Introduction

Sensory descriptive analysis is one of the most powerful, sophisticat-ed and most extensively used tools in sensory science. Its application hassteadily grown in the end of the 20th century and the beginning of the21st. This methodology enables to measure the sensory reaction to thestimuli resulting from the consumption of a product, providing a descrip-tion of the qualitative and quantitative aspects of human perception, andallowing correlations to other parameters (Lawless & Heymann, 2010;Moussaoui & Varela, 2010; Murray, Delahunty, & Baxter, 2001; Stone &Sidel, 2004). Describing the sensory characteristics of a product hasbeen common practice in the food and beverage industry since longago, allowing informed business decisions, guiding product developmentto match a consumers' ideal, to get closer to a benchmark, to check theeffect of ingredients or processes, for quality control purposes, to trackproduct changes over time, and to correlate with instrumental measure-ments. In academic research it has been a valuable resource as well, en-abling the establishment of correlations with analytical measurements,helping to explain how changes in texture, flavor, aroma or structuraland microstructural features determine different sensory characters,and allowing to better understand the mechanisms underlying sensoryperception (Gacula, 1997; Moussaoui & Varela, 2010; Stone & Sidel,2004). Importantly, both in industrial and in academic fields, descriptiveanalysis has served as a link between product characteristics and con-sumer reaction.

1.1. Sensory characterization, from classical descriptive analysis to theemergence of novel profiling techniques

There are various ways to perform classical sensory descriptiveanalysis in practice, and there is plenty of literature extensivelyreviewing them in the last years. Murray et al. (2001) have publishedone of the latest reviews on classical methods of descriptive analysis,mentioning the Flavor Profile Method (Cairncross & Sjöstrom, 1950),the Texture Profile Method (Brandt, Skinner, & Coleman, 1963), Quan-titative Descriptive Analysis™ — QDA™ (Stone, Sidel, Oliver, Woolsey,& Singleton, 1974), the Spectrum™ method (Meilgaard, Civille, &Carr, 1991), Quantitative Flavor Profiling (Stampanoni, 1993a, 1993b)and what they called a “generic descriptive analysis”, which is general-ly a mixed approach taking ideas from some of these methods. Themost utilized generic descriptive analysis technique comprises acombination of the basic elements of QDA™ and Spectrum™(Lawless & Heymann, 2010), to make it more flexible and allowingmeeting project specific objectives. As a summary, it requires as afirst step the selection, training and maintenance of a panel of8–20 assessors. Once the assessor panel is selected for each projector sample group it would: (a) generate specific attributes that de-scribe the similarities and differences between products,(b) determine and agree on the evaluation procedure for each of theselected attributes, (c) be trained in the evaluation and scaling of theselected attributes for the particular sample set, and (d) would fi-nally quantitatively evaluate the samples, generally with the use of10–15 cm unstructured line scales, where samples would be evaluatedindividually in a sequential monadic, balanced randomized presentation.

The obtained data would be in the form of intensity scores of all theattributes, which can be analyzed individually, by attribute and sampleas a sensory signature or profile of each product. It is common practiceto look at descriptive data through sensorymapping, reducing the num-ber of variables, and obtaining oneormore biplots representing the per-ceptual space of interest. This space provides a representation of thesamples according to the similarities and differences in the intensityof the evaluated sensory attributes.

The high specialization of descriptive panels allows obtaining verydetailed, robust and consistent, reproducible results, stable in timeand within a certain sensory space (Moussaoui & Varela, 2010). Cre-ating and maintaining a well-trained, calibrated sensory panel canbe quite expensive, though; small food companies usually cannot af-ford it, and it could even mean a significant expenditure for big com-panies if they have a wide range of products that require variouspanels working in parallel. Furthermore, the training step can be rel-atively long, as it should be detailed and extensive, varying between10 and 120 h, depending on the range and complexity of the sampleset, which might result in a time constraint, particularly when indus-tries require quick responses to market (Lawless & Heymann, 2010;Murray et al., 2001). The high economic and time consuming aspectsof having a trained descriptive panel could be a problem in academicresearch as well, when a short project does not justify the training of apanel from scratch, or the lack of funding does not allow it.

All things considered, it was natural that sensory science wouldtransition, at certain point, to less time consuming, more rapid senso-ry methods, that would be more flexible and give extra agility to sen-sory description, both in terms of timing and training requirements.And that is exactly what happened, starting with the development offree choice profiling (FCP) and repertory grid (RG)methods in the eight-ies, which could be used with non-trained assessors (Thomson &Mcewan, 1988; Williams & Arnold, 1985). In FCP, consumers developtheir own attributes to describe the products, with their own vocabularyand in any number, limited only by their sensory skills, they then quan-tify their personal attributes using line scales; the method is basedon the assumption that panelists do not differ in their perceptions butsolely in the to describe them (Murray et al., 2001). The developmentof Generalized Procrustes Analysis (GPA) as statistical tool (Gower,1975) allowed the possibility of analyzing the data coming from datasets which differed in the number of attributes per consumer and alsohaving differences in the use of the scale. The RG method quantifiesproducts in the sameway, but the step of development of the attributesis done by constructs generation using triads of products through a ver-sion of Kelly's repertory grid (Kelly, 1955). The emergence of these twomethods opened the way to the use of consumers for product sensorydescription, with the realization that by allowing panelists to selecttheir own attributes it was possible to identify characteristics (whichmay not have been considered using the traditional approach), togetherwith the economization of time and resources.

Development of descriptive techniques continued since the eight-ies to our days with an array of different methods of sensory charac-terization, developed in the last ten years, which can be used withsemi trained assessors (i.e. trained in sensory recognition and charac-terization but not in the specific category of products or in scaling)

Fig. 1. Example of the response provided of an assessor to a free sorting task with 7samples of milk custards.

895P. Varela, G. Ares / Food Research International 48 (2012) 893–908

and even naïve consumers, with a great success, obtaining sensorymaps very close to a classic descriptive analysis with highly trainedpanels. Those novel techniques are, namely: sorting (Lawless, Sheng, &Knoops, 1995; Schiffman, Reynolds, & Young, 1981), flash profiling(Dairou & Sieffermann, 2002), projective mapping or Napping® (Pagès,2005; Risvik, McEvan, Colwill, Rogers, & Lyon, 1994), Check-all-that-applies (CATA) questions (Adams, Williams, Lancaster, & Foley,2007), and other techniques less frequently used for sensory profiling(use of intensity scales with consumers, evaluation of open-endedquestions, paired comparisons) or still in early development at thispoint in time as polarized sensory positioning (PSP) (Teillet, Schlich,Urbano, Cordelle, & Guichard, 2010) and preferred attribute elicitationmethod (Grygorczyk, Lesschaeve, Corredig, & Duizer, in press). In gen-eral, novel techniques are based on different approaches: in the lineof conventional profiling or free choice profiling there are methodsbased on the evaluation of individual attributes (intensity scales,CATA, flash profiling, paired comparisons); methods based on the eval-uation of global differences (sorting, Napping®); methods based onthe comparison with product references (PSP), and based on a free,global evaluation of the individual products (open-ended questions).

Conventional descriptive analysis nevertheless, has not beensubstituted by novel profiling methods, as trained panel descriptivemeasurements usually perform better in various cases, for examplewhen there is a need to compare samples in different moments intime, when comparing different sample sets with a few samples incommon, or when a very detailed sensory description is required.The new tools emerged, in fact, as complementary tools to sensoryand consumer science, as they can be applied to gather product de-scriptions directly from consumers, with the added benefit of havingdirect feedback from them, and sometimes with their own vocabulary(Moussaoui & Varela, 2010).

The hypothesis that consumers are able to accurately describeproducts is more and more accepted within the sensory science com-munity and diverse product profiling methods are used as never be-fore in the food industry: the line between sensory and consumerscience is becoming blurred. This paper aims at reviewing theory, im-plementation, advantages and disadvantages of the novel productprofiling techniques developed in the last ten years.

2. Novel methods for product characterization

2.1. Sorting

2.1.1. Theory and implementationClassification, i.e. putting a group of things into categories

according to an established criterion, is one of the most common op-erations in thinking (Coxon, 1999). In the context of social sciences,the process by which a person classifies objects is called sorting(Coxon, 1999). Sorting has been extensively used as a systematicmethod for data collection in psychology, anthropology and sociolo-gy (Coxon, Davies, & Jones, 1986; Miller, Wiley & Wolfe, 1986). Theaim of the technique is to study how people classifies objects,which provides information of how people perceive the objects andwhat characteristics they attend to when making classification be-tween a series of objects (Black, 1963).

Sorting tasks have also been used to get information aboutthe sensory characteristics of food products in sensory and consumerscience (Lawless et al., 1995; Schiffman et al., 1981). The aim of themethodology is to measure the global degree of similarity betweensamples by sorting them into groups. Assessors are asked to try thewhole set of samples and to sort them into groups according totheir similarities and differences, using their own personal criteria.Assessors are told that two samples which are perceived as similarshould be placed in the same group, whereas two samples that aremarkedly different should be placed in different groups. In order toavoid trivial answers, assessors are usually told that they should

sort the samples in at least two groups and that they could not haveonly one sample in all groups. In order to gather information aboutthe sensory characteristics of the samples which are responsiblefor the similarities and differences between the samples, once thesorting has been completed, assessors are asked to provide descrip-tive words for each of the groups they formed (Lawless et al., 1995;Popper & Heymann, 1996). A typical classification provided by an as-sessor in a sorting task is shown in Fig. 1.

When assessors are not trained they might find it difficult toprovide a description of the sensory characteristics of each group ofsamples. Therefore, in order to make the description phase easier,Lelièvre, Chollet, Abdi, and Valentin (2008) provided to the assessorsa list of pre-defined sensory characteristics from which they couldselect those they consider appropriate to describe the samples.

The number of assessors used in sorting tasks depends on theirtraining:whenworkingwith trained assessors the usual number rangesfrom 9 to 15 (Cartier et al., 2006; Chollet, Lelièvre, Abdi, & Valentin,2011), whereas when untrained assessors or naive consumers are con-sidered the number of assessors ranges from 9 to 98 (Chollet et al.,2011; Cadoret, Lê, & Pagès, 2009). Despite the variability in the numberof untrained assessors considered in sorting tasks, most studies workwith 20–50 (Ares, Varela, Rado, & Gimenez, 2011a; Cartier et al.,2006; Falahee & MacRae, 1997; Moussaoui & Varela, 2010).

2.1.2. Data analysisData analysis of sorting tasks aims at providing a spatial map that

represents the similarities and differences between samples in termsof their sensory characteristics (Lawless et al., 1995). In this map,the distance between a pair of samples is related to their degree ofdifference, which means that two samples which are representedclose to each other are similar, whereas two samples which are repre-sented far from each other correspond to dissimilar products. Differ-ent approaches have been proposed for analyzing data and to get asample map.

The most common approach for analyzing data from sorting tasksis Multidimensional Scaling (Lawless et al., 1995). When using thisapproach a similarity matrix is created by counting the number oftimes that each pair of samples is sorted within the same group, asshown in Fig. 2. Non-metric or metric Multidimensional Scaling (MDS)is carried out on this similarity matrix in order to get a 2-dimensionalrepresentation of the samples. A typical sample representation fromMDS is shown in Fig. 3.

The main disadvantage of MDS is that information about differ-ences in the perception of the individual assessors is lost since thesimilarity matrix computes the similarity and differences betweensamples for the entire group of assessors (Lawless et al., 1995). There-fore, it is not possible to determine if assessors sorted the samples sim-ilarly or if they had different perceptions and used different criteriato group them. Abdi, Valentin, Chollet, and Chrea (2007) proposedthe application of a different statistical technique, called DISTATIS, to

Fig. 2. Example of a similarity matrix for analyzing data from a free sorting task usingMultidimensional Scaling. Each cell indicates the number of times that each pair ofsamples was placed together within the same group in the free sorting task.

896 P. Varela, G. Ares / Food Research International 48 (2012) 893–908

overcome this limitation. This technique allows the analysis of 3-waydistance tables and takes into account the sample grouping providedby each assessor. DISTATIS first analyzes the individual co-occurrencesmatrices of the participants, providing an optimal representation ofthe assessors based on their resemblance. Then, a diagonalization ofthe linear combination of individual matrices is performed to providea consensus sample representation. Finally, the words used by theassessors to describe the groups are projected by using barycentricproperties.

Cadoret et al. (2009) proposed the application of FAST for analyz-ing sorting data. This approach provides an optimal representation ofthe samples based on Multiple Correspondence Analysis (MCA), andan optimal representation of the assessors based on Multiple FactorAnalysis (MFA). In this technique all the assessors have the same im-portance when constructing the sample map. An example of the sam-ple representation provided by FAST on data from a sorting task isshown in Fig. 4.

The main advantage of DISTATIS and FAST is that they provide arepresentation of the assessors, which enables the visualization ofindividual differences. Besides, by applying these techniques, thewords used by the assessors to describe the samples could easily beprojected into the sample space, which improves interpretation andprovides more actionable results.

Within the novel approaches for sensory characterization of foodproducts, sorting is one of the most popular approaches.

Fig. 3. Typical sample representation of data from a free sorting task of 7 orange juicesamples (named A to G) with 50 consumers using Multidimensional Scaling (MDS).

2.1.3. Application, advantages and limitationsThismethodology has been applied to awide range of productswith

different sensory complexities, including cheese (Lawless et al., 1995),drinking water (Falahee & MacRae, 1997), beer (Chollet & Valentin,2001), wine (Gawel, Iland, & Francis, 2001), yogurt (Saint-Eve, PaàiKora, & Martin, 2004), breakfast cereals (Cartier et al., 2006), olive oil(Santosa, Abdi, & Guinard, 2010), coffee (Moussaoui & Varela, 2010)and orange-flavored powdered drinks (Ares et al., 2011a).

Free sorting has several advantages which makes it an interestingmethodology for sensory characterization. Firstly, it corresponds tonatural and common mental activity, being an easy and enjoyabletask for participants (Coxon, 1999). Besides, it does not require exten-sive training and produces little fatigue and boredom, which makes itappropriate for both trained assessors and consumers (Bijmolt &Wedel, 1995). As suggested by Cartier et al. (2006) another advantageof the methodology is that it does not require the use of scales orother quantitative systems.

Although sorting tasks can be applied to a large sample set it is im-portant to take into account that all samples should be presented simul-taneously in a single session. Thus, when dealing with complex andfatiguing products, the number of products to be evaluated should belimited. Furthermore, it should be highlighted that when sorting tasksare performed by untrained assessors, the descriptions provided couldbe difficult to interpret in order to get actionable information.

2.1.4. Modifications to the original methodologyTwomodifications to the original methodology have been recently

proposed. Santosa et al. (2010) applied a modified sorting task to in-vestigate consumer perception of extra virgin olive oils. These authorsasked consumers to perform a two-stage sorting task to encouragethem to further discriminate between samples, after participants fin-ished sorting samples into groups, they asked them to complete a sec-ond sorting task, which consisted of sorting samples within eachgroup according to their similarities and differences. Data was analyzedusing DISTATIS considering the hierarchical nature of data from the firstand the second task.

In order to get information about the structure of the groups formedby each assessor, Courcoux, Qannari, Taylor, Buck, and Greenhoff(2012) proposed the application of taxonomic sorting task. In this ap-proach, after assessors have completed the first sorting task, they areasked to organize the groups into a hierarchical structure. In this laststep, assessors are instructed to put together the two groups that theythink aremost similar. This step is repeated until all groups are reducedto one, yielding a hierarchical structure similar to that shown in Fig. 5.Data analysis is performed using a non-metric MDS on an averaged dis-similarity matrix (Courcoux et al., 2012).

2.2. Flash profiling

2.2.1. Theory and implementationDairou and Sieffermann (2002) suggested the use of flash profiling

(FP) for sensory description, developed as a variant of free choiceprofiling. It was defined as a combination of FCP with a comparativeevaluation of the products via ranking, based on the simultaneouspresentation of the whole sample set. FP is a flexible method meantto rapidly profile products according to their most salient sensoryattributes. It has proven to be as satisfactory as conventional profilingin many applications (Dairou & Sieffermann, 2002).

FP can be done in two sessions, or in one session with two steps. Inpractice, coded samples are presented all together. In a first step con-sumers have to taste them comparatively in order to generate alldescriptors that they consider appropriate to discriminate betweenthe samples. In a second step, they rank all samples from “low” to“high” on each selected attribute, where ties are allowed. As in FCP,each consumer generates his/her own set of attributes; no indicationis given regarding the number of attributes (Dairou & Sieffermann,

Fig. 4. Representation of 7 orange juice samples and descriptive terms from a free sorting task with 50 consumers using FAST.

897P. Varela, G. Ares / Food Research International 48 (2012) 893–908

2002; Delarue & Sieffermann, 2004; Lassoued, Delarue, Launay, &Michon, 2008; Moussaoui & Varela, 2010). Fig. 6 shows an exampleof how a ballot of one consumer would look like.

When products are tested blind, and the samples permit (noteasily recognizable by shape, color, etc.), it is usual to include a repeatedblind control within the sample set to examine individual assessorperformance, subsequently checking in the final perceptual map thatsample and blind control are close (Ferrage, Nicod, & Varela, 2010).Another possibility would be repeating the whole evaluation withthe same assessors, but this is more difficult when the number of con-sumers is considerable.

The simultaneous comparison of all the samples could allow bet-ter product discrimination. Furthermore, when the tested productsbelong to the same or to similar product categories, flash profilingcan be more discriminating than conventional profiling (Delarue &Sieffermann, 2004; Mazzucheli & Guinard, 1999).

Flashprofile has been carried outwith trainedpanels of 6 to 12 trainedor semi-trained assessors (Albert, Varela, Salvador, Hough, & Fiszman,2011; Dairou & Sieffermann, 2002; Delarue & Sieffermann, 2004;Moussaoui & Varela, 2010; Tarea, Cuvelier, & Siefffermann, 2007) andwith consumer panels of 20 to 40 participants (Lassoued et al., 2008;Moussaoui & Varela, 2010; Veinand, Godefroy, Adam, & Delarue, 2011).

2.2.2. Data analysisThe analysis is based on ranking data, Fig. 7a shows how data is

computed, with ties; an easy way of verifying the ranks are entered

Fig. 5. Typical hierarchical structure obtained from a taxono

correctly is the sum of ranks, as the sum is the same for all attributesand all assessors, as it depends only on the number of products.For data collection, individual matrices are built for each consumer(products in rows×attributes in columns), the table is structured sothat each consumer has a table with his/her own attributes, wereproduct rankings are inputted (Fig. 7b).

A Generalized Procrustes Analysis (GPA) is run on all the matricesin order to obtain the product and attribute configurations. GPA de-livers a consensus configuration and provides, like in PCA, a productbiplot and an attribute plot. In the case of the attributes, consensuscomes from the usage of the same/similar attributes by different asses-sors, i.e. “sweet”, “sweetness” and “sugary”, etc. (Moussaoui & Varela,2010). Hierarchical Cluster Analysis (HCA) can be applied to identifyclusters of attributes that are correlated (Lassoued et al., 2008).

Ferrage et al. (2010) put to discussion the topic of how to checkpanel performance in novel profiling methods, particularly becauseof the use of consumers, and proposed a method to detect bad per-formers in a FP test, with the use of a blind control, and HCA analysisper consumer. Also, Veinand et al. (2011) proposed the use of HCA onthe FP product consensus configuration to check the clustering of theblind control as a performance measure.

2.2.3. Application, advantages and limitationsFlash profile has been applied to describe different foods, includ-

ing jams (Dairou & Sieffermann, 2002), dairy products (Delarue &Sieffermann, 2004), commercial apple and pear purees (Tarea et al.,

mic sorting task of an assessor asked to sort 8 samples.

Fig. 6. Example of a ballot for one assessor describing 7 yogurt samples via FP, ranking 3attributes from “low” to “high”. “Pete” ranked some samples in the same position whenassessing “sweet flavor” (D and E) and “grainy” (C, E, F), as ties are allowed.

898 P. Varela, G. Ares / Food Research International 48 (2012) 893–908

2007), flavor perception of bread odor (Poinot et al., 2007), jellies(Blancher et al., 2007), bread (Lassoued et al., 2008), wines (Perrinet al., 2008), hot beverages (Moussaoui & Varela, 2010), lemon icedteas (Veinand et al., 2011) and fish nuggets (Albert et al., 2011).Also, it has been successfully applied for non-food products as concerthall acoustics (Lokki, Pätynen, Kuusinen, Vertanen, & Tervo, 2011).

FP is a rapid sensory mapping tool, easy to comprehend for con-sumers, suited for applications when a quick response is needed butalso as initial screening tool for a new product set or category andto study a specific market (Dairou & Sieffermann, 2002; Delarue &Sieffermann, 2004; Tarea et al., 2007).

A limitation of FP, being a comparative method, is that the numberof samples that can be assessed is limited, and it would depend on theproduct category. However, Tarea et al. (2007), reported successfullydescribing via FP, 49 samples of apple and pear purees, in one session

Fig. 7. (a) Schematic view of how to collect FP data for one attribute. Tied attributes share thesample, so they share position 4.5 ((4+5)/2=4.5); (b) structure of the data matrix for FP cathat the number of attributes usually differ between consumers (n≠m≠p).

lasting between 2 and 5 h, where assessors could take breaks; it isnoteworthy though, that they were 6 trained, motivated assessors.

Another drawback of FP is that each assessor has his/her ownattribute list, so the semantic interpretation can be complex (Dairou& Sieffermann, 2002; Veinand et al., 2011). However, it has beenproved that even if this method produces a large amount of variedattributes, the core attributes for the description of the samples setare well covered when using FP even with consumers speaking differ-ent languages, what makes the method especially suited for crosscountry comparisons (Moussaoui & Varela, 2010). Within the sameline, consumers often use hedonics or benefit-related terms togetherwith sensory attributes; this fact can be seen as a limitation becauseit complicates the analysis (Veinand et al., 2011), nevertheless, thisinformation could be interesting to relate product characteristics tomarketable features and consumer preference.

2.3. Projective mapping and Napping®

2.3.1. Theory and implementationProjective mapping, and its special case Napping®, are projective-

type methods that collect bi-dimensional perceptual maps for eachassessor in a single sensory session. There were originally derivedfrom psychology and previously used in qualitative market research,to obtain associations between products (Pagès, 2005; Risvik et al.,1994). Risvik et al. (1994) proposed to use it with consumers, andlinked the results to trained panel data to explain product description.

In projective mapping, samples are simultaneously presented, tobe positioned by each assessor on a bi-dimensional space as a “table-cloth” (“nappe” in French, which gave the name to Napping®), or moreoften in an A4 or A3 blank paper. Samples are arranged by the partici-pant according to the differences and similarities between them insuch a way that the more similar they are, the closer would be twosamples on the sheet (Perrin et al., 2008). Assessors are asked to ob-serve, smell and taste the samples and to position them according todifferences/similarities. The positioning criteria and their importance

same rank, in the examples D and E were ranked together between the 3rd and the 6thlculations, ranks are inputted for each of the attributes selected for the consumers, note

899P. Varela, G. Ares / Food Research International 48 (2012) 893–908

are chosen individually by each assessor, which makes projectivemapping a flexible and spontaneous procedure. After positioning thesamples, the participants are sometimes asked to write down com-ments to describe samples or groups of samples, the reasons behindtheir placement, method generally known as “Ultra Flash Profiling”(Pagès, 2003; Perrin et al., 2008). When the exercise is blind, and thesamples allow (not easily recognizable by shape, color, etc.), it is com-mon practice to add a repeated blind control sample in the set, tocheck assessor performance, subsequently checking in the final percep-tual map that sample and blind control are close (Ferrage et al., 2010).Fig. 8 shows an example of how the configuration of one of the assessorswould look like. Via multivariate statistical analysis of the data, allthe individual maps are collated in a consensus configuration that,together with the comments, would determine the sensory profile ofthe sample set, in terms of distances and descriptions, as perceived bythe group of assessors.

When working with trained panels projective mapping has beenperformed with 9 to 15 assessor, as in traditional descriptive analysis(Perrin et al., 2008; Risvik et al., 1994), whereas when untrained asses-sors or consumers complete this type of task the number has been in-creased to 15–50 (Albert et al., 2011; Ares, Deliza, Barreiro, Giménez,& Gámbaro, 2010; Nestrud & Lawless, 2008).

2.3.2. Data analysisFor data collection, the coordinates of the location of the products

are measured for each consumer (Fig. 9a); also, the comments givenfor each of the samples are counted across consumers, for example,how many consumers mentioned the term “sweet” for sample 1, howmany for sample 2, etc. The data would be structured in three tables,with the products in rows and the coordinates (x,y) and the mentionedattributes in columns. The first table would have the x coordinates ofconsumer 1 to consumer n, the second table the y coordinates of con-sumer 1 to consumer n, and the third tablewould include the frequencyof mention of all the attributes across consumers per each sample(Fig. 9b).

Pagès (2005) proposed to analyze Napping® data by MultipleFactor Analysis (MFA), in which after directly collecting the Euclideanconfiguration of each subject, the simultaneous processing of all mapsprovides a graphical display of the products (biplots) in which twoproducts are near if they were perceived similar by the whole panelof subjects, each one having used and weighted its own criteria.

MFA is an enriched PCA — or Multiple Correspondence Analysisin the case of categorical variables (Pagès, 2005). It analyzes severaltables of variables differing in number and nature, with the require-ment that within a table, that the variables must be of the same

Fig. 8. Example of a bi-dimensional map or “nappe” configuration of one of the asses-sors for 7 coffee based beverages.

nature (quantitative or qualitative). MFA integrates different tablesof variables describing the same observations (Albert et al., 2011;Moussaoui & Varela, 2010). The difference with PCA is that MFAtakes into account individual differences rather than averaging thedata (Nestrud & Lawless, 2008). When comments are added to thesheet to describe the groups (ultra flash profiling), the qualitativedata is analyzed as another data table that is accounted for as supple-mentary variables (Perrin et al., 2008). Pagès (2005), Perrin et al.(2008) and Moussaoui and Varela (2010) present schematic exempli-fying views of how to collect and treat the Napping® data.

An extension of MFA, Hierarchical Multiple Factor Analysis (HMFA),can be applied when the data are organized in a hierarchical manner;it balances the role of each table of data and allows the interpretationin terms of the different levels of hierarchy (Le Dien & Pagès, 2003).

2.3.3. Application, advantages and limitationsNapping® is based on the global perception of the sample set

differences, and it has been described as a natural, intuitive, holisticway for consumers to describe products, closer to what happens infront of the supermarket shelf (Ares et al., 2011a; Carrillo, Varela, &Fiszman, 2012a). It has been applied to various food products like choc-olate (Risvik et al., 1994), commercial dried soup samples (Risvik,McEwan, & Rodbotten, 1997), snack bars (King, Cliff, & Hall, 1998),ewe milk cheeses (Bárcenas, Pérez Elortondo, & Albisu, 2004), citrusjuices (Nestrud& Lawless, 2008),wines (Perrin & Pagès, 2009), hot bev-erages (Moussaoui & Varela, 2010), milk desserts (Ares, Deliza, et al.,2010), fish nuggets (Albert et al., 2011) and powdered drinks (Ares etal., 2011a). Furthermore, Napping® has also been applied with successto study other stimuli than sensory, like the influence of packaginginfo and nutritional claims on consumer perception (Carrillo, Varela,& Fiszman, 2012a; Carrillo, Varela, & Fiszman, 2012b). The methodcan be quite good to understand the relation to consumer hedonic per-ception and uncover drivers of liking, Ares, Varela, Rado, and Giménez(2011b) even proposed it as a method to identifying the consumers'ideal product or flavor space.

Napping has proved to be quite easy and understandable for con-sumers to perform, Risvik et al. (1997) even suggested the possibilityto use it with children, for the possibility of turning it into a game.However, some authors report that other novel techniques could beeasier to understand than Napping® to naïve consumers (Ares et al.,2011a).

The greatest limitation of napping, like in other comparative tech-niques, is the number of products that can be tested at the same time,that would much depend on the sensory characteristics of the prod-uct, but generally with a maximum of 12 (Pagès, 2005). Other limita-tion is the reproducibility, validity or robustness of the methods, likemost novel techniques, has not been studied in detail so far. Ferrageet al. (2010) put to discussion the topic of panel performance innovel profiling methods, particularly with the use of non-trained as-sessors, and proposed a method to detect bad performers in a nap-ping test, with the use of a blind control; they stated that StructuredNapping was quite a robust method, even when bad performerswere present in the set. This could be owed to the pre-set dimensions,that is not always possible to use or relevant for all categories of prod-ucts or sample sets. There is still much to do in this area and one ofthe topics that we expect to appear more often in scientific researchabout Napping® and other novel methods of sensory characterization.

2.3.4. Modifications to the original methodologyNapping® is still a developing method as its flexibility allows some

changes in implementation to suit different objectives or depending onthe complexity of the products. Pagès, Cadoret, and Lê (2010) proposedthe sorted Napping® procedure, combining napping with a categoriza-tion task: after the Napping® exercise panelists are asked to identify ex-plicit groups by circling products on the sheet. Also, in its initial paper,Pagès (2003) suggested as a good idea to perform napping by modality

Fig. 9. (a) Schematic view of how to collect Napping® data for 7 coffee based beverages; (b) structure of the data matrix: as an example, x1 and y1 would be the coordinates of P2for consumer 1, x2 and y2 are the coordinates of P2 for consumer 2, etc. Attributes were recorded with the frequency of mention per product; “bitter” was mentioned 21 times forP2, “watery” 15 times, “sweet” 3 times, etc.

900 P. Varela, G. Ares / Food Research International 48 (2012) 893–908

(appearance, odor, flavor, texture), which was later on implementedby Pfeiffer and Gilbert (2008) and presented in Sensometrics2008.Afterwards it has also been performed by Dehlholm, Brockhoff,Meinert, Aaslyng, and Bredi (2012) with success; this variation isknown as Partial Napping. Finally, Ferrage et al. (2010) suggested intheir communication in Eurosense2010 the possibility of performingStructured Napping, where pre-named axis would be given to the pan-elists, signaling the 2 main directions of the sensory flavor space (x andy axes of the “nappe”), this variant has not been further investigated sofar.

2.4. Check-all-that-apply questions

2.4.1. Theory and implementationCheck-all-that-apply (CATA) questions are multiple choice ques-

tions which are commonly used in marketing research in order to re-duce response burden (Rasinski, Mingay, & Bradburn, 1994). Thesequestions consist of a list of words or phrases from which respondentsshould select those they consider that apply to answer a certain ques-tion. The main advantage of this type of question is that it allows mul-tiple options to be selected, instead of limiting respondents to selectonly one answer or forcing consumers to focus their attention and eval-uate specific attributes (Smyth, Dillman, Christian, & Stern, 2006).

Adams et al. (2007) proposed the application of CATA questions asa simple method to gather information about consumers' perceptionof the sensory characteristics of food products. In this approach, prod-ucts are presented to consumers in monadic sequence, following a

balanced rotation order. Consumers are asked to try the productsand to answer a CATA question by selecting all the terms that theyconsider appropriate to describe each of the samples. There are usual-ly no constraints on the number of attributes that could be selected bythe consumers. The list of words or phrases in the CATA questioncould be exclusively related to the sensory characteristics of the prod-uct (Fig. 10a) or also include terms related to non-sensory character-istics, such as usage occasions, product positioning and emotions(Fig. 10b).

The selection of the list of words or phrases included in the CATAquestion is one of the main challenges of the methodology. Sensoryterms could correspond to the descriptors used by trained assessorspanels to characterize the products or could be selected consideringresults from previous focus groups or quantitative consumer studies.

As this methodology is mainly used with consumers, the number ofassessors necessary to perform a sensory characterization using CATAquestions ranges from 50 to 100 (Adams et al., 2007; Ares, Barreiro,Deliza, Giménez, & Gámbaro, 2010; Dooley, Lee, & Meullenet, 2010).

2.4.2. Data analysisThe first step when analyzing data from CATA questions is deter-

mining if consumers detected significant differences between sam-ples for each of the terms of the CATA question. This analysis isperformed using Cochran's Q test (Parente, Manzoni, & Ares, 2011),which is a nonparametric statistical test used in the analysis oftwo-way randomized block designs to check whether k treatmentshave identical effects, when the response variable is binary. For

Fig. 10. Examples of check-all-that-apply (CATA) questions including: (a) sensory and(b) non-sensory terms.

Fig. 12. Example of the frequency table used for analyzing data from a term ofcheck-all-that-apply (CATA) questions using correspondence analysis.

901P. Varela, G. Ares / Food Research International 48 (2012) 893–908

each term of the CATA question a data matrix is created containingsamples in columns, consumers in rows. In this matrix each cell indi-cates if the term was checked or not (1/0 respectively) by each con-sumer to describe each sample (Fig. 11).

Then, in order to get a bi-dimensional representation of the sam-ples Multiple Correspondence Analysis (MCA) is used on a matrixcontaining the number of consumers who checked each term fromthe CATA question to describe each sample (Fig. 12). This analysisprovides a sensory map of the samples, which enables to determinethe similarities and differences between the samples, aswell as the sen-sory attributes that characterize their sensory attributes (Ares, Barreiro,et al., 2010; Ares, Deliza, et al., 2010; Ares, Giménez, Barreiro andGámbaro, 2010).

2.4.3. Application, advantages and limitationsCATA questions have been used for the sensory characterization of

several food products: snacks (Adams et al., 2007), strawberry cultivars(Lado, Vicente, Manzzioni, & Ares, 2010), ice-cream (Dooley et al.,2010), milk desserts (Ares, Barreiro, et al., 2010), orange-flavoredpowdered drinks (Ares et al., 2011a, 2011b), and citrus-flavored sodas(Plaehn, 2012).

Some studies have compared the sensory maps generated byCATA questions with those provided by classic descriptive analysis(QDA) with a trained assessor panel, reporting similar results (Ares,Barreiro, et al., 2010; Bruzzone et al., 2012; Dooley et al., 2010). Con-sidering these results and the fact that checking terms from a list doesnot require much effort for consumers, CATA questions have beenreported to be a quick, simple and easy method to gather information

Fig. 11. Example of the data matrix used for analyzing data from check-all-that-apply(CATA) questions using Cochran's Q test.

about consumer perception of the sensory characteristics of foodproducts. Moreover, Adams et al. (2007) have reported that CATAquestions have a smaller influence on liking scores thanjust-about-right or intensity questions.

However, it is important to take into account that despite the factthat frequency of mention of the terms from CATA questions havebeen reported to be closely related to attribute intensity, they donot provide quantitative information since consumers only evaluateif a term is appropriate or not to describe the product. When usedin marketing research, Sudman and Bradburn (1982) indicated thatrespondents may not select a term in a check-all question due tothree main reasons: because they consider that the term does notapply, because they are neutral or undecided about it, or becausethey did not pay attention to it. Therefore, if consumers do not selecta term in a CATA question it cannot be concluded that they considerthat it does not apply to the product. Moreover, it has been reportedthat respondents do not usually involve in a deep processing and givequick answers, tending to select the terms that appear near the topof the list rather than those that appear near the bottom of the list(Krosnick, 1999). This effect has been also reported for the applicationof CATA questions to sensory characterization of food products.Castura (2009) reported that term position strongly affect results,which suggests that fixed choice order CATA ballots skew results. Forthis reason, rotation of terms within a CATA question is recommendedfor getting valid results. However, further research is needed to evaluatethe influence of the number and type of terms, as well as the design ofthe questionnaire, in results from sensory characterization providedby this methodology.

Another limitation of CATA questions is that they provide qualita-tive data and therefore might have smaller discriminative capacity thanranking tasks or intensity scales (Dooley et al., 2010). Moreover, CATAquestions require a relatively large number of consumers, not beingrecommended with trained assessors. However, frequency-based tech-niques have been used to characterize the aroma of complex productssuch as wine (Campo, Ballester, Langlois, Dacremont, & Valentin, 2010;Campo, Do, Ferreira, & Valentin, 2008; Le Fur, Mercurio, Moio, Blanquet,& Meunier, 2003). Campo et al. (2010) have recommended that this ap-proach could be a practical alternative to conventional descriptive analy-sis for characterizing products with complex aroma. Also, Nicod et al.(2010) recommended the application of CATA questions with trained as-sessors for the evaluation of complex products. In particular, theysuggested its use for attributes present in the sample in low concentra-tions. These attributes cannot be easily discriminated via scaling sincetheir perception is more related to presence/absence than to differentintensities.

Furthermore, due to the nature of the response provided by partici-pants, if the products are very similar the same terms will be selectedfor all the evaluated samples. In these cases frequency data from CATA

902 P. Varela, G. Ares / Food Research International 48 (2012) 893–908

questions would not be able to detect significant differences betweensamples. Thus, this approach is not recommended when evaluatingproducts with subtle differences in their sensory characteristics.

2.5. Other methodologies

Several other less commonmethodologies have been used for sen-sory characterization of food products. Despite most of these method-ologies have been applied to a limited number of products, they havebeen reported to provide valid results.

2.5.1. Intensity scalesOne of the main disadvantages of QDA and other classical descrip-

tive methods is the time and resources needed to train the assessorpanel. Therefore, some authors have stated that the training periodcould be omitted and have performed sensory characterization withthe use of consumers (Husson, Le Dien, & Pagès, 2001; Worch, Lê, &Punter, 2010). In this approach consumers are asked to rate the inten-sity of a fixed set of sensory attributes using intensity scales, as it iscommonly done with trained assessors in descriptive analysis. Themain difference with the traditional approach is that descriptors areprovided to consumers by the researcher and that no training in attri-bute recognition or quantification is performed.

Despite this approach has been traditionally not recommended(Lawless & Heymann, 2010; Stone & Sidel, 2004), recent studieshave reported that results from sensory characterization performedby 50–100 consumers with intensity scales are similar to those pro-vided by trained assessor panels (Husson et al., 2001; Worch et al.,2010). These authors concluded that consumers and trained assessorspanel provided similar results in terms of discrimination, consensus,and reproducibility and that the product spaces obtained from bothpanels were similar.

However, Ares, Bruzzone, & Giménez (2011) compared global and in-dividual performance of a consumer and trained assessor panels for tex-ture evaluation of milk desserts, concluding that both showed similardiscriminative capacity and reproducibility and were able to detect thesame differences in the texture of the evaluated milk desserts. On theother hand, consumer agreementwas low and themajority of consumerswere not able to give scores that significantly discriminated among sam-ples. Thus, the lack of consensus in the consumer panel and the high var-iability in their evaluations were compensated by the large sample size.For these reasons, sensory characterizationwith consumers using intensi-ty scales would not be recommended, except for specific situations inwhich information about the intensity of sensory attributes is neededand the cost and time involved in the selection and training of the asses-sors might be higher than those needed to perform a study with 50–150consumers. In particular, the evaluation of sensory attributes using inten-sity scales by consumers might be a good option in specific applicationswhen food companies do not have a trained panel or when the productis not evaluated on a regular basis.When, information about the intensityof sensory attributes is not needed other sensory characterization meth-odologies are recommended.

2.5.2. Open-ended questionsIn many methodologies assessors' descriptions have been consid-

ered as supplementary information in order to better understandthe similarities and differences between products (Bécue-Bertaut,Álvarez-Esteban, & Pagès, 2008; Lawless et al., 1995; Pagès, 2005).However, sensory characterization has also been performed using ex-clusively consumer descriptions of products, as suggested by ten Kleijand Musters (2003). In this methodology open-ended questions areused to ask assessors to provide a description of the sensory charac-teristics of a set of products, which aims at understanding the maincharacteristics that determine consumer perception of the productsand especially what motivates their liking scores.

Product descriptions have been gathered using three main ap-proaches. In the original application of the methodology ten Kleijand Musters (2003) allowed consumers to voluntarily write downremarks after their overall liking evaluations. Alternatively, Ares,Giménez, et al. (2010) asked consumers to provide up to four wordsto describe the samples after their overall liking evaluation, as part ofthe task they had to complete. More recently Symoneaux, Galmarini,and Mehinagic (2012) gave consumers the option to freely statewhat they liked and/or disliked about the evaluated products. Allthese options enabled consumers to provide a description about thesensory characteristics of food products.

Considering that this methodology is applicable to consumer studies,the number of assessors necessary to perform a sensory characterizationusing open-ended questions ranges from 50 to 100 (Ares, Giménez, et al.,2010; Symoneaux et al., 2012; ten Kleij & Musters, 2003).

Consumer responses to open-ended questions are not subjected torestrictions from the researchers and therefore contain rich informa-tion that could underscore and complement quantitative findingsfrom trained assessor panels (ten Kleij & Musters, 2003). However,due to the inherent complexity of textual data, data analysis is oftendifficult, labor-intensive and time-consuming.

Consumers answer open-ended questions in their own style,without any specific guidance, even with typing, orthographic andgrammatical mistakes, which makes it necessary to transform thedata into accurate sensory terms (Symoneaux et al., 2012). Analysisof text data consists of the following stages: removing mistakes, elim-ination of connectors and auxiliary terms, identification of phrasesand terms which make them up, regrouping synonyms, managingambiguous words, and marking terms of interest for the researcher(Rostaing, Ziegelbaum, Boutin, & Rogeaux, 1998). The first step ofthe analysis usually consists of deleting stopwords, auxiliary termsand other irrelevant words. Then, words with similar meaning aregrouped into the same category according to word synonymy asdetermined by a dictionary and the personal interpretation of the re-searchers. This classification is usually performed consensually bythree researchers (Modell, 2005). Categories mentioned by morethan 5% or 10% of the consumers are selected and frequency of men-tion of each category is determined by counting the number of partic-ipants that used each category to describe each product. A frequencytable is constructed and analyzed using Chi-square test and corre-spondence analysis. Global Chi-square test could be used to deter-mine significant differences in the description of the evaluatedsamples by studying the independence between rows and columns(Ares, Giménez, et al., 2010). Moreover, a Chi-square per cell testcan be used to identify significant differences between samples foreach of the sensory characteristics used by consumers to describethe evaluated products (Symoneaux et al., 2012). Finally, correspon-dence analysis can be used to get a 2-dimensional representation ofthe samples and the attributes (Ares, Giménez, et al., 2010; ten Kleij& Musters, 2003).

This methodology has been used in a limited number of foodproducts: mayonnaise (ten Kleij & Musters, 2003), milk desserts(Ares, Giménez, et al., 2010), and apples (Symoneaux et al., 2012).In these studies, sample maps gathered from open-ended questionshave been reported to be similar to those obtained from classic de-scriptive analysis (QDA) with trained assessor panels.

It is important to take into account that despite its simplicity andease of use for consumers, the descriptions provided by consumersare usually vague and difficult to interpret, which makes data analysistedious and difficult. Moreover, the information provided by thismethodology is not as precise as the information provided by descrip-tive analysis or other methodologies, particularly in those cases inwhich differences between the products are small.

Open-ended questions can be considered as complementary totraditional descriptive approach with trained assessors. Consumers'responses to open-ended questions could be used to get an insight

903P. Varela, G. Ares / Food Research International 48 (2012) 893–908

on the vocabulary used by consumers to describe the products, whichcould be interesting when designing communication strategies (Ares,Giménez, et al., 2010; Symoneaux et al., 2012).

A particular case in which this methodology is recommended iswhen working with experts in the particular product under study,since they are usually familiar with providing a verbal description ofproducts. In this sense, Thuillier (2007) proposed a variation of thismethod, called Pivot© profile to characterize the sensory propertiesof champagne with wine experts. The method is based on the free de-scription of the differences between samples and a reference product,which is called “pivot”. Assessors are asked to describe the sensoryattributes that they perceive as less intense in the product comparedto the pivot (e.g. less bitter), as well as those that they perceive asmore intense (more acid). They are asked to use only descriptivewords and to avoid writing complete sentences. An example of theevaluation sheet is shown in Fig. 13.

2.5.3. Preferred attribute elicitation methodGrygorczyk et al. (in press) proposed the preferred attribute elicita-

tion method as a novel approach to identify the key attributes whichdrive consumer liking. In this method, consumers are asked to agreeon a set of attributes to describe a group of samples within a productcategory, and to rank them according to how important they considerthey are in determining their liking of the product category.

Grygorczyk et al. (in press) presented a set of commercial vanillayogurts to consumers and asked them to rate their texture likingand to write down the sensory attributes they liked and dislikedabout the samples. Then, through a round-table discussion with amoderator consumers were asked to group the attributes, based ontheir similarity, to select anchors for their evaluation, and to rankthe groups according to their importance in driving their liking.After a short break, consumers were asked to individually rate prod-ucts on 7-point scales for each attribute.

Intensity data was analyzed using GPA as in free choice profiling.When characterizing yogurt texture, results from preferred attribute elic-itationmethodwith 42 consumerswere similar to those providedby con-ventional descriptive profiling with 10 trained assessors (Grygorczyk etal., in press). Regarding attribute importance, consumers consistentlyranked texture and flavor as very important to their liking, whereas ap-pearance attributes were ranked as the least important.

According to the authors, the advantage of this method is that itdirectly identifies the most relevant attributes for consumers, provid-ing product developers a smaller set of terms to be considered duringproduct optimization.

2.5.4. Polarized sensory positioningPolarized sensory positioning is based on the comparison of food

samples to a fixed set of reference products, or poles. This methodology

Fig. 13. Example of an evaluation sheet used in pivot profile.

was developed by Teillet et al. (2010) to explore the sensory character-istics of water. Despite the fact that these authors applied PSP with 15trained assessors, this methodology could be applied with semi-trainedassessors or naive consumers.

Assessors are asked to evaluate the degree of similarity of the sam-ples to a set of reference products. Teillet et al. (2010) selected threepoles that represented three typical profiles of mineral water. Whenevaluating a sample assessors should quantify the overall differencebetween it and each of the references using unstructured scales rang-ing from “exactly the same taste” to “totally different taste”, as shownin Fig. 14. A description phase should be performed in order to get in-formation about the sensory characteristics responsible for the simi-larities and differences between products.

Data analysis could be performed using MDS or PCA. In the firstapproach assessors ratings are considered as a measure of the dis-tance from each pole (Fig. 15). Ratings are averaged by sample and an-alyzed using Multidimensional Scaling unfolding techniques (Busing,Groenen, & Heiser, 2005) on the samples by poles matrix, to get atwo-dimensional representation of the samples. In the second approachthe poles are considered as descriptors, data is analyzed by calculatingaverage scores, and sample representation is obtained by PrincipalComponent Analysis (Teillet et al., 2010).

Polarized sensory positioning is an easy and quick methodologywhich could be performed with trained and untrained assessors. Itsmain advantage is that it enables to compare all products with fixedreferences, even if they are not evaluated in the same session. Howev-er, it is important to note that research is necessary to determine howreference samples should be selected, and particularly how manysamples are necessary and which their characteristics should be.

2.5.5. Paired comparisonPaired comparisons are one of the most common methodologies in

sensory science (Lawless & Heymann, 2010). They are usually used todetermine if two samples are perceived as equal or different in a specificsensory characteristic.

Poirson, Petiot, and Richard (2010) proposed the application ofpaired-comparison tasks for gathering information about consumers'perceptual space of diesel motor sounds. These authors asked consumersto perform a series of paired comparison tasks according to a list of attri-butes. For each attribute a comparison matrix was created, which con-tained all samples in rows and columns. Each intersection of columnsand rows corresponds to a paired comparison (Fig. 16), which has to befilled up using a 7-point scale (□□, □, □~, =, □~, □, □□). Assessorsare asked to complete some of the comparisons of each matrix by com-paring each pair of samples and to assess their difference by using thescale. The authors worked with two different panels: 10 experts and 30naive consumers.

Data is analyzed to determine the discriminative power of the at-tributes and the consensus between panelists by using least squareslogarithmic regression (LSLR).

Poirson et al. (2010) reported that paired comparisons providedbetter agreement between consumers and were more discriminatingthan ratings tasks. The results of these two tests with the naives werethen compared with the conventional sensory profile of the expertsusing Generalized Procrustes Analysis. Moreover, good consensuswas found between results from paired comparison task and thosefrom descriptive analysis with a trained assessor panel.

2.6. Comparison of the methodologies

Several studies have focused on the comparison of the sensorycharacterization provided by conventional descriptive analysis andnovel methodologies in a wide range of food products with differentcomplexity, ranging from simple products such as mineral water(Teillet et al., 2010) to complex products as wine (Perrin et al.,2008) or fish nuggets (Albert et al., 2011). Most studies have reported

Fig. 14. Example of an evaluation sheet used in polarized sensory positioning to compare one sample with three reference products (R1, R2 and R3).

904 P. Varela, G. Ares / Food Research International 48 (2012) 893–908

that, compared to results provided by conventional descriptive analysiswith trained assessors, novel methodologies provide similar informa-tion on the main sensory characteristics responsible for differencesbetween products, as well as similar sensory maps (Albert et al., 2011;Ares, Barreiro, et al., 2010; Ares, Giménez, et al., 2010; Ares et al.,2011; Bruzzone et al., 2012; Cartier et al., 2006; Chollet et al., 2011;Dairou & Sieffermann, 2002; Delarue & Sieffermann, 2004; Dooley etal., 2010; Guàrdia, Aguiar, Claret, Arnau, & Guerrero, 2010; Jack &Piggott, 1991; Moussaoui & Varela, 2010; Risvik et al., 1994, 1997;Symoneaux et al., 2012; Teillet et al., 2010).

However, it is important to stress that the information providedby descriptive analysis is clearly different from that gathered fromnovel methodologies. Descriptive analysis provides quantitative in-formation about the intensity of specific sensory attributes, enablingto identify significant differences between samples in each of theevaluated attributes. On the other hand, it is not possible to gatherthis information using novel methodologies. From a statistical pointof view, descriptive analysis is more robust than most novel methodol-ogies, which makes it possible to identify small and subtle differencesbetween products (Albert et al., 2011). Furthermore, considering asses-sors' training, descriptive analysis is more appropriate for comparingdifferent sample sets or for comparing samples evaluated at differentmoments in time (Moussaoui & Varela 2010).

Another disadvantage of many novel methodologies is related tothe interpretation of the sensory terms provided by assessors. Analy-sis of assessor descriptions in free profiling, flash profile, open-endedquestions, or holistic methodologies is in general a time-consuming,labor-intensive and difficult task. Due to the heterogeneity of con-sumers' descriptions, the large number of terms used and the lack

Fig. 15. Example of the data matrix used for analyzing data from polarized sensory po-sitioning using multidimensional scaling. Each couple of columns R1, R2, and R3 repre-sent the degree of difference between a sample and each of the references (R1, R2 andR3 respectively) for each of the assessors.

of definitions and evaluation procedures information about specificsensory attributes could be difficult to interpret. In particular, the in-terpretation of consumers' vocabulary can be difficult for complexmultiparameter sensations as for example “creaminess”, when morethan one modality is involved and it is not known if they are referringto flavor, texture or aroma (Moussaoui & Varela, 2010). Therefore, re-sults from descriptive analysis are usually more actionable for prod-uct developers than those from novel methodologies; being thelatter particularly useful when the objective is to identify the mostsalient attributes and the most relevant characteristics responsiblefor the similarities and differences between products.

Added to the vocabulary complexity, consumers sometimes usehedonics or benefit-related terms which in principle can be seen as alimitation because it complicates the analysis (Veinand et al., 2011),however, this information could be interesting to relate product charac-teristics to marketable features and consumer preference.

Despite the fact that descriptive analysis provides more accurate andreliable information inmost cases, some clear advantages of novelmeth-odologies could be mentioned. In the first place, the time needed for theimplementation of novelmethodologies for sensory characterization of aset of products is considerably shorter than for descriptive analysis,which makes the novel approach an interesting alternative, particularlyfor thoseworking in the industry. Another advantage of novelmethodol-ogies is that they do not require consensus from the panel, which couldpotentially lead to some loss of information due to the fact that if the per-ception of the minority of the assessors differs from that of the majority,it is not taken into account (Albert et al., 2011). The lack of need for con-sensus between panelists allows a diversity of points of views, whichcould provide richer information (Dairou & Sieffermann, 2002).

Fig. 16. Example of the evaluation sheet used by consumers in the paired comparisonmethodology.

905P. Varela, G. Ares / Food Research International 48 (2012) 893–908

In a few cases novel methodologies have been reported to providebetter information than descriptive analysis. For example, Delarueand Sieffermann (2004) stated that when working with similar prod-ucts flash profile was more discriminating than descriptive analysis.Similarly, Albert et al. (2011) reported that flash profile performedwith semi-trained assessors provided a more detailed description ofthe sensory characteristics of fish nuggets than descriptive analysis.Furthermore the lack of consensus could also be advantageous inthe case of very heterogeneous products where it is very difficult toreach.

Novel methodologies clearly differ in the way in which they gath-er information about the sensory characteristics of food products,which leads to differences in the information they provide aboutproducts and the applications for which they are recommended(Blancher et al., 2007). Holistic methodologies, such as sorting andprojective mapping, are based on assessors' global perception of theproducts, which might enable to identify the main attributes respon-sible for differences in how they perceive the samples. In free sortingand projective mapping assessors focus their attention on the globalperception of the products, which enables to identify the most salientsensory characteristics of the products. On the other hand, whenusing methods based on the evaluation of specific attributes assessors'perception is focused on specific features. This leads to differences inthe information provided by similarity based methodologies andthose that rely on the evaluation of specific attributes. The Focusingon attributes can be of use when specific or more detailed informationabout the product is needed, however, the evaluation would be moreartificial than that of holistic approaches. Methods that are not basedon attributes have the extra advantage that can be used forcross-country studies with consumers who speak different languages,without translation problems.

Many studies have reported that information provided by differentmethodologies is similar. When comparing CATA questions and projec-tive mapping for sensory characterization of milk desserts Ares, Deliza,et al. (2010) concluded that bothmethodologies provided similar infor-mation. Similarly, when evaluating powdered orange-flavored drinksAres et al. (2011a) reported that CATA questions, sorting, projectivemapping and intensity scales were equivalent.

On the other hand, some authors have reported that similarity-basedmethods are less discriminative than those from methodologiesbased on the evaluation of specific sensory attributes, particularlywhen small differences between samples are considered. For example,Veinand et al. (2011) compared three methodologies (free choice pro-filing, flash profile and projective mapping) for consumer profiling oflemon iced teas and concluded that flash profile showed the highestdiscriminative ability, whereas projective mapping showed the lowest.Albert et al. (2011) reported that flash profile provided more detailedinformation about the sensory characteristics of fish nuggets thanprojective mapping due to the fact that the latter was based on asses-sors' global perception of the products. Moussaoui and Varela (2010)reported thatflash profile and free choice profile provided richer vocab-ularies and more accurate sample maps than similarity-based method-ologies such as projective mapping and free sorting when workingwith hot beverages. Moreover, these authors reported that untrainedassessors were more repeatable when working with flash profilecompared to projective mapping or free sorting. Nestrud and Lawless(2010) compared projective mapping and free sorting and concludedthat, despite the sensory maps provided by both methodologies weresimilar for apples and for cheese, the identification of samples with sim-ilar sensory characteristicswas easier to interpret for projectivemapping.

When comparing novel methodologies it is also important to takeinto account practical issues since they clearly differ in the difficultythat assessors encounter when completing the tasks. Holistic meth-odologies can be considered more intuitive and less rational thanother methodologies based on the evaluation of specific sensory attri-butes. However, Ares et al. (2011a) reported that although consumers

were able to understand projective mapping and sorting tasks, theyfound them much more difficult than CATA questions or intensityscales. In agreement with this result, Veinand et al. (2011) reportedthat projective mapping was more difficult to perform with con-sumers than flash profile. These authors stated that when performinga projective mapping task consumers found it difficult to use thesheet of paper to locate the samples according to their similaritiesand differences. Besides, Ares, Deliza, et al. (2010) reported that, inorder to assure that consumers understood the task, it was necessaryto provide further explanations when working with projective map-ping compared to CATA questions.

Regarding the time needed by assessors to complete the task, in-tensity scales, CATA questions, open-ended questions and pivot pro-file are usually less time-consuming than projective mapping, freechoice profiling, flash profile, and polarized sensory positioning. Freechoice profiling and flash profile require two separate sessions, onefor descriptor generation and another for evaluating the sample set.Meanwhile, the rest of the methodologies could be carried out in asingle session. Projectivemapping and sorting tasks have been reportedto be more time-consuming than CATA or open-ended questions.According to Ares, Deliza, et al. (2010), consumers needed between 5and 15 min to complete a CATA question for sensory characterizationof 8 milk desserts, whereas they needed between 18 and 25 min tocomplete a projective mapping task with the same samples.

Thus, holistic methodologies such as projective mapping and freesorting seem to be more difficult and time-consuming for consumers.Considering that trained assessors with previous experience withsensory evaluation techniques could more easily understand thesemethodologies, Veinand et al. (2011) recommended performing pro-jective mapping with expert panelists.

Another disadvantage of projective mapping when using paperballots is that measuring the products' coordinates in the sheet ofeach assessor is tedious and tiresome for the researchers, particularlywhen a large number of consumers are used (Veinand et al., 2011).

The methods also differ in the number of samples that could be in-cluded within a set in a single session. Free choice profiling, flash pro-file, free sorting and projective mapping require that all productsare evaluated by assessors simultaneously in the same session, sincecomparisons between them are made. Thus, in order to avoid fatigueand adaptation, the number of samples to be evaluated in a singlesession is limited when compared to other methodologies such as in-tensity scales, CATA questions or polarized sensory positioning. Forthis reason, it could be complicated to use the former methodologiesthem when working with products that require careful temperaturecontrol or that have intense and persistent sensory characteristics. Inparticular, polarized sensory positioning requires the use of a fixedreference product, which makes it appropriate for comparing productsover time or when dealing with evaluations performed on differentsessions. However, the criteria for the selection of stable and easilyavailable reference products should be carefully taken into account;also, the fact of having to compare with a reference sample makes itmore tiresome for the panelists, as the technique requires repeated tast-ing of various samples.

3. Conclusions and recommendations

A summary of the characteristics of the novel methodologies forsensory characterization reviewed in the present article is providedin Table 1.

Novel methodologies consist of valid, reliable, simple and quick al-ternatives for sensory characterization of food products. They havebeen reported to provide similar information to classical descriptiveanalysis performed with trained assessor panels. However, it is im-portant to highlight that they could not be considered a replacementfor classic descriptive analysis since it is always more accurate due tothe fact that assessors are extensively trained in the identification and

Table 1Summary of the characteristics of the methodologies reviewed in the present article.

Method Type of evaluation Vocabulary Statistical method Limitations

Sorting Classification of samples based on their similaritiesand differences

Elicited by the assessors orprovided by the researcher

MDS, DISTATIS orFAST(MCA and MFA)

All samples should be presentedsimultaneously

Flash profiling Ranking of samples on a set of selected attributes Elicited by the assessors GPA Two sessions are requiredAll samples should be presentedsimulteaneously

Projective mappingor Napping®

Generating samples on a two-dimensional map according totheir similarities and differences

Elicited by the assessors MFA All samples should be presentedsimulteaneouslyIt could be difficult to understandfor naïve consumers

Check-all-that-apply(CATA) questions

Selection of terms from a list that are appropriate todescribe the samples

Provided by the researcher Cochran Q test,MCA, MFA

The design of the attribute listcould strongly affect the responsesNot recommended for evaluatingvery similar samples

Intensity scales Rating the intensity of a set of attributes using scales Provided by the researcher ANOVA, PCA Lack of consensus in consumers'responses

Open-ended questions Verbal description of the samples Elicited by the assessors Content analysis,Chi-square and MCA

Difficulties for analyzing verbaldescriptions

Preferred attributeelicitation

Ranking of attributes according to their importance andrating of products using structured scales

Elicited by the assessors GPA A round-table discussion is necessaryAll samples should be presentedsimultaneously

Polarized sensorypositioning

Evaluation of global differences between samples and aset of fixed references

Not gathered in the originalmethodIt could be elicited by theassessors

MDS or PCA Stable and readily-available refer-ences are neededSelection of the references couldstrongly affect the results

Paired comparison Paired comparisons between samples in a set of attributes Provided by the researcher LSLR Complicated experimental design

906 P. Varela, G. Ares / Food Research International 48 (2012) 893–908

quantification of sensory attributes. Therefore, descriptive analysis ismore appropriate when the objective of the sensory characterizationis to identify small differences between products or to detect differ-ences in the intensity of specific sensory attributes, as it happens inmany cases during the optimization step of the development of newproducts.

On the other hand, when non-detailed information about the sen-sory characteristics of food products is required, novel methodologiesoffer a good and quick alternative. They could be considered a valu-able alternative to get information about the sensory characteristicsof food products for companies which do not have enough time or re-sources to train a panel for evaluating a specific product, which iscommon in many small companies or developing countries. In thesesituations the cost and time required for the selecting and trainingsensory assessors might be higher than those needed to perform a con-sumer study. Novel methodologies are also useful when conductingpreliminary studies for getting information about the sensory charac-teristics of food products or when performing screening tests for the se-lection of products or conditions for the design of a larger experiment.

Moreover, sensory characterization with consumers could be use-ful for uncovering consumer perception of food products, with theirown vocabulary, which could provide valuable information duringnew food product development or when designing marketing orcommunication campaigns. In this context, holistic methodologies,free choice profiling and flash profile enable the identification ofconsumers' vocabulary to describe the sensory characteristics of theproducts. On the other hand, CATA questions and intensity scalesrely on previous studies to identify consumers' relevant terms.

The selection of a novel methodology for a particular applicationdepends on the type of assessors to be considered, practical issuesand the specific objectives of the studies. However, when workingwith consumers it would be generally easier to work with simplemethodologies such as CATA questions, open-ended questions orpivot profile. On the other hand, when a trained assessor panel isavailable and quick information about the sensory characteristics offood products is needed, the recommended approach would be toapply flash profile, sorting, projective mapping or polarized sensorypositioning due to their higher complexity. Holistic methods based

on global similarity, such as sorting and projective mapping seemmore appropriate when summarized sensory information is needed;also, they can be an interesting approach when analyzing the percep-tion of external cues like packaging information, as holistic techniquesallow a more realistic setting, closer to what a consumer would dowhen buying. Polarized sensory positioning or pivot profile is a goodoption when the aim of the study is to compare new products withknown or reference products or when the sensory characteristics ofsamples evaluated over time are to be compared.

Finally, it is important to take into account that most of the novelmethodologies for sensory characterization have been used for a rel-atively short period of time and have been applied in a limited numberof applications. For this reason, further research on the applicability,reliability and reproducibility of new approaches for sensory character-ization is still strongly needed, particularly when dealing with complexproducts.

References

Abdi, H., Valentin, D., Chollet, S., Chrea, C., Abdi, H., Valentin, D., et al. (2007). Analyzingassessors and products in sorting tasks: DISTATIS, theory and applications. FoodQuality and Preference, 18, 627–640 (Food Quality and Preference, 20, 410–417).

Adams, J., Williams, A., Lancaster, B., & Foley, M. (2007). Advantages and uses ofcheck-all-that-apply response compared to traditional scaling of attributes forsalty snacks. 7th Pangborn Sensory Science Symposium. Minneapolis, USA, 12–16August, 2007.

Albert, A., Varela, P., Salvador, A., Hough, G., & Fiszman, S. (2011). Overcoming the is-sues in the sensory description of hot served food with a complex texture. Applica-tion of QDA®, flash profiling and projective mapping using panels with differentdegrees of training. Food Quality and Preference, 22, 463–473.

Ares, G., Barreiro, C., Deliza, R., Giménez, A., & Gámbaro, A. (2010). Application ofa check-all-that-apply question to the development of chocolate milk desserts.Journal of Sensory Studies, 25, 67–86.

Ares, G., Bruzzone, F., & Giménez, A. (2011). Is a consumer panel able to reliably eval-uate the texture of dairy desserts using unstructured intensity scales? Evaluationof global and individual performance. Journal of Sensory Studies, 26, 363–370.

Ares, G., Deliza, R., Barreiro, C., Giménez, A., & Gámbaro, A. (2010). Comparison oftwo sensory profiling techniques based on consumer perception. Food Qualityand Preference, 21, 417–426.

Ares, G., Giménez, A., Barreiro, C., & Gámbaro, A. (2010). Use of an open-ended ques-tion to identify drivers of liking of milk desserts. Comparison with preference map-ping techniques. Food Quality and Preference, 21, 286–294.

907P. Varela, G. Ares / Food Research International 48 (2012) 893–908

Ares, G., Varela, P., Rado, G., & Gimenez, A. (2011a). Are consumer profiling techniquesequivalent for some product categories? The case of orange-flavoured powdereddrinks. International Journal of Food Science and Technology, 46, 1600–1608.

Ares, G., Varela, P., Rado, G., & Giménez, A. (2011b). Identifying ideal products usingthree different consumer profiling methodologies. Comparison with external pref-erence mapping. Food Quality and Preference, 22, 581–591.

Bárcenas, P., Pérez Elortondo, F. J., & Albisu, M. (2004). Projective mapping in sensoryanalysis of ewes milk cheeses: A study on consumers and trained panel perfor-mance. Food Research International, 37, 723–729.

Bécue-Bertaut, M., Álvarez-Esteban, R., & Pagès, J. (2008). Rating of products throughscores and free-text assertions: Comparing and combining both. Food Quality andPreference, 19, 122–134.

Bijmolt, T., & Wedel, M. (1995). The effects of alternative methods of collecting similar-ity data for multidimensional scaling. International Journal of Research in Marketing,12, 363–371.

Black, M. B. (1963). On formal ethnographic procedures. American Anthropologist, 65,1347–1351.

Blancher, G., Chollet, S., Kesteloot, R., Nguyen, D., Cuvelier, G., & Sieffermann, J. -M.(2007). French and Vietnamese: How do they describe texture characteristics ofthe same food? A case study with jellies. Food Quality and Preference, 18, 560–575.

Brandt, M. A., Skinner, E. Z., & Coleman, J. A. (1963). Texture profile method. Journal ofFood Science, 28, 404–409.

Bruzzone, F., Ares, G., & Giménez, A. (2012). Consumers' texture perception of milkdesserts II- Comparison with trained assessors' data. Journal of Texture Studies,43, 214–226.

Busing, F. M. T. A., Groenen, P. J. F., & Heiser, W. J. (2005). Avoiding degeneracy inmultidimensional unfolding by penalizing on the coefficient of variation. Psychometrika,70, 49–76.

Cadoret, M., Lê, S., & Pagès, J. (2009). A factorial approach for sorting task data (FAST).Food Quality and Preference, 20, 410–417.

Cairncross, S. E., & Sjöstrom, L. B. (1950). Flavour profiles: A new approach to flavourproblems. Food Technology, 4, 308–311.

Campo, E., Ballester, J., Langlois, J., Dacremont, C., & Valentin, D. (2010). Comparison ofconventional descriptive analysis and a citation frequency-based descriptive meth-od for odor profiling: An application to Burgundy Pinot noir wines. Food Qualityand Preference, 21, 44–55.

Campo, E., Do, B. V., Ferreira, V., & Valentin, D. (2008). Aroma properties of young Spanishwhite wines: A study using sorting task, list of terms and frequency of citation.Australian Journal of Grape and Wine Research, 14, 104–115.

Carrillo, E., Varela, P., & Fiszman, S. (2012a). Packaging information as a modulator ofconsumers' perception of enriched and reduced-calorie biscuits in tasting andnon-tasting tests. Food Quality and Preference, 25, 105–115.

Carrillo, E., Varela, P., & Fiszman, S. (2012b). Effects of food package information andsensory characteristics on the perception of healthiness and the acceptability ofenriched biscuits. Food Research International, 48, 209–216.

Cartier, R., Rytz, A., Lecomte, A., Poblete, E., Krystlik, J., Belin, E., et al. (2006). Sortingprocedure as an alternative to quantitative descriptive analysis to obtain a productsensory map. Food Quality and Preference, 17, 562–571.

Castura, J. C. (2009). Dopanellists donkey vote in sensory choose-all-that-apply questions?8th Pangborn Sensory Science Symposium, July 26–30, Florence.

Chollet, S., & Valentin, D. (2001). Impact of training on beer flavour perception anddescription: Are trained and untrained subjects really different. Journal of SensoryStudies, 16, 601–618.

Chollet, S., Lelièvre, Abdi, H., & Valentin, D. (2011). Sort and beer: Everything youwanted to know about the sorting task but did not dare to ask. Food Quality andPreference, 22, 507–520.

Courcoux, P., Qannari, E. M., Taylor, Y., Buck, D., & Greenhoff, K. (2012). Taxonomic freesorting. Food Quality and Preference, 23, 30–35.

Coxon, A. P. M. (1999). Sorting data: Collection and analysis. Thousand Oaks, CA: Sage.Coxon, A. P.M., Davies, P.M., & Jones, C. L. (1986). Images of social stratification. London: Sage.Dairou, V., & Sieffermann, J. -M. (2002). A comparison of 14 jams characterized by con-

ventional profile and a quick original method, flash profile. Journal of Food Science,67, 826–834.

Dehlholm, C., Brockhoff, P., Meinert, L., Aaslyng, M., & Bredi, W. (2012, December). Sortand beer: Everything you wanted to know about the sorting task but did not dareto ask. Food Quality and Preference, 26(2), 267–277.

Delarue, J., & Sieffermann, J. -M. (2004). Sensory mapping using Flash profile. Compar-ison with a conventional descriptive method for the evaluation of the flavour offruit dairy products. Food Quality and Preference, 15, 383–392.

Dooley, L., Lee, Y. -S., & Meullenet, J. -F. (2010). The application of check-all-that-apply(CATA) consumer profiling to preference mapping of vanilla ice cream and its com-parison to classical external preference mapping. Food Quality and Preference, 21,394–401.

Falahee, M., & MacRae, A. W. (1997). Perceptual variation among drinking waters: Thereliability of sorting and ranking data for multidimensional scaling. Food Qualityand Preference, 8, 389–394.

Ferrage, A., Nicod, L., & Varela, P. (2010). How does assessors' performance influencethe outcome of alternative sensory methodologies? Eurosense 2010 — A sense ofquality. Vitoria-Gasteiz, Spain, 5–8 September, 2010.

Gacula, M. C. (1997). Descriptive sensory analysis in practice. Trumbull, CT: Food andNutrition Press.

Gawel, R., Iland, P. G., & Francis, I. L. (2001). Characterizing the astringency of red wine:A case study. Food Quality and Preference, 12, 83–94.

Gower, J. C. (1975). Generalised procrustes analysis. Psychometrika, 40, 33–50.Grygorczyk, A., Lesschaeve, I., Corredig, M., & Duizer, L. (in press). Extraction of

consumer texture preferences for yogurt: Comparison of the preferred attribute

elicitation method to conventional profiling. Food Quality and Preference, http://dx.doi.org/10.1016/j.foodqual.2012.02.017.

Guàrdia, M. D., Aguiar, A. P. S., Claret, A., Arnau, J., & Guerrero, L. (2010). Sensory charac-terization of dry-cured ham using free-choice profiling. Food Quality and Preference,21, 148–155.

Husson, F., Le Dien, S., & Pagès, J. (2001). Which value can be granted to sensory pro-files given by consumers? Methodology and results. Food Quality and Preference,12, 291–296.

Jack, F. R., & Piggott, J. R. (1991). Free choice profiling in consumer research. Food Qualityand Preference, 3, 129–134.

Kelly, G. A. (1955). The psychology of personal constructs. New York: Norton.King, M. C., Cliff, M. A., & Hall, J. W. (1998). Comparison of projective mapping and sorting

data collection and multivariate methodologies for identification of similarity-of-useof snack bars. Journal of Sensory Studies, 13, 347–358.

Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537–567.Lado, J., Vicente, E., Manzzioni, A., & Ares, G. (2010). Application of a check-all-that-apply

question for the evaluation of strawberry cultivars from a breeding program. Journalof the Science of Food and Agriculture, 90, 2268–2275.

Lassoued, N., Delarue, J., Launay, B., & Michon, C. (2008). Baked product texture: Corre-lations between instrumental and sensory characterization using flash profile.Journal of Cereal Science, 48, 133–143.

Lawless, H. T., & Heymann, H. (2010). Sensory evaluation of food. Principles and prac-tices. (2nd edition). New York: Springer.

Lawless, H. T., Sheng, N., & Knoops, S. S. C. P. (1995). Multidimensional scaling ofsorting data applied to cheese perception. Food Quality and Preference, 6, 91–98.

Le Dien, S., & Pagès, J. (2003). Hierarchical multiple factor analysis: Application to thecomparison of sensory profiles. Food Quality and Preference, 14, 397–403.

Le Fur, Y., Mercurio, V., Moio, L., Blanquet, J., & Meunier, J. M. (2003). A new approach toexamine the relationships between sensory and gas chromatography–olfactometrydata using generalized procrustes analysis applied to Six French Chardonnay wines.Journal of Agriculture and Food Chemistry, 51, 443–452.

Lelièvre, M., Chollet, S., Abdi, H., & Valentin, D. (2008). What is the validity of thesorting task for describing beers? A study using trained and untrained assessors.Food Quality and Preference, 19, 697–703.

Lokki, T., Pätynen, J., Kuusinen, A., Vertanen, H., & Tervo, S. (2011). Concert hall acous-tics assessment with individually elicited attributes. Journal of the Acoustical Societyof America, 130, 835–849.

Mazzucheli, R., & Guinard, J. X. (1999). Comparison of monadic and simultaneous samplepresentation modes in a descriptive analysis of milk chocolate. Journal of SensoryStudies, 14, 235–248.

Meilgaard, M. C., Civille, G. V., & Carr, B. T. (1991). Sensory evaluation techniques (2ndedition). Boca Raton, FL: CRC Press.

Miller, D. M., Wiley, D. E., & Wolfe, R. G. (1986). Categorization methodology: An ap-proach to the collection and analyses of certain classes of qualitative information.Multivariate Behavioral Research, 21, 135–167.

Modell, S. (2005). Triangulation between case study and survey methods in manage-ment accounting research: An assessment of validity implications. ManagementAccounting Research, 16, 231–254.

Moussaoui, K. A., & Varela, P. (2010). Exploring consumer product profiling techniquesand their linkage to a quantitative descriptive analysis. Food Quality and Preference,21, 1088–1099.

Murray, J. M., Delahunty, C. M., & Baxter, I. A. (2001). Descriptive sensory analysis: Past,present and future. Food Research International, 34, 461–471.

Nestrud, M., & Lawless, H. (2008). Perceptual mapping of citrus juices using projectivemapping and profiling data from culinary professionals and consumers. Food Qualityand Preference, 19, 431–438.

Nestrud, M. A., & Lawless, H. T. (2010). Perceptual mapping of apples and cheeses usingprojective mapping and sorting. Journal of Sensory Studies, 25, 390–405.

Nicod, L., Varela, P., & Ferrage, A. (2010). Could too much training ever be a limitationin products' characterization? Oral presentation. Eurosense 2010 — A sense of quality.Vitoria-Gasteiz, Spain, 5–8 September, 2010.

Pagès, J. (2003). Recueil direct de distances sensorielles: Application à l'évaluation dedix vins blancs du Val-de-Loire. Sciences des Aliments, 23, 679–688.

Pagès, J. (2005). Collection and analysis of perceived product inter-distances usingmultiple factor analysis: Application to the study of 10 white wines from theLoire Valley. Food Quality and Preference, 16, 642–649.

Pagès, J., Cadoret, M., & Lê, S. (2010). The sorted napping: A new holistic approach insensory evaluation. Journal of Sensory Studies, 25, 637–658.

Parente, M. E., Manzoni, A. V., & Ares, G. (2011). External preference mapping of com-mercial antiaging creams based on consumers' responses to a check-all-that-applyquestion. Journal of Sensory Studies, 26, 158–166.

Perrin, L., & Pagès, J. (2009). Construction of a product space from the ultra-flash pro-filing method: Application to 10 red wines from the Loire Valley. Journal of SensoryStudies, 24, 373–395.

Perrin, L., Symoneaux, R., Maître, I., Asselin, C., Jourjon, F., & Pagès, J. (2008). Compari-son of three sensory methods for use with the Napping® procedure: Case of tenwines from Loire Valley. Food Quality and Preference, 19, 1–11.

Pfeiffer, J., & Gilbert, C. (2008). Napping by modality: A happy medium between analyticand holistic approaches. Oral communication. Sensometrics 2008.

Plaehn, D. (2012). CATA penalty/reward. Food Quality and Preference, 24, 141–152.Poinot, P., Grua-Priol, J., Arvisenet, G., Rannou, C., Semenou, M., Le Bail, A., et al. (2007).

Optimisation of HS-SPME to study representativeness of partially baked breadodorant extracts. Food Research International, 40, 1170–1184.

Poirson, E., Petiot, J. -F., & Richard, F. (2010). A method for perceptual evaluation ofproducts by naive subjects: Application to car engine sounds. International Journalof Industrial Ergonomics, 40, 504–516.

908 P. Varela, G. Ares / Food Research International 48 (2012) 893–908

Popper, P., & Heymann, H. (1996). Analyzing differences among products and panelistsby multidimensional scaling. In T. Naes, & E. Risvik (Eds.), Multivariate analysis ofdata in sensory science (pp. 159–184). Amsterdam: Elsevier.

Rasinski, K. A., Mingay, D., & Bradburn, N. M. (1994). Do respondents really mark allthat apply on self-administered questions? Public Opinion Quarterly, 58, 400–408.

Risvik, E., McEvan, J. A., Colwill, J. S., Rogers, R., & Lyon, D. H. (1994). Projective mapping:A tool for sensory analysis and consumer research. Food Quality and Preference, 5,263–269.

Risvik, E., McEwan, J. A., & Rodbotten, M. (1997). Evaluation of sensory profiling andprojective mapping data. Food Quality and Preference, 8, 63–71.

Rostaing, H., Ziegelbaum, H., Boutin, E., & Rogeaux, M. (1998). Analyse de commentaireslibres par la techniques des réseaux de segments. Fourth International Conference onthe Statistical Analysis of Textual Data, JADT'98.

Saint-Eve, A., Paài Kora, E., & Martin, N. (2004). Impact of the olfactory quality andchemical complexity of the flavouring agent on the texture of low fat stirredyogurts assessed by three different sensory methodologies. Food Quality andPreference, 15, 655–668.

Santosa, M., Abdi, H., & Guinard, J. -X. (2010). A modified sorting task to investigateconsumer perceptions of extra virgin olive oils. Food Quality and Preference, 21,881–892.

Schiffman, S. S., Reynolds, M. L., & Young, F. W. (1981). Introduction to multidimensionalscaling. New York: Academic Press.

Smyth, J. D., Dillman, D. A., Christian, L. M., & Stern, M. J. (2006). Comparing check-all andforced-choice question formats in web surveys. Public Opinion Quarterly, 70, 66–77.

Stampanoni, C. R. (1993b). Quantitative flavour profiling: An effective tool in flavourperception. Food and Marketing Technology.

Stampanoni, C. R. (1993a). The quantitative profiling technique. Perfumer Flavourist, 18,19–24.

Stone, H., & Sidel, J. L. (2004). Sensory evaluation practices. London: Elsevier Academic Press.Stone, H., Sidel, J. L., Oliver, S., Woolsey, A., & Singleton, R. C. (1974). Sensory evaluation

by quantitative descriptive analysis. Food Technology, 28(11), 24–33.Sudman, S., & Bradburn, M. B. (1982). Asking questions. San Francisco: Jossey-Bass.Symoneaux, R., Galmarini, M. V., & Mehinagic, E. (2012). Comment analysis of

consumer's likes and dislikes as an alternative tool to preference mapping. A casestudy on apples. Food Quality and Preference, 24, 59–66.

Tarea, S., Cuvelier, G., & Siefffermann, J. -M. (2007). Sensory evaluation of the texture of49 commercial apple and pear purees. Journal of Food Quality, 30, 1121–1131.

Teillet, E., Schlich, P., Urbano, C., Cordelle, S., & Guichard, E. (2010). Sensory methodol-ogies and the taste of water. Food Quality and Preference, 21, 967–976.

ten Kleij, F., & Musters, P. A. D. (2003). Text analysis of open-ended survey responses:A complementary method to preference mapping. Food Quality and Preference, 14,43–52.

Thomson, D., & Mcewan, J. (1988). An application of the repertory grid method to in-vestigate consumer perceptions of foods. Appetite, 10, 181–193.

Thuillier, B. (2007). Rôle du CO2 dans l'Appréciation Organoleptique des Champagnes—Expérimentation et Apports Méthodologiques. Thèse de l'URCA, Reims, France. Avail-able at http://www.abt-sensory-analysis.com/docs/methode_profil_pivot.pdf.

Veinand, B., Godefroy, C., Adam, C., & Delarue, J. (2011). Highlight of important productcharacteristics for consumers. Comparison of three sensory descriptive methodsperformed by consumers. Food Quality and Preference, 22, 474–485.

Williams, A., & Arnold, G. (1985). A comparison of the aromas of 6 coffees characterizedby conventional profiling, free-choice profiling and similarity scaling methods.Journal of the Science of Food and Agriculture, 36, 204–214.

Worch, T., Lê, S., & Punter, P. (2010). How reliable are the consumers? Comparison ofsensory profiles from consumers and experts. Food Quality and Preference, 21,309–318.