1.1 The standard approach to model preferences in consumer research

Conjoint analysis is based on seminal work from Luce and Tukey (Reference Luce and Tukey1964). Green developed the method further and adapted it to marketing and product-development problems (e.g, Green & Rao, Reference Green and Rao1971; Green & Wind, Reference Green and Wind1975). Today, conjoint analysis is regarded as the most prevalent tool to measure consumer preferences (Wittink & Cattin, Reference Wittink and Cattin1989; Wittink, Vriens & Burhenne, Reference Wittink, Vriens and Burhenne1994). In a survey among market research institutes, 65% of the institutes indicated having used conjoint analysis within the last 12 months, and growing usage frequency was forecasted (Hartmann & Sattler, Reference Hartmann and Sattler2002). Conjoint analysis is used to analyze how different features of products contribute to consumers’ preferences for these products. This is accomplished by decomposing the preference for the whole product into partitions assigned to the product’s constituent features. The established way to collect preference data is the full-profile method. Product profiles consisting of all relevant product features are presented to respondents. These profiles are evaluated either by rating or ranking or by discrete choice (i.e., buy or non-buy) decisionsFootnote ¹.

The assumption behind the decompositional nature of conjoint analysis is that people weigh and add all available pieces of product information, thus deriving a global utility value for each option as the sum of partworth utilities. Options with higher utility are preferred — either deterministically or probabilistically — over options with lower utility. Clearly, this assumption rests on traditional conceptions of what constitutes rational decision making. Homo economicus is assumed to carefully consider all pieces of information and to integrate them into some common currency, such as expected utility, following a complex weighting scheme.

For rating- and ranking-based conjoint methods,Footnote ² the basic weighted additive model (WADD) can be stated as follows:

(1)

with

r_k = response for option k;

β_jm = partworth utility of level m of attribute j;

x_jm = 1 if option k has level m on attribute j;

else x_jm = 0; and

ε_k = error term for response for option k.

The partworth utilities are estimated, usually by applying multiple regression, such that the sum of squares between empirically observed responses r_k (ratings or rankings) and estimated responses r_i is minimal.

1.2 Simple decision heuristics

The traditional view of rational decision making as utility maximization has been challenged in the judgment and decision making literature. Many authors propose alternative accounts of human decision making processes and argue that people are equipped with a repertoire of decision strategies from which to select depending on the decision context (e.g., Beach & Mitchell, Reference Beach and Mitchell1978; Einhorn, Reference Einhorn1971; Gigerenzer, Todd, & the ABC Research Group, Reference Gigerenzer and Todd1999; Payne, Reference Payne1976, Reference Payne1982; Payne, Bettman, & Johnson, Reference Payne, Bettman and Johnson1988, Reference Payne, Bettman and Johnson1993; Rieskamp & Otto, Reference Rieskamp and Otto2006; Svenson, Reference Svenson1979). According to Payne et al. (Reference Payne, Bettman and Johnson1988, Reference Payne, Bettman and Johnson1993), decision makers choose strategies adaptively in response to different task demands, and often apply simplified shortcuts — heuristics — that allow fast decisions with acceptable losses in accuracy. Moreover, simple heuristics are often more or at least equally accurate in predicting new data compared to more complex strategies (e.g., Czerlinski, Gigerenzer & Goldstein, Reference Czerlinski, Gigerenzer, Goldstein, Gigerenzer and Todd1999; Gigerenzer, Czerlinski & Martignon, Reference Gigerenzer, Czerlinski, Martignon, Shanteau, Mellers and Schum1999). The explanation is that simple heuristics are more robust, extracting only the most important and reliable information from the data, while complex strategies that weigh all pieces of evidence extract much noise, resulting in large accuracy losses when making predictions for new data — a phenomenon called overfitting (Pitt & Myung, Reference Pitt and Myung2002).

Lexicographic strategies are a prominent category of simple heuristics. A well-known example is Take The Best (TTB; Gigerenzer & Goldstein, Reference Gigerenzer and Goldstein1996), for inferring which of two alternatives has a higher criterion value by searching sequentially through cues in the order of their validity until one discriminating cue is found. The alternative with the positive cue value is selected. TTB is “noncompensatory” because a cue cannot be outweighed by any combination of less valid cues, in contrast to “compensatory” strategies, which integrate cue values (e.g., the WADD model). Applied to a consumer choice context, a lexicographic heuristic would prefer a product that is superior to another product on the most important aspectFootnote ³ for which the two options have different values, regardless of the aspects that follow in the aspect hierarchy.

1.3 Inferring lexicographic decision processes

Choices can be forecast by linear compensatory models even though the underlying decision process has a different structure (e.g., Einhorn, Kleinmuntz & Kleinmunutz, Reference Einhorn, Kleinmuntz and Kleinmuntz1979). A WADD model, for example, can theoretically reproduce a non-compensatory decision process if, in the ordered set of weights, each weight is larger than the sum of all weights to come (e.g., aspect weights of 2^1-n with n = 1, …, N and N = number of aspects; Martignon & Hoffrage, Reference Martignon, Hoffrage, Gigerenzer and Todd1999, Reference Martignon and Hoffrage2002). Despite its flexibility in assigning weights, however, Yee et al. (Reference Yee, Dahan, Hauser and Orlin2007 showed in Monte Carlo simulations that WADD models fall short of capturing the non-compensatory preference structure and are outperformed by lexicographic models when the choice is made in a perfectly non-compensatory fashion. Moreover, the goal is not only to generate high prediction performance but also insight into the process steps of decision making. Although conclusions from consumer self-reports and process tracing studies are limited (see below), several such studies suggest that only a minority of participants use a weighted additive rule, thus questioning the universal application of conjoint analysis (Denstadli & Lines, Reference Denstadli and Lines2007; Ford, Schmitt, Schechtman, Hults & Doherty, Reference Ford, Schmitt, Schechtman, Hults and Doherty1989).

Most users of conjoint models are well aware of their status as “as if” models (Dawkins, Reference Dawkins1976), and do not claim to describe the underlying cognitive process but only aim to predict the outcome. Consequently, many researchers call for psychologically more informed models (e.g., Bradlow, Reference Bradlow2005; Louviere, Eagle & Cohen, Reference Louviere, Eagle and Cohen2005). However, these rightful claims suffer from the lack of data analysis tools that estimate heuristics based on preference data. Self-reports (e.g., Denstadli & Lines, Reference Denstadli and Lines2007) are an obvious tool for tracking decision strategies but have questionable validity (Nisbett & Wilson, Reference Nisbett and Wilson1977). A more widely accepted way to deduce heuristics from people’s responses are process tracing techniques, such as eye tracking and mouse tracking (e.g., Payne et al., Reference Payne, Bettman and Johnson1988, Reference Payne, Bettman and Johnson1993; see Ford et al., Reference Ford, Schmitt, Schechtman, Hults and Doherty1989, for a review), or response time analyses (e.g., Bröder & Gaissmeier, Reference Bröder and Gaissmaier2007). However, these techniques are very expensive for examining large samples, as often required in consumer research. Moreover, data collection methods such as information boards tend to interfere with the heuristics applied and might induce a certain kind of processing (Billings & Marcus, Reference Billings and Marcus1983). Finally, it is unclear how process measures can be integrated into mathematical prediction models, as the same processing steps can be indicative of several strategies (Svenson, Reference Svenson1979).

In inference problems where the task is to pick the correct option according to some objective external criterion, such as inferring which of two German cities is larger, heuristics can be deduced by using datasets with known structure — in this case, a data set of German cities including their description in terms of features such as existence of an exposition site or a soccer team in the major league (Gigerenzer & Goldstein, Reference Gigerenzer and Goldstein1996). Based on the data set, one can compute the predictive validity of the different features, or cues, and thus derive cue weights. This way, competing inference strategies that process these cues in compensatory or noncompensatory fashions, including their predictions, can be specified a priori. These predictions can then be compared to the observed inferences that participants have made and the strategy that predicts most of these responses can be determined (see, e.g., Bröder, Reference Bröder2000; Bröder & Schiffer, Reference Bröder and Schiffer2003b; Rieskamp & Hoffrage, Reference Rieskamp, Hoffrage, Gigerenzer and Todd1999, Reference Rieskamp and Hoffrage2008).

In preferential choice, by contrast, the individual attribute weighting, or ordering structure, does not follow some objective outside criterion but depends on subjective preference and has to be deduced in addition to the decision strategy people use. Standard conjoint analyses estimate individual weighting structure assuming a weighted additive model, as laid out above. Approaches assuming alternative models are rare. Gilbride and Allenby (Reference Gilbride and Allenby2004) model choices in a two-stage-process and allow for compensatory and noncompensatory screening rules in the first stage. Elrod et al. (Reference Elrod, Johnson and White2004) suggest a hybrid model that integrates compensatory decision strategies with noncompensatory conjunctive and disjunctive heuristics. However, these approaches are basically modifications of the standard WADD model, allowing for noncompensatory weighting and conjunctions and disjunctions of aspects. This model, however, is not a valid representation of human decision making; its flexibility not only makes psychologically implausible computational demands, but also technically requires huge processing capacity. In contrast, the greedoid algorithm we focus on is intriguingly simple. It incorporates the principles of lexicography and noncompensatoriness rather than just adapting weighting schemes to imitate the output of lexicographic heuristics.

Yee et al. (Reference Yee, Dahan, Hauser and Orlin2007Footnote ⁴ developed the greedoid algorithm for deducing lexicographic processes from observed preference data, applicable to rating, ranking and choice alike. The algorithm rests on the assumption that the aspects of different options are processed lexicographically. It discloses the aspect sorting order that best replicates the observed (partial) preference hierarchy of options. Generally, the algorithm can be used to estimate various lexicographic heuristics. By introducing specific restrictions, aspects can be treated as acceptance or elimination criteria to model the well-known elimination-by-aspects heuristic (Tversky, Reference Tversky1972). For our purposes, we will implement the most flexible lexicographic-by-aspects (LBA) process that allows aspects from different attributes to be freely ranked as acceptance or elimination criteria.

The lexicographic-by-aspects process can be illustrated by a simple example. Given a choice between holiday options differing in travel location, with the three aspects Spain, Italy, and France, and means of transport, with the two aspects plane and car, a person may express the following preference order:

1. (1) Spain by plane;

2. (2) Spain by car;

3. (3) France by plane;

4. (4) Italy by plane;

5. (5) France by car;

6. (6) Italy by car.

The person’s preferences in terms of location and transport are quite obvious. She prefers Spain regardless of how to get there. Means of transport becomes determining when considering other countries, with a preference for flying. A restrictive lexicographic-by-attributes process could not predict this preference order without mistake. Using country as first sorting criteria, with the aspect order [Spain — France — Italy], would produce one mistake (i.e., wrong order for options 4 and 5), and sorting by means of transport, with the aspect order [plane — car], would produce two mistakes (i.e., wrong order for options 2 and 3 as well as 2 and 4). A lexicographic-by-aspects process, in contrast, by allowing aspects from different attributes to be ordered after each other, can predict the observed preference ranking perfectly. This way, the sorting order [Spain — plane — France] becomes possible, reproducing the observed order without mistakes.Footnote ⁵ This is the result that would be produced by the lexicographic-by-aspects implementation of the greedoid algorithm.

The algorithm is an instance of dynamic programming with a forward recursive structure. As goodness-of-fit criterion, a violated-pairs metric is used, counting the number of pairs of options that are ranked inconsistently in the observed and the predicted preference order. Basically, the algorithm creates optimal aspect orders for sorting alternatives — for example, various product profiles — by proceeding step-by-step from small sets of aspects to larger ones until the alternatives are completely sorted. First, the algorithm determines the inconsistencies that would be produced if the alternatives were ordered by one single aspect. This is repeated for each aspect. Then, starting from a set size of n = 2 aspects, the algorithm determines the best last aspect within each set, before moving forward to the next larger set size, and so forth until the set size comprises enough aspects to rank all options. Maximally, all 2^N possible sets of N aspects are created and searched through (only if the set of profiles cannot be fully sorted by fewer aspects than N). This sequential procedure exploits the fact that the number of inconsistencies induced by adding an aspect to an existing aspect order depends only on the given aspects within the set and is independent from the order of those aspects. Compared to exhaustive enumeration of all N! possible aspect orders, dimensionality is reduced to the number of possible unordered subsets, 2^N, decreasing running time by a factor of the order of 10⁹ (Yee et al., Reference Yee, Dahan, Hauser and Orlin2007).

In an empirical test of the new algorithm, participants had to indicate their ordinal preferences for 32 SmartPhones (Yee et al., Reference Yee, Dahan, Hauser and Orlin2007). The products were described by 7 attributes; 3 attributes had 4 levels, 4 attributes had 2 levels, resulting in 20 aspects.Footnote ⁶ The Greedoid approach was compared to two methods based on the weigh-and-add assumption: hierarchical Bayes ranked logit (e.g., Rossi & Allenby, Reference Rossi and Allenby2003) and linear programming (Srinivasan, Reference Srinivasan, Aronson and Zionts1998).Footnote ⁷ Overall, the greedoid approach proved superior to both benchmark models in predicting hold-out data,Footnote ⁸ and thus seems to represent a viable alternative to standard estimation models. We aim to find out more about the conditions under which this methodology can be fruitfully applied, as well as about its limitations.

1.4 External and internal factors affecting strategy selection

According to the adaptive-strategy-selection view (Payne et al., Reference Payne, Bettman and Johnson1993), people choose different strategies depending on characteristics of the decision task. For instance, simple lexicographic heuristics predict decisions well when retrieval of information on the available options is associated with costs, such as having to pay for information acquisition (Bröder, Reference Bröder2000; Newell & Shanks, Reference Newell and Shanks2003; Newell, Weston & Shanks, Reference Newell, Weston and Shanks2003; Rieskamp & Otto, Reference Rieskamp and Otto2006), deciding under time pressure (Rieskamp & Hoffrage, Reference Rieskamp, Hoffrage, Gigerenzer and Todd1999, Reference Rieskamp and Hoffrage2008), or having to retrieve information from memory (Bröder & Schiffer, Reference Bröder and Schiffer2003a). The exact same studies, however, show that, when participants are given more time and can explore information for free, weighted-additive models usually outperform lexicographic strategies in predicting people’s decision making. Information integration seems to be a default applied by most people when faced with new decision tasks unless circumstances become unfavorable for extensive information search (Dieckmann & Rieskamp, Reference Dieckmann and Rieskamp2007).

Additionally, the mode in which options are presented as well as the required response mode affect strategy selection (for a review, see Payne, Reference Payne1982). Simultaneous display of options facilitates attribute-wise comparisons between alternatives.Footnote ⁹ In contrast, sequential presentation promotes alternative-wise, and thus more holistic, additive processing, as attribute-wise comparisons between options become difficult and would require the retrieval of previously seen options from memory or the application of internal comparison standards per attribute (Dhar, Reference Dhar1996; Lindsay & Wells, Reference Lindsay and Wells1985; Nowlis & Simonson, Reference Nowlis and Simonson1997; Schmalhofer & Gertzen, Reference Schmalhofer, Gertzen, Brehmer, Jungermann, Lourens and Sevlon1986; Tversky, Reference Tversky1969). Regarding response mode effects, Westenberg and Koele (Reference Westenberg and Koele1992) propose that the more differentiated the required response, the more differentiated and compensatory the evaluation of the alternatives. Following this proposition, ranking — which additionally is associated with simultaneous presentation of options — requires ordinal comparisons between options and is thus supposed to foster lexicographic processing, while rating requires evaluating one option at a time on a metric scale, which should promote compensatory processing. Indeed, there is empirical evidence that people use strategies that directly compare alternatives, such as elimination-by-aspects, more often in choice than in rating (e.g., Billings & Scherer, Reference Billings and Scherer1988; Schkade & Johnson, Reference Schkade and Johnson1989). Note that ranking tasks are often posed as repeated choice tasks, requiring participants to sequentially choose the most preferred option from a set that gets smaller until only the least preferred option remains. For such ranking tasks, we therefore expect similar differences in comparison to rating tasks as for choice tasks, and anticipate higher predictive accuracy of a lexicographic model relative to a compensatory model.

This prediction agrees with another result reported by Yee et al. (Reference Yee, Dahan, Hauser and Orlin2007. Besides their own ranking data, they re-analyzed rating data by Lenk, DeSarbo, Green and Young (Reference Lenk, DeSarbo, Green and Young1996). Unlike in the ranking case, the Greedoid approach produced slightly lower predictive accuracy for hold-out data than a hierarchical Bayes ranked logit model. However, the two studies differed in several aspects, so the performance difference cannot be unambiguously attributed to the difference in respondents’ preference elicitation task.

Among the internal factors affecting the selection of decision strategies are prior knowledge and expertise (Payne et al., Reference Payne, Bettman and Johnson1993). Experts tend to apply more selective information processing than non-experts (e.g., Bettman & Park, Reference Bettman and Park1980; see Shanteau, Reference Shanteau1992, for an overview). Shanteau (Reference Shanteau1992) reports results demonstrating that experts are more able than non-experts to ignore irrelevant information. Ettenson, Shanteau, and Krogstad (Reference Ettenson, Shanteau and Krogstad1987) found that professional auditors weighted cues far more unequally, relying primarily on one cue, than students. Similarly, in a study on rating mobile phones, participants that reported having used a weighted additive strategy had the lowest scores on subjective and objective product category knowledge compared to other strategy users (Denstadli & Lines, Reference Denstadli and Lines2007). In short, experts seem to be better able to prioritize attributes, thus possibly giving rise to a clear attribute hierarchy with noncompensatory alternative evaluation. In contrast, non-experts might be less sure about which attribute is most important and therefore apply a risk diffusion strategy by integrating different pieces of information.

To summarize, we compared two models of decision strategies — weighted additive and lexicographic — in terms of their predictive accuracy for ranking versus rating data. Our hypothesis was that, relative to compensatory strategies, lexicographic processes predict participants’ preferences better in ranking than in rating tasks. Our second hypothesis was that, regardless of the required response, predictive accuracy of the lexicographic model is higher for experts than for non-experts, because experts are better able to prioritize attributes (Shanteau, Reference Shanteau1992).