In previous chapters we introduced a variety of different recommendation techniques and systems developed by researchers or already in use on commercial platforms. In the future, many new techniques will claim to improve prediction accuracy in specific settings or offer new ways for users to interact with each other, as in social networks and Web 2.0 platforms.

Therefore, methods for choosing the best technique based on the specifics of the application domain, identifying influential success factors behind different techniques, or comparing several techniques based on an optimality criterion are all required for effective evaluation research. Recommender systems have traditionally been evaluated using offline experiments that try to estimate the prediction error of recommendations using an existing dataset of transactions. Some point out the limitations of such methods, whereas others argue that the quality of a recommender system can never be directly measured because there are too many different objective functions. Nevertheless, the widespread use of recommender systems makes it crucial to develop methods to realistically and accurately assess their true performance and effect on the users. This chapter is therefore devoted to discussing existing evaluation approaches in the light of empirical research methods from both the natural and social sciences, as well as presenting different evaluation designs and measures that are well accepted in the research community.