Hostname: page-component-76fb5796d-25wd4 Total loading time: 0 Render date: 2024-04-26T09:16:15.665Z Has data issue: false hasContentIssue false
  • English
  • Français

inML Kit: empowering the prototyping of ML-enhanced products by involving designers in the ML lifecycle

Part of: Perspectives on Data-Enabled Design - Design meets Data Science

Published online by Cambridge University Press:  09 February 2022

Lingyun Sun ,
Yuyang Zhang ,
Zhuoshu Li ,
Zihong Zhou  and
Zhibin Zhou Open the ORCID record for Zhibin Zhou [Opens in a new window]
Lingyun Sun
Affiliation:
State Key Laboratory of CAD&CG at Zhejiang University, Hangzhou, China Alibaba-Zhejiang University Joint Institute of Frontier Technologies, Zhejiang University, Hangzhou, China
Yuyang Zhang
Affiliation:
Alibaba-Zhejiang University Joint Institute of Frontier Technologies, Zhejiang University, Hangzhou, China
Zhuoshu Li
Affiliation:
State Key Laboratory of CAD&CG at Zhejiang University, Hangzhou, China Alibaba-Zhejiang University Joint Institute of Frontier Technologies, Zhejiang University, Hangzhou, China
Zihong Zhou
Affiliation:
Alibaba-Zhejiang University Joint Institute of Frontier Technologies, Zhejiang University, Hangzhou, China
Zhibin Zhou*
Affiliation:
Alibaba-Zhejiang University Joint Institute of Frontier Technologies, Zhejiang University, Hangzhou, China
*
Author for correspondence: Zhibin Zhou, E-mail: zhibinzhou@zju.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

Machine learning (ML) is increasingly used to enhance intelligent products in the field of product design. However, ML has a never-ending lifecycle in which its capabilities and technical properties iteratively change as new annotated data are utilized. The never-ending lifecycle of ML (which includes data annotation, model training, and other steps) has led to challenges to the prototyping of ML-enhanced products and requires a high level of ML literacy in designers. To facilitate the prototyping of ML-enhanced products and improve the ML literacy of designers, we draw inspiration from a design method called Material Lifecycle Thinking (MLT), which regards ML as a continuously evolving design material. Based on the MLT, we proposed a cyclical prototype workflow and developed inML Kit, a toolkit enabling designers to make functional ML prototypes and improve ML literacy by involving them in the never-ending ML lifecycle. The toolkit was designed, iterated, and implemented through the participatory design process with experienced designers in this field. We evaluated inML Kit by conducting a controlled user study where our toolkit was compared with Google AIY. The evaluation results imply that our inML Kit helps designers to make functional ML prototypes while improving their ML literacy.

Keywords

Artificial intelligencedesignlifecyclemachine learningprototype tool
Type
Research Article
Information
AI EDAM , Volume 36 , 2022 , e8
Check for updates
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abadi, M, Barham, P, Chen, J, Chen, Z, Davis, A, Dean, J, Devin, M, Ghemawat, S, Irving, G, Isard, M, Kudlur, M, Levenberg, J, Monga, R, Moore, S, Murray, DG, Steiner, B, Tucker, P, Vasudevan, V, Warden, P and Zheng, X (2016) TensorFlow: A System for Large-Scale Machine Learning, pp. 265–283. Available at https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi.Google Scholar
Alpaydin, E (2020) Introduction to Machine Learning. Cambridge, MA: MIT Press.Google Scholar
Amershi, S, Cakmak, M, Knox, WB, Kulesza, T and Lau, T (2013) IUI workshop on interactive machine learning. Proceedings of the Companion Publication of the 2013 International Conference on Intelligent User Interfaces Companion, pp. 121–124. doi:10.1145/2451176.2451230CrossRefGoogle Scholar
Amershi, S, Cakmak, M, Knox, WB and Kulesza, T (2014) Power to the people: the role of humans in interactive machine learning. AI Magazine 35, 105–120. doi:10.1609/aimag.v35i4.2513CrossRefGoogle Scholar
Amershi, S, Inkpen, K, Teevan, J, Kikin-Gil, R, Horvitz, E, Weld, D, Vorvoreanu, M, Fourney, A, Nushi, B, Collisson, P, Suh, J, Iqbal, S and Bennett, PN (2019) Guidelines for human-AI interaction. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ‘19, pp. 1–13. doi:10.1145/3290605.3300233CrossRefGoogle Scholar
Camburn, B, Viswanathan, V, Linsey, J, Anderson, D, Jensen, D, Crawford, R, Otto, K and Wood, K (2017) Design prototyping methods: state of the art in strategies, techniques, and guidelines. Design Science 3. doi:10.1017/dsj.2017.10CrossRefGoogle Scholar
Cerejo, J and Carvalhais, M (2019) Design after the Rise of AI-Driven Services: Learning from Literature Review. DIGICOM International Conference on Digital Design & Communication, Barcelos, Portugal.Google Scholar
Choudhury, MD and Kiciman, E (2018) Integrating artificial and human intelligence in complex, sensitive problem domains: experiences from mental health. AI Magazine 39, 69–80. doi:10.1609/aimag.v39i3.2815CrossRefGoogle Scholar
Davis, FD (1985) A Technology Acceptance Model for Empirically Testing New End-user Information Systems: Theory and Results (Thesis). Massachusetts Institute of Technology. Available at https://dspace.mit.edu/handle/1721.1/15192.Google Scholar
Domingos, P (2012) A few useful things to know about machine learning. Communications of the ACM 55, 78–87. doi:10.1145/2347736.2347755CrossRefGoogle Scholar
Dove, G, Halskov, K, Forlizzi, J and Zimmerman, J (2017) UX design innovation: challenges for working with machine learning as a design material. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 278–288. doi:10.1145/3025453.3025739CrossRefGoogle Scholar
Fails, J and Olsen, D (2003) Interactive machine learning. International Conference on Intelligent User Interfaces, Proceedings IUI. doi:10.1145/604045.604056CrossRefGoogle Scholar
Fiebrink, R and Cook, PR (2010) The Wekinator: A System for Real-Time, Interactive Machine Learning in Music. Proceedings of The Eleventh International Society for Music Information Retrieval Conference (IS-MIR 2010).Google Scholar
Google (2019) Do-it-yourself artificial intelligence. Available at https://aiyprojects.withgoogle.com/.Google Scholar
Grecu, DL and Brown, DC (1998) Dimensions of machine learning in design. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 12, 117–121. doi:10.1017/S0890060498122047CrossRefGoogle Scholar
Intel (2018) Intel® Movidius™ Neural Compute Stick. Intel. Available at https://www.intel.com/content/www/us/en/develop/articles/intel-movidius-neural-compute-stick.html.Google Scholar
Kelley, JF (1984) An iterative design methodology for user-friendly natural language office information applications. ACM Transactions on Information Systems 2, 26–41. doi:10.1145/357417.357420CrossRefGoogle Scholar
Kleinsmann, M and Bhömer, M (2020) The (New) roles of prototypes during the co-development of digital product service systems. International Journal of Design 14, 15.Google Scholar
Krueger, RA (1998) Designing and Conducting Focus Group Interviews: Notes. Minneapolis, MN: University of Minnesota.Google Scholar
Ledo, D, Houben, S, Vermeulen, J, Marquardt, N, Oehlberg, L and Greenberg, S (2018) Evaluation strategies for HCI toolkit research. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–17. doi:10.1145/3173574.3173610CrossRefGoogle Scholar
Lim, Y-K, Stolterman, E and Tenenberg, J (2008) The anatomy of prototypes: prototypes as filters, prototypes as manifestations of design ideas. ACM Transactions on Computer-Human Interaction 15, 7.1–7.27. doi:10.1145/1375761.1375762CrossRefGoogle Scholar
Long, D and Magerko, B (2020) What is AI literacy? Competencies and design considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–16. doi:10.1145/3313831.3376727CrossRefGoogle Scholar
Mierswa, I, Wurst, M, Klinkenberg, R, Scholz, M and Euler, T (2006) YALE: rapid prototyping for complex data mining tasks. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 935–940. doi:10.1145/1150402.1150531CrossRefGoogle Scholar
Miorandi, D, Sicari, S, De Pellegrini, F and Chlamtac, I (2012) Internet of things: vision, applications and research challenges. Ad Hoc Networks 10, 1497–1516. doi:10.1016/j.adhoc.2012.02.016CrossRefGoogle Scholar
Mitchell, TM (1999) Machine learning and data mining. Communications of the ACM 42, 30–36.CrossRefGoogle Scholar
Muller, MJ and Kuhn, S (1993) Participatory design. Communications of the ACM 36, 24–28. doi:10.1145/153571.255960CrossRefGoogle Scholar
Nachar, N (2008) The Mann-Whitney U: a test for assessing whether two independent samples come from the same distribution. Tutorials in Quantitative Methods for Psychology 4. doi:10.20982/tqmp.04.1.p013CrossRefGoogle Scholar
Patel, K, Fogarty, J, Landay, JA and Harrison, B (2008) Investigating statistical machine learning as a tool for software development. Proceeding of the Twenty-Sixth Annual CHI Conference on Human Factors in Computing Systems - CHI ‘08, p. 667. doi:10.1145/1357054.1357160CrossRefGoogle Scholar
Polyakov, EV, Mazhanov, MS, Rolich, AY, Voskov, LS, Kachalova, MV and Polyakov, SV (2018) Investigation and development of the intelligent voice assistant for the internet of things using machine learning. 2018 Moscow Workshop on Electronic and Networking Technologies (MWENT), pp. 1–5. doi:10.1109/MWENT.2018.8337236CrossRefGoogle Scholar
Raspberry Pi Foundation (2021) Teach, Learn, and Make with Raspberry Pi. Raspberry Pi. Available at https://www.raspberrypi.org/.Google Scholar
Ribeiro, M, Grolinger, K and Capretz, MAM (2015) MLaas: machine learning as a service. 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pp. 896–902. doi:10.1109/ICMLA.2015.152CrossRefGoogle Scholar
Richert, W and Coelho, LP (2013) Building machine learning systems with Python: master the art of machine learning with Python and build effective machine learning systems with this intensive hands-on guide. Packt Publ.Google Scholar
Shapiro, SS and Wilk, MB (1965) An analysis of variance test for normality (complete samples). Biometrika 52, 591–611. doi:10.1093/biomet/52.3-4.591CrossRefGoogle Scholar
Spencer, D and Warfel, T (2004) Card Sorting: A Definitive Guide. Boxes and Arrows. Available at https://boxesandarrows.com/card-sorting-a-definitive-guide/.Google Scholar
Sun, L, Zhou, Z, Wu, W, Zhang, Y, Zhang, R and Xiang, W (2020) Developing a toolkit for prototyping machine learning-empowered products: the design and evaluation of ML-rapid. International Journal of Design 14, 01.Google Scholar
Swain, J (2018) A Hybrid Approach to Thematic Analysis in Qualitative Research: Using a Practical Example. SAGE Publications Ltd. doi:10.4135/9781526435477.Google Scholar
van Allen, P (2018) Prototyping ways of prototyping AI. Interactions 25, 46–51. doi:10.1145/3274566CrossRefGoogle Scholar
van der Lelie, C (2006) The value of storyboards in the product design process. Personal and Ubiquitous Computing 10, 159–162. doi:10.1007/s00779-005-0026-7CrossRefGoogle Scholar
Verlinden, J and Horváth, I (2009) Analyzing opportunities for using interactive augmented prototyping in design practice. AI EDAM 23, 289–303. doi:10.1017/S0890060409000250Google Scholar
Yang, Q (2017) The role of design in creating machine-learning-enhanced user experience. 2017 AAAI Spring Symposia.Google Scholar
Yang, Q (2018) Machine learning as a UX design material: how can we imagine beyond automation, recommenders, and reminders? 2018 AAAI Spring Symposium.Google Scholar
Yang, Q, Banovic, N and Zimmerman, J (2018 a) Mapping machine learning advances from HCI research to reveal starting places for design innovation. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ‘18, pp. 1–11. doi:10.1145/3173574.3173704CrossRefGoogle Scholar
Yang, Q, Scuito, A, Zimmerman, J, Forlizzi, J and Steinfeld, A (2018 b) Investigating how experienced UX designers effectively work with machine learning. Proceedings of the 2018 on Designing Interactive Systems Conference 2018 - DIS ‘18, pp. 585–596. doi:10.1145/3196709.3196730CrossRefGoogle Scholar
Yang, Q, Steinfeld, A, Rosé, C and Zimmerman, J (2020) Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–13. doi:10.1145/3313831.3376301CrossRefGoogle Scholar
Zhou, Z, Sun, L, Zhang, Y, Liu, X and Gong, Q (2020) ML lifecycle canvas: designing machine learning-empowered UX with material lifecycle thinking. Human–Computer Interaction, 1–25. doi:10.1080/07370024.2020.1736075Google Scholar
Zhu, H, Yu, B, Halfaker, A and Terveen, L (2018) Value-sensitive algorithm design: method, case study, and lessons. Proceedings of the ACM on Human-Computer Interaction 2, 23. Article No. 194. doi:10.1145/3274463CrossRefGoogle Scholar