Are Data-Driven Personas Considered Harmful? : Diversifying User Understandings With More Than Algorithms

Joni O Salminen; Soon-gyo Jung; Bernard J. Jansen

doi:10.21153/psj2021vol7no1art1236

Authors

Joni O Salminen Hamad Bin Khalifa University, Qatar & University of Turku, Finland
Soon-gyo Jung Hamad Bin Khalifa University, Qatar
Bernard J. Jansen Hamad Bin Khalifa University, Qatar

DOI:

https://doi.org/10.21153/psj2021vol7no1art1236

Keywords:

personas, data-driven personas, harm

Abstract

In this work, we build on research on data-driven personas to present what might be “wrong with them”. From wrong assumptions by the client and wrong applications of methods to imbalanced, messy, or superficial data; a lack of communication regarding how these personas are created; and issues with usability, there are a plethora of issues that plague data-driven personas. We conclude by contemplating whether data-driven personas are even worthwhile and, if they are, then what are some of the immediate remedies required from the human-computer interaction community to make data-driven personas a viable tool for user understanding.

Downloads

Download data is not yet available.

References

Adlin T & Pruitt J 2009, ‘Putting personas to work: Using data-driven personas to focus product planning, design, and development; in Sears A and Jacko JA (eds) Human-Computer Interaction: Development Process, CRC Press, New York, pp. 95–120.

— 2010, The Essential Persona Lifecycle: Your Guide to Building and Using Personas. 1st ed. Morgan Kaufmann Publishers Inc. San Francisco.

An J, Kwak H, Jung S-G, Salminen J & Jansen BJ 2018, ‘Customer segmentation using online platforms: isolating behavioral and demographic segments for persona creation via aggregated user data’, Social Network Analysis and Mining, vol. 8, no. 1: 54. https://doi.org/10.1007/s13278-018-0531-0.

An J, Kwak H, Salminen J, Jung S-G & Jansen BJ 2018, ‘Imaginary People Representing Real Numbers: Generating Personas from Online Social Media Data’, ACM Transactions on the Web (TWEB), vol. 12, no. 4: 27. https://doi.org/10.1145/3265986.

Anvari F & Tran HMT 2013, ‘Persona ontology for user centred design professionals’, in The ICIME 4th International Conference on Information Management and Evaluation, Ho Chi Minh City, Vietnam, 2013, pp. 35–44.

Anvari F, Richards D, Hitchens M & Tran H 2019, ‘Teaching user centered conceptual design using cross-cultural personas and peer reviews for a large cohort of students’, in Proceedings of the 41st International Conference on Software Engineering: Software Engineering Education and Training, IEEE Press, Piscataway, NJ, 2019, pp. 62–73.

Blažica B 2014, Use of UX and HCI tools among start-ups. Working paper. Ljubljana, Slovenia: XLAB Research.

Blomquist A & Arvola M 2002, ‘Personas in action: Ethnography in an interaction design team’, in Proceedings of the second Nordic conference on Human-computer interaction, ACM Press, New York, pp. 197–200.

Bødker S, Christiansen E, Nyvang T & Zander P 2012, ‘Personas, people and participation: challenges from the trenches of local government’, in Proceedings of the 12th Participatory Design Conference on Research Papers: Volume 1 - PDC ’12, ACM Press, Roskilde, p. 91. ACM Press. https://doi.org/10.1145/2347635.2347649.

Brickey J, Walczak S & Burgess T 2012, ‘Comparing Semi-Automated Clustering Methods for Persona Development’, IEEE Transactions on Software Engineering, vol. 38, no. 3, pp. 537–546. https://doi.org/10.1109/TSE.2011.60.

Carroll JM 1997, ‘Chapter 17 - Scenario-Based Design’, in Helander MG, Landauer TK, and Prabhu PV (eds) Handbook of Human-Computer Interaction (Second Edition). Amsterdam: North-Holland, pp. 383–406. https://doi.org/10.1016/B978-044481862-1.50083-2.

Chapman C & Feit EM 2019, R For Marketing Research and Analytics. Springer.

Chapman C & Milham RP 2006, ‘The Personas’ New Clothes: Methodological and Practical Arguments against a Popular Method’, in Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1 October 2006, pp. 634–636. https://doi.org/10.1177/154193120605000503.

Chapman C, Love E, Milham RP, ElRif P & Alford J 2008, ‘Quantitative Evaluation of Personas as Information’, in Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1 September 2008, pp. 1107–1111. https://doi.org/10.1177/154193120805201602.

Chapman C, Krontiris K & Webb J 2015, ‘Profile CBC: Using Conjoint Analysis for Consumer Profiles’, in Sawtooth Software Conference Proceedings, Google Research. Available at: https://research.google.com/pubs/archive/44167.pdf.

Cooper A 1999, The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity. 1st ed. Sams-Pearson Education, Indianapolis.

De Voil N 2010, Personas considered harmful. Industry report. London, UK: De Voil Consulting. Available at: http://www.devoil.com/papers/PersonasConsideredHarmful.pdf. Retrieved November 2019.

Diaz M, Johnson I, Lazar A, Piper A & Gergle D 2018, ‘Addressing Age-Related Bias in Sentiment Analysis’, in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, New York, pp. 1–14. CHI ’18. Association for Computing Machinery. https://doi.org/10.1145/3173574.3173986.

Dijkstra EW 1968, ‘Letters to the editor: go to statement considered harmful’, Communications of the ACM, vol.11, no. 3, pp.147–148.

Eslami M, Krishna Kumaran SR, Sandvig C & Karahalios K 2018, ‘Communicating Algorithmic Process in Online Behavioral Advertising’, in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, ACM, Montréal, p. 432.

Friedman B & Nissenbaum H 1996, ‘Bias in computer systems’, ACM Transactions on Information Systems (TOIS), vol.14, no. 3, pp. 330–347.

Friess E 2012, ‘Personas and decision making in the design process: an ethnographic case study’, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1209–1218. https://doi.org/10.1145/2207676.2208572.

Goodman E, Kuniavsky M & Moed A 2013, Observing the User Experience: A Practitioner’s Guide to User Research. Morgan Kaufmann.

Goodman-Deane J, Waller S, Demin D, González de Heredia A, Bradley M & Clarkson J 2018, ‘Evaluating Inclusivity using Quantitative Personas’, in In the Proceedings of Design Research Society Conference 2018, Limerick. https://doi.org/10.21606/drs.2018.400.

Gosling SD, Rentfrow PJ & Swann WB 2003, ‘A very brief measure of the Big-Five personality domains’, Journal of Research in personality, vol. 37, no.6, pp: 504–528.

Grudin J 2006, ‘Why Personas Work: The Psychological Evidence’, in Pruitt J and Adlin T (eds) The Persona Lifecycle. Elsevier, pp. 642–663. https://doi.org/10.1016/B978-012566251-2/50013-7.

Guo H & Razikin KB 2015, ‘Anthropological User Research: A Data-Driven Approach to Personas Development’, in Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction, OzCHI ’15. ACM, New York, pp. 417–421. https://doi.org/10.1145/2838739.2838816.

Haag M & Marsden N 2019, ‘Exploring personas as a method to foster empathy in student IT design teams’, International Journal of Technology and Design Education, vol. 29, no. 3, pp.565–582. https://doi.org/10.1007/s10798-018-9452-5.

Hajian S, Bonchi F & Castillo C 2016, ‘Algorithmic Bias: From Discrimination Discovery to Fairness-aware Data Mining’, in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16. Association for Computing Machinery San Francisco, pp. 2125–2126. https://doi.org/10.1145/2939672.2945386.

Hertzberg HT, Daniels GS & Churchill E 1954, Anthropometry of Flying Personnel-1950. Antioch, Yellow Springs.

Howard TW 2015, ‘Are Personas Really Usable?’, Communication Design Quarterly Review, vol. 3, no. 2, pp. 20–26. https://doi.org/10.1145/2752853.2752856.

Huang X, Wu L & Ye Y 2019, ‘A Review on Dimensionality Reduction Techniques’, International Journal of Pattern Recognition and Artificial Intelligence, vol. 33, no. 10. https://doi.org/10.1142/S0218001419500174.

Jansen BJ, Jung S-G & Salminen J 2019, ‘Creating Manageable Persona Sets from Large User Populations’, in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, Glasgow, pp. 1–6. https://doi.org/10.1145/3290607.3313006.

Jansen BJ, Salminen J, Jung S-G, & Guan K 2021, Data-Driven Personas. 1st ed. Synthesis Lectures on Human-Centered Informatics. Morgan & Claypool Publishers. Available at: https://www.morganclaypool.com/doi/abs/10.2200/S01072ED1V01Y202101HCI048 (accessed 10 February 2021).

Jansen BJ, Salminen J & Jung S-G 2020, ‘Data-Driven Personas for Enhanced User Understanding: Combining Empathy with Rationality for Better Insights to Analytics’, Data and Information Management, vol. 4, no. 1, pp. 1–17. https://doi.org/10.2478/dim-2020-0005.

Jensen I, Hautopp H, Nielsen L & Madsen S 2017, ‘Developing international personas: A new intercultural communication practice in globalized societies’, Journal of Intercultural Communication, vol. 43.

Jung S-G, An J, Kwak H, Ahmad M, Nielsen L & Jansen BJ 2017, ‘Persona Generation from Aggregated Social Media Data’ in Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA ’17. ACM, Denver, pp. 1748–1755.

Jung S-G, Salminen J, Kwak H & Jansen BJ 2018, ‘Automatic Persona Generation (APG): A Rationale and Demonstration’, in CHIIR ’18: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval, ACM, New Jersey, pp. 321–324. ACM. https://doi.org/10.1145/3176349.3176893.

Jung S-G, Salminen J, An J & Jansen BJ 2018, ‘Automatically Conceptualizing Social Media Analytics Data via Personas’, in Proceedings of the International AAAI Conference on Web and Social Media (ICWSM 2018), San Francisco, p. 2.

Jung S-G, Salminen J & Jansen BJ 2020, ‘Giving Faces to Data: Creating Data-Driven Personas from Personified Big Data’, in Proceedings of the 25th International Conference on Intelligent User Interfaces Companion, IUI ’20. Association for Computing Machinery, Cagliari, pp. 132–133. https://doi.org/10.1145/3379336.3381465.

Kliman-Silver C, Siy O, Awadalla K, Lentz A, Convertino G & Churchill E 2020, ‘Adapting User Experience Research Methods for AI-Driven Experiences’, in Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–8.

Kwak H, An J & Jansen BJ 2017, ‘Automatic Generation of Personas Using YouTube Social Media Data’, in Proceedins of the Hawaii International Conference on System Sciences (HICSS-50), Waikoloa, pp. 833–842.

Lingel J 2012, ‘Ethics and dilemmas of online ethnography’, in CHI’12 Extended Abstracts on Human Factors in Computing Systems, pp. 41–50.

Marsden N & Haag M 2016, ‘Stereotypes and politics: reflections on personas’, in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 4017–4031.

Marsden N & Pröbster M 2019, ‘Personas and Identity: Looking at Multiple Identities to Inform the Construction of Personas’, in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19, ACM Press, Glasgow, pp. 1–14. https://doi.org/10.1145/3290605.3300565.

Matthews T, Judge T & Whittaker S 2012, ‘How do designers and user experience professionals actually perceive and use personas?’, in Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI ’12, ACM Press, Austin, p. 1219. https://doi.org/10.1145/2207676.2208573.

McGinn JJ & Kotamraju N 2008, ‘Data-driven persona development’, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, Florence, pp. 1521–1524. https://doi.org/10.1145/1357054.1357292.

Miaskiewicz T & Luxmoore C 2017, ‘The Use of Data-Driven Personas to Facilitate Organizational Adoption–A Case Study’, The Design Journal, vol. 20, no. 3, pp. 357–374.

Mijač T, Jadrić M & Ćukušić M 2018, ‘The potential and issues in data-driven development of web personas’, in 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 1237–1242. https://doi.org/10.23919/MIPRO.2018.8400224.

Nielsen L 2010, ‘Personas in Cross-Cultural Projects’, in Katre D, Orngreen R, Yammiyavar P, et al. (eds) Human Work Interaction Design: Usability in Social, Cultural and Organizational Contexts. IFIP Advances in Information and Communication Technology. Springer, Berlin, pp. 76–82. https://doi.org/10.1007/978-3-642-11762-6_7.

— 2019a, ‘Going Global—International Personas’, in Nielsen L (ed.) Personas - User Focused Design. Human–Computer Interaction Series. Springer, London, pp. 123–133. https://doi.org/10.1007/978-1-4471-7427-1_7.

— 2019b, Personas - User Focused Design. 2nd ed. 2019 edition. Springer, New York.

Nielsen L & Storgaard Hansen K 2014, ‘Personas is applicable: a study on the use of personas in Denmark’, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, Toronto, pp. 1665–1674.

Nielsen L, Hansen KS, Stage J & Billestrup J 2015, ‘A Template for Design Personas: Analysis of 47 Persona Descriptions from Danish Industries and Organizations’, International Journal of Sociotechnology and Knowledge Development, vol. 7, no. 1, pp. 45–61. https://doi.org/10.4018/ijskd.2015010104.

Nielsen L, Jung S-G, An J, Salminen J, Kwak H & Jansen BJ 2017, ‘Who Are Your Users?: Comparing Media Professionals’ Preconception of Users to Data-driven Personas’, in Proceedings of the 29th Australian Conference on Computer-Human Interaction, Brisbane, ACM, Queensland, pp. 602–606. OZCHI ’17. ACM. https://doi.org/10.1145/3152771.3156178.

Pruitt J & Grudin J 2003, ‘Personas: Practice and Theory’, in Proceedings of the 2003 Conference on Designing for User Experiences, DUX ’03. ACM, San Francisco, pp. 1–15. https://doi.org/10.1145/997078.997089.

Rönkkö K 2005, ‘An Empirical Study Demonstrating How Different Design Constraints, Project Organization and Contexts Limited the Utility of Personas’, in Proceedings of the Proceedings of the 38th Annual Hawaii International Conference on System Sciences - Volume 08, HICSS ’05. IEEE Computer Society, Washington. https://doi.org/10.1109/HICSS.2005.85.

Rönkkö K, Hellman M, Kilander B & Dittrich Y 2004, ‘Personas is Not Applicable: Local Remedies Interpreted in a Wider Context’, in Proceedings of the Eighth Conference on Participatory Design: Artful Integration: Interweaving Media, Materials and Practices - Volume 1, PDC 04 ACM, Toronto, pp. 112–120. https://doi.org/10.1145/1011870.1011884.

Sakata J, Zhang M, Pu S,Xing J & Versha K 2014, ‘Beam: a mobile application to improve happiness and mental health’, in CHI’14 Extended Abstracts on Human Factors in Computing Systems, pp. 221–226.

Salminen J, Jansen BJ, An J, Kwak H & Jung S-G 2018, ‘Are personas done? Evaluating their usefulness in the age of digital analytics’, Persona Studies, vol. 4, no. 2, pp. 47–65. https://doi.org/10.21153/psj2018vol4no2art737.

Salminen J, Jansen BJ, An J, Jung S-G, Nielsen L & Kwak H 2018, ‘Fixation and Confusion – Investigating Eye-tracking Participants’ Exposure to Information in Personas’, in Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2018), ACM, New Jersey, pp. 110–119. https://doi.org/10.1145/3176349.3176391.

Salminen J, Jung S-G, An J, Kwak H & Jansen BJ 2018, ‘Findings of a User Study of Automatically Generated Personas’, in Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems – CHI ’18, ACM Press, Montreal, pp. 1–6. https://doi.org/10.1145/3170427.3188470.

Salminen J, Jung S-G, An J, Kwak H, Nielsen L & Jansen BJ 2019, ‘Confusion and information triggered by photos in persona profiles’, International Journal of Human-Computer Studies, vol. 129, pp. 1–14. https://doi.org/10.1016/j.ijhcs.2019.03.005.

Salminen J, Jung S-G & Jansen BJ 2019, ‘Detecting Demographic Bias in Automatically Generated Personas’, in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, CHI EA ’19. ACM, New York, pp. LBW0122:1-LBW0122:6. https://doi.org/10.1145/3290607.3313034.

Salminen J, Nielsen L, Jung S-G, An J, Kwak H & Jansen BJ 2018, ‘“Is More Better?”: Impact of Multiple Photos on Perception of Persona Profiles’, in Proceedings of ACM CHI Conference on Human Factors in Computing Systems (CHI2018), ACM, Montréal, https://doi.org/10.1145/3173574.3173891.

Salminen J, Sengün S, Jung S-G & Jansen BJ 2019, ‘Design Issues in Automatically Generated Persona Profiles: A Qualitative Analysis from 38 Think-Aloud Transcripts’, in Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR), ACM, Glasgow, pp. 225–229. https://doi.org/10.1145/3295750.3298942.

Salminen J, Şengün S, Kwak H, Jansen BJ, An J, Jung S-G, Vieweg S & Harrell DF 2018, ‘From 2,772 segments to five personas: Summarizing a diverse online audience by generating culturally adapted personas’, First Monday, vol. 23, no. 6. https://doi.org/10.5210/fm.v23i6.8415.

Salminen J, Santos Joao M., Jung S-G, Eslami M Jansen BJ 2019, ‘Persona Transparency: Analyzing the Impact of Explanations on Perceptions of Data-Driven Personas’, International Journal of Human–Computer Interaction, vol. 0, no. 0, pp.1–13. https://doi.org/10.1080/10447318.2019.1688946.

Salminen J, Guan Kathleen, Jung S-G, Chowdhury SA & Jansen BJ 2020, ‘A Literature Review of Quantitative Persona Creation’, in CHI ’20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, ACM, Honolulu, pp. 1–14. https://doi.org/10.1145/3313831.3376502.

Salminen J, Jung S-G & Jansen BJ 2020, ‘Explaining Data-Driven Personas’, in Proceedings of the Workshop on Explainable Smart Systems for Algorithmic Transparency in Emerging Technologies co-located with 25th International Conference on Intelligent User Interfaces (IUI 2020), CEUR Workshop Proceedings, Cagliari, p. 7. https://doi.org/urn:nbn:de:0074-2582-4.

Salminen J, Jung S-G, Chowdhury SA, Sengün S & Jansen BJ 2020, ‘Personas and Analytics: A Comparative User Study of Efficiency and Effectiveness for a User Identification Task’, in Proceedings of the ACM Conference of Human Factors in Computing Systems (CHI’20), ACM, Honolulu, https://doi.org/10.1145/3313831.3376770.

Salminen J, Liu Ying-Hsang, Sengün S, Santos JM, Jung S-G & Jansen BJ 2020, ‘The Effect of Numerical and Textual Information on Visual Engagement and Perceptions of AI-Driven Persona Interfaces’, in IUI ’20: Proceedings of the 25th International Conference on Intelligent User Interfaces, ACM, Cagliary, pp. 357–368. https://doi.org/10.1145/3377325.3377492.

Salminen J, Froneman Willemien, Jung S, Chowdhury S & Jansen BJ 2020, ‘The Ethics of Data-Driven Personas’, in Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems Extended Abstracts, CHI ’20. Association for Computing Machinery, Honolulu, pp. 1–9. https://doi.org/10.1145/3334480.3382790.

Seidelin C, Jonsson A, Høgild M, Rømer J & Diekmann P 2014, ‘Implementing personas for international markets: a question of UX maturity’, in Proceedings at SIDER.

Siegel DA 2010, ‘The Mystique of Numbers: Belief in Quantitative Approaches to Segmentation and Persona Development’, in CHI ’10 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’10. ACM, New York, pp. 4721–4732. https://doi.org/10.1145/1753846.1754221.

Thomas TW, Tabassum M, Chu B & Lipford H 2018, ‘Security during application development: An application security expert perspective’, in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–12.

Turner P & Turner S 2011, ‘Is stereotyping inevitable when designing with personas?’ Design studies, vol. 32, no. 1, pp. 30–44.

Watanabe Y, Washizaki H, Honda K, Noyori Y, Fukazawa Y, Morizuki A, Shibata H, Ogawa K, Ishigaki M, Shiizaki S, Yamaguchi T & Yagi T 2017, ‘ID3P: Iterative Data-driven Development of Persona Based on Quantitative Evaluation and Revision’, in Proceedings of the 10th International Workshop on Cooperative and Human Aspects of Software Engineering, CHASE ’17. IEEE Press, Piscataway, pp. 49–55. https://doi.org/10.1109/CHASE.2017.9.

Wright P & McCarthy J 2008, ‘Empathy and Experience in HCI’, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’08. ACM, Florence, pp. 637–646. https://doi.org/10.1145/1357054.1357156.

Zhang X, Brown H-F & Shankar A 2016, ‘Data-driven Personas: Constructing Archetypal Users with Clickstreams and User Telemetry’, in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16. ACM, San Jose, pp. 5350–5359.

Zhu H, Wang H & Carroll JM 2019, ‘Creating Persona Skeletons from Imbalanced Datasets - A Case Study using U.S. Older Adults’ Health Data’, in Proceedings of the 2019 on Designing Interactive Systems Conference - DIS ’19, ACM Press, San Diego, pp. 61–70. https://doi.org/10.1145/3322276.3322285.