(Mis)communication? Social listening and the exclusion of marginalised voices


  • Irene Scott Internews




Social Media Analytics, Community perceptions, Data


This article aims to contribute to the growing scholarship on the use of social media by humanitarian organisations in a crisis. Although social media’s role in times of crisis has been rigorously studied, much of this work looks at the distribution or collection of information by first-responders or relief organisations. However, there is a growing interest in the analysis of social media content to understand community perceptions and to guide public health and risk communication interventions. This article aims to explore some key limitations of data collected using Social Media Analytics (SMA) tools in fairly representing community-wide perceptions. Through a review of ‘social listening reports’ produced by UN bodies and international aid organisations, this article will explore whether these data deficiencies are fairly represented. This article concludes that while there are many well documented limitations in the use of social media discourse to holistically represent community perceptions, these limitations are not adequately discussed in the reporting produced from this data. Consequentially, users of this analysis cannot adequately weigh the quality of the data when using it to influence policy decisions.


Download data is not yet available.


Metrics Loading ...

Author Biography

Irene Scott, Internews

Irene is a former journalist who specialises in building humanitarian information systems to connect crisis affected communities with quality information to inform decision making and influence humanitarian response efforts. She is currently the Senior Community Engagement Project Manager at Internews


Abdouli, A., Chaffai, A., Hassouni, L., Anoun, H., Khalid, R. (2016). Current trends in Moroccan Social Networks, March 2016. International Journal of Computer Science and Information Security, 14(3). https://www.researchgate.net/publication/324839604_Current_Trends_in_Moroccan_Social_Networks

Albergotti, R., MacMillan, D., Rusli, E. (2014). Facebook to Pay $19 Billion for WhatsApp. Wall Street Journal. pp. A1, A6. ISSN 0099-9660. https://www.wsj.com/articles/facebook-to-buy-whatsapp-for-16-billion-1392847766

Appling, S., Briscoe, E., Ediger, D., Poovey, J., McColl, R. (2014). Deriving Disaster-Related Information from Social Media. Georgia Tech Research Institute. https://www.academia.edu/10282640/Deriving_Disaster-Related_Information_from_Social_Media

Berman, H. (2019, 20 June). What’s up with WhatsApp? How WhatsApp is used and misused in Africa. The Economist https://www.economist.com/middle-east-and-africa/2019/07/18/how-whatsapp-is-used-and-misused-in-africa

Bies, A., Song, Z., Maamouri, M., Grimes, S., Lee, H., Wright, J., Strassel, S., Habash, N., Eskander, R., Rambow, O. (2014). Transliteration of arabizi into Arabic orthography: developing a parallel annotated arabizi-arabic script sms/chat corpus. Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pp. 93–103. https://aclanthology.org/W14-3612

Bowles, J., Larreguy, H., Liu, S. (2021). Countering formation via WhatsApp: Preliminary evidence from the COVID-19 pandemic in Zimbabwe. PLoS ONE. 10.14.2020. pp. 1-11. DOI: 10.1371/journal.pone.0240005

Broniatowski, D., Jamison, A., Qi, S., AlKulaib, L., Chen, T., Benton, A., Quinn, S., Dredze, M. (2018). Weaponized Health Communication: Twitter Bots and Russian Trolls Amplify the Vaccine Debate. Am J Public Health. DOI: 10.2105/AJPH.2018.304567

Brown, R., Barram, D., Irving, L. (1995). Falling through the net: a survey of the “have nots” in rural and urban America. US Department of Commerce, National Telecommunications and Information Administration. https://www.ntia.doc.gov/ntiahome/fallingthru.html

Cassa, C., Chunara, R., Mandl, K., Brownstein, J (2013). Twitter as a Sentinel in Emergency Situations: Lessons from the Boston Marathon Explosions PLoS Currents, no. JUNE. DOI: currents.dis.ad70cd1c8bc585e9470046cde334ee4b

Chanely, S., Benjamin, P., Mechae, P. (2021). Finding the Signal through the Noise: A landscape review and framework to enhance the effective use of digital social listening for Immunization demand generation, Health Enabled with technical guidance and support from Gavi, the Vaccine Alliance, UNICEF and WHO. https://www.gavi.org/sites/default/files/2021-06/Finding-the-Signal-Through-the-Noise.pdf

Cohen, S. E. (2013). Lessons Learned: Social Media and Hurricane Sandy. Virtual Social Media Working Group and DHS First Responders Group. https://www.dhs.gov/sites/default/files/publications/Lessons%20Learned%20Social%20Media%20and%20Hurricane%20Sandy.pdf

Darwish, K. (2014). A Panoramic Survey of Natural Language Processing in the Arab World. Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pp.217–224., https://eds.b.ebscohost.com/eds/detail/detail?vid=40&sid=63fecaea-5a6a-4c76-a561-abb6e3ed2c12%40pdc-v-sessmgr02&bdata=JmF1dGh0eXBlPXNzbyZjdXN0aWQ9ZGVha2luJnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0ZQ%3d%3d#AN=149459802&db=aci

Dashtian, H., Murthy, D. (2021). CML-COVID: A large-scale covid-19 twitter dataset with latent topics, sentiment and location information. Academia Letters, Article 314. https://doi.org/10.20935/AL314

Davies, W. (2020, 2 July). What’s wrong with WhatsApp? The Guardian online. https://www.theguardian.com/technology/2020/jul/02/whatsapp-groups-conspiracy-theories-disinformation-democracy

Dotto, C., Cubbon, S. (2021). Disinformation exports: How foreign anti-vaccine narratives reached West African communities online. First Draft. https://firstdraftnews.org/long-form-article/foreign-anti-vaccine-disinformation-reaches-west-africa/

Duarte, N., Llanso, E., Loup, A. (2018). Mixed Messages? The Limits of Automated Social Media Content Analysis. 2018 Conference on Fairness, Accountability, and Transparency. https://cdt.org/wp-content/uploads/2017/12/FAT-conference-draft-2018.pdf

Ethnologue. (2021). Country language profiles for: Kenya, Nigeria, South Africa, Ivory Coast, Burkina Faso, Senegal, Democratic Republic of Congo, Niger, and Mali. https://www.ethnologue.com/browse/countries>

Gilmore, B., Ndejjo, R., Tchetchia, R., de Claro, V., Mago, E., Diallo, A., Lopes, C., Bhattacharyya, S. (2020). Community engagement for COVID-19 prevention and control: a rapid evidence synthesis. BMJ Global Health 2020;5:e003188. https://gh.bmj.com/content/5/10/e003188

Goh, Dion Hoe-Lian., Chua, Alton Y. K., Shi, Hanyu., Wei, Wenju., Wang, Haiyan., Lim, Ee-peng. (2017). An analysis of rumor and counter-rumor messages in social media, Digital libraries: Data, information, and knowledge for digital lives. 19th International Conference on Asia-Pacific Digital Libraries, ICADL 2017, Proceedings, pp. 256-266. Research Collection School Of Information Systems. https://ink.library.smu.edu.sg/sis_research/3875

Goncalves, A. (2017). Social media analytics strategy: using data to optimize business performance. APress, https://eds.a.ebscohost.com/eds/detail/detail?vid=1&sid=4

Guellil, I., Saâdane, H., Azouaou, F., Gueni. B., Nouvel, D. (2021). Arabic natural language processing: An overview. Journal of King Saud University - Computer and Information Sciences, 33(5): pp. 497-507. https://doi.org/10.1016/j.jksuci.2019.02.006

Hampton, K.. Rainie, L., Weixu, L., Dwyer, M., Shin, I., Purcell, K. (2014). Social Media and the ‘Spiral of Silence’. Pew Research Center. https://www.pewresearch.org/internet/2014/08/26/social-media-and-the-spiral-of-silence/

Hargittai, E. (2015) Is Bigger Always Better? Potential Biases of Big Data Derived from Social Network Sites. Annals of the American Academy of Political and Social Science. 659(1): pp. 63-76. DOI: 10.1177/0002716215570866

Hirschberg, J., Manning, C. (2015). Advances in Natural Language Processing. Science 261. American Association for the Advancement of Science, Stanford University. https://cs224d.stanford.edu/papers/advances.pdf.

Hoffmann, C., Bublitz, W. (2017). Pragmatics of Social Media. De Gruyter Mouton, Berlin. https://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=1595340&authtype=sso&custid=deakin&site=eds-live&scope=site

Hossain, L., Kam, D., Kong, F., Wigand, R.T., Bossomaier, T. (2016). Social media in Ebola outbreak Epidemiol Infect. 144(10): pp. 2136-43. DOI: 10.1017/S095026881600039X

Hou, Z., Tong, Y., Du, F., Lu, L., Zhao, S., Yu. K., Piatek, S.J., Larson, H.J., Lin, L. (2021). Assessing COVID-19 Vaccine Hesitancy, Confidence, and Public Engagement: A Global Social Listening Study. Journal of Medical Internet Research. 23(6): pp. e27632. https://eds.a.ebscohost.com/eds/detail/detail?vid=4&sid=8a33421d-e516-4d8c-ac59-9c83af16a7f8%40sessionmgr4007&bdata=JmF1dGh0eXBlPXNzbyZjdXN0aWQ9ZGVha2luJnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0-ZQ%3d%3d#db=mdc&AN=34061757

Johansson, F., Kaati, L., Sahlgren, M. (2016). Detecting Linguistic Markers of Violent Extremism in Online Environments. Combating Violent Extremism and Radicalization in the Digital Era. pp. 374–90. https://www.foi.se/download/18.3bca00611589ae7987820d/1480076542059/FOI-S--5452--SE.pdf

Kenworth, S. (2019). How a Russian Troll Farm Aims to Upset the Current Hegemonic, Unipolar World Order. DOI: 10.13140/ RG.2.2.32736.74248. https://www.researchgate.net/publication/333092238_How_a_Russian_Troll_Farm_Aims_to_Upset_the_Current_Hegemonic_Unipolar_World_Order

Kietzmann, J.H., Hermkens, K., McCathy, I.P., Silvestre, B.S. (2011). Social media? Get serious! Understanding the functional building blocks of social media. Business Horizons. 54(3): pp. 241-25. https://www.sciencedirect.com/science/article/abs/pii/S0007681311000061

Lazer, D.M.J., Baum, M.A., Benkler, Y., Berinsky, A.J., Greenhill, K.M., Menczer, F. (2018). The science of fake news. Science. 359(6380): pp. 1094–1096. https://doi.org/10.1126/science.aao2998

Landers, C. (2017). The Digital Divide: Issues, Recommendations and Research. Hauppauge, New York, Nova Science Publishers, Inc. https://eds.a.ebscohost.com/eds/detail/detail?vid=4&sid=4ace7791-db24-43d3-97b7

Liebermann, M. (2013). Social: why our brains are wired to connect. Oxford, Oxford University Press. https://eds.b.ebscohost.com/eds/detail/detail?vid=32&sid=63fecaea-5a6a-4c76-a561-abb6e3ed2c12%40pdc-v-sessmgr02&bdata=JmF1dGh0eXBlPXNzbyZjdXN0aWQ9ZGVha2luJnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0ZQ%3d%3d#AN=deakin.b3460813&db=cat00097a

Lindsay, B. (2011). Social Media and Disasters: Current Uses, Future Options, and Policy Considerations, Congressional Research Service report for Congress. https://sgp.fas.org/crs/homesec/R41987.pdf

Mancosou, M., Vegetti, F. (2020). What You Can Scrape and What Is Right to Scrape: A Proposal for a Tool to Collect Public Facebook Data. Journal of Social Media and Society. 6(3). https://doi.org/10.1177/2056305120940703

Markham, A., Buchanan, E. (2012). Association for Internet Researchers’ Ethics Working Committee—Ethical decision-making and Internet research: Version 2.0. Association of Internet Researchers https://aoir.org/reports/ethics2.pdf

Murthy, D., Gross, A.J. (2016). Social media processes in disasters: Implications of emergent technology use, Social Science Research. DOI: 10.1016/j.ssresearch.2016.09.015

Noelle-Neumann, E. (1974). The Spiral of Silence: A Theory of Public Opinion. Journal of Communication. 24(2): pp. 43-51. https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1460-2466.1974.tb00367.x

Noveck, B., Button, J., Gambrell, D., Paulson. L., Spada, P., Subramanian, L. (2021). The Power of Virtual Communities. The Gov Lab. https://virtual-communities.thegovlab.org/files/DTR_report_en_EN.pdf

Nutbeam, D. (2021). The vital role of meaningful community engagement in responding to the COVID-19 pandemic. Public health research and practice. DOI: https://doi.org/10.17061/phrp3112101

Obar, J., Wildman S. (2015). Social media definition and governance challenge, University of Ontario Institute of Technology and Michigan State University, working paper. https://d1wqtxts1xzle7.cloudfront.net/38260771/Obar_Wildman_GOSM_TP_Editorial_July_22_2015-with-cover-page-v2.pdf

Oh, O., Agrawal, M., Rao, R. (2013). Community Intelligence and Social Media Services: A Rumor Theoretic Analysis of Tweets During Social Crises. Management Information Systems Quarterly. 37(2): pp. 407-426. https://aisel.aisnet.org/misq/vol37/iss2/7/

O’Mathuna, D., Siriwardhana, S. (2017). Research ethics and evidence for humanitarian health. The Lancet. 390(10109):pp. 2228-9. DOI: https://doi.org/10.1016/S0140-6736(17)31276-X

Orfan, S. (2020). Political participation of Afghan Youths on Facebook: A case study of Northeastern Afghanistan. Cogent Social Sciences. 7(1) https://www.tandfonline.com/doi/full/10.1080/23311886.2020.1857916

Pasquetto, I., Jahani, E. (2020). Mobile Instant Messengers (MIMs) in Developing Countries, Shorenstein Center on Media, Politics and Public Policy, Harvard Kennedy School. https://shorensteincenter.org/misinformation-on-mims

Perrin, A., Anderson, M. (2019). Share of US adults using social media, including Facebook, is mostly unchanged since 2018, Pew Research Center. https://pewrsr.ch/2VxJuJ3

Plexico-Sinclair, H., Vestrheim, K. (2020). Engaging Communities during a Pandemic: Experiences of Community Engagement during the COVID-19 Response in Camps and Out-of-Camp Settings, Norwegian Refugee Council. https://reliefweb.int/sites/reliefweb.int/files/resources/engaging-communities-during-a-pandemic.pdf

Ragnedda, M., Muschert, G. (2013). The Digital Divide : The Internet and Social Inequality in International Perspective, Routledge Advances in Sociology Series, Taylor & Francis Group. https://ebookcentral-proquest-com.ezproxy-f.deakin.edu.au/lib/deakin/detail.action?docID=1221501

Rai, A. (2019). Digital Divide: How Do Women in South Asia Respond?. International Journal of Digital Literacy and Digital Competence. 10(1). DOI: 10.4018/IJDLDC.2019010101

Ravn, S., Barnwell, A., Neves, B. (2019). What Is “Publicly Available Data”? Exploring Blurred Public–Private Boundaries and Ethical Practices Through a Case Study on Instagram. Journal of Empirical research on Human research ethics. 15(1-2): pp. 40-45. DOI: https://doi.org/10.1177/1556264619850736

Remy, E. (2019). How public and private Twitter users in the U.S. compare - and why it might matter for your research. Pew Research Center. https://medium.com/pew-research-center-decoded/how-public-and-private-twitter-users-in-the-u-s-d536ce2a41b314

Rosler, P., Mainka, C., Schwenk, J. (2018). More is Less: On the End-to-End Security of Group Chats in Signal, WhatsApp, and Threema. 2018 IEEE European Symposium on Security and Privacy. https://eds.b.ebscohost.com/eds/detail/detail?vid=13&sid=63fecaea-5a6a-4c76-a561-abb6e3ed2c12%40pdc-v-sessmgr02&bdata=JmF1dGh0eXBlPXNzbyZjdXN0aWQ9ZGVha2luJnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0ZQ%3d%3d#AN=edseee.8406614&db=edseee

Ross, P., Zaidi, N. (2019). Limited by our limitations. Perspect Med Educ. 8(4): pp. 261–264. DOI: 10.1007/s40037-019-00530-x

Salvatore, C., Biffignandi, S., Bianchi, A. (2020). Social media and twitter data quality for new social indicators. Social Indicators Research. 156(2-3): pp. 601-630. https://eds.b.ebscohost.com/eds/detail/detail?vid=35&sid=63fecaea-5a6a-4c76-a561-abb6e3ed2c12%40pdc-v-sessmgr02&bdata=JmF1dGh0eXBlPXNzbyZjdXN0aWQ9ZGVha2luJnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0ZQ%3d%3d#db=her&AN=151632112

Sasu, D. (2021). Number of active social media users in Morocco 2017-2021, Morocco Country report. https://www.statista.com/statistics/1172771/number-of-social-media-users-morocco/

Simon, T., Goldberg, A., Aharonson-Daniel, L., Leykin, D., Adini, B. (2014). Twitter in the Cross Fire—The Use of Social Media in the Westgate Mall Terror Attack in Kenya. PLoS ONE. 9(8): e104136. DOI: 10.1371/journal.pone.0104136

Simon, T., Avishay, G., Bruria, A. (2015). Socializing in emergencies—A review of the use of social media in emergency situations. International Journal of Information Management. 35: pp. 609–619. https://www.academia.edu/20379786/Socializing_in_emergencies_A_review_of_the_use_of_social_media_in_emergency_situations

Snider, M. (2018, 16 February). Robert Mueller investigation: What is a Russian troll farm? USA Today https://www.usatoday.com/story/tech/news/2018/02/16/robert-muellerinvestigation-what-russian-troll-farm/346159002/

Statista Analytics. (2021). Most popular social networks worldwide as of July 2021, ranked by number of active users, Q3 2008 to Q2 2021 industry report. Statista Research Department. https://www.statista.com/statistics/272014/global-social-networks-ranked-by-number-of-users/

Stewart, M., Arnold, C. (2018). Defining Social Listening: Recognizing an Emerging Dimension of Listening. International Journal of Listening. 32: pp. 85–100. DOI: https://doi.org/10.1080/10904018.2017.1330656

Talafha, B., Abuammar, A., Al-Ayyoub, M. (2021). ATAR: Attention-based LSTM for Arabizi transliteration. International Journal of Electrical & Computer Engineering. 11(3): pp. 2327-2334 https://eds.a.ebscohost.com/eds/detail/detail?vid=29&sid=4ace7791-db24-43d3-97b7-66badfb05b47%40sessionmgr4008&bdata=JmF1dGh0eXBlPXNzbyZjdXN0aWQ9ZGVha2luJnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0ZQ%3d%3d#AN=148685260&db=aci

UNICEF. (2021). UNICEF mission statement. https://www.unicef.org/about-us/mission-statement

Utomo, M., Adji, T., Ardiyanto, I. (2018). Geolocation prediction in social media data using text analysis: A review. 2018 International Conference on Information and Communications Technology (ICOIACT). DOI: 10.1109/ ICOIACT.2018.8350674

Uyheng, J., Carley, K. (2020). Bots and online hate during the COVID-19 pandemic: case studies in the United States and the Philippines. Journal of Computational Social Science. 3(2): pp. 445-46. https://eds.b.ebscohost.com/eds/detail/detail?vid=20&sid=63fecaea-5a6a-4c76-a561-abb6e3ed2c12%40pdc-vsessmgr02&bdata=JmF1dGh0eXBlPXNzbyZjdXN0aWQ9ZGVha2luJnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0ZQ%3d%3d#AN=edssjs.36F3AF94&db=edssjs

Vashisha, N., Zubiaga, A. (2021). Online Multilingual Hate Speech Detection: Experimenting with Hindi and English Social Media. Information. 12(1). https://eds.a.ebscohost.com/eds/detail/detail?vid=24&sid=4ace7791-db24-43d3-97b7-66badfb05b47%40sessionmgr4008&bdata=JmF1dGh0eXBlPXNzbyZjdXN0aWQ9ZGVha2luJnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0ZQ%3d%3d#AN=148316730&db=lls

Vieweg, S., Hughes, A., Starbird, K., Panel, L. (2010). Microblogging During Two Natural Hazards Events: What Twitter May Contribute to Situational Awareness. CHI 2010: Crisis Informatics April 10–15, 2010. http://www.pensivepuffin.com/dwmcphd/syllabi/insc547_wi13/papers/microblog/vieweg.et.al.TwitterAwareness.CHI10.pdf

Walker, T. (2017). Humanitarian futures for messaging apps: understanding the opportunities and risks for humanitarian action, ICRC. https://reliefweb.int/sites/reliefweb.int/files/resources/4299_002_Humanitarian-Futures-for-Messaging-Apps_WEB_.pdf

Wiegmann, M., Kerston, J., Senaratne, H., Potthast, M., Klan, F., Stein, B. (2021). Opportunities and risks of disaster data from social media: a systematic review of incident information. Natural Hazards and Earth System Sciences. 21: pp. 1431-1444. https://eds.b.ebscohost.com/eds/detail/detail?vid=42&sid=63fecaea-5a6a-4c76-a561-abb6e3ed2c12%40pdc-v-sessmgr02&bdata=JmF1dGh0eXBlPXNzbyZjdXN0aWQ9ZGVha2luJnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0ZQ%3d%3d#AN=edsdoj.1a8e1c70ba6f4675a6353c1ed092632c&db=edsdoj

Winhill, L. (2018). How Africa Tweets 2018, Portland Communications. https://portland-communications.com/pdf/How-Africa-Tweets-2018.pdf

Wojcik, S., Messing, S., Smith, A., Rainie, L., Hitlin, P. (2018). Bots in the Twittersphere. Pew Research Center. https://www.pewresearch.org/internet/2018/04/09/bots-in-the-twittersphere

Wojcik, S., Adam, H. (2019). Sizing up Twitter users. Pew Research Center. https://pewrsr.ch/2VUkzj4

World Health Organization. (2020). COVID-19 Global Risk Communication and Community Engagement Strategy. https://www.who.int/teams/risk-communication/the-collective-service

World Health Organization. (2020). Landmark alliance launches in Africa to fight COVID-19 misinformation. https://www.afro.who.int/news/landmark-alliance-launches-africa-fight-covid-19-misinformation>

Yang, K., Pierri, F., Hui, P., Axelrod, A., Torres-Lugo, C., Bryden, J., Menczer, F. (2021). The COVID-19 Infodemic: Twitter versus Facebook, Big Data and society. Sage Journals. https://doi.org/10.1177/20539517211013861

Zimmer, M. (2010). But the data is already public: on the ethics of research in Facebook, Ethics and Information Technology. Ethics and Information Technology. 12(4): pp. 313-325. DOI: 10.1007/s10676-010-9227-5




How to Cite

Scott, I. (2022) “(Mis)communication? Social listening and the exclusion of marginalised voices”, The Humanitarian Leader, p. Working Paper 023, March 2022. doi: 10.21153/thl2022art1558.