Evaluation of Shopee Food Usability Using Usability Testing Methods and System Usability Scale
Keywords:
Usability Testing, System Usability Scale (SUS), Mobile Applications, Usability EvaluationAbstract
This study aims to evaluate the Usability of the Shopee Food application using Usability Testing (UT) and the System Usability Scale (SUS) methods. The backdrop of this research is the significance of information technology in daily life, particularly in the use of mobile applications that facilitate various human activities. Shopee Food, as one of the popular food ordering applications in Indonesia, was chosen as the research object. Despite having a significant market share, the application still faces challenges in terms of navigation and ease of use. The methodology employed in this study includes a qualitative approach through UT conducted on 5 new users and a quantitative approach through SUS questionnaires filled out by 30 experienced Shopee Food application users. UT was conducted to identify issues faced by new users while using the application, whereas the SUS questionnaire was used to measure the satisfaction level of active users. The results indicate significant Usability issues with the Shopee Food application, particularly regarding the addition of vouchers and changing payment methods, which often confuse users. However, in terms of Learnability, Efficiency, Errors, and Satisfaction, the application demonstrates good performance. The Success Rate reached 98.57%, indicating users could complete tasks effectively. The average Time-Based Efficiency was 0.049 goals per second, the Error Rate at 22.8% is considered low, and the SUS score of 70 is above average, indicating a satisfactory level of user satisfaction. The conclusion of this study is that despite some Usability issues, the overall Usability level of the Shopee Food application is good. This research suggests improvements in the voucher addition feature and payment methods, as well as continuous Usability evaluations to ensure the application remains user-friendly and efficient
References
1. Abuaddous, H. Y., Saleh, A. M., Enaizan, O., Ghabban, F., & Al-Badareen, A. B. (2022). Automated User Experience (UX) Testing for Mobile Application: Strengths and Limitations. International Journal of Interactive Mobile Technologies, 16(4).
2. Alail, I. J., & Pertiwi, A. (2022). View of Evaluating User Experience of Online Food Delivery ‘ShopeeFood’using User Experience Questionnaire and Heuristic Evaluation. International Research Journal of Advanced Engineering and Science, 7(4), 49-54.
3. Andreasen, M. S., Nielsen, H. V., Schrøder, S. O., & Stage, J. (2007, April). What happened to remote usability testing? An empirical study of three methods. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 1405-1414).
4. Badran, O., & Al-Haddad, S. (2018). The impact of software user experience on customer satisfaction. Journal of Management Information and Decision Sciences, 21(1), 1-20.
5. Demirkan, H. (2015). Enhancing e-commerce outcomes with IT service innovations. International Journal of Electronic Commerce, 19(3), 2-6. https://doi.org/10.1080/10864415.2015.1000214.
6. Fakri, N. Z. A., Ismail, A., & Lokman, A. M. (2022). User Requirements and Usability Testing on a Mobile Application for Food Ordering Canteens. Malaysian Journal of Medicine & Health Sciences, 18.
7. Fernandez, A., Insfran, E., & Abrahão, S. (2011). Usability evaluation methods for the web: A systematic mapping study. Information and software Technology, 53(8), 789-817.
8. Fersellia, F., Utami, E., & Yaqin, A. (2023). Sentiment analysis of shopee food application user satisfaction using the c4. 5 decision tree method. Sinkron: jurnal dan penelitian teknik informatika, 7(3), 1554-1563.
9. Gan, B., Menkhoff, T., & Smith, R. (2015). Enhancing students’ learning process through interactive digital media: New opportunities for collaborative learning. Computers in Human Behavior, 51, 652-663. https://doi.org/10.1016/j.chb.2014.12.048.
10. Ghasemifard, N., Shamsi, M., Kenari, A. R. R., & Ahmadi, V. (2015). A new view at usability test methods of interfaces for human computer interaction. Global Journal of Computer Science and Technology, 15(1), 17-24.
11. Gonzalez-Holland, E., Whitmer, D., Moralez, L., & Mouloua, M. (2017, September). Examination of the use of Nielsen’s 10 usability heuristics & outlooks for the future. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 61, No. 1, pp. 1472-1475). Sage CA: Los Angeles, CA: SAGE Publications.
12. Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International journal of human-computer studies, 64(2), 79-102.
13. Isomursu, M., Häikiö, J., Wallin, A., & Ailisto, H. (2008). Experiences from a Touch‐Based Interaction and Digitally Enhanced Meal‐Delivery Service for the Elderly. Advances in Human‐Computer Interaction, 2008(1), 931701.
14. Juliá-Nehme, B., & Rosell, J. (2024). Interaction and Design Barriers for Older Adults in Food Delivery Apps: A Usability Study. International Journal of Human–Computer Interaction, 1-18. https://doi.org/10.1080/10447318.2024.2365484.
15. Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E., & Sinnelä, A. (2011). UX Curve: A method for evaluating long-term user experience. Interacting with computers, 23(5), 473-483. https://doi.org/10.1016/j.intcom.2011.06.005.
16. Leung, R., & Loo, P. T. (2022). Co-creating interactive dining experiences via interconnected and interoperable smart technology. Asian Journal of Technology Innovation, 30(1), 45-67. https://doi.org/10.1080/19761597.2020.1822748.
17. Lin, H. X., Choong, Y. Y., & Salvendy, G. (1997). A proposed index of usability: a method for comparing the relative usability of different software systems. Behaviour & information technology, 16(4-5), 267-277.
18. Liu, C. F., & Lin, C. H. (2020). Online food shopping: a conceptual analysis for research propositions. Frontiers in Psychology, 11, 583768.
19. Md Saad, N. H., Mei Nie, F., & Yaacob, Z. (2023). Exploring sentiment analysis of online food delivery services post COVID-19 pandemic: grabfood and foodpanda. Journal of Foodservice Business Research, 1-25.
20. Mitra, A., & Debnath, S. (2023, May). A Set of Usability Constructs and Indicators for UI/UX Research on Mobile Food Ordering Applications in India. In International Conference on Data Analytics and Insights (pp. 431-441). Singapore: Springer Nature Singapore.
21. Nielsen, 2012. “Usability 101: Introduction to Usability.”. [Online]. Link Online: https://www.nngroup.com/articles/usability-101-introduction-to-usability.
22. Ntoa, S. (2024). Usability and User Experience Evaluation in Intelligent Environments: A Review and Reappraisal. International Journal of Human–Computer Interaction, 1-30. https://doi.org/10.1080/10447318.2024.2394724.
23. Pakaja, F Panjiranatha, G, D., Rahman, M, A, HM Amir HM. Adoption of Technology Acceptance and Interfaces for Academic Information System Applications. 2024. Qeios. https://doi.org/10.32388/TYZLOL.
24. Rusu, C., Rusu, V., Roncagliolo, S., & González, C. (2015). Usability and user experience: What should we care about?. International Journal of Information Technologies and Systems Approach (IJITSA), 8(2), 1-12.
25. Shaeeali, N. S., Mohamed, A., & Mutalib, S. (2020). Customer reviews analytics on food delivery services in social media: A review. IAES Int. J. Artif. Intell, 9(4), 691.
26. Taherdoost, H. (2018). Development of an adoption model to assess user acceptance of e-service technology: E-Service Technology Acceptance Model. Behaviour & Information Technology, 37(2), 173-197. https://doi.org/10.1080/0144929X.2018.1427793.
27. Taimouri, A., Emamisaleh, K., & Mohammadi, D. (2019). Assessing the usability of online food ordering websites using a new fuzzy kano method: Implications for improvement. International Journal of Business and Management, 14(10).
28. Vlachogianni, P., & Tselios, N. (2022). Perceived usability evaluation of educational technology using the System Usability Scale (SUS): A systematic review. Journal of Research on Technology in Education, 54(3), 392-409. https://doi.org/10.1080/15391523.2020.1867938.
29. Wilujeng S. SP., Yahya, T. Y., Pratama, W. M., Hardiana, D., Andy Putr, V. W., & Naibaho, T. M. (2022). Shopee (ShopeeFood) Application Quality Factors and Their Impact on Loyalty through Consumer Satisfaction. Central Asia & the Caucasus (14046091), 23(1).
