Predictors of AI Literacy and Productivity: Instrument Development, Scaling and Validation
DOI:
https://doi.org/10.24191/jikm.v15iSI1.6155Keywords:
Artificial Intelligence, Ai Literacy, Productivity , Ai Applications, Survey Instrument ValidationAbstract
Artificial Intelligence (AI) literacy refers to the degree of knowledge, skills, and ability that a person needs to effectively develop, manage, and understand the capabilities of AI in a variety of fields, including education. These days, the idea of AI is gaining popularity and needs further study. Nevertheless, prior research indicates a lack of studies on AI literacy, particularly regarding its determinants and impacts. Therefore, the objective of this study is to establish and validate an instrument that can determine AI literacy and its productivity. The study covered several stages of empirical phases and a content validity evaluation. A set of variables was chosen based on previous studies. Then, a pilot instrument was developed according to the variables chosen. Lastly, the instrument was validated using an expert review method. The reliability and face validity of potential responders were examined. Therefore, a valid instrument consisting of eight variables and forty-five items was developed.
References
Agarwal, R., & Karahanna, E. (2000). Time flies when you're having fun: Cognitive absorption and beliefs about information technology usage. MIS quarterly, 665-694.
Al-Mughairi, H., & Bhaskar, P. (2024). Exploring the factors affecting the adoption AI techniques in higher education: insights from teachers' perspectives on ChatGPT. Journal of Research in Innovative Teaching & Learning.
Alzahrani, L. (2023). Analyzing students’ attitudes and behavior toward artificial intelligence technologies in higher education. International Journal of Recent Technology and Engineering (IJRTE), 11(6), 65-73.
Asio, J. M. R. (2024). Artificial Intelligence (AI) Literacy and Academic Performance of Tertiary Level Students A Preliminary Analysis. https://doi.org/10.25273/she.v5i2.20862
Bancoro, J. (2024). The Relationship Between Artificial Intelligence (AI) Usage and Academic Performance of Business Administration Students. International Journal of Asian Business and Management, 3, 27-48. https://doi.org/10.55927/ijabm.v3i1.7876
Biagini, G., Cuomo, S., & Ranieri, M. (2024). Developing and validating a multidimensional AI literacy questionnaire: Operationalizing AI literacy for higher education. Proceedings of the First International Workshop on High-performance Artificial Intelligence Systems in Education,
Cardon, P., Fleischmann, C., Aritz, J., Logemann, M., & Heidewald, J. (2023). The challenges and opportunities of AI-assisted writing: Developing AI literacy for the AI age. Business and Professional Communication Quarterly, 86(3), 257-295.
Chai, C. S., Lin, P.-Y., Jong, M. S.-Y., Dai, Y., Chiu, T. K., & Qin, J. (2021). Perceptions of and behavioral intentions towards learning artificial intelligence in primary school students. Educational Technology & Society, 24(3), 89-101.
Chaka, C. (2023). Fourth industrial revolution—a review of applications, prospects, and challenges for artificial intelligence, robotics and blockchain in higher education. Research and Practice in Technology Enhanced Learning, 18, 002-002.
Chen, L., Jiang, M., Jia, F., & Liu, G. (2022). Artificial intelligence adoption in business-to-business marketing: toward a conceptual framework. Journal of Business & Industrial Marketing, 37(5), 1025-1044.
Cheng, X., Sun, J., & Zarifis, A. (2020). Artificial intelligence and deep learning in educational technology research and practice. British Journal of Educational Technology, 51(5), 1653-1656.
Chiu, T. K., Ahmad, Z., Ismailov, M., & Sanusi, I. T. (2024). What are artificial intelligence literacy and competency? A comprehensive framework to support them. Computers and Education Open, 6, 100171.
Chiu, T. K. F., & Sanusi, I. T. (2024). Define, foster, and assess student and teacher AI literacy and competency for all: Current status and future research direction. Computers and Education Open, 100182. https://doi.org/https://doi.org/10.1016/j.caeo.2024.100182
Crabtree, M. (2023). Artificial Intelligence (AI) vs Machine Learning (ML): A Comparative Guide. datacamp. https://www.datacamp.com/blog/the-difference-between-ai-and-machine-learning
Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: the state of the field. International Journal of Educational Technology in Higher Education, 20(1), 22.
Dergunova, Y., Aubakirova, R. Z., Yelmuratova, B. Z., Gulmira, T. M., Yuzikovna, P. N., & Antikeyeva, S. (2022). Artificial Intelligence Awareness Levels of Students. International Journal of Emerging Technologies in Learning (iJET), 17(18), 26-37. https://doi.org/10.3991/ijet.v17i18.32195
Du, H., Sun, Y., Jiang, H., Islam, A. Y. M. A., & Gu, X. (2024). Exploring the effects of AI literacy in teacher learning: an empirical study. Humanities and Social Sciences Communications, 11, 1-10. https://doi.org/10.1057/s41599-024-03101-6
Duan, L., & Da Xu, L. (2012). Business intelligence for enterprise systems: a survey. IEEE Transactions on Industrial Informatics, 8(3), 679-687.
Erfanmanesh, M., Abrizah, A., & Karim, N. H. A. (2012). Development and validation of the Information Seeking Anxiety scale. Malaysian Journal of Library and Information Science, 17(1), 21-39.
Falloon, G. (2020). From digital literacy to digital competence: the teacher digital competency (TDC) framework. Educational technology research and development, 68(5), 2449-2472.
Grassini, S. (2023). Development and validation of the AI attitude scale (AIAS-4): a brief measure of general attitude toward artificial intelligence. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1191628
Hanushek, E. A., & Ettema, E. (2017). Defining productivity in education: Issues and illustrations. The American Economist, 62(2), 165-183.
Heng, W. N. (2023). Adoption of AI technology in education among UTAR students: the case of Chatgpt UTAR].
Holmes, W. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. Center for Curriculum Redesign.
Huda, N., Abdurrahman, A., & Hidayati, N. (2023). Enhancing Accounting Learning Motivation Through Artificial Intelligence Driven Optimization Strategies For Students. Jurnal Sistem Informasi, Akuntansi dan Manajemen, 3(2), 280-289.
Hwang, G.-J. (2014). Definition, framework and research issues of smart learning environments-a context-aware ubiquitous learning perspective. Smart Learning Environments, 1, 1-14.
Hwang, G.-J., Xie, H., Wah, B. W., & Gašević, D. (2020). Vision, challenges, roles and research issues of Artificial Intelligence in Education. In (Vol. 1, pp. 100001): Elsevier.
Kandlhofer, M., Steinbauer-Wagner, G., Hirschmugl-Gaisch, S., & Huber, P. (2016). Artificial intelligence and computer science in education: From kindergarten to university. https://doi.org/10.1109/FIE.2016.7757570
Kay, J. (2012). AI and education: Grand challenges. IEEE Intelligent Systems, 27(5), 66-69.
Keefe, E. B., & Copeland, S. R. (2011). What is literacy? The power of a definition. Research and practice for persons with severe disabilities, 36(3-4), 92-99.
Klein, A. (2023). AI Literacy, Explained. Education Week. https://www.edweek.org/technology/ai-literacy-explained/2023/05
Kong, S.-C., Cheung, M.-Y. W., & Tsang, O. (2024). Developing an artificial intelligence literacy framework: Evaluation of a literacy course for senior secondary students using a project-based learning approach. Computers and Education: Artificial Intelligence, 6, 100214. https://doi.org/https://doi.org/10.1016/j.caeai.2024.100214
Kong, S.-C., Cheung, W. M.-Y., & Zhang, G. (2021). Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds. Computers and Education: Artificial Intelligence, 2, 100026.
Laupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023). Development of the “Scale for the assessment of non-experts’ AI literacy”–An exploratory factor analysis. Computers in Human Behavior Reports, 12, 100338.
Lin, P., & Van Brummelen, J. (2021). Engaging teachers to co-design integrated AI curriculum for K-12 classrooms. Proceedings of the 2021 CHI conference on human factors in computing systems,
Linna, P., Pekkola, S., Ukko, J., & Melkas, H. (2010). Defining and measuring productivity in the public sector: managerial perceptions. International Journal of Public Sector Management, 23(5), 479-499.
Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. Proceedings of the 2020 CHI conference on human factors in computing systems,
Luckin, R., Cukurova, M., Kent, C., & Du Boulay, B. (2022). Empowering educators to be AI-ready. Computers and Education: Artificial Intelligence, 3, 100076.
Massy, W. F., Sullivan, T. A., & Mackie, C. (2013). Improving measurement of productivity in higher education. Change: The magazine of higher learning, 45(1), 15-23.
Mills, K., Ruiz, P., & Lee, K.-w. (2024). Revealing an AI Literacy Framework for Learners and Educators. Digital Promise. https://digitalpromise.org/2024/02/21/revealing-an-ai-literacy-framework-for-learners-and-educators/#:~:text=AI%20literacy%20includes%20the%20knowledge,in%20an%20increasingly%20digital%20world.
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041.
Ng, D. T. K., Leung, J. K. L., Su, M. J., Yim, I. H. Y., Qiao, M. S., & Chu, S. K. W. (2022). AI education and AI literacy. In AI literacy in K-16 classrooms (pp. 9-19). Springer.
Nunnally, J. C. (1975). Psychometric theory—25 years ago and now. Educational researcher, 4(10), 7-21.
Perry, E. (2023). What is productivity? Definition and ways to improve. BetterUp. https://www.betterup.com/blog/what-is-productivity
Poth, R. D. (2024). Guiding Students to Develop AI Literacy. edutopia. https://www.edutopia.org/article/ai-literacy-students/
Rosman, M. R. M., Ismail, M. N., & Masrek, M. N. (2022). Development and validation of a tool for measuring digital library engagement. Int. J. Electr. Comput. Eng.(IJECE), 12, 4146-4154.
Rosman, M. R. M., Ismail, M. N., Masrek, M. N., Branch, K., & Campus, M. (2019). Investigating the determinant and impact of digital library engagement: a conceptual framework. Journal of Digital Information Management, 17(4), 215.
Rosman, M. R. M., Razlan, N. M., Shukry, A. I. M., Baharuddin, N. S., Rosli, N. N. I. N., & Alimin, N. A. (2022). Development and validation of instrument measuring referencing competencies and individual performance. AIP Conference Proceedings,
Sair, S. A., & Danish, R. Q. (2018). Effect of performance expectancy and effort expectancy on the mobile commerce adoption intention through personal innovativeness among Pakistani consumers. Pakistan Journal of Commerce and social sciences (PJCSS), 12(2), 501-520.
Schepman, A., & Rodway, P. (2020). Initial validation of the general attitudes towards Artificial Intelligence Scale. Computers in Human Behavior Reports, 1, 100014. https://doi.org/https://doi.org/10.1016/j.chbr.2020.100014
Shofiah, N., Putera, Z. F., & Solichah, N. (2023). Challenges and opportunities in the use of artificial intelligence in education for academic writing: A scoping review. Conference Psychology and Flourishing Humanity (PFH 2023),
Shukry, A. I. M., Rosman, M. R. M., Rosli, N. N. I. N., Alias, N. R., Razlan, N. M., & Alimin, N. A. (2023). " Bring-Your-Own-Device"(BYOD) and Productivity: Instrument Development and Validation. International Journal of Interactive Mobile Technologies, 17(11).
So, H.-J., Jong, M. S.-Y., & Liu, C.-C. (2020). Computational thinking education in the Asian Pacific region. In (Vol. 29, pp. 1-8): Springer.
Southworth, J., Migliaccio, K., Glover, J., Reed, D., McCarty, C., Brendemuhl, J., & Thomas, A. (2023). Developing a model for AI Across the curriculum: Transforming the higher education landscape via innovation in AI literacy. Computers and Education: Artificial Intelligence, 4, 100127.
Torkzadeh, G., & Doll, W. J. (1999). The development of a tool for measuring the perceived impact of information technology on work. Omega, 27(3), 327-339. https://doi.org/https://doi.org/10.1016/S0305-0483(98)00049-8
Walter, Y. (2024). Embracing the future of Artificial Intelligence in the classroom: the relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21(1), 15. https://doi.org/10.1186/s41239-024-00448-3
Wang, B., Rau, P.-L., & Yuan, T. (2022). Measuring user competence in using artificial intelligence: validity and reliability of artificial intelligence literacy scale. Behaviour & Information Technology, 42, 1-14. https://doi.org/10.1080/0144929X.2022.2072768
Wang, S., Yu, H., Hu, X., & Li, J. (2020). Participant or spectator? Comprehending the willingness of faculty to use intelligent tutoring systems in the artificial intelligence era. British Journal of Educational Technology, 51(5), 1657-1673.
Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. MIS quarterly, xiii-xxiii.
Zhang, C., & Lu, Y. (2021). Study on artificial intelligence: The state of the art and future prospects. Journal of Industrial Information Integration, 23, 100224. https://doi.org/https://doi.org/10.1016/j.jii.2021.100224
Zhang, H., Lee, I., Ali, S., Dipaola, D., Cheng, Y., & Breazeal, C. (2023). Integrating Ethics and Career Futures with Technical Learning to Promote AI Literacy for Middle School Students: An Exploratory Study. International Journal of Artificial Intelligence in Education, 33(2), 290-324. https://doi.org/10.1007/s40593-022-00293-3
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Nur Arifah Syahadah Yusnilzahri, Mohamad Rahimi Mohamad Rosman, Farrah Diana Saiful Bahry
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Copyright of articles that appear in the journal belongs exclusively to Faculty of Information Science, Universiti Teknologi MARA (Publisher). This copyright covers the rights to reproduce the article, including reprints, electronic reproductions or any other reproductions of similar nature.
