Empirical Analysis of the Validity and Reliability of an Energy Literacy Instrument for Indonesian Junior Secondary School Students

Musawwir Usman; Yulianti Yusal; Zulqifli Alqadri

doi:10.59890/ijasse.v4i2.390

Authors

Musawwir Usman Universitas Negeri Makassar
Yulianti Yusal Universitas Negeri Makassar
Zulqifli Alqadri Universitas Negeri Makassar

DOI:

https://doi.org/10.59890/ijasse.v4i2.390

Keywords:

Empirical Analysis, Validity, Reliability, Energy Literacy Instrument, Indonesian Junior Secondary School

Abstract

This study aimed to empirically evaluate the validity and reliability of an energy literacy assessment instrument for junior secondary school students. The instrument covers cognitive, affective, and behavioral domains of energy literacy. A quantitative approach was employed using classical test theory analysis. Data were collected from 134 Grade VIII students from four junior secondary schools. Item difficulty (p, proportion of correct responses) and item discrimination (r_pbis, point-biserial correlation) were analyzed for the cognitive domain, while item–total correlation (r, Pearson correlation) was used to examine the affective and behavioral domains. Internal consistency reliability was estimated using Cronbach’s alpha (α) and KR-20. The results showed that most items demonstrated acceptable difficulty, discrimination, and item–total correlation values, and the overall instrument exhibited high reliability. It can be concluded that the refined energy literacy instrument is psychometrically sound and suitable for assessing students’ energy literacy in junior secondary education.

References

Ahmed, I., & Ishtiaq, S. (2021). Reliability and Validity: Importance in Medical Research. Journal of the Pakistan Medical Association, 71(10), 2401–2406.

Akitsu, Y., & Ishihara, K. N. (2018). An Integrated Model Approach: Exploring the Energy Literacy and Values of Lower Secondary Students in Japan. International Journal of Educational Methodology, 4(3), 161–186.

Cai, Y., Li, Y., Zou, J., Zhang, Jie, Luo, W., Zhang, Jingping, & Qu, C. (2024). Cross-cultural adaptation and reliability of the inventory of vicarious posttraumatic growth and research of its influencing factors: a cross-sectional study. BMC Nursing, 23(1), 763.

Cappelleri, J. C., Jason Lundy, J., & Hays, R. D. (2014). Overview of Classical Test Theory and Item Response Theory for the Quantitative Assessment of Items in Developing Patient-Reported Outcomes Measures. Clinical Therapeutics, 36(5), 648–662.

Chen, K. L., Huang, S. H., & Liu, S. Y. (2013). Devising a framework for energy education in Taiwan using the analytic hierarchy process. Energy Policy, 55, 396–403. https://doi.org/10.1016/j.enpol.2012.12.025

Chyung, S. Y. Y., Barkin, J. R., & Shamsy, J. A. (2018). Evidence-Based Survey Design: The Use of Negatively Worded Items in Surveys. Performance Improvement, 57(3), 16–25. https://doi.org/10.1002/pfi.21749

DeWaters, J. E., & Powers, S. E. (2011). Energy literacy of secondary students in New York State (USA): A measure of knowledge, affect, and behavior. Energy Policy, 39(3), 1699–1710.

DeWaters, J., & Powers, S. (2013). Establishing Measurement Criteria for an Energy Literacy Questionnaire. Journal of Environmental Education, 44(1), 38–55. https://doi.org/10.1080/00958964.2012.711378

Fiskawarni, T. H., Rahmawati, R., Widiasih, W., & Saad, R. (2024). The Design and Validation of the Four-Tier Test Instrument for Energy Literacy Using the Rasch Model Analysis. JIPF (Jurnal Ilmu Pendidikan Fisika), 9(1), 74.

Gul, N., Shagufta, S., & Parveen, S. (2022). Item Difficulty in Item Analysis of Intelligence Test Items. Pakistan Social Sciences Review, 6(II).

Guyer, R., & Thompson, N. A. (2013). User’s Manual for Iteman 4.3. Assessment Systems Corporation.

Haroz, E. E., Kane, J. C., Nguyen, A. J., Bass, J. K., Murray, L. K., & Bolton, P. (2020). When less is more: reducing redundancy in mental health and psychosocial instruments using Item Response Theory. Global Mental Health, 7, e3. https://doi.org/10.1017/gmh.2019.30

Hartelt, T., & Martens, H. (2025). Development and validation of an instrument measuring students’ self-efficacy regarding explaining evolutionary changes. Journal of Biological Education, 1–35.

Hendinata, L. K., Ardiwinata, T., & Pratama, F. K. T. (2022). The Role of Energy Literacy in Supporting Energy Conservation: Perspective from Indonesian Citizens. Indonesian Journal of Energy, 5(2).

Hu, J.-L., & Yang, P.-S. (2024). Interactive Cycles between Energy Education and Energy Preferences: A Literature Review on Empirical Evidence. Energies, 17(20), 5092. https://doi.org/10.3390/en17205092

Huda, N., & Wahyuni, T. S. (2020). PENGGUNAAN APLIKASI ITEM AND TEST ANALYSIS (ITEMAN) PADA SOAL TRY OUT UN IPA SMP TAHUN 2019. Jurnal Pembelajaran Sains, 4(1), 1–8.

Ibrahim, S. E., & Ismail, M. R. (2025). Item Analysis of Information Literacy Test for Open, Distance and Digital Education New Learners. E-Learning Innovations Journal, 3 (2), 31–54.

Izah, S. C., Sylva, L., & Hait, M. (2023). Cronbach’s Alpha: A Cornerstone in Ensuring Reliability and Validity in Environmental Health Assessment. ES Energy & Environment, 23. https://doi.org/10.30919/esee1057

Kalkbrenner, M. T. (2021). A Practical Guide to Instrument Development and Score Validation in the Social Sciences: The MEASURE Approach. Practical Assessment, Research, and Evaluation, 26(1).

Karim, S. A., Sudiro, S., & Sakinah, S. (2021). Utilizing test items analysis to examine the level of difficulty and discriminating power in a teacher-made test. EduLite: Journal of English Education, Literature and Culture, 6(2), 256. https://doi.org/10.30659/e.6.2.256-269

Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G., & Srinivas, V. (2021). Item analysis of multiple choice questions: A quality assurance test for an assessment tool. Medical Journal Armed Forces India, 77, S85–S89.

Kurniawan, A., Istiyono, E., & Naba, S. D. (2024). Item Quality Analysis of Physics Concept Understanding Test with Rasch Model. JIPF (Jurnal Ilmu Pendidikan Fisika), 9(3), 474. https://doi.org/10.26737/jipf.v9i3.5692

Mukherjee, S., & Kittur, J. (2025). Learning through cognitive, affective, and psychomotor domains: Understanding undergraduate engineering students’ perspectives in the United States. Engineering Education Review, 3. https://doi.org/10.54844/eer.2025.0916

Mura, K. B., Sujito, Taufiq, A., Irawan, I. D. A., & Basri, N. A. (2025). Empirical Validation of a Conceptual Understanding Test on Dynamic Electricity for High School Students. Jurnal Penelitian Pendidikan IPA, 11(11), 229–235.

Ningsih, S. N., Istiyono, E., & Widihastuti. (2024). An Application of Classical Test Theory for Item Characteristic Analysis of Chemical Literacy Instruments. Jurnal Pendidikan Kimia Indonesia, 7(2), 58–68.

Putri, I. R., Setiawan, A., & Nasrudin, D. (2022). Energy Literacy Profile of Vocational High School Teacher Candidates for Renewable Energy Engineering Expertise Program. Jurnal Ilmiah Pendidikan Teknik Dan Kejuruan, 15(2), 99. https://doi.org/10.20961/jiptek.v15i2.67623

Quaigrain, K., & Arhin, A. K. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1), 1301013.

Rabiatul Adawiah. (2020). THE APPLICATION-BASED ANALYSIS OF QUESTIONS ITEM QUALITY IN JUNIOR HIGH SCHOOL. JISAE: Journal of Indonesian Student Assessment and Evaluation, 6(2), 140–148.

Ridwan, M. R., Istiyono, E., & Widihastuti, W. (2021). Test Items Analysis of Mathematical Problem-Solving Ability using a Classical Test Theory Approach. Jurnal Pendidikan MIPA, 22(1), 98–111.

Rohmatdi, A., Martono, M., Zafrullah, Z., & Safitri, R. (2024). Optimizing Question Quality in Junior High School Exams: Classical Test Theory Evaluation with ITEMAN 4.3. AL-ISHLAH: Jurnal Pendidikan, 16(4), 4363–4375. https://doi.org/10.35445/alishlah.v16i4.5671

Setiadi, I. (2021). Empirical Test Of Content Validity And Construct Validity of the Mathematical Self-Efficacy Scale. Tarbiyah: Jurnal Ilmiah Kependidikan, 10(2), 64–72.

Sudaryono, Rahardja, U., Aini, Q., Isma Graha, Y., & Lutfiani, N. (2019). Validity of Test Instruments. Journal of Physics: Conference Series, 1364(1), 012050. https://doi.org/10.1088/1742-6596/1364/1/012050

Sukmawati, Sudarmin, & Salmia. (2023). DEVELOPMENT OF QUALITY INSTRUMENTS AND DATA COLLECTION TECHNIQUES. Jurnal Pendidikan dan Pengajaran Guru Sekolah Dasar (JPPGuseda), 6(1), 119–124. https://doi.org/10.55215/jppguseda.v6i1.7527

U.S. Department of Energy. (2017). Energy Literacy: Essential Principles and Fundamental Concepts for Energy Education.

Usman, M., Pramudawardani, H., Umam, R., & Takahashi, H. (2025). Synthesizing A Framework and Establishing Content Validity of An Energy Literacy Instrument for Indonesian Students. Islamic Journal of Integrated Science Education (IJISE), 4(3), 175–185.

Usman, M., Suyanta, Pujianto, & Huda, K. (2021). Energy Literacy of Junior High School Students in Indonesia: A Preliminary Study. Proceedings of the 6th International Seminar on Science Education (ISSE 2020), 609–614.

Usman, M., Syakur, A., & Sangkala, N. R. (2025). IDENTIFIKASI PENCAPAIAN LITERASI ENERGI PESERTA DIDIK PADA PEMBELAJARAN BERBASIS STEM DENGAN BANTUAN SIMULASI PhET. Jurnal Biogenerasi, 10(4), 2354–2362.

Vincent, W., & Shanmugam, S. K. S. (2020). The Role of Classical Test Theory to Determine the Quality of Classroom Teaching Test Items. Pedagogia: Jurnal Pendidikan, 9(1), 5–34.

Yusup, M. (2021). Using the Rasch model for the development and validation of an energy literacy assessment instrument for prospective physics teachers. Journal of Physics: Conference Series, 1876(1), 012056.

Zhang, L., Liu, X., & Feng, H. (2023). Development and validation of an instrument for assessing scientific literacy from junior to senior high school. Disciplinary and Interdisciplinary Science Education Research, 5(1), 21. https://doi.org/10.1186/s43031-023-00093-2