Psychometric Evaluation of a Cognitive Test for Accreditation Assessors in Academic Libraries
DOI:
https://doi.org/10.37640/jip.v18i1.2816Keywords:
Library accreditation, Assessors, Higher-education Library accreditation instrument, Cognitive instrument, Item analysisAbstract
Limited empirical evidence exists regarding the validity of instruments used to assess evaluative competence in academic libraries accreditation. The 2025 revision of the Indonesian higher-education library accreditation instrument assigns the highest weighting to service components, requiring assessors to interpret relationships across indicators. This study examined the psychometric properties of a cognitive test instrument designed to assess such competence. A total of 32 items were administered to 30 assessors, and item characteristics were analyzed using classical test theory. Results indicate highly homogeneous response patterns: 90% of respondents scored in the high category, 81.25% of items were classified as easy, 46.88% had low discrimination indices, and 65.63% contained non-functional distractors. Although the KR-20 reliability coefficient was high (0.84), internal consistency was driven by response homogeneity rather than true variance in ability. These findings suggest that high internal consistency in cognitive assessment instruments may arise from response uniformity rather than reflecting actual variation in underlying ability, particularly in professional evaluation contexts.
Downloads
References
Ansari, M., Sadaf, R., Akbar, A., Rehman, S., Chaudhry, Z. R., & Shakir, S. (2022). Assessment of distractor efficiency of MCQs in item analysis. Professional Medical Journal, 29(5), 730–734. https://doi.org/10.29309/TPMJ/2022.29.05.6955
Chhetri, D. B., & Khanal, B. (2024). A pilot study approach to assessing the reliability and validity of relevancy and efficacy survey scale. Janabhawana Research Journal, 3(1), 35–49. https://doi.org/10.3126/jrj.v3i1.68384
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). SAGE Publications.
Dayadaya, R. M. N., & Sermona. (2024). Analysis of distractor efficiency of the multiple-choice comprehensive examination for technical-vocational pre-service students. International Journal for Multidisciplinary Research, 6(6), 1–6. https://doi.org/10.36948/ijfmr.2024.v06i06.32251
Fadhli, R. (2018). Peran perpustakaan perguruan tinggi dalam research data management untuk mendukung scholarly communication. Khizanah Al-Hikmah: Jurnal Ilmu Perpustakaan, Informasi, dan Kearsipan, 6(2), 122–131. https://doi.org/10.24252/kah.v6i2a4
Fernanda, J. W., & Hidayah, N. (2018). Analisis kualitas soal ujian statistika menggunakan classical test theory dan rasch model. SQUARE: Journal of Mathematics and Mathematics Education, 2(1), 49–60. http://dx.doi.org/10.21580/square.2020.2.1.5363
Haryono, B. S., & Cahyono, D. (2020). Implementasi kebijakan standar nasional perpustakaan perguruan tinggi di Perpustakaan Universitas Negeri Malang. BACA: Jurnal Dokumentasi dan Informasi, 42(2), 179–191. https://doi.org/10.14203/j.baca.v41i2.640
Ikhsanudin, I., Novaliah, N., Hidayatullah, H., & Almizi, M. (2023). A practical using of the QUEST program to analyze the characteristics of test items in educational measurement. JISAE (Journal of Indonesian Student Assessment and Evaluation), 9(1), eP2307. https://doi.org/10.21009/jisae.v9i1.31163
Karim, S. A., Sudiro, S., & Sakinah, S. (2021). Utilizing test items analysis to examine the level of difficulty and discriminating power in a teacher-made test. EduLite: Journal of English Education, Literature, and Culture, 6(2), 256–269. http://dx.doi.org/10.30659/e.6.2.256-269
Lintangesukmanjaya, R. T., Rizka, A. M., Damarsha, A. B., & Dwikoranto. (2025). Rasch model analysis: Case study in SDGs trend point 7 in physics learning based domicile. The Journal of Indonesia Sustainable Development Planning, 6(2), 173–184. https://doi.org/10.46456/jisdep.v6i2.608
May, L. (2019). Dynamic research support for academic libraries [Book review]. Journal of Librarianship and Scholarly Communication, 7(1), eP2307. https://doi.org/10.7710/2162-3309.2307
Moto, A., Musyarofah, L., & Taufik, K. S. (2022). Developing item analysis of teacher-made test for summative assessment of seventh grade of SMPN 8 Komodo in academic year 2020/2021. Budapest International Research and Critics Institute Journal, 5(1), 5305–5315. https://doi.org/10.33258/birci.v5i1.4237
Nugroho, P. A. (2022). Korelasi antara jumlah publikasi dosen dengan penggunaan layanan RemoteXs Perpustakaan Universitas Airlangga. Daluang: Journal of Library and Information Science, 2(1), 62–70. https://doi.org/10.21580/daluang.v2i1.2022.10060
Rezigalla, A. A. (2022). Item analysis: Concept and application. In M. S. Firstenberg & S. P. Stawicki (Eds.), Medical education for the 21st century. IntechOpen.
Setiawan, F., Andhika, B., & Yudiarso, A. (2025). Evaluation of validity and reliability of the eight-item Body Shape Questionnaire (BSQ-8C) Indonesian version: Mixture Rasch model approach. Jurnal Sains Psikologi, 14(1), 39–55. http://dx.doi.org/10.17977/um023v14i12025p39-55
Shakurnia, A., Ghafourian, M., Khodadadi, A., Ghadiri, A., Amari, A., & Shariffat, M. (2022). Evaluating functional and non-functional distractors and their relationship with difficulty and discrimination indices in four-option multiple-choice questions. Education in Medicine Journal, 14(4), 55–62. https://doi.org/10.21315/eimj2022.14.4.5
Wahab, A., Hasibuan, A., Siregar, R., Risnawaty, & Ningsih, T. Z. (2023). Item analysis of final examination questions for social studies in junior high schools through the ITEMAN program. Journal of Educational Research and Evaluation, 7(3), 526–536. https://doi.org/10.23887/jere.v7i3.59239
Wind, S. A. (2022). Identifying problematic item characteristics with small samples using Mokken scale analysis. Educational and Psychological Measurement, 82(4), 747–756. https://doi.org/10.1177/00131644211045347
Yudiarso, A., Ardhiani, I. W., Surya, R., Watimena, F. Y., & Kanzaki, M. (2025). Psychometric properties of the 18-item Indonesian mental toughness questionnaire using the Rasch model and machine learning. Psikohumaniora: Jurnal Penelitian Psikologi, 10(1), 747–756. https://doi.org/10.21580/pjpp.v10i1.23214
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Trimo Septiono, Awaludin Tjalla
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors agree that this article remains permanently open access under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License
