Assessing the Psychometric Properties of an AI and Big Data Analytics Adoption Readiness Instrument in Primary Health Care

Abstract

Primary health care (PHC) centres in Saudi Arabia are expected to adopt artificial intelligence (AI) and big data analytics (BDA) for chronic disease management, yet no validated instrument measures readiness for these technologies among practising PHC professionals. This study assessed the content validity, face validity, feasibility and internal consistency of a 67-item AI and BDA Adoption Readiness Instrument developed for PHC centres in Al Madinah. Items were adapted from published scales and organised into eight subscales grounded in the Technology Acceptance Model (perceived usefulness, perceived ease of use) and the Technology-Organisation-Environment framework (organisational readiness, organisational culture and climate, technological readiness, environmental readiness), together with AI-related barriers and overall adoption readiness. Six experts rated each item for relevance and clarity, and a pilot study was conducted with 30 professionals (8 physicians, 8 nurses, 7 public health specialists and 7 chronic disease coordinators) from five PHC centres. All 67 items achieved an item-level content validity index (I-CVI) of at least .83 (modified kappa ≥ .82) on both relevance and clarity. The scale-level index (S-CVI/Ave) was .93 for relevance and .92 for clarity; clarity fell slightly below .90 for the barriers (.89) and adoption readiness (.86) subscales. Mean completion time was 15.0 minutes, and participants rated overall clarity at 4.20 and relevance at 4.40 on a 5-point scale. Cronbach's alpha ranged from .740 (environmental readiness) to .907 (organisational readiness), although the lower 95% confidence limits for organisational culture (.689) and environmental readiness (.564) fell below .70. Three items had corrected item-total correlations below .30, and four items (PEOU10, BR3, BR12, ER2) were revised and re-rated by the expert panel with an I-CVI of 1.00. Perceived usefulness and ease of use correlated with each other (r = .69) and with adoption readiness (r = .51 and .55); organisational readiness and culture overlapped strongly (r = .74), and the barriers subscale was unrelated to the other scales. A one-factor solution for the adoption readiness subscale explained 49.9% of variance. The instrument shows strong content validity, acceptable feasibility and adequate preliminary reliability. Its factor structure, discriminant validity between organisational subscales and test-retest stability should be established in the main survey before it is used to guide AI and BDA adoption in Saudi health clusters.