Development of a Evaluation Tool for Age-Appropriate Software in Aging Environments: A Delphi Study

Abstract: Objective: We aimed to develop a dependable reliable tool for assessing software ageappropriateness. Methods: We conducted a systematic review to get the indicators of technology ageappropriateness from studies from January 2000 to April 2023.This study engaged 25 experts from the fields of anthropology, sociology,and social technology research across, three rounds of Delphi consultations were conducted. Experts were asked to screen, assess, add and provide feedback on the preliminary indicators identified in the initial indicator pool. Result: We found 76 criterias for evaluating quality criteria was extracted, grouped into 11 distinct domains. After completing three rounds of Delphi consultations,experts drew upon their personal experiences,theoretical frameworks,and industry insights to arrive at a three-dimensional structure for the evaluation tooluser experience,product quality,and social promotion.These metrics were further distilled into a 16-item scale, and a corresponding questionnaire was formulated.The developed tool exhibited strong internal reliability(Cronbach's Alpha is 0.867)and content validity(S-CVI is 0.93). Conclusion: This tool represents a straightforward,objective,and reliable mechanism for evaluating software's appropriateness across age groups. Moreover,it offers valuable insights and practical guidance for designing and developing of high-quality age-appropriate software,and assisst age groups to select software they like.

• The paper introduces STAGE, a multidimensional tool that uses Delphi and AHP to rigorously evaluate the age-appropriateness of software.
• The study refines 76 potential indicators to 16 final metrics through expert consensus and systematic literature review, ensuring strong reliability.
• Empirical testing confirms robust psychometric properties and emphasizes the integration of social inclusivity and ethical considerations.

Development of an Evaluation Tool for Age-Appropriate Software in Aging Environments

Context and Motivation

Rapid increases in the global aged population and technological advancement have amplified the importance of digital inclusion for older adults. Traditional evaluation of software suitability in aging populations has focused primarily on usability and functionality, whereas broad social impacts and humanistic aspects have been systematically neglected. Recent evidence underscores that less than 0.1‰ of software in China undergo age-appropriate modifications, revealing a severe inadequacy in actionable evaluation standards for age-oriented software. This study targets this gap through the development of a multidimensional, theory-driven tool specifically designed to assess the age-appropriateness of software for elderly care and participation.

Methodology

A systematic literature review, spanning over two decades, established a comprehensive pool of 703 potential evaluation indicators, later refined to 76. The construction and validation of the evaluation tool relied on a rigorous, three-round Delphi process, engaging a panel of 25 interdisciplinary experts in anthropology, sociology, and social technology. The iterative Delphi rounds employed stringent statistical cutoffs for indicator inclusion—including thresholds for means, full-score frequencies, and coefficients of variation—ensuring statistical robustness and consensus-driven selection.

The study further applied the Analytic Hierarchy Process for indicator weighting and implemented advanced psychometric validation: internal reliability was quantified with Cronbach’s alpha, and content validity was calculated via item- and scale-level CVI. The resulting framework, Software Technology And Geriatric Evaluation (STAGE), comprises a validated 21-question instrument reflecting 16 final indicators after expert consensus.

Structure and Theoretical Underpinnings

Dimensionality

Expert consultation resolved a three-dimensional architecture for the evaluation tool:

• User Experience: Encompassing usability, intelligibility, cost consideration, and service experience.
• Product Quality: Centered on security and innovation.
• Social Promotion: Capturing ethics and social integration.

All dimensions and sub-indicators were weighted according to quantified expert consensus, and two bonus indicators, compliance and sociability, provide broader social policy and adaptability assessment.

Integration of Social Technology Theory

Unlike paradigms that narrowly operationalize 'age-appropriateness' as ease of use, this tool operationalizes the assessment within the "social science and technology" framework. This expands the notion of age-appropriateness to integrate social inclusion, humanistic engagement, ethical considerations, and systemic impacts on older adults, thus aligning the tool’s output with equitable and society-scale objectives.

Evaluation and Empirical Results

The STAGE index underwent empirical pretesting with 26 elderly software users. The overall internal consistency was robust; total Cronbach’s alpha was 0.867, exceeding typical cutoffs for reliability. Sub-dimensions such as “Perceptibility” (0.831) and “Usability” (0.742) demonstrated substantial homogeneity, underlining their primacy for aging-user interaction. Content validity was also exceptional: item-level CVI exceeded 0.78 for all 16 items, and the scale-level CVI attained 0.93.

Strong user-expert alignment was observed on critical indices, notably “Function Learnability” and “Operation Simplicity” (both I-CVI = 1.00). Notably, ethical, cost, and social integration indicators had relatively lower reliability, suggesting either sample-dependent limitations or inherent measurement challenges in these less tangible domains. The tool enables both consumer and expert evaluation; the dual-path methodology facilitates both subjective and objective assessment, enhancing overall validity.

Implications and Future Research

Practical

The tool provides a generalizable, scalable mechanism for assessing, benchmarking, and guiding the development of age-appropriate software for older adults, aiming to catalyze transformation in digital product strategy, regulatory policy, and inclusive technology design. Its explicit integration of user feedback overcomes the traditional expert-driven bias and acknowledges the heterogeneity of older users' lived experiences.

Theoretical

By embedding social inclusivity, interdisciplinary perspectives, and ethical frameworks into the formal assessment structure, the tool establishes a new methodological norm for age-appropriateness that can scale beyond software to other sociotechnical systems. It operationalizes social technology theory within empirical assessment.

Limitations and Prospects

Methodologically, the study is constrained by initial small-sample testing; further validation on varied populations and real-world application scenarios is crucial. Future work prioritizes expansion to larger, more diverse cohorts, field testing, detailed manual development, and adaptation to hardware and system-level technologies. The refinement roadmap is clearly articulated: items with sub-optimal domain Cronbach’s alpha or corrected item-total correlations will be iteratively redeveloped in large-sample settings, contingent on expert consensus.

Conclusion

This study introduces a multidimensional, rigorously validated tool for evaluating the age-appropriateness of software, addressing a critical gap in the digital inclusion of aging populations. The tool’s strong psychometric properties support its immediate applicability and pave the way for deploying social technology frameworks in gerontechnology evaluation. As digital transformation pervades senior care and societal participation, tools such as STAGE will be instrumental for appraising and promoting technologies that are not only technically accessible but socially and ethically aligned with diverse aging cohorts.