Source attributes

Developers of SMART Apps can specify if their application qualifies as a Decision Support Intervention (DSI) when creating a new app in the Developer Sandbox. After doing this developers define key source attributes.

Evidence DSIs source attributes

1. Bibliographic citation of the intervention

2. Developer of the intervention (translation from clinical research or guideline)

3. Funding source of the technical implementation for the intervention(s) development

4. Release and, if applicable, revision dates of the intervention or reference source

5. Use of race

6. Use of ethnicity

7. Use of language

8. Use of sexual orientation

9. Use of gender identity

10. Use of sex

11. Use of date of birth

12. Use of social determinants of health data

13. Use of health status assessments data

Predictive DSIs source attributes

• Details and output of the intervention, including:

  • Name and contact information for the intervention developer;

  • Funding source of the technical implementation for the intervention(s) development;

  • Description of value that the intervention produces as an output; and

  • Whether the intervention output is a prediction, classification, recommendation, evaluation, analysis, or other output type.

• Purpose of the intervention, including:

  • Intended use of the intervention;

  • Intended patient population(s) for the intervention’s use;

  • Intended user(s); and

  • Intended decision-making role for which the intervention was designed to be used/for (e.g., informs, augments, replaces clinical management).

• Cautioned out-of-scope use of the intervention, including:

  • Description of tasks, situations, or populations where a user is cautioned against applying the intervention; and

  • Known risks, inappropriate settings, inappropriate uses, or known limitations.

• Intervention development details and input features, including at a minimum:

  • Exclusion and inclusion criteria that influenced the training data set;

  • Use of variables in paragraph (b)(11)(iv)(A)(5)-(13) as input features;

  • Description of demographic representativeness according to variables in paragraph (b)(11)(iv)(A)(5)-(13) including, at a minimum, those used as input features in the intervention:

    • Race

    • Ethnicity

    • Language

    • Sexual orientation

    • Gender identity

    • Sex

    • Date of birth

    • Social determinants of health data

    • Health status assessment data

  • Description of relevance of training data to intended deployed setting; and

• Process used to ensure fairness in the development of the intervention, including:

  • Description of the approach the intervention developer has taken to ensure that the intervention’s output is fair; and

  • Description of approaches to manage, reduce, or eliminate bias.

• External validation process, including:

  • Description of the data source, clinical setting, or environment where an intervention’s validity and fairness has been assessed, other than the source of training and testing data

  • Party that conducted the external testing;

  • Description of demographic representativeness of external data according to variables in paragraph (b)(11)(iv)(A)(5)-(13) including, at a minimum, those used as input features in the intervention:

    • Race

    • Ethnicity

    • Language

    • Sexual orientation

    • Gender identity

    • Sex

    • Date of birth

    • Social determinants of health data

    • Health status assessment data

  • Description of external validation process.

• Quantitative measures of performance, including:

  • Validity of intervention in test data derived from the same source as the initial training data;

  • Fairness of intervention in test data derived from the same source as the initial training data;

  • Validity of intervention in data external to or from a different source than the initial training data;

  • Fairness of intervention in data external to or from a different source than the initial training data;

  • References to evaluation of use of the intervention on outcomes, including, bibliographic citations or hyperlinks to evaluations of how well the intervention reduced morbidity, mortality, length of stay, or other outcomes;

• Ongoing maintenance of intervention implementation and use, including:

  • Description of process and frequency by which the intervention’s validity is monitored over time;

  • Validity of intervention in local data;

  • Description of the process and frequency by which the intervention’s fairness is monitored over time;

  • Fairness of intervention in local data; and

• Update and continued validation or fairness assessment schedule, including:

  • Description of process and frequency by which the intervention is updated; and

  • Description of frequency by which the intervention’s performance is corrected when risks related to validity and fairness are identified.