Real-Time Delphi Consultation on Medication Adherence Technologies: Consistency of Views Among Different Stakeholder Groups

Urska Nabergoj Makovec (Coresponding Author), Ivana Tadic, Catherine Goetzinger, Janette Ribaut, Pilar Barnestein-Fonseca, Frederik Haupenthal, Maria Teresa Herdeiro, Sean Patrick Grant, Cristina Jácome, Fatima Roque, Dins Šmits, Alexandra Dima

Research output: Contribution to conferenceAbstractpeer-review


Objectives: ENABLE COST Action (CA19132) aims to develop an online repository for
medication adherence technologies (MATech). We explored stakeholders’ level of
agreement on relevance and clarity of the proposed MATech definition and repository
structure as well as the consistency among their views. Methods: Researchers,
healthcare practitioners (HCP), policy makers, patient representatives and health IT
developers from 29 countries participated in a real-time online Delphi survey to
consult on the relevance and clarity of the MATech definition and 21 attributes on
product and provider information, medication adherence content, and evaluation and
implementation. The Interpercentile Range Adjusted for Symmetry analysis from the
RAND/UCLA Appropriateness Method was used to determine the level of agreement
on 9-point scale for each stakeholder group. Responses of participants identifying
themselves with more than one group were used in all identified groups. Consistency
of results was calculated with intraclass correlation coefficient (ICC) for relevance and
clarity, overall and separately. Results: Among the 83 participants, 50 identified as
researchers, 45 HCP, 19 policy makers, 18 patient representatives and 24 health IT
developers. Overall ICC was 0.97, ICC for relevance and clarity was 0.95 and 0.97,
respectively, indicating good consistency across groups. The median level of agree-
ment with MATech definition ranged from 6.74–7.05 (patients-researchers) and clarity
from 7.09-7.59 (patients-IT). Implementation reached highest relevance in 4 groups
(medians 7.54, 7.76, 7.75, 7.72 for research, policy, patient, IT respectively) and clarity
in 2 groups (7.62, 7.76 for research, IT respectively). ISO certification was the least
relevant attribute for all groups (median range 6.19-6.33, patient-HCP), and with
highest clarity for policy (8.01). The least clear were attributes on medication
adherence intervention features (5.63, 5.67, 5.26 for research, policy, patients
respectively) and phase (5.59, 5.51 for HCP and IT respectively). Conclusions: High
consistency across groups indicates a solid common ground for the development of
the MATech repository.
Original languageEnglish
Number of pages1
Publication statusPublished - 1 Dec 2022
EventISPOR (Professional Society for Health Economics and Outcomes Research) Europe Conference 2022: Collaborating Across Borders: Building & Using Evidence to Enable Access - Vienna & Virtual participation, Vienna, Austria
Duration: 6 Nov 20229 Nov 2022


ConferenceISPOR (Professional Society for Health Economics and Outcomes Research) Europe Conference 2022
Abbreviated titleISPOR Europe 2022
Internet address

Field of Science*

  • 3.3 Health sciences

Publication Type*

  • 3.3. Publications in conference proceedings indexed in Web of Science and/or Scopus database


Dive into the research topics of 'Real-Time Delphi Consultation on Medication Adherence Technologies: Consistency of Views Among Different Stakeholder Groups'. Together they form a unique fingerprint.

Cite this