Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

OBJECTIVE: To develop and validate a questionnaire to assess the usability of clinical decision support systems (CDSS) and to assist in the early identification of usability issues that may impact patient safety and quality of care. DESIGN: Mixed research methods were used to develop and validate the questionnaire. The qualitative study involved scale item development, content and face validity. Pilot testing established construct validity using factor analysis and facilitated estimates for reliability and internal consistency using the Cronbach's alpha coefficient. SETTING: Two hospitals within a single National Health Service Trust. PARTICIPANTS: We recruited a panel of 7 experts in usability and questionnaire writing for health purposes to test content validity; 10 participants to assess face validity and 78 participants for the pilot testing. To be eligible for this last phase, participants needed to be health professionals with at least 3 months experience using the local hospital electronic patient record system. RESULTS: Feedback from the face and content validity phases contributed to the development and improvement of scale items. The final Healthcare Systems Usability Scale (HSUS) proved quick to complete, easy to understand and was mostly worded by potential users. Exploratory analysis revealed four factors related to patient safety, task execution, alerts or recommendations accuracy, the effects of the system on workflow and ease of system use. These separate into four subscales: patient safety and decision effectiveness (seven items), workflow integration (six items), work effectiveness (five items) and user control (four items). These factors affect the quality of care and clinician's ability to make informed and timely decisions when using CDSS. The HSUS has a very good reliability with global Cronbach's alpha 0.914 and between 0.702 and 0.926 for the four subscales. CONCLUSION: The HSUS is a valid and reliable tool for usability testing of CDSS and early identification of usability issues that may cause medical adverse events.

Original publication




Journal article


BMJ Open

Publication Date





Health & safety, Health informatics, Information technology, QUALITATIVE RESEARCH, Quality in health care, Telemedicine, Humans, Decision Support Systems, Clinical, Reproducibility of Results, State Medicine, Electronic Health Records, Delivery of Health Care, Surveys and Questionnaires