The article describes the methodological approach to, and results of an evaluation of a comprehensive electronic health record (EHR) in the shake down phase, shortly after its implementation at a regional hospital in Denmark.
A formative evaluation based on a mixed-methods case study, designed to be interactive and concurrent was conducted at two hospital departments based on the updated DeLone and McLean framework for evaluating information systems success.
To ascertain user assessments of the EHR, we distributed a questionnaire two months after implementation to four groups of staff (physicians, nurses, medical secretaries, and physiotherapists; n=244), and at the same time we conducted thirteen individual, semi-structured interviews with representatives from these four groups. Subsequently, seven follow-up focus group interviews were conducted with the four above-mentioned groups, in order to go deeper into specific user assessments. Simultaneously, focus group interviews with two IT departments and the implementation team were conducted, to gain insight into system provider assessments of the implementation process and the EHR. Before, during, and after implementation, 88 h of ethnographic observation were carried out, to give the researchers an understanding of the daily routine of staff, and their use of health records. Finally, daily system performance data were obtained, to gather factual information on system response and downtime.
Overall, staff had positive experiences with the EHR and its operational reliability, response time, login and support. Performance was acceptable. Medical secretaries found the use of the patient administration module cumbersome, and physicians found the establishment of the overview of professionally relevant data challenging. There were demands for improvements to these and other functionalities, and for the EHR to be integrated with other systems and databases.
Evaluations immediately following implementation are inherently difficult, but was required because a key role was to inform decision-making upon enrollment at other hospitals and systematically identify barriers in this respect. The strength of the evaluation is the mixed-methods approach. Further, the evaluation was based on assessments from staff in two departments that comprise around 50% of hospital staff. A weakness may be that staff assessment plays a major role in interviews and survey. These though are supplemented by performance data and observation. Also, the evaluation primarily reports upon the dimension 'user satisfaction', since use of the EHR is mandatory. Finally, generalizability may be low, since the evaluation was not based on a validated survey. All in all, however, the evaluation proposes an evaluation design in constrained circumstances.
Despite inherent limitations, evaluation of a comprehensive EHR shortly after....