An estimated 500–600 children are born with CHD in Norway each year. Approximately 25% of them have severe defects and require early diagnosis and treatment. Reference Leirgul, Fomina and Brodwall1 CHD is still a major cause of infant death, and approximately 10% of Norwegian children with severe heart disease die during the first two years of life. Reference Wik, Jortveit, Sitras, Døhlen, Rønnestad and Holmstrøm2 Recent research shows that 29% of these deaths occurred unexpectedly outside of/unrelated to surgery, 60% of which after gradual deterioration at home. Reference Wik, Jortveit, Sitras, Døhlen, Rønnestad and Holmstrøm3 Many parents find it difficult to recognise worsening symptoms in their children. They also struggle to describe the physiological and behavioural changes that occur and decide what action to take. Reference Tregay, Brown and Crowe4
In Britain, an expert group suggested actions to improve services and reduce adverse events after discharge. They recommended education and training of parents before discharge, clear guidance to families and health professionals on “what is normal” for that child as well as signs and symptoms to look for and how to respond to them, and the distribution of important contact numbers. They also recommend early warning tools that should be nationally standardised to improve navigation of the complex services pathway. Reference Crowe, Knowles and Wray5
To support parents’ ability to perform their new caregiving responsibilities and detect deterioration after discharge, comprehensive interstage home monitoring programmes have been developed for parents of infants with single ventricle. Reference Gaskin, Wray and Barron6,Reference Rudd, Ghanayem, Hill, Lambert, Mussatto and Nieves7 Such programmes use combinations of written material, films, and digital solutions that aims to educate parents, support the health professionals teaching them, and offer tools for detecting deterioration. Some of these programmes exchange data digitally with healthcare teams. In Norway today, the population of infants with single ventricle is small, but recent data show that other infants with CHD also are in need of supportive initiatives. Reference Wik, Jortveit, Sitras, Døhlen, Rønnestad and Holmstrøm3 In addition, strategies to support parents at home must be compatible with existing health services.
Based on these challenges for parents, one of our project group members suggested the concept of a digital tool for decision support in 2018. A multidisciplinary team further developed this concept. Hence, this article aims to describe the development and usability of a personalised application for mobile phones that seeks to interactively increase parents’ awareness of their infant’s normal (habitual) condition, help them recognise signs of deterioration, and help decide who and when to contact for help.
Materials and methods
An interdisciplinary group from Oslo University Hospital developed the Heart OBServation app to support parents of infants with severe CHD. This group of infants includes a large clinical spectrum. Some infants require extensive monitoring at home, whereas others can be managed through parental care.
Development of intervention
The Heart OBServation app was developed using an iterative systematic evaluation process (Fig 1). It included a combination of: 1) exploration and stakeholder involvement; 2) literature review and theoretical conceptualisation; 3) development of features and content; 4) study of features’ usability; and 5) revision of HOBS, incorporating this study’s findings.
Exploration and stakeholder involvement
The initial goal was to reduce stress and mortality by providing parents with a decision support tool. This idea was presented using paper prototypes to the Norwegian Association for Children with Congenital Heart Disease, health professionals at the Department of Pediatric Cardiology, and Neonatal Intensive Care Unit at Oslo University Hospital. Users and medical professionals endorsed this concept and the clinic formally approved it. We established a multidisciplinary and user-centred project group to guide the development of content and features in the application. To ensure that the user perspective was captured, the mother of a child with CHD participated in the development process throughout the project. The chief adviser in the Norwegian Association for Children with CHD provided additional contact with parents for feedback during the development process. To ensure a flexible and intuitive system of features, we included a user experience designer. Through an iterative process of exploration and stakeholder involvement, one test group included six parents of children with CHD and four test groups consisted of 28 nurses and two neonatologists from the neonatal ICU. The patients installed a prototype of the mobile application, completed tasks using a questionnaire, and provided written and oral feedback about the features. Following revision based on this feedback, four parents of infants with CHD hospitalised in the neonatal ICU used a test phone for one day, which also gave us valuable feedback
Figure 1. Iterative stages in the development of the Heart OBServation (HOBS) app.
Literature review and theoretical conceptualisation
During the iterative development process, we explored two main theoretical perspectives on how to build supportive features in the application. The application was initially intended as a decision support tool. Such tools usually include a combination of general and individual health information to support decisions regarding patients’ health. They can advise users if something is normal, show information relevant to the problem, and recommend actions based on an algorithm. Reference Makhlysheva, Kristian, Ruiz, Bakkevoll and Pedersen8 Infants with CHD are vulnerable, and their symptoms may be subtle and similar to normal physiological and developmental variations. Hence, we acknowledge the risk of providing clear recommendations of actions based on an algorithm. With the educational goals in mind and to prevent incorrect decisions, we designed the application as an educational tool. HOBS suggests which signs to look for, how parents could assess them, and when to contact health professionals based on their own interpretations. Reference Nystrom9 Since no automated recommendations are made, the HOBS app does not qualify as a medical software device. 10
Developing the application as a capability-enhancing decision support tool raised the importance of including features to support discharge preparation and readiness. Parents in the target group for HOBS were in transition from hospital to home care for infants with CHD. Readiness for discharge includes competence managing self-care at home, receiving adequate support to cope with life after leaving the hospital, the psychological ability to manage the process, and adequate information and knowledge to respond to common problems. Reference Galvin, Wills and Coffey11 To promote competence managing the infant’s care, available support and adequate knowledge were within the scope of the application and intertwined in the features.
Development of features and content
After the initial development phase, we decided to include the following six features:
My Child
In this feature, parents in collaboration with health professionals register information about their child’s birth (weight and date), diagnosis, treatment, and needs after discharge, including surgery, pathophysiology, nutritional demands, medications, and need for health-related equipment. The application uses this information to individualise observations in “Normal for my child”, questions in the “Assessment function”, and to adapt personalised “Information” (Fig 2, Image 1, upper square). A list of tasks to be completed before discharge is included in this section to ensure the completion of settings and guidance of use.
Figure 2. Features in Heart OBServation app (HOBS).
Normal for my child
This feature is built upon the concept that awareness of the infant’s normal condition enhances the recognition of signs of deterioration. Reference Crowe, Knowles and Wray5 The consequences of CHD vary, and the “normal” status must be personalised. Reference Daily, FitzGerald, Downing, King, del Rey and Ittenbach12 Hence, in this feature, parents select their infant’s condition and behaviour from predetermined alternative descriptions of respiration, circulation, elimination, nutrition, sleeping, and satisfaction. There are 6–10 categories depending on the surgical procedures and monitoring equipment. In each category, parents choose from a list of options that best matches their child. The healthiest choice is at the top of the list, making symptoms of deterioration comprehensible (Fig 2, Images 1 and 2). Parents add concrete numbers for oxygen saturation, respiratory rate, and heart rate.
Information
Individualised information based on the settings from “My child” is allocated to a reading list. The list contains links to nationally approved information for parents, developed by cardiologists and other healthcare professionals working with infants with CHD. This feature aims to support parents in managing self-care at home and responding to common problems Reference Galvin, Wills and Coffey11 (Fig 2, Image 3).
Contact
Telephone numbers to the national centre at Oslo University Hospital are listed in this feature. Information about who and when to call on different occasions is explained, and parents can call directly from the application (Fig 2, Image 4). When the infant is ready for discharge, parents should register the allocated telephone numbers to local health services before leaving the hospital. Reference Galvin, Wills and Coffey11
Assessment of my child
The assessment feature covers knowledge to support what to look for. Reference Crowe, Knowles and Wray5 It consists of three areas.
-
1) The general condition was based on the normal condition of the infant. Based on the settings, parents receive 7–14 questions. The selection of questions includes measures from the paediatric early warning score and more specific symptoms of deterioration in infants with CHD. Reference Tregay, Brown and Crowe4,Reference Lambert, Matthews, MacDonell and Fitzsimons13 Parents assess respiration, circulation, elimination, eating, sleeping, satisfaction, and well-being. They answer “yes” or “no” to questions about deterioration, and a tip-button (light bulb) may instruct them on how to do it, how to interpret the result, and what to do (Fig 2, Image 5a). At the end of the assessment, they receive a summary of worsening symptoms and a general advice to contact healthcare professionals if they are still uncertain or uncomfortable (Fig 2, Image 5b).
-
2) Wound assessment: Parents may take and save pictures of wounds or other visual objects of interest they want to follow. The tip button provides advice about signs of infection.
-
3) Weight registration and other measurements: Weight may be plotted, and weight gain is calculated as sufficient or not visualised by a red cross or a green tick in the summary function.
Summary
The Summary feature provides an overview of completed assessments. Previous assessments of the infants’ conditions are presented as bar charts, and other measurements are presented as curves (Fig 2, Image 6). This function provides an overview and may support communication with health professionals during consultation and follow-up.
Legal aspects
The Information Security Department at the Oslo University Hospital approved the data protection impact assessment and risk analysis of the application for privacy considerations and information security.
Study of features’ usability
After the initial development, we examined the experiences of HOBS features among parents and nurses in a usability study
Study intervention procedure
The eligibility criteria for participation among the parents were that they owned a smartphone, were parenting an infant hospitalised with severe CHD at Oslo University Hospital, gestational age above 34 weeks, and had appropriate Norwegian communication skills. We recruited families after the infant’s cardiac surgery or the final diagnosis if no intervention was conducted before discharge. They signed a written consent. Parents got the Heart OBServation app on their own phones and received a 10–15-min introduction of its main features. Because infants with CHD varies in severity and need of monitoring, each family was advised to make assessments until they were confident in what to look for, before consultations and if they felt unsure of the infant’s condition. An electronic reminder in the hospital’s electronic system for monitoring and ordinations popped up twice a day for the nurses to ensure continuation of guidance before discharge. A checklist of nursing tasks to prepare parents for discharge was attached to the patient’s binder.
Nurses who engaged in family guidance and patient care received a 20-min lecture on the purpose and use of Heart OBServation, its features, and the tasks to complete together with the families. They were encouraged to consult the e-learning course on how to guide the parents and other resources available on the Heart OBServation website, established to support health professionals nationwide (www.hobs.no). A test phone with the application was also available in the unit to make nurses confident with the Heart OBServation app features and content.
Semi-Structured parent interviews
Parents participated in two semi-structured interviews via phone: at the time of discharge from OUH and one month later. The researcher followed a semi-structured interview protocol. The topics in the interviews at discharge were questions about app features and functionality or any other comments. The second interview focused on the usability of the Heart OBServation app features and change requests.
System usability and system use
After both interviews, the System Usability Scale was sent electronically to the parents to measure system usability of the Heart OBServation app. Reference Bangor, Kortum and Miller14 The instrument gives a general score of system usability and consists of 10 items, with five response options from strongly disagree to strongly agree. Total scores range from 0 to 100, with 100 indicating the most positive response.
In addition, we collected system user logs to capture the use of the assessment function and information links.
Focus group interviews with nurses
Eight nurses from three departments that followed the families in the study were recruited to two focus group interviews after the last family had completed their participation. Their work experience was 1–35 years. The moderator followed a semi-structured interview guide and used a PowerPoint presentation of the Heart OBServation app features to refresh memories and avoid misconceptions. Each feature was discussed, and requests for additions and changes were encouraged. An observer noted the ambiguity and wrapped up the discussion to clarify the interpretations.
Deductive framework analysis
A deductive framework analysis was used to evaluate the features of the application. Reference Green and Thorogood15 Comments were sorted as they related to the Heart OBServation app features: 1) my child, 2) normal for my child, 3) measure of condition, 4) wound observation, 5) weight, 6) information, 7) summary, 8) contact, and 9) overall impression. Comments about each feature were interpreted, discussed, and condensed into a meaningful unit representing the patients’ views. Initially, we analysed data from parents and nurses separately. Next, we merged the data sets and compared them to determine the necessary revisions to the Heart OBServation app. Finally, all data in the framework was analysed to explore ideas for new features and changes to the existing design.
Results
Patients of the usability study
Eleven families were consecutively included over a period of three months. One infant recovered before discharge, and one family did not respond to phone calls or questionnaires and did not use the application after hospital discharge (Table 1). The diagnoses represented among the infants were “tetralogy of Fallot”, “Ebstein anomaly”, “aortic stenosis”, “coarctation of the aorta”, “interrupted aortic arch”, and “truncus arteriosus”. For more demographic information regarding the parents and infants, see Table 1. All infants attended follow-up after discharge, and no adverse events occurred during the study.
Table 1. Parent and infant demographics, clinical characteristics, and user logs (n = 9)
Results regarding system usability and system use
The overall system usability scores after discharge were 82.3 and 81.7 after one month, indicating good system usability (70–100). Reference Bangor, Kortum and Miller16
Five parents used the assessment function to evaluate whether they should contact health professionals or not. Only one parent made contact and that was related to observation of increased respiratory distress. The amount of assessment varied depending on CHD severity and cardiologist recommendations. Each parent entered 2–9 different information links (median, 6), but many of the links were entered several times, with a peak at the beginning of the study (Table 1). The most frequently used link was about the infant’s diagnosis, postoperative care, and consequences of the infant’s condition. Users overlooked the possibility of reading extended information in additional tabs within the allocated information.
Semi-Structured interviews and focus groups
Both parents were invited to participate in semi-structured interviews, but only three fathers joined after discharge from Oslo University Hospital and two after one month at home. The first interview lasted for a mean 12 min (range, 7–14 min). Five of these interviews occurred when the child was still at a local hospital and four when they were at home. The second interview lasted for a mean 18 min (range, 10–26 min). The focus group interviews with nurses lasted for 75 and 90 minutes.
The Heart OBServation features were considered as intuitive and easy to use. Users found the contents and features valid and the information relevant, available, and easy to understand. They requested only minor revisions to the features and content. Table 2 presents an overview of the results after the framework analysis, illustrating quotes regarding each feature
Table 2. Results from deductive analysis of parents‘ and nurses‘ experiences of the functionality of the heart OBServation (HOBS) app
Final revision of the Heart OBServation app
Overall, the results from the qualitative analysis, System Usability Scale, and log data provided useful information for the revision of Heart OBServation. The project group held digital workshops to review, select, and decide on revisions. The final revisions included nuance some categories in “normal to my child”, more explicit advice in some areas of “the assessment feature”, add an interactive discharge preparation list, and redesign “the information feature” to be more personalised. See Table 2 for details of the requested and fulfilled revisions.
Discussion
In this paper, we report on the process of development, usability testing, and revision of a smartphone application to support the parents of infants with severe CHD as an alternative to traditional paper-based information. The results of this usability study are promising, as the usability scores were high, and the parents and nurses evaluated Heart OBServation baby as easy to use. Users considered the content and features valid and requested only minor revisions. Interviews with parents and nurses provided useful information about their use of the application and necessary revisions.
An important aim of the Heart OBServation application was to help parents know what was normal for their child and use it as a baseline for assessments. Reference Crowe, Knowles and Wray5 Parents and nurses expressed that the use of Heart OBServation drew attention to and provided control over something parents would normally have been less aware of. Choosing between defined categories does not precisely describe the infant’s condition, but the added nuances may refine the parents’ apprehension about their child. The parents did not express any problems when assessing changes from their registered normal condition. Our interpretation is that the personalised design facilitated the parents’ awareness of “what is normal” versus not. Reference Crowe, Knowles and Wray5
Support for discharge preparation is an important aim of the Heart OBServation app. The existing discharge checklist for the application was neither interactive nor personalised, and nurses were unsure about parents’ ability to execute the right initial settings by themselves. To agree on completion of learning tasks is important for the empowerment of parents and discharge preparations. Reference Galvin, Wills and Coffey11 Such empowerment has been facilitated in programmes to support parents of infants with single ventricle in home monitoring programmes. Reference Tanem, Pridham and Schroeder17 Hence, an interactive discharge feature for parents based on personalised settings in “My child” to confirm readiness for discharge was included and advanced underneath the home icon.
Parents evaluated the information function as relevant, available, and easy to understand. Further investigations revealed that some information headings were general, and user logs revealed that tabs used to explore additional electronic information were overlooked. Hence, we revised the information feature to use a more personalised design and removed the tabs to obtain hidden information.
Requests for new functionality, such as sharing information and settings between parents, were not included because of financial limitations and data privacy matters. We also rejected the request of a calendar in Heart OBServation to track events and consultations because this function is available on all mobile phones.
For unknown reasons, one family did not use the application after discharge. As mentioned by a nurse in one of the focus groups, this could relate to the fact that parents not necessarily appreciate mobile applications, or it might be overwhelming to capture in a chaotic situation. Reference Woolf-King, Anger, Arnold, Weiss and Teitel18 We do not know whether paper information and contact with health professionals would have been preferred in this case. However, the availability of information in a mobile app may be beneficial to all parents as experienced by parents and nurses in this study. An ongoing study will compare whether Heart OBServation or written information is preferred in discharge preparation and follow-up.
Fathers participated in only five of the 17 interviews. Nevertheless, these fathers were positive about the features of the application. Reasons for limited participation may have been that mothers are still primary caregivers on paid leave, hospitals’ coronavirus disease 2019 precautions and visit restrictions disfavoured fathers, and fathers had started working by the time of the second interview. In this situation, the possibility of sharing content could have been favourable to utilise Heart OBServation for both parents.
There are limitations to the present work related to the short trial period and small number of patients. One of the aims of the application is to empower parents to recognise deterioration in their children. Although many parents acknowledged the benefit of having a checklist of symptoms to look for, only one family experienced deterioration during the trial period, which is not enough to claim that Heart OBServation increases their capability to recognise and act on possible deteriorations. On the other hand, half of the included families used the assessment function when managing uncertainty of symptoms like crying and vomiting and felt reassured and did not contact healthcare professionals. This reassurance could be a potential benefit.
Conclusion
This study is the first to evaluate usability of features in a mobile application, to support parents to infants with a broad spectrum of severe CHD diagnosis. The Heart OBServation app combines new and already established strategies to prepare and support parents with severe CHD in one interactive application. This application aims to increase parents’ awareness of their infant’s normal (habitual) condition, help them assess signs of deterioration, decide who and when to contact for health services if necessary, and what to report. Heart OBServation was well received by parents and nurses in this usability study. The feasibility and benefits of this application in clinical practice will be investigated in further studies.
Acknowledgements
The authors acknowledge the solution architect and developer Henning Harmens in Hena AS, who collaborated with user experience designer Snorre Berge in Knirkefritt and illustrator Flu Hartberg, who drew the icons. We also acknowledge Pia Bråss, chief adviser in the Norwegian Association for Children with CHD, and Eli Våbenø, mother of a child with CHD, who provided valuable feedback in the development process. Other contributors in the process have been physician Astri Lang and nurse Ragnhild Hillestad Andersen. SAGE Author services performed the language editing before the last version.
Financial support
The Dam Foundation (grant number 2019/HEl-261465) supported the development of the Heart OBServation app. The Association of Children with Cardiac Disease Research Foundation (grant number 119) financed the present acceptance and usability study.
Conflicts of interest
None.
Ethical standards
Ethical approval for this study was obtained from the Regional Committee for Medical and Health Research Ethics, South East, Norway (2019/1271). All patients received written information about the study before giving their written consent to participate and were informed that they could withdraw from the study at any time.
Open access
