NASS cat help

Don’t make the mistake of treating a new technology as a plug-and-play solution. You need to ask a lot of questions about it before you can be sure it’s the right tool for the job. New technologies often look appealing and promising until we consider all aspects of the innovation process.

Find out more about the technology and assess its quality and implications.  If you are not the creator of the technology, familiarise yourself with all relevant aspects of it or ask an expert. Look at it; play with it; do a ‘walk through’ the imagined use case. Will this product really help with what you are planning to achieve? Could a different technology (perhaps one that is already tried and tested) do a similar job with less hassle?

NHS apps library – a searchable database of quality-assured smartphone health apps

Publicly available ‘curated’ databases of apps – for example:

• Psyberguide for mental health apps
• ORCHA, an independent organisation that evaluates apps

Find out more about where the technology will come from and associated challenges.

Ideally the building blocks for your chosen technology e.g. coding platform, devices etc can be accessed or purchased easily (no long waiting periods or unreliable supply chains). Ideally, the technology should not depend on a single vendor/device/coding language etc, but work (or have the potential to work with or easily change to) others as well. They will have been tested extensively so you don't have to worry about these components being dependable. Conflicts of interest and claims to intellectual property (IP) should be sorted out before the project begins. It should be clear who will fund the technology, what it will cost and which costs are covered (set up, maintenance, updates etc).

Identify and address the key points where technical complexity will impact on success. 

Find out about any unknowns and dependencies as soon as possible, and develop a plan to deal with them, including alternatives or workarounds. Reduce unnecessary technical integration.  Integration between multiple systems makes everything more complex. Ask whether it is really necessary or if there are ways to avoid or delay this, especially during initial testing. But bear in mind that some forms of technical integration (e.g. to make a new piece of software accessible from within a patient's existing electronic record) may make the technology simpler for a clinician to use.

Consider how the technology will disrupt the system. 

Map possible disruptions and take steps to avoid or mitigate them. Can you modify the technology to make it less disruptive? Can you reduce knock-ons by adjusting other systems or processes? What measures might you put in place (e.g. small-scale pilot running in parallel with the old service, on-the-job training, help desk) to deal with the disruption until systems and processes have evolved to accommodate the new technology?  We pick up this important point again under ‘the organisation’ below.

This project is only going to work if all stakeholders gain something of value from it.

Consider how to increase the technology’s appeal to investors. If the technology is at an early stage of development, what is its likely upstream value as viewed by investors (especially the business case for generating profits, further spin-offs, and highly qualified jobs), drug and device regulators (preliminary evidence of efficacy and safety), and financial regulators (auditable business processes and governance)? Can the technology be ‘de-risked’ by removing costly but inessential features? See the Guidance and Impact tracking System (GAITS) resource linked above.

 Consider how to increase the technology’s value to patients or citizens. If a technology is meant to be used by patients or lay people, its potential benefits must be weighed against its costs (and the person’s willingness and ability to contribute to these), the work needed to use it (and whether the person or their carer is able and willing to do that work), and the desirability of medicalisation and surveillance.  Can the design be improved to make the technology more appealing? Can the data be visualised in a way patients or carers can engage with?

See links above under ‘Responding to complexity in the illness’

Getting the most out of PROMS – A guide to using patient-reported outcome measures to assess whether an intervention or technology is actually improving outcomes that are valued by patients

A guide to PROMs methodology from NHS Digital (using hip and knee replacement as an example)

Identify evidence of effectiveness and cost-effectiveness. If the technology is at a more advanced stage of development, there may be research evidence comparing its effectiveness (does it work?) and cost-effectiveness (is it good value for money?) with ‘usual care’ and measuring an outcome that is important to patients.  Try these resources:

NICE Evidence Standards for digital health technologies – These cover both effectiveness and economic impact.

Consider real-world value issues. Is there a realistic assessment of the challenges of implementing this innovation at scale in a particular public-sector health or care environment? Even when something has been shown to be cost-effective, it may not be locally affordable or a funding priority.

The NICE Evidence Standards website linked above offers a budget impact guide and budget impact template for local cost planning.

This project is only going to work if the people you want to use the technology are able and willing to do so.

Address acceptability, accessibility and usability for patients and citizens. If the technology requires input from a patient, carer or other lay person, will they find the product aesthetically pleasing and easy to use? Does the technology make sense, for example, in the context of how patients and carers already do things, their routines and existing tools they use to support their work? Remember, everyone is different. Some people have limited vision or dexterity; some people find instructions hard to understand. Can you make the product more accessible? Is it worth building design changes in now or planning to do so in the future (e.g. after proof of concept testing)?  If the technology includes several components, can users select what is most relevant for them? These resources may help:

• How to do research on user needs in the ‘discovery phase’ of technology design – a website from the UK government.
• International Design Foundation – a US site offering tips and resources for making websites and apps more accessible.
• How to design websites for older people – a guide from the Alzheimers Society.

Address staff motivation and concerns. Assess the level of enthusiasm for the technology from different staff groups, and also how motivated teams are to take on the new technology. Have any of them had experience of using this technology elsewhere? Listen to staff concerns – which may be legitimate – and to their ideas for increasing the project’s success.  This resource may help:

Higher Education England Digital Capabilities Framework for assessing the digital capability of staff.

Modify staff roles and provide training. Develop new roles and job descriptions where needed, perhaps by adapting ones already in use elsewhere. Set learning objectives (some of which will be about building confidence to make judgements, not about mechanically following protocol). Design and develop training courses. Remember: using a technology usually needs on-the-job and team-based training, not just sitting in classrooms.  Allocate sufficient budget for this work, and consider issues such as backfill.

Promote social learning. One way to become confident in using a technology is to shadow someone in the same role who is already an enthusiast for it (‘champion’) and confident in using it (‘super user’). Learning in this way not only develops skills but also helps people develop a positive attitude and identity.

Support collective sensemaking and communities of practice. People need to make sense of new technologies – sometimes by coming together to complain about them initially! Surfacing one’s irritation with a technology may be the first step to coming to terms with it. Both staff and patients may benefit from being in ‘communities of practice’ (groups or networks of people who share an interest in something and are trying to get better at it). Online communities of patients, for example, are often good sources of knowledge and wisdom about how to manage a condition. Try to get these communities on board if introducing a patient-facing technology.

The Kings Fund guide to engaging NHS staff may provide some practical ways of achieving the above.

The project is only going to work if the organisation has the capacity to take on innovations and if there is good ‘innovation-system fit’. The tips below may help if you are trying to support an organisation to implement a new, technology-supported care model.

Assess the organisation’s capacity to innovate. An innovative organisation has strong leadership, good clinician-managerial relations, a devolved management structure, slack resources (money and/or staff) that can be channelled into new projects, good lines of communication and an ethos where it’s OK to take risks and learn from failures. If an organisation appears to lack these essential prerequisites for innovation, consider whether you need to strengthen its capacity before pressing ahead.  Here are some questions to help you assess capacity to innovate:

• Is there a culture that supports innovation and change (e.g. are staff trusted to introduce new ideas)?
• Does the organization have systems and processes in place that support innovation and change e.g. effective information and communication systems, opportunities for networking and learning across departments/teams?
• Do the senior management team actively seek opportunities for improvement and encourage ideas and feedback from patients, the public and staff?
• Are the organisation’s leaders helping to create a facilitative context through providing motivation and support, creating a vision and reinforcing the change process?
• Is there a distributed and devolved style of management?
• Is there a history of introducing successful change in comparable projects at a local level?
• Are there mechanisms in place to support learning and evaluation and to embed changes in routine practice e.g. regular team meetings, audit and feedback processes, professional development opportunities and performance review systems?

Assess innovation-system fit. Even when an organisation is capable of running a successful project to implement a new technology, it might be the wrong technology to introduce in this organisation right now. Has the organisation successfully adopted similar technologies in the past? Are its strategic priorities aligned with the use of the proposed technology? Or are other projects more pressing?

Assess the implications of the technology for the organisation. Careful mapping out of tasks and processes is necessary to surface how the technology or other innovation is likely to change these. The pathway in which the technology is used directly (e.g. clinical care) may have indirect knock-ons for other processes and pathways (e.g. booking, correspondence, billing). You need to estimate costs (both initial and recurrent), and consider how money will need to flow across the system. Before signing off on a project, boards generally want to know how much will it cost up-front, what the likely savings will be, and when these savings will occur. These resources may help:

Process mapping guide from NHS Improvement. Ideas and tools for mapping the steps in a care pathway.  A full list of additional service improvement and redesign tools from NHS Improvement is available here.

Using costing information to support better outcomes – a guide from NHS Improvement.

Assess the level of ‘political’ backing for the innovation. For an organisational-level adoption decision to be approved, it needs support from both top management (a ‘senior sponsor’) and the rank-and-file. Supporters of the change must outnumber opponents and be more strategically placed. People with ‘wrecking power’ can block progress and may need to be brought on board (or worked around). To assess all this, use the NASSS-CAT PROJECT tool and also:

Stakeholder analysis guide from NHS Improvement. This guide will help you construct a table or chart listing all the stakeholders who will need to accept (and, in many cases, start to use) the technology. Consider each key stakeholder’s perspective (and their potential wrecking power).

Consider inter-organisational relationships. Costs and benefits of technology projects are hard to predict, and savings may accrue elsewhere in the system. When there is no pre-existing contractual relationship between organisations, it can be hard to reach a satisfactory arrangement for how to manage these uncertainties. 

Think how (and by whom) success will be evaluated.  If this project is going to happen, you will need to monitor how well the change is going. You will almost certainly need both quantitative metrics (to answer the “how many…?” and “are we on track…?” questions) and also qualitative measures (to answer the “how do people feel about this…?” questions). Evaluation is everyone’s job, and data are often best collected by people doing the job. Extensive data collection can be time-consuming and slow the project down (i.e. the perfect may be the enemy of the good).

Evaluation: what to consider – A guide by the Health Foundation. This basic guide includes qualitative and quantitative approaches.

The ‘rainbow framework’ for evaluation and monitoring by Michael Quinn Patton. It takes you through 7 colour-coded steps, namely Manage (e.g. define stakeholders, secure funding), Define (set a scope for the evaluation), Frame (intended users of the evaluation, what they will use it for, what success will look like), Describe (sample, measures/metrics, data sources, analytic approaches), Understand Causes (deeper analysis to produce explanatory models), Synthesise (combining results), and Report & Support Use (publishing and disseminating).

Evaluation Works and Evidence Works toolkits to guide commissioning decisions, produced by West of England Academic Health Sciences Network and their partners.

Allocate funding. Studies of ‘failed’ technology projects often identify inadequate funding as a leading cause. You will probably need substantial set-up funding and possibly a recurrent budget line (for things like licences and IT support). Budget adequately for staff to learn and adjust as the transition occurs (see ‘Responding to complexity in the intended adopters’ above).

Manage the transition. Good change management involves a combination of ‘hard’ and ‘soft’ approaches. As well as setting goals and milestones and using agreed metrics to monitor progress, you also need to create opportunities for staff to come together and talk about the technology and new care model. As noted above, collective sensemaking, training (especially on-the-job training for both individuals and teams) and social learning from champions and super-users is crucial for building capacity. Use creative tools such as flip-charts and post-it exercises to surface people’s interpretations and concerns. Invite them to come up with creative ideas and solutions to any problems they identify.  Allocate sufficient budget for this work, and consider issues such as backfill. This guide may help:

Leading large-scale change: a practical guide from NHS England.

Plans for technology-supported change locally are unlikely to work out if there is a major mis-match with national policy or the prevailing political, economic or professional environment.

Try to align your project with current policy priorities. If the technology is actively supported in policy, it will be easier to introduce. If priorities are elsewhere, it may be worth trying to ‘rebrand’ the work to fit these.

Address regulatory issues and challenges. Consider which regulations (from which regulatory bodies) are relevant to the introduction of this technology. Are all approvals already in place? If not, who do you need to work with to make progress in this regard? See ‘Due diligence’ section on page 12.

Get the professions on board.  If clinicians or social workers believe that the technology compromises the care of their patients or clients, or if they view it as demeaning to their role or a threat to their professional jurisdiction or income, their professional bodies may oppose it. Early dialogue with such bodies may avert such a situation.

Establish inter-organisational networks or collaboratives. Complex, organisation-wide change is a lot easier if change agents in one organisation can network with their opposite numbers in comparable organisations – for example in quality improvement collaboratives or learning sets. Here’s a resource for that:

Improvement collaboratives in health care – A guide from the Health Foundation.

Keep a close eye on the outer context. External shocks to an organisation (such as economic turbulence) make change precarious. Whist such shocks are often hard to predict, it is a good idea to see what’s on the horizon.  The following questions may help you:

• Does the new technology and the proposed changes to services align with the strategic priorities for the wider health system e.g. in terms of current health policy, national priorities for action and improvement?
• Are there incentives in the wider health system that reinforce the proposed change e.g. pay for performance schemes, regulatory requirements etc.?
• Are there existing inter-organisational networks (e.g. specialised clinical networks) that will be helpful in terms of supporting the proposed changes?
• How much stability/instability is there in the wider health system – and how might this likely influence the implementation project?

The point about emergent change is it’s difficult if not impossible to predict. So this domain is really about how you might build resilience in your staff and your organisation to enable them to respond to things that come up in the future.

Acknowledge unpredictability.  Have you left open the possibility that the project might unfold in one of several different ways?  Can you flesh out these different possible futures and talk them through with your stakeholders?

Recognise and support self-organisation. Front-line teams will ‘tinker’ – that is, try to adapt the technology and the work process to make them work better locally. Are you able to capture data to evaluate and support these efforts?

Facilitate interdependencies.  Have you identified the key interdependencies in the project? Is there anything you can do to strengthen existing interdependencies or develop and strengthen new ones? 

Maintain space for experimentation and sensemaking. As complex projects unfold, staff will need to tinker more, and also talk about what’s happening. Encourage them to admit ignorance, explore paradoxes, exchange different viewpoints (there’s no need for them to agree on a single version of the ‘truth’!) and reflect collectively.

Develop adaptive capability in staff and teams. Train your staff to be creative and to adapt to change as it happens. They will sometimes need to make judgements in the light of incomplete or ambiguous data.

Attend to human relationships. Dealing with emergent problems requires give-and-take. It’s sometimes a matter of muddling through.  This will happen more easily if people know, like and trust each other.

Harness conflict productively. There is rarely a single, right way of addressing a complex problem, so view conflicting perspectives as the raw ingredients for producing multifaceted solutions.