PACS downtime comes under growing scrutiny

2014 07 07 13 19 51 987 2014 07 08 Pacs Joachim Zaers Thumb

As more and more core care processes move from paper-based to e-support, system or service uptime now plays a critical role in provider organizations, and planned or unplanned PACS downtime can be a serious problem, according to German expert Joachim Zaers.

"For most patient cases in acute care, imaging information is vital," said Zaers, an engineer for Munich-based medtech consultants Birkholz und Partner. "Whereas physicians and nursing staff may be rather creative in surviving downtime of the administrative and documentation systems, a PACS lights-out scenario tends to be challenging."

Joachim Zaers has been working for Birkholz und Partner since 2004, and since 2008 has headed its IT project implementation team.Joachim Zaers has been working for Birkholz und Partner since 2004, and since 2008 has headed its IT project implementation team.

Zaers has worked as an IT systems architect for IBM Global Services, and has been in the PACS business for more than 15 years, his largest PACS installation generating more than a million studies annually. He has been working for Birkholz und Partner since 2004, and since 2008 has headed its IT project implementation team.

With high availability nowadays being sold as part of a server virtualization deal, the need for continuity plans may start to fade. In some provider organizations, high PACS uptime is safeguarded by a second backup or "shadow PACS." While this approach may be appropriate for larger organizations, smaller providers will wish to avoid the additional cost, including human resources, he told delegates at the 16th DICOM Meeting and HIS-RIS-PACS workshop, held at the end of last month in Mainz, Germany.

"The annual 'HIT and Miss' sessions at the HIMSS congress outline that we will keep experiencing ever new threats, and that our risk inventory is a tool of limited efficiency," Zaers said, adding there is no 100% uptime guarantee. The human factor will always contribute errors that may lead to unplanned downtimes, and everybody knows spectacular stories about what can go wrong in the complex and highly integrated IT landscape. "Even with an effective risk management, we will only reduce that risk -- this is what we should be aiming at."

Cut risks, increase lights-out tolerance

However, experience garnered from organizations that have implemented, and are executing regularly, their continuity plans reveals users can learn to tolerate unplanned downtime. In addition, robust and trained downtime procedures can lead to lower requirements regarding service levels, and this situation translates directly into lower operating costs.

Today's IT applications are designed to support users unobtrusively, and this can lead to an underestimation of the degree of process support delivered by IT. In addition, use of IT applications in hospitals grows continuously.

"The full scope of utilization is often only realized when the IT systems are shut down for maintenance," Zaers explained. "With the advances in high availability technology, these planned downtimes are reduced, and users are not really forced to implement complete and sustainable continuity plans."

As for the regulatory framework in Germany, PACS continuity plans are required for the operation of teleradiology services in rural areas; for PACS installations in hospitals, however, there are no explicit standardized requirements for continuity plans. There is a rising interest in protecting critical IT systems -- including healthcare -- and the German Federal Office for Information Security BSI has developed a standard for IT continuity planning (BSI 100-4), he explained.

"We often find that the effect of an unplanned PACS downtime is underestimated and proactive planning for such downtimes is not really effective. A proper indicator for the organization's maturity in this respect is the simple question: When is your next planned PACS downtime that is longer than one hour?" he said.

Today's PACS solutions are complex, multitier integrated IT systems, and recovery often involves second-level support from the vendor. Even worse, IT diagnostics -- as in medicine -- has turned into a multidisciplinary activity where PACS admin, network, and medical engineering staff try to track down the issue. Furthermore, not all of these team members are available 24/7.

"In this context, it is anything but an easy task to identify the 20% of effort that will deliver 80% of the benefit induced by such proactive planning -- in line with the Pareto Principle," Zaers said. "So when we work on continuity plans and downtime procedures, we resort to a simple and robust environment which is easy to operate and which is tested against the following worst case scenario or the chief information officer's nightmare: Imagine Saturday morning, 2 a.m., and images of the current emergency case are not readable in the trauma room -- this scenario is marked by low quantity and quality in staff across all tasks involved, with damage potential on the patient side at the maximum."

This worst-case scenario is a real-world example. The facility was looking back on three years of excellent PACS operation, with no unplanned downtime during the service hours, and downtime procedures were never documented nor trained as the PACS was running smoothly. The technician on duty on the Saturday had only joined the organization six months after PACS rollout. As a consequence of the incident, the head radiologist implemented a process, allowing him to ask for a downtime drill at any time.

Integrating IT health checks

PACS vendors, indeed all healthcare IT vendors, should add "health check" features to the software to improve overall uptime of certain functionalities, Zaers urged. Continuity plans are often product-specific and can be provided as templates by the vendor, and open system technologies available could use an optimization for resilience and simple handling. On behalf of hospitals for whom his team is active, he has begun requiring monitoring specifications for the system and its environment to allow for a real-time health-check supporting users in detecting abnormal system status.

"With the advent of end-to-end electronic documentation by physicians and nurses -- electronic medical records -- it is high time to develop methods to create and maintain continuity plans for all these digitally supported processes," he underlined. "In our experience, the current staffing of IT departments is not taking into account the additional resources needed."

Page 1 of 1243
Next Page