9 Changing and stopping registries

From Parent-Wiki
Jump to: navigation, search

Methodological guidelines » 9 Changing and stopping registries

Content
9.1 Changing an existing registry
9.2 Time to stop? - Stopping a registry


A registry sometimes needs to be modified in order to continue to successfully accomplish its goals. The activity of changing a registry should not be underestimated, there are number of points that have to be taken into account. Stopping a registry may not be seen as a crucial task, however there are still factors that contribute to the successful shutdown of a registry.


Key principles:

  • A clear rationale for the registry change should exist. Modifications should be done only if essentially needed.
  • The scope of the registry modification should be carefully determined and understood; the feasibility of a registry modification has to be assessed.
  • It is necessary to carefully elaborate the consequences before implementing a registry modification.
  • It is essential to try to retain the comparability over time as much as possible; too ground-breaking changes might hurt time series.
  • All relevant stakeholders have to be informed about the registry change.
  • It should be considered whether modification to a registry requires any training or other support for a successful application of a change.
  • The effect of any modification should be monitored.
  • Clear decision on stopping a registry should exist; it is recommended to set the criteria for a registry stop in advance.
  • It has to be decided what will happen to the registry data when a registry is stopped (e.g. dissemination of results, archiving)
  • When storing/archiving a registry data the retention period, security, costs, monitoring and evaluation of the archiving process should be taken into account
  • Final report and other documentation that supports future (re)use of registry data should be prepared.


9.1 Changing an existing registry

A registry is a living system that evolves over time. In order to remain or become more useful and successful a registry sometimes needs to be modified. In general, it is important that a registry is flexible and adaptive, with a sense of continuous development. Regular checks and evaluations (e.g. internal or external reviews) of whether any of the registry’s components need to be modified are important factors that affect the sustainable success of a registry.

There are various reasons for a registry to undertake a modification or adaptation process. Unmet registry stakeholder needs, failure to meet certain standards, reducing the burden of the registry team or participants, new regulatory or legal requirements, innovations and changes in medicine and health care (i.e. new products, procedures, and services), innovations in information technology, or changes in the financing of the registry, for example, are some of them.

In connection with this a registry can undergo many different modifications. Maybe a registry needs to change its general purpose, goals or outputs, change the mode of data collection (e.g. moving from paper-based data collection to electronic data collection), introduce a new technology (e.g. web-based data entry), modify the target population or cohort (e.g. geographical expansion, the expansion of the age range, additional excluding criteria), adapt the outcomes or exposures (e.g. inclusion of knee implants as an exposure in addition to hip implants), modify any of the data elements (e.g. removing a redundant data element, adapting the outdated one, or adding a new one), change the case report form (e.g. to develop a more user-friendly form that is less subject to human error), modify the data collection protocol (e.g. different time points for follow-up), improve data analysis or data dissemination (e.g. more appropriate analytical techniques, different graphical representation of data, or different frequency of the dissemination), adapt a registry team or governing board, change the funding source or find new stakeholders (e.g. a move to public-private partnership), improve the overarching processes (e.g. quality assessment, auditing) etc. Some minor changes in a registry can be implemented more easily and quickly, but modifying a registry can be also a complex task that requires more effort, time and money. Therefore, it is highly recommended that a registry team, wanting to modify a registry, considers various elements in order to implement changes successfully and run the transition smoothly.

This chapter does not provide guidance for each and every change that can occur and be implemented within a registry, and does not cover in detail every step of the registry modification. The modification of existing registry is in many ways similar to a process of establishing a new one, the latter already being covered in other chapters. Thus, a reader is encouraged to read other parts of the guidelines for additional information. For example, when reconsidering legal and ethical obligations during the modification process a reader is encouraged to see chapter 5 ‘General requirements for cross-border use of patient registries’; or when modifying data elements a reader should read the chapters 6.3 ‘Registry dataset’, and 6.5.6 ‘Data modelling’, on how to adequately develop data elements for a registry. However, as already mentioned, there are various points that should be considered/emphasized while planning and/or implementing the modification of a registry:

  1. It is important that there is a clear rationale for the registry change since not every change is a good one. A registry team needs to understand why the change is necessary, what exactly needs to be changed, and what the change will bring. Thus, a clear purpose and goals of the change should be developed. If there is no solid reason for a registry change, it should not be implemented.
  2. It is essential to know how major and complex the change will be. The fact that changing one element of a registry can lead to the changing of other elements here should be taken into account. For example, changing dataset can lead to the adaptation of case report form, the data collecting process, statistical analysis and data dissemination. When modifying a registry this can be a good opportunity to make some other changes that are necessary as well. Moreover, it is important to be aware that registries are not just the documentation centre, but a network of data providers, so any modification has to be implemented in all satellites/units. Therefore, a registry team needs to carefully determine and understand the scope of the registry modification. Furthermore, an assessment of the feasibility of registry modification is a crucial part. Costs, time, effort, skills, and other resources are essential factors to consider. It is necessary to be aware of potential limitations and risks (e.g. technical breakdown or incompatibility, delays) as well. As for creating a new registry, good planning contributes to the successful implementation of a registry modification/transition. Thus, it is recommended to develop a thorough and realistic action plan and strategy for a registry modification/transition. It is worth mentioning here that piloting and testing are activities that should not be underestimated.
  3. Once it is clear what modification in a registry will be implemented, a registry needs to have a team for transition. In that case it is important to consider the skills and knowledge that members possess, and how the effort of the specific team member will be increased during the modification process, because a member will probably have to perform his or her regular work simultaneously. It is also important to establish continuous, honest and open communication between all members, because effective collaboration between them can identify some unexpected barriers or risks that can be suitably addressed during the planning phase. The role of leadership here cannot be overemphasized since, as in many other areas, it is one of the key factors for a successful implementation of the modification process [1][2][3].
  4. The elaboration of the consequences before implementing a registry modification is a very important task and should be undertaken carefully. Knowing where and what differences will occur with the registry modification, and how this will affect the further registry operation, will help in making the right decisions during the registry modification. When thinking about these consequences a registry team should look at every step and part of the registry operation. Are the changes going to reduce the quality of data (e.g. greater number of errors, new biases, lower statistical power, incompatibility etc.), increase the burden on data providers or registry participants, cause delays in reporting of results, increase the operating costs etc. are just a few examples that need to be considered. Similarly, it is crucial to monitor the effect of any modification after the implementation phase. The evaluation of differences in data prior to and after a change would be an example.
  5. The registry team should use experiences that were acquired with the existing registry. Which things worked well and which did not (bearing in mind every component of the registry operation) represents an important feedback that can be used as an advantage when undertaking a registry change.
  6. The registry needs to develop a good notification protocol for informing key stakeholders about the registry change. If the stakeholders are not engaged in a decision-making process they certainly have to be adequately informed to understand the rationale for the change, and its benefits. This is especially true for the participating sites/data providers which must be kept informed also about the timeline and implications that registry adaptation will have on the users. For any additional clarifications a registry team should be available, knowing that the change can take people out of their comfort zone and raise their stress and anxiety levels [1][4][5]. If it proves necessary (e.g. in case of the additional follow-up), patients have to be provided with information about the change as well.
  7. It is crucial to reconsider the ethical and legal requirements. A registry holder needs to be aware of how the changes in a registry affect the privacy, confidentiality and data access. It is necessary to consider whether the modifications require a new (or first) review from the ethics committee, if the inclusion of informed consent or change in informed consent form is needed, or if the re-consenting is required. In case of changes in stakeholder composition a registry holder must also determine whether the previous stakeholders should have access to data and if so, to which one [1]. It is important to look from the other point of view as well. A registry can be modified as a result of (new) legal requirements. It is therefore important that registry holders actively follow the potential changes in this area (e.g. regulation updates) and comply with them if necessary.
  8. When implementing changes to a registry dataset (e.g. removing redundant data element, adding a new category/permissible value or modifying a whole value set, introducing a brand new data element, adapting data element’s definition, changing a relationship between data elements etc.) a registry team should be aware that comparability over time (i.e. longitudinal comparability) can be a great advantage in obtaining new information and knowledge. Therefore, it is advisable to try to retain the comparability over time as much as possible. If a registry team is changing a value set/categories of a specific data element, a mapping between the old and new value sets usually needs to be done and a so-called conversion table designed to clearly show the link between the prior and new value set. It is important that the conversion table is accessible and understandable to every user. The mapping may be a lengthy and intensive process (e.g. problems with the equivalence of prior and new categories) which needs to involve well qualified personnel. Certain changes may make it difficult to match a prior value set with the new value set which can have the result that missing (“unknown”) data for subjects, on which data collection has already been done, can appear. In that case, these subjects can be reviewed/re-evaluated to update the missing value with the valid one. When this is not feasible it means that longitudinal comparability is not preserved. This is especially the case when significantly changing a definition on one of the key data elements, where the reality often is that everything must start again, meaning there is no comparability with the previous registry period, unless some well-established and validated conversions exist that enable making approximate comparisons.
  9. As a result of continuous development in technology, and also due to some other reasons (e.g. moving from one database vendor to another) a registry may go through the process of data migration which is a process of transferring data between storage types, formats, or computer systems [6]. Data migration is a complex process that should be carefully managed as, due to its iterative nature, it can easily lead to schedule and cost overruns. First, data on the old system needs to be mapped to the new system. Next, data are extracted from the old system, and, at this point, thorough data cleansing is recommended. If there is any redundant data, it should be removed. When the data are loaded/imported into the new system a data validation needs to be performed to check whether data were accurately transferred causing no errors or data loss. As already mentioned, mapping, loading, and validation steps will probably need to be repeated several times [7][8][9]. Last but not least, a registry holder must ensure that the data migration process complies with the legal requirements.
  10. Appropriately documenting the registry modification will allow registry users (and other stakeholders) to understand changes that have been implemented in the registry, provide insights into the history of changes and increase the transparency of the registry. Rationale for a registry change, a description of a change and its practical implications, were there any unexpected problems and how they were solved, are there any other changes that need to be done in a future as a result of a recent change are important items that should be documented.
  11. A registry holder should think about whether a modification to a registry requires any training or other support for a successful implementation/application of this change. Changing software for data entry or changing the analytical approach, for example, will probably require more comprehensive training than some other change, such as changing the data element’s value set. A registry holder therefore needs to carefully consider how extensive the training should be, who has to be trained (e.g. data providers, registry’s staff), what is the most appropriate way of training, and if any supporting material is needed.

9.2 Time to stop? - Stopping a registry

Partly also due to the fact that registries are often open-ended, the activity of stopping a registry does not seem so crucial, time and resource consuming as planning and setting up a registry. However, this does not mean that this activity should be neglected and that there are no important points that contribute to the correct and successful stopping of the registry.

When stopping a registry (with this we mean stopping data collection and ending all other sequential and overarching processes of a registry), first there must be a clear decision on stopping it. Setting the tangible and measurable goals/criteria for a registry stop in advance (in a registry planning phase) will help the registry holder to decide on whether the registry should continue with the operation, or if it is time to stop. Such criteria/goals might be, for example, to obtain a certain number of cases in the registry, achieve the desired precision of estimates and/or simply to fulfil the general purpose of the registry[note 1]. However, the registry is not stopped only when certain goals are accomplished but it should be looked at from the opposite side as well - failure to meet a registry’s predefined objectives or the fact that a registry appears to be unable to meet them in a reasonable time, poor operating results, loss of registry’s relevance, lack of a purpose for the continuation, or other serious problems (e.g. discontinued funding, lack of personnel, poor data quality, low patient accrual or significant withdrawal of the registry’s participants, ethical issues) could also represent the rationale for ending a registry [1][10].

When a registry holder together with other stakeholders involved in the decision-making process decides to stop the registry he or she should establish the communication with the data providers, and inform also registry users, personnel and, if necessary, any other stakeholders (e.g. patients that are enrolled in the registry and regulators) about the registry stop. It is important that the key stakeholders understand the rationale for the registry stop and the consequences that this decision will bring. In case of regulatory issue that may arise when a registry is stopped before the regulatory question has been answered, the regulators might be even involved in discussions about stopping a registry, if applicable.

Furthermore, a registry team has to decide on what will happen to the registry data. Will the registry aggregate and disseminate the collected data (as a kind of final report) and/or will archive the data, meaning the data will continue to be available in the future? If preserving registry data brings important benefits (e.g. to have insights into the historical data; possibility to perform additional analyses and address the questions that were not covered in a prior registry’s reports) then archiving might be the right decision. However, it is recommended that the decision about data archiving is discussed in a registry planning phase and not only when it comes to the registry stop.

When storing and archiving registry data a registry holder should take into account several points of data preservation:

  • Retention period (how long the registry will retain the data, considering regulatory requirements if they exist)
  • Security (following the norms of data protection and confidentiality of information a registry should establish policies and procedures to safeguard all data against loss, destruction, unauthorized use, or inappropriate alteration, and if necessary, also policies for proper and secure destruction of data. Some practical procedures for the above issues are authentication of system users, firewalls, back-ups, use of appropriate technology/storage media, policies that address copying data, disaster preparedness, emergency response, disaster recovery and training [11])
  • Data for archiving (in addition to the main registry data that are usually obtained by the case report form, the registry should preserve also a data entry log that tracks changes and users who made them, allowing registry to find the sources of the potential errors easier. To ensure that data can be correctly (re)used in the future, especially by others, data that are selected to be preserved must be packed with sufficient metadata. According to ICPSR[11] preservation metadata include all the information that is required by an organisation to preserve data, namely descriptive, structural, administrative and technical metadata.)
  • Monitoring and evaluation (monitoring and assessment of the quality and effectiveness of the data maintaining/archiving process enables controlling of the process, finding out if everything is going according to the plan, whether any system errors are occurring, and enabling the adaptation or improvement of internal operations themselves.)
  • Costs (data preservation requires financial, human and IT resources; registry holder should assess whether the funding is and will be available for the long-term maintenance of the registry data).

Finally, it is recommended that a registry prepares a final report in which its work, achievements, any encountered obstacles, rationale for the stop, and any implications for future work are clearly described. Along with the report, a registry should provide all the necessary documentation that supports the potential future (re)use of collected data.

Notes

  1. Planning and consideration of the registry's anticipated size and duration is covered in subchapter 6.2.2.

References

  1. 1.0 1.1 1.2 1.3 Gliklich R, Dreyer N, Leavy M, eds. Registries for Evaluating Patient Outcomes: A User's Guide. 3rd edition. 2014. Two volumes. Available from: http://www.effectivehealthcare.ahrq.gov/registries-guide-3.cfm.
  2. Newton R. Managing Change Step by Step: All You Need to Build a Plan and Make It Happen. Pearson Education, 2007.
  3. Queensland Government. Change Management Best Practices Guide: Five key factors common to success in managing organisational change. Available from: http://www.psc.qld.gov.au/publications/subject-specific-publications/assets/change-management-best-practice-guide.pdf
  4. Harding P., Pooley J. Resource Efficiency and Corporate Responsibility – Managing Change. 2004. available from http://www.oursouthwest.com/SusBus/mggchange.pdf
  5. Queensland Government. Change Management Best Practices Guide: Five key factors common to success in managing organisational change. Available from: http://www.psc.qld.gov.au/publications/subject-specific-publications/assets/change-management-best-practice-guide.pdf
  6. Bal Gupta S., Mittal A. Introduction to Database Management System. Laxmi Publications, 2009
  7. AHIMA. "Data Mapping Best Practices." Journal of AHIMA 82, no.4 (April 2011): 46-52. Available from: http://library.ahima.org/xpedio/groups/public/documents/ahima/bok1_048788.hcsp?dDocName=bok1_048788#Notes
  8. Computer Economics. Ensuring Success of Data Migration (April 2008). Available from: http://www.computereconomics.com/article.cfm?id=1329
  9. SAS Institute Inc. Enhancing Your Chance for Successful Data Migration - Critical steps for creating data migration solutions that balance cost and rapid delivery. 2009. Available from: http://www.sas.com/resources/whitepaper/wp_5969.pdf
  10. Rothman J. K., Haas J. When Should a Patient Registry End? Draft White Paper for AHRQ Patient Registries Handbook II. 2009. Available from: http://www.effectivehealthcare.ahrq.gov/repFiles/draftDocuments/2009_0817StoppingARegistry.pdf
  11. 11.0 11.1 Inter-university Consortium for Political and Social Research (ICPSR). 2009. “Principles and Good Practice for Preserving Data”, International Household Survey Network, IHSN Working Paper No 003, December 2009. Available from: http://www.ihsn.org/home/sites/default/files/resources/IHSN-WP003.pdf