6.1 Planning a registry

From Parent-Wiki
Jump to: navigation, search

Methodological guidelines » 6 Creating a registry » 6.1 Planning a registry

6.1.1 Defining the Purpose, Objectives and Outputs of the Registry Purpose(s) Objectives Outputs Process outcome
6.1.2 Data Considerations Data Quality
6.1.3 Overview of the Current State and the Importance of Interoperability Overview of Current State Interoperability Planning for integration
6.1.4 Considering Legal Aspects and Confidentiality Privacy and Privacy Impact Assessments Data Protection Policy Data Ownership, Access and Intellectual Property
6.1.5 Eliciting Expert Opinion & Generating an Advisory Board
6.1.6 Defining the Scope of the Registry & Building a Registry Development Team
6.1.7 Performing Stakeholder Engagement and Analysis Identification of Stakeholders Engagement Recording Stakeholder (& expert participation) Content of the Stakeholder Evaluation Stakeholder Evaluation Output
6.1.8 Re-defining the Scope of the Registry
6.1.9. Governance, Oversight and Registry Teams Project management team Scientific Committee Quality assurance Committee
6.1.10 Resource requirements Human Resources Information Technology Resources Financial Resources Other Resources
6.1.11 Funding Strategy
6.1.12 Risks and feasibility
6.1.13 Developing an Implementation Plan

Considerable resources and support are required to develop a successful, sustainable patient registry, the extent of which may not be obvious at the outset of planning. This section outlines an approach to registry planning that is believed to be logical and capable of enabling registry planners to propose feasible development strategies that will avoid duplication of effort and incorporate best evidence and expertise, while involving the extensive stakeholder support required to create an abiding, exemplary registry that can become a valuable, interoperable cog in the broader health ecosystem.

Key principles:

  • Maintaining a focused, open and transparent approach to registry development is vital.
  • The purpose, objectives and outputs of the registry should be clearly and succinctly defined, in a manner supported by best evidence and guidance, and that is aware of whether there is overlap with other projects nationally and internationally.
  • Key determinants of high-quality data should be considered and essential means of improving data quality embraced.
  • The broad health information landscape in which the registry will operate should be examined to ensure that the registry is interoperable and adheres to relevant standards, datasets and terminology.
  • Clear guidance with respect to legal obligations should be obtained. This should include consideration of privacy impact assessments, data protection policies, data ownership, data access and intellectual property.
  • Obtaining guidance from key experts and stakeholders is crucial and should be facilitated by engaging an advisory board.
  • Broad stakeholder engagement can generate involved registry champions and committee members who can facilitate the smooth running and enduring success of a registry.
  • The scope of the registry should be maintained as the project develops.
  • A governance plan should be established. This should be supported by registry teams that include, at a minimum, a project management team, a scientific committee and a quality assurance committee. Amongst other things, these groups can identify datasets required to ensure the registry meets its predefined outputs, in addition to creating clear data access policies and ensuring that quality assurance is maintained.
  • Inclusion of opinions of patients and real-world users of the proposed registry is essential.
  • The feasibility of the registry should be iteratively considered as the project progresses and the funding strategy should aim to ensure lasting success of the registry. This should include proactive risk identification.
  • Should the proposed project prove to be feasible, an implementation plan that includes a proof-of-concept pilot project to test the registry in a real-world environment is recommended.

This guide assumes that a registry is designed to fulfil a need that can be met through the scientific analysis of predefined data, collected in a real-world setting. Though this data might ultimately be utilised to answer other questions, it is essential that registry establishment is an organised, well governed and purposeful scientific process rather than a purposeless exercise in data collection. This will ensure the creation of a resource that maximises resource allocation and efficiency, and has well-defined, valuable outputs that can be measured, so that the quality and success of the registry can be verified.

Though this section evolves in what we believe is a logical, sequential process, components might be best addressed in tandem or may need to be revisited in an iterative fashion as further information becomes available. We do, however, feel that addressing each section will add value to the registry and increase the likelihood of developing a successful registry.

During each phase of planning, we advise considering how it may fit into the bigger picture, not just of the registry that is being created, but also with respect to the local, regional, national and international environment in which it is created. As the digital world becomes more connected we envisage the role of registries becoming progressively more valuable. This will only happen if they are developed in a manner that is cognisant of the importance of interoperability.

We also suggest that there is a wealth of experience to be gained from regulatory authorities, other registry groups and registry experts whose contributions could not only be helpful in the construction of a successful registry, but also critical to its implementation. We endorse in particular the creation of a resource such as PARENT’s Registry of Registries or the AHRQ’s Registry of Patient Registries (RoPR) which are helping to connect registries, while raising the standard of registries considerably [1][2]. We strongly advocate ensuring that any registry created forthwith joins such initiatives.

It should be noted that two resources in particular were of considerable help in informing the authors and in structuring this chapter. We highly recommended their utilisation as reference documents of extraordinarily high standard. These are “Registries for Evaluating Patient Outcomes: A User’s Guide” and the “ISPOR Taxonomy of Patient Registries: Classification, Characteristics and Terms” [3][4].


6.1.1 Defining the Purpose, Objectives and Outputs of the Registry Purpose(s)

The first step is to clearly define the overarching aim(s) or purpose(s) for which the registry is being established. This may emerge from a clinical need, a post-marketing requirement, or an interest of patients or clinicians, but the purpose(s) should be capable of being realised through the prospective, non-interventional, scientific approach that a registry should adhere to.

So far as possible, the purpose(s) should be limited in scope and number to ensure focus. As will become apparent, the expansion of ideas is likely to occur rapidly once stakeholders become involved and it is important to limit this at an early stage so as to prevent it becoming too unwieldy to manage. As with any scientific endeavour, this process will be greatly facilitated by conducting a literature review to analyse what information already exists within the scope of the proposed registry. This might demonstrate that while the purpose and objectives are reasonable, a clinical trial or other study design might be a more appropriate means of delivering the required outputs. Furthermore, a literature review will highlight relevant experts and stakeholders in the field of interest who might be contacted as part of a stakeholder evaluation or for expert advice. Objectives

To facilitate the generation of a valid scientific question, the registry’s purpose(s) should be divided into specific objectives, which together will achieve that overarching purpose(s) of the registry. It is worth considering how each objective might translate into a dataset and to imagine whether a scientific methodology could be applied to help validate whether the objectives are achievable. Outputs

Ultimately, a registry’s findings are only valuable if the data they generate can be translated into information capable of improving health outcomes. This is more likely to occur if outputs are considered at an early stage, so as to achieve the following objectives:

  • Ensure that a registry is purposeful as there will be measurable end-points against which its success can be judged
  • Identify potential experts required to advise the development of the registry
  • Identify potential stakeholders
  • Facilitate buy-in through the identification of outcomes of interest
  • Identify the target audience for whom the information gained from a registry might be valuable. This will facilitate the most effective dissemination of results and will also help identification of unforeseen requirements. For example, the primary purpose of a drug registry might be to identify its effectiveness in a real world setting; however, mandatory reporting of adverse effects will also need to be considered. Process outcome

Defining the purpose, objectives and outputs of a registry will typically clarify a registry as belonging to one of three groups (or a combination of these). Conversely, by considering a registry as belonging to one of these groups, may facilitate defining a registry’s purpose, objectives and outputs.

  1. Condition based registry
    • Purpose: Though there are many listing registers, which identify patients suffering from a particular condition, ‘condition-based registries’ in this document refer to registries that aim to describe outcomes related to a particular condition.
    • Purpose example: description of the natural history of chronic obstructive pulmonary disease (COPD).
    • Objective example: identification of depression in patients with severe COPD
    • Output example: Defining the prevalence of depression in COPD and examining how this might be more effectively be detected in the COPD care pathway.
    • Registry example:The Malta National Cancer Registry.
  2. Product based registry
    • These registries typically focus on medical devices or pharmaceutical products.
    • Purpose example: The assessment of the real-world effectiveness; safety and cost effectiveness of a new device; outcome monitoring of performance and potential safety issues over the entire life cycle; early signal detection of inferior outcome of device and surgical techniques; the impact of patient profile/comorbidities/risk classes on patient side on the outcome; market monitoring concerning implants and health care providers; feedback to health care providers; identification of fields for improvement and monitoring of effects.
    • Objective example: Defining the number of post-operative complications related to the device insertion to facilitate feedback to stakeholders in order to support decision-making.
    • Output example: demonstrating that a device or surgical technique is associated with increased postoperative complications; fulfilment of post-marketing obligations; validation of realisation of expected value by innovations and/or premium products; transparent ranking of quality achieved by implants and health care provider
    • Registry example: Arthroplasty Registries; there are multiple registries in the vast majority of EU Member States in different stages of development, which monitor approximately 4 million patients worldwide at present. Shortcuts to websites of relevant arthroplasty registries for further information are available at http://www.ear.efort.org/registers.aspx
  3. Services based registry
    • These registries aim to evaluate the quality of service provision or correlate an intervention with outcomes.
    • Purpose example: The assessment of: all hospital discharges (one day or longer) due to illnesses, injuries, poisonings, childbirths, stillbirths, sterilizations and new-borns in all Slovenian hospitals; information for monitoring, planning, management and development of health care system; health status of the population; cost effectiveness; patient safety quality; other quality indicators.
    • Objective example: to assess the health status of the population and specific subgroups; setting the priorities for developing national policies for improvements of health care system; to asses potential inequalities in health.
    • Output example: prevalence in the population for a certain disease or condition in a specific time period (e.g. year); calculation of burden of specific diseases; determination of quality level for certain quality indicators:
      • patient safety indicators, e.g. postoperative complications, obstetric traumas,
      • quality indicators related to acute care, e.g. 30 day in-hospital and/or out of hospital mortality
    • Registry example: The Slovenian Hospital Discharge Registry
  4. Combination
    • As is obvious from the examples above, some registries may have aspects that belong to more than one registry type.

6.1.2 Data Considerations

The success of a registry will ultimately be judged on its ability to meet the goals it was created for. This requires the collection and analysis of sufficiently high quality, targeted data specified by research hypotheses and the dissemination of the results of these analyses. High-quality data is also a key component in enabling interoperability (discussed later in this chapter). Though this section might be expected to occur later in the sequence of registry planning, success-by-design warrants the consideration of the determinants of high-quality data at an early stage of planning a registry. This should result in a focus on instilling key building blocks of quality, making the process of verifying the quality of the registry much easier when audit and other quality assurance processes are conducted post-implementation.

Knowledge of key determinants of data quality and how to achieve it will raise awareness of possible obstacles that might threaten the creation of a registry, such as the absence of an electronic health record to provide useful data. This knowledge also serves to underline the importance of a considered stakeholder evaluation that avails of input from groups that the registry planners may not have considered. Data Quality

Firstly it is worth recalling that data quality is influenced by a number of factors, categorized into four groups. These factors, including data quality, are considered in much greater depth in chapter 4 ‘Quality dimensions of registries’, which the reader is strongly advised to review before proceeding. In this chapter, these components are integrated within a suggested sequence of steps in planning a registry rather than factor by factor. Data and Information types

Data or information may be considered primary or secondary. Primary data, or information, refers to data collected to “provide health or social care to the data subject” [5]. “Secondary use of information relates to information collected in the course of providing care, being used for purposes other than direct service-user care” [5]. The use of data for secondary purposes, such as research, is governed by legislation, which varies across Member States. As such it is advised that legal expertise is sought. It might be prudent to adopt a position that secondary use of data requires explicit consent from patients or full anonymisation, which should be performed in keeping with local data protection regulation. Data Quality Dimensions

“The delivery of safe and effective healthcare depends on access to, and use of information that is accurate, valid, reliable, timely, relevant, legible and complete” [5]. Data quality dimensions are presented in Table 4.1 of chapter 4, so that registry planners might consider what dimensions are significant in the context of the purpose and objectives of the registry that they are planning. ‘Seven essential means of improving data quality’ are also described there (Table 4.2) so that registry planners might consider at a high-level, how these will be addressed by their registry [5][6]. Method of data capture

The quality of data will be significantly affected by the manner in which data are collected. Data collection can be considered with respect to two major domains; data source and data provider:

  1. Data sources
    • Paper-based
      • Questionnaire
      • Paper health record review
      • Documentation review
      • Laboratory reports
      • Other
    • Electronic
      • Questionnaire
      • Electronic Health Record
      • Laboratory reports
      • Databases
      • Mobile applications
      • Health devices
      • Social media
      • Other
  2. Data provider
    • Clinical units
    • Laboratories/central services
    • Discharge registries
    • Patients and families
    • Patients user groups (associations/federations)
    • Disability registries
    • Centres of expertise
    • Birth registries
    • Cause of death registries
    • Insurance funds (public and private)
    • Other registries
    • Other
Paper-based methods

Paper-based records have the advantage of being relatively inexpensive to create and distribute. However, in an era where health is becoming progressively more connected, paper is potentially very restrictive and does not take advantage of many error avoidance techniques that electronic methods offer. It is also worth considering that at some point, the data will need to be collated electronically to facilitate analysis. Paper can still play a core role in registry design however. Questionnaires and process flows can be created using paper which can be far more accessible for primary stakeholders. Once a prototype has been created using this method, it can facilitate the development of an electronic solution.

Electronic-based methods

Though the design of bespoke electronic solutions can be expensive, their advantage is that of connectivity, error minimization and reduction of duplication. While Electronic Health Records are still in evolution they are certainly not ubiquitous and they still have significant difficulties associated with their use. When they are available, and adhere to appropriate interoperability and terminology standards, they can offer an exceptional source of data for a registry. The list of potential electronic sources is large and it is for this reason that it is highly recommended that registry designers make considerable efforts to liaise with national, and possibly international, health, information and registry bodies to maximize resource utilisation.

Future developments

The recent explosion in mobile Health (mHealth) warrants consideration. As noted previously, there is an ever-increasing facility to utilize technology to connect data that has previously been unimaginable. Similarly, social media has established an almost ubiquitous presence and the extensive data networks that have resulted are of enormous potential to registries. It may be advisable to contact universities and connected health centres to consider what initiatives and ecosystems a registry could form part of to maximize the potential of mHealth and social media.

6.1.3 Overview of the Current State and the Importance of Interoperability Overview of Current State

Having an appreciation for the organizational structure of registries and other healthcare information networks or ecosystems nationally and internationally is of vital importance in ensuring that a registry is best placed to make use of existing resources. This knowledge will also help orientate how a registry’s role can best be positioned to fit into “the bigger picture” and contribute to the direction of health policy. There may be relevant data sources that could be integrated within your registry or vice versa. It may be that your proposal has previously been assessed, but was determined not to be feasible. Furthermore, significant resources might be spared through the identification of existing solutions the proposed registry might otherwise have replicated. Interoperability

Interoperability is the means of ensuring that a registry will be able to integrate within “the bigger picture”. Interoperability is defined by the Institute of Electrical and Electronics Engineers (IEEE) as the “ability of a system or a product to work with other systems or products without special effort on the part of the customer” [7]. Interoperability is a core component of good communication and as a result, effectiveness and safety.

Meta-analysis has demonstrated the importance of good communication within healthcare scenarios, suggesting that “interventions to improve the quality of information exchange increases effectiveness” [8]. In addition, the value of improving information transfer has been noted by major organizations, such as the Agency for Healthcare Research and Quality (AHRQ) in the United States, as an “important patient safety practice” [9]. Another US organization, the Institute of Medicine, having identified the extent of the risk posed by medical error in the landmark paper “To Err is Human”, have suggested the development of improved communication systems as core components of modern healthcare systems [10][11]. Planning for integration

As “interoperability is made possible by the implementation of standards”, liaising with national regulation/quality improvement authorities, which can be a valuable source of advice regarding access to and appropriate use of relevant standards, is essential [7].

Of particular relevance from a registry development perspective, is the selection of standard datasets and terminology to facilitate local and cross-border interoperability. For general areas, such as demographics, PARENT is an excellent source of guidance with respect to standard datasets and terminology or to facilitate contact with a registry in another state with a structure and composition that can be adapted or adopted for a new registry’s needs. At a national level, regulatory bodies will typically be able to advise best use of classification systems such as the World Health Organisation’s (WHO) International Classification of Diseases (ICD) or terminologies such as the International Health Terminology Standards Development Organisation’s (IHTSDO) Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT®). For more specific areas, national or international professional clinical groups can be a rich source of information. It should only be a last resort that non-standardised terminology/datasets are developed by a registry team and this should only be considered after discussion with appropriate experts/standards bodies to advise on how the dataset should be best developed.

An ultimate end point of achieving interoperability is to prevent potentially valuable data becoming trapped in information “silos” and facilitate more accurate representation of concepts and comparison of data across international borders.

The same connections that will facilitate interoperability are likely to be able to provide information regarding the current state of the art in registry design. In addition, we suggest contacting health authorities that are likely to have useful guidance regarding supportive ecosystems, including other registry groups. They may also be able to provide a clear picture of current and planned national health and information strategies and existing projects that could provide data sources for the registry, such as an electronic health record, or a national data architecture. PARENT will be able to offer further registry establishment advice and tools for registry development.

“Connected health” is a developing concept which “encompasses terms such as wireless, digital, electronic, mobile, and tele-health and refers to a conceptual model for health management where devices, services or interventions are designed around the patient’s needs, and health related data are shared in such a way that the patient can receive care in the most proactive and efficient manner possible. All stakeholders in the process are ‘connected’ by means of timely sharing and presentation of accurate and pertinent information regarding patient status through smarter use of data, devices, communication platforms and people” [12]. As connected health democratizes health information, there is significant potential for a registry to increase:

  • Awareness and interest
  • Dissemination and impact of outputs
  • Collaboration opportunities
  • The volume and variety of data sources available
  • Resource sharing

It is therefore worth liaising with centres promoting connected health, such as universities, or non-profit-making groups such as the ECHAlliance to establish how a registry might integrate in the process [13]. Conversely, the considerable organization required to develop a registry may facilitate the development of an ecosystem that can drive connected health.

6.1.4 Considering Legal Aspects and Confidentiality

While there are many important aspects to planning a registry, ensuring compliance with data protection regulations is not only vital, but a legal requirement, the breach of which may result in termination of the registry project. Furthermore, adopting a gold standard, transparent data protection practice is likely to increase the confidence that registry participants will place in a registry and add to its value. As such it is essential to prioritise contacting the relevant national Data Protection Commissioner’s Office early in the design of the registry. Contact details for EU member Data Protection Commissioners are outlined in Table 6.1. More information about the legal aspect is available in chapter 5.

Table 6.1 Data protection authorities and contact details for EU Member States
EU Member State Data Protection Authority email
Austria Österreichische Datenschutzbehörde dsb@dsb.gv.at
Belgium Commission de la protection de la vie privée commission@privacycommission.be
Bulgaria Commission for Personal Data Protection kzld@cpdp.bg; kzld@cpdp.bg
Croatia Croatian Personal Data Protection Agency azop@azop.hr; info@azop.hr
Cyprus Commissioner for Personal Data Protection commissioner@dataprotection.gov.cy
Czech Republic The Office for Personal Data Protection posta@uoou.cz
Denmark Datatilsynet dt@datatilsynet.dk
Estonia Estonian Data Protection Inspectorate viljar.peep@aki.ee
Finland Office of the Data Protection tietosuoja@om.fi
France Commission Nationale de l'Informatique et des Libertés /
Germany Der Bundesbeauftragte für den Datenschutz und die Informationsfreiheit poststelle@bfdi.bund.de
Greece Hellenic Data Protection Authority contact@dpa.gr
Hungary Data Protection Commissioner of Hungary peterfalvi.attila@naih.hu
Ireland Data Protection Commissioner nfo@dataprotection.ie
Italy Garante per la protezione dei dati personali garante@garanteprivacy.it
Latvia Data State Inspectorate info@dvi.gov.lv
Lithuania State Data Protection ada@ada.lt
Luxembourg Commission nationale pour la protection des données info@cnpd.lu
Malta Office of the Data Protection Commissioner commissioner.dataprotection@gov.mt
Netherlands Dutch Data Protection Authority info@cbpweb.nl
Poland The Bureau of the Inspector General for the Protection of Personal Data sekretariat@giodo.gov.pl
Portugal Comissão Nacional de Protecção de Dados geral@cnpd.pt
Romania The National Supervisory Authority for Personal Data Processing anspdcp@dataprotection.ro
Slovakia Office for Personal Data Protection of the Slovak Republic statny.dozor@pdp.gov.sk
Slovenia Information Commissioner gp.ip@ip-rs.si
Spain Agencia de Protección de Datos internacional@agpd.es
Sweden Datainspektionen datainspektionen@datainspektionen.se
United Kingdom Information Commissioner's Office casework@ico.org.uk
    England Information Commissioner's Office casework@ico.org.uk
    Northern Ireland Information Commissioner's Office ni@ico.org.uk
    Scotland Information Commissioner's Office Scotland@ico.org.uk
    Wales Information Commissioner's Office wales@ico.org.uk
EU Data Protection Officer of the EU DATA-PROTECTION-OFFICER@ec.europa.eu Privacy and Privacy Impact Assessments

“Privacy is the right of individuals to keep information about themselves from being disclosed” [14][15]. A privacy impact assessment (PIA) is a process that “facilitates the protection and enhancement of the privacy of individuals” and is best conducted at a planning stage to protect the registry and its participants from potentially irreconcilable personal and organisational breaches that may be damaging at a later stage [15]. This will facilitate the identification of risks to privacy breaches and examination of how these risks can be allayed. Detailing the process involved in a PIA is beyond the scope of this chapter, however the Health Information and Quality Authority of Ireland provide an excellent range of resources in this area, including a review of international PIAs, a tool to establish whether a PIA is required, details regarding how it should be conducted and a sample report [15][16][17]. Data Protection Policy

Even following a PIA, it is advisable to develop a data protection policy for the registry project and ensure that all involved with design and implementation of the registry are appropriately trained in this regard and regularly made aware of their responsibilities. A local Data Protection Commissioner’s Office or health authority may provide links with groups who have a policy that can be adapted for the purpose of the registry. Data Ownership, Access and Intellectual Property

While considering data security it is prudent to consider data ownership, access and intellectual property. This is likely to require dedicated expert guidance and, to ensure transparency; it is advised that the outcome of this process is formalized in a policy document. This document should also consider the scenario in which the registry project is terminated so that it is clear how the data might best be protected.

6.1.5 Eliciting Expert Opinion & Generating an Advisory Board

Expert elicitation refers to the “solicited exchange of knowledge, information, or opinion from an expert” [18]. If the initial planning processes suggest that there is a valid opportunity to establish a registry, further planning can be greatly facilitated by expert guidance. We suggest the establishment of an Advisory Board consisting of a knowledgeable panel with expertise relevant to the registry domain and those who are committed to the establishment of the registry. This will not only facilitate the implementation of best practice, it will also help identify stakeholders who might not be immediately apparent to the group establishing the registry. Finally, the selection of appropriate representatives for an advisory board is likely to increase the engagement of potential stakeholders with the project by virtue of their involvement, which can be vital to the success of the project.

While, from a practical perspective, it is most likely that experts, with a view to establishing an Advisory Board, are likely to be selected from the country in which the registry is to be implemented, it is advised that input, where possible, should be sought from international experts who have established registries in relevant domains. These experts can contribute vital, hard-won experience, resources and support that can improve the efficiency, sustainability and cost-effectiveness of developing a registry, as well as identifying potential unseen obstacles, thereby significantly increasing the chance of successful implementation of a registry and the relevance of its outputs.

6.1.6 Defining the Scope of the Registry & Building a Registry Development Team

It is advisable, at this point, to consider with the advisory board and funders what the scope of the registry will be. Though this may seem obvious once the purpose(s), objectives and outputs have been defined, these may be challenged by the open nature of stakeholder involvement and there is a significant risk of losing focus if clear limitations are not imposed. In addition, though an open stakeholder engagement process is likely to engage stakeholders’ imaginations and promote innovative ideas and engagement with the project, false promises can lead to significant disappointment at a later stage in the project.

The scope should aim to highlight the value of achieving the purpose(s), objectives and outputs of the registry with the minimal complexity possible, and in a manner that is most likely to be successfully accepted by users. Financial resources should be defined and a rough timeframe be agreed to give invited stakeholders an opportunity to plan when they can engage.

As there will be considerable time and preparation involved in developing the registry from this point, it is advised that a project development team is established that is proportional in size to the level of resources available to the registry. This might be a registry champion or person with an interest in the area in question, but thought might also be given to involving a research fellow with an interest in registries, healthcare informatics or the area targeted by the registry. Ideally, this person would be a primary stakeholder with a long-term interest in the area the registry is focused on. This will facilitate development of skills that can improve the long-term success of the registry, while also ensuring that the registry is designed in a fashion cognizant of end-users requirements.

6.1.7 Performing Stakeholder Engagement and Analysis

The Health Information and Quality Authority of Ireland (HIQA) have produced a document entitled “Guidelines for Stakeholder Engagement in Health Technology Assessment” which provides a comprehensive overview of stakeholder assessment that is extremely relevant to registry planning [18].

HIQA note that “Stakeholder engagement is an iterative process of actively soliciting the knowledge, experience, judgment and values of individuals selected to represent a broad range of direct interests in a particular issue“ [18][19]. Though stakeholder analysis might also involve the experts identified in 4.1.3, the aim in stakeholder analysis is to avoid solicited advice and instead facilitate wider engagement on the topic [18].

The process of stakeholder engagement should also be seen as an inclusive “hearts and minds” campaign. An effort to be inclusive and respectful of all stakeholders’ contributions can significantly improve the registry’s later adoption and success. Identification of Stakeholders

Though the definition of what constitutes a stakeholder varies, for the purpose of a registry, two subtypes can be considered [3]:

  • Primary stakeholders are intrinsically involved in the design and funding of the registry, but may also include parties with a regulatory capacity.
  • Secondary stakeholders may be affected by and involved in using and operating the registry, but do not have direct involvement in its design. Engagement

As the stakeholders of a registry may be extremely diverse, it is recommended that a flexible approach is adopted towards engagement. None-the-less, to facilitate transparency, consistency and relevance it is advised that a standard information document is prepared and distributed in advance, where feasible. Ideally, this document would support the conduct of a semi-structured interview. Recording Stakeholder (& expert participation)

Even with a focused registry, the number of potential stakeholders and registry contacts can increase significantly beyond the expected scope. As such we would advocate using a tool to monitor involvement at a high level. The table in Appendix B is a re-usable means of collecting information about possible registry stakeholders and recording high-level outcomes from meeting with them, relevant for the purpose of designing the registry.

High-level categories of contacts include:

  • Clinical groups
  • Public health and regulatory groups
  • Product and device manufacturers
  • Health care service providers
  • Health funding and insurance groups
  • Patient and advocacy groups
  • Academia
  • Relevant experts
  • Professional groups and societies
  • Registry groups
  • Registry sponsor groups
  • Development groups (informatics and management)
  • Other international groups Content of the Stakeholder Evaluation

Though the content of the evaluation will vary greatly depending on the nature of the registry and its stakeholders, we suggest a process that has been adapted from Registries for Evaluation Patient Outcomes: A User’s Guide” [3]. We recommend providing the stakeholder with a document, striking a balance between delivering information and being concise, that consists of the following components:

  1. Introduction to the group designing the registry, the current state and the motivation for developing a registry
  2. A brief introduction to Registries
  3. The purpose of the document
  4. Engagement requesting input from stakeholders regarding
    • Purpose & Objectives
    • Available relevant information sources
    • Key stakeholders
    • Feasibility – barriers and motivators to establishing a registry
    • Registry team membership
    Description of the further steps required to establish the registry

Though direct feedback is likely to be limited given the time constraints of busy stakeholders, this process is likely to not only be an exercise in clarifying the registry design process, but can create a template for a semi-structured interview with stakeholders at a later stage and also helps develop awareness of the project and confidence in the design process. Stakeholder Evaluation Output

Given the diverse nature of stakeholders it is difficult to ensure consistency and as such a scientific document is unlikely to be produced. None-the-less, by following the process described above, the opportunity has been presented for frank and honest engagement and useful information and requirements will be made apparent.

The analysis of the stakeholder evaluation also provides a useful opportunity to personalize further interactions with stakeholders and to provide relevant information at conferences or with other stakeholders to increase awareness and further engagement.

6.1.8 Re-defining the Scope of the Registry

Following stakeholder assessment it is advisable to reconsider the scope of the project. While factors likely to improve stakeholder engagement and ultimately increase the chance of the registry's success are important, these should be weighed against the considerable expense the extra scope is likely to add. It is also worth noting that increasing the volume of data collection is typically associated with a decrease in completeness of data entry.

A final scoping document will facilitate the creation of a business case and will better inform selection of data elements of the registry and the registry data model.

From this point, changes to the scope may result in significant resource utilisation and, as such, a change management strategy should be created which outlines how further adaptations to the scope should occur in the future.

6.1.9 Governance, Oversight and Registry Teams

Before considering data elements for the registry and beginning to focus on the practical implementation of the registry, it is advisable to establish a governance plan and to develop teams that will facilitate design of the registry and maintenance following implementation.

This serves a number of purposes:

  1. Creating teams can involve end-users, increasing buy-in.
  2. Facilitating a better understanding of how the registry will operate and how intellectual property will be handled.
  3. Creating the governance framework for data sharing and dissemination of data or information created by the registry.
  4. Ensuring oversight and that the registry development is progressing as planned.

Particularly when the scope of the registry is small, there may need to be overlap, however, at a minimum, we suggest prioritisation of a project management team, scientific committee and a quality assurance committee. It is suggested that, though specific teams/committees will benefit from members with specific skill sets, that members be selected to ensure that all stakeholder groups are adequately represented. In particular, it is advised that patient groups should be asked to contribute to ensure that the patient’s voice is represented appropriately as their data are the subject of the registry process. Project management team

The involvement of a person skilled and experienced in project management is advised. If this is not possible, it would be worthwhile considering training for a project manager and consideration given to the use of project management software. Table 6.2 outlines a tool to facilitate registry team organisation and selection. Scientific Committee

The aim of the scientific committee should be to ensure that the registry is outcomes- driven and that the data collected is disseminated effectively. It is suggested that the committee aim to meet four main objectives:

  • Question identification
  • Data element identification and selection
  • Dissemination of results
  • External data access/study proposal adjudication

As such, this group should consist of subject matter experts, ideally with a track record in publication of scientific results. It would also be ideal to include members of the group with statistical/epidemiological and health outcomes analyses experience, so that these factors remain in focus throughout the design, implementation and life of the registry. It is suggested that a transparent approach is undertaken with respect to member and topic selection. Question identification

Based on the scope identified by the advisory board and the input of the stakeholder evaluation, the committee should identify specific questions that the registry will address. These questions will inform the selection of data fields that the registry will record. Data element identification & selection

It is suggested that this process be considered an iterative one that considers the dimensions of data quality discussed previously.

Rough selection:

In the first instance, it is advised that the scientific committee consider a rough map of possible data fields. This should then be submitted for statistical analysis based on the scientific questions that have been proposed.

Statistical and Epidemiological analysis:

This process is vital to ensure that the registry is developed to an appropriate scale that ensures that the purpose and objectives it was created for are met.

Extra data fields add considerable complexity and cost because of data validation requirements. A statistical analysis can help highlight the essential fields for registry success and help maintain as much simplicity as possible; reducing the resources required ensuring completeness of data entry when the registry is implemented. It will also reduce the effort required to validate and analyse data.

It is advisable that this process is conducted by statisticians and epidemiologists trained in registry science. If the registry development group has no formal attachment with experts with skills in this area, it is worth checking with universities or other registry groups, who might identify relevant experts.

Health Outcomes/Pharmacoeconomics analysis:

At the same time as a statistical analysis review of potential registry outcomes from a health outcomes and pharmacoeconomics perspective should be considered. Increasingly, the relevance of real-world effectiveness is being prioritised and the relevance and attractiveness of a registry can be greatly increased by engraining it within national and international strategies. In Sweden, for example, the establishment of a hip and arthroplasty registry resulted in the avoidance of 7,500 revisions between 2000 and 2009, with a saving of $140 million in costs.[note 1] This is also a mechanism of scientifically establishing the potential economic worth of the registry and as a means of creating a benchmark against which the registry might later be evaluated as a marker of success. This can be of particular consequence when funding organisations are approached with a view to ensuring the long-term feasibility of the registry project.

Final Selection of elements:

The final selection of data elements is only likely to occur at the time of implementation of the registry, or ideally, after a pilot project has been conducted and after a financial analysis has identified the scope that can realistically be supported. The aim of the data selection process at the planning stage should therefore be to outline the data fields that will be required to a level adequate to conduct a feasibility study. Dissemination of Results

Dissemination of registry data increases the potential impact of a registry and facilitates peer review. This process enables registry methods and data to be independently scrutinized, which in turn can validate the quality of the registry. Planning how registry data will be disseminated can help develop a timeline for implementation as well as ensuring that adequate funding is considered for this purpose. External data access/study proposal adjudication

If a registry collects high-quality data, it is both likely and desirable that external requests will be received requesting access to data or proposing studies that can utilize registry data. To ensure transparency and facilitate best use of data, it is suggested that the scientific committee establish a formal plan to adjudicate on such requests. This might involve defining the grounds for collaborative agreements where external parties, in addition to gaining access to data, can benefit from the experience and expertise of committee members aware of the context in which the data were collected. Quality assurance Committee

Ensuring that the registry’s quality is validated will increase the value of the registry. Though the project management team and scientific committee will together increase a registry’s quality, it is advisable to have an independent committee established to assess whether this is the case through the creation of a formal audit and quality assurance plan. In addition, this group might be well placed to handle complaints or to ensure that ethical and legal obligations are being met in the absence of a specific group to manage this.

This group would ideally comprise experts familiar with registry analysis and who have experience of audit and quality assurance. There should also be consultation with regulatory groups to ensure that all regulatory requirements are met; this is of particular relevance when the registry is focused on safety assessment, such as devices. Conflicts of interest should be considered and declared during this process.

Table 6.2: Re-usable table to facilitate selection and recording of possible registry team members

6.1.10 Resource requirements

Resource requirements will vary significantly depending on the scope of the registry project. The steps followed to this point should identify the extent of resourcing that will be required to meet the objectives outlined. Resources to consider include: Human Resources

Though registry committees can perform a large quantity of work, there is likely to be either dedicated or intermittent need for staff to ensure proper set-up and maintenance of a high-quality registry. Depending on the scope of the registry project, this might include staff to meet needs in:

  • Administration
  • Project management
  • Data management
  • Data collection
  • Study design, epidemiology & statistical support
  • Data dissemination
  • Programming
  • Design (question and graphic)
  • Training
  • Financial
  • Legal/data security & protection
  • Clinical Information Technology Resources

Depending on the environment in which the registry is to be established, requirements can range from analysis software to an extensive hardware and software budget. It should be stressed that information technology support with experience of registry design is extremely valuable. Gaining advice from other registries, registry groups such as PARENT and local regulatory bodies is invaluable and should be sought to ensure any system delivered is designed appropriately and with interoperability in mind. Financial Resources

Though the outlay for the initial design and implementation of a registry is the most obvious requirement, consideration should be given to the long-term sustainability of the registry project. Financial resources will vary significantly depending on the scope of the registry; however, by following a planning process with an inclusive stakeholder assessment, it is more likely to identify appropriate funding avenues and collaborations that may maximise financial investments in addition to the financial value of registry outputs. Examples of funding avenues include public-private partnerships, governmental funding, patient groups, and sponsorship from charities or pharmaceutical companies.

Finally, it is necessary to take account of the financial implications of closing down a registry and what arrangements would need to be made to ensure that data security is maintained in this scenario. Other Resources

The list of potential other resources is extensive, however, particular note should be drawn to office space. It is ideal if the registry emerges from a group that can provide accommodation, which is important for a number of reasons, including from a data security perspective.

6.1.11 Funding Strategy

It is likely that for each group which proposes a registry, there is a funding source that has helped bring the idea to this stage. By including a directed stakeholder evaluation, it is likely that further opportunities might present themselves. Of particular significance, however, is the need to consider how funding might influence how the outputs of the registry are interpreted. At all times, funding should be arranged in a manner that is transparent and without conditions that might undermine the validity of the scientific study.

6.1.12 Risks and feasibility

Risks accompany each component of the registry establishment and maintenance process, from excessive dataset selection and lack of adherence to recognized standards through to a failure to consider a registry termination strategy. Of all these, however, we suggest that a failure to be aware of the extensive preparations required to develop and maintain a registry are the most significant. The process described in this section may seem over-burdensome, but it can present a myriad of advantages, such as identifying collaborative opportunities and identifying funding opportunities. Apart from this, as registry science evolves, regulation is likely to follow and create obstacles that might threaten the creation and survival of registries that have already consumed significant resources. As such, we recommend that an extensive planning process be undertaken under the guidance of experts familiar with the process of registry design and with stakeholders. Once this has been completed, an informed feasibility assessment can be undertaken. This should review whether the objectives and purpose of the registry are likely to be met within the timeline considered, the budget available, the scientific model proposed and within the environment in which the proposed registry is due to be implemented.

6.1.13 Developing an Implementation Plan

If the feasibility study reaches a positive conclusion, it is likely that most of the components will now be adequately developed to create an implementation plan, which should also include a plan that details how and when the registry will be evaluated. It is suggested that a further review of the steps involved in planning the registry is undertaken to develop an action plan and timeframe for each step in conjunction with the appropriate expert or stakeholder identified by the planning process. Within this, rate-limiting steps should be identified to help determine the “critical path” which will dictate how long the project is likely to take. It is suggested that at this point, particularly in the case of a large registry project, an experienced project manager is involved to help deliver the project on schedule.

As part of the implementation plan, it may be useful to consider a pilot project as a proof-of-concept model before proceeding with a full implementation. This can generate significant support for a registry, create useful outcomes and identify significant obstacles that may not have been initially obvious. It can also create a wealth of knowledge and experience at a manageable level that can increase the chances of ultimate success.

A project proposal should be formalized with firm time and budgetary constraints outlined to facilitate regular oversight by the project management committee (or similar). Though numerous measures of quality have been mentioned, ultimately, the registry will need to be regularly evaluated against the objectives and purpose it was designed to meet. This can facilitate review and adjustment of the registry that can further improve outcomes, efficiency and ensure that relevance is maintained.

Figure 6.1: Planning a Registry Process


  1. National disease registries for advancing health care. Lancet. 2011; 378(9809):2050.


  1. PARENT (PAtient REgistries iNiTiative). PARENT Pilot Registry of Registries 2014 [24th May 2014]. Available from: http://www.parent-ror.eu/ - !state/list_all.
  2. Agency for Healthcare Research and Quality. RoPR | Registry of Patient Registries: U.S. Department of Health & Human Services; 2014 [24th May 2014]. Available from: https://patientregistry.ahrq.gov/.
  3. 3.0 3.1 3.2 Gliklich R, Dreyer N, Leavy M, eds. Registries for Evaluating Patient Outcomes: A User’s Guide. Third edition. Two volumes. (Prepared by the Outcome DEcIDE Center [Outcome Sciences, Inc., a Quintiles company] under Contract No. 290 2005 00351 TO7.) AHRQ Publication No. 13(14)-EHC111. Rockville, MD: Agency for Healthcare Research and Quality. April 2014.
  4. ISPOR Taxonomy of Patient Registries: Classification, Characteristics and Terms. Lawrenceville, NJ2013.
  5. 5.0 5.1 5.2 5.3 Health Information and Quality Authority. Guiding Principles for National Health and Social Care Data Collections. Dublin, Ireland: HIQA; 2013.
  6. Health Information and Quality Authority. What you should know about Data Quality. Dublin, Ireland: HIQA, 2012.
  7. 7.0 7.1 IEEE Standards Glossary: IEEE; [22 May 2014]. Available from: http://www.ieee.org/education_careers/education/standards/standards_glossary.html.
  8. Foy R, Hempel S, Rubenstein L, Suttorp M, Seelig M, Shanman R, et al. Meta-analysis: effect of interactive communication between collaborating primary care physicians and specialists. Annals of Internal Medicine. 2010;152(4):247-58.
  9. Shojania KG, Duncan BW, McDonald KM, Wachter RM, Markowitz AJ. Making health care safer: a critical analysis of patient safety practices: Agency for Healthcare Research and Quality Rockville, MD; 2001.
  10. Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. Washington: National Academies Press, 2001 0309072808.
  11. Levit L, Balogh E, Nass S, Ganz PA. Delivering high-quality cancer care: charting a new course for a system in crisis. Institute of Medicine Washington, DC: Institute of Medicine. 2013.
  12. Caulfield BM, Donnelly SC. What is Connected Health and why will it change your practice? QJM. 2013;106(8):703-7.
  13. ECHAlliance | Delivering leadership for the development of Connected and MHealth market and practice across Europe 2014 [24th May 2014]. Available from: http://www.echalliance.com/.
  14. Erickson JI. Caring for patients while respecting their privacy: renewing our commitment. Online journal of issues in nursing. 2005;10(2).
  15. 15.0 15.1 15.2 Health Information and Quality Authority. Guidance on Privacy Impact Assessment in Health and Social Care. Dublin, Ireland: HIQA; 2010.
  16. Health Information and Quality Authority. International Review of Privacy Impact Assessments. Dublin, Ireland2010.
  17. Health Information and Quality Authority. Sample Privacy Impact Assessment Report Project. Dublin, Ireland 2010.
  18. 18.0 18.1 18.2 18.3 Health Information and Quality Authority. Guidelines for Stakeholder Engagement in Health Technology Assessment in Ireland. Dublin, Ireland: HIQA; 2014.
  19. Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. Journal of comparative effectiveness research. 2012;1(2):181-94.
  1. Irish Platform for Patients' Organisations, Science & Industry [07/12/2013]. Available from: http://www.ipposi.ie/.
  2. Irish Platform for Patient Organisations, Science and Industry,. Towards a National Strategy for Patient Registries in Ireland. Dublin, Ireland: 2008.
  3. Donohue F. Presentation IPPOSI Meeting Dublin 10 May. Dublin 2011.