4 Quality dimensions of registries

From Parent-Wiki
Jump to: navigation, search

Methodological guidelines » 4 Quality dimensions of registries

4.1 Governance
4.1.1 Procedures and methods for registry operation
4.1.2 Education and training
4.1.3 Resource planning and financial sustainability
4.1.4 Interoperability as a quality dimension
4.1.5 Self-assessment
4.2 Data quality
4.2.1 Data quality dimensions and its assessment
4.2.2 Mode of data collection and impact on data quality
4.2.3 Improving data quality
4.3 Information quality
4.4 Confidentiality, security, privacy, ethical issues, secondary use of information
4.4.1 Privacy impact assessment (PIA) – a method to assess privacy

The primary dimension of registries’ quality is the quality of the data. Data quality is influenced by a number of other identifiable registry features. Four basic categories of factors influencing registry’s quality are:

  • Governance, as an organizational foundation of patient registries, is mostly concerned with guidance and decision making. Adequate governance model makes sure to address issues such as overall direction and operations (procedures and processes), communication, scientific content, ethics, safety, data access, transparency, publications, change management and registry life-span planning.
  • Data quality is assured by defined requirements/standards for data collection and management. Data quality is also to be assessed against a list of dimensions which can be defined and measured.
  • Information quality is an output of a data collection process. It is measured by the amount and impact of scientific publications based on registry data.
  • Quality is also influenced by features like confidentiality, security, privacy and ethical issues. These influence a registry’s interoperability capability and information dissemination. Meeting ethical and legal requirements concerning privacy influences registry’s interoperability capability and information dissemination. Privacy component of the registry is measured by privacy impact assessments (PIAs).

Integrally addressing advices indicated within stated categories during registry planning and creation but also while running a registry, should ensure high level of registry performance.

In broader terms, quality can be defined as “the standard of something as measured against other things of a similar kind; the degree of excellence of something[1]”. In that light, other quality dimensions can also be (and should be) assessed. This way data quality remains the primary dimension within registry quality evaluation, but acknowledging that it is influenced by other identifiable registry features. Based on such a rather holistic view and through conducting a literature review we have identified numerous “quality influencing factors” and categorized them into four groups, which are not to be viewed separately. These are: 1) governance; 2) data quality; 3) information; 4) ethical issues, security and privacy (Figure 4.1).

It is useful to consider these categories while planning and evaluating registries, since they should, rounded all together, provide a rough estimate basis for assessing registry performance. [note 1]

Figure 4.1 Quality dimensions of registries
  • procedures and methods for registry operation
  • education and training
  • resource planning and financial sustainability
  • interoperability as a quality dimension
  • self-assessment
Data quality
  • data quality dimensions
  • data standardisation
Information quality
  • surveillance
  • outcomes
  • scientific publication
Ethical issues, security and privacy
  • adherence to privacy legislation
  • ensuring data and information security
  • ethical and privacy issues with secondary use of data

4.1 Governance

Governance and management are the organisational foundations of patient registries, by:

  • providing the framework to ensure that the registry achieves objectives set on its establishment
  • driving the registry’s functioning in terms of securing resources (financial, human, technical), measuring performance and ensuring sustainability
  • influencing data quality and registry outputs regarding dissemination of information
  • complying with legal pre-requisites

Applying proper governance principles should ensure that robust operational procedures and processes are in place, clearly communicated, and easy to access for everyone involved in data collection. Besides basic managerial and operative functions, the goal of apt governance should also be transparency to stakeholders in operations, decision making, and finally in reporting of results.

Governance is thus mostly concerned with guidance and decision making, which include the topics of registry concept, funding and dissemination of information. A governance plan is important at a registry’s onset as it substantially determines future functioning. Therefore, the plan for registry governance and oversight should clearly address issues such as overall direction and operations, scientific content, ethics, safety, data access, publications, and change management. It is also helpful to plan for the entire lifespan of a registry, including how and when the registry will end and any plans for transition at that time [2].

Specific elements of the governance quality dimension are presented below.

4.1.1 Procedures and methods for registry operation

In order to justify the holding of personal health data, the establishment of a registry first requires a clearly stated purpose. The stated purpose should contain a brief description of why the registry is established and what its intended use is (e.g. program administration, service delivery or research). This purpose should also be subject to review and change should the objectives and aims of the registry change. A statement of purpose should also contain information such as: full (legal) name of the registry, contact details, name of responsible registry holder, year started, overall function, objectives, list of data providers, legal basis for establishment, legislation and standards (privacy, national, international) that the registry must adhere to [3].

Determining the appropriate scope of the registry, data set and target population, along with a study plan or protocol is fundamental to proper data collection and to the future quality functioning of the registry. At a registry’s outset proper documentation managing should be upheld, meaning that the goals of the registry, its design, target population, all procedures related to data (methods and procedures for data collection, clearly defined data elements and items, data management, appropriate data analysis and reporting practice procedures), and how human subjects will be protected (privacy legislation) should be documented.

It is very important for a patient registry to have a complete and detailed manual containing descriptions of protocols, policies, structures and procedures. Documenting registry policies and procedures enables it to become more process dependent than person dependent, potentially enhancing data quality stability and reliability. Document management should be an active process, maintaining and updating documentation through the registry’s further operational period. One more feature closely linked to document management is the registry’s overall adaptability, as technical, regulatory and ethical frameworks of the registry should be periodically reviewed in order to address possible newly emerging issues.

4.1.2 Education and training

Staff education and training is another important aspect of registry quality. Inadequate registry staff training may cause data quality issues as well as security breaches and/or privacy violation. Sufficient staff qualification and training is thus necessary, and this can be achieved through training and education. All staff should receive training and education relating to their roles and specific job responsibilities, as well as proper training on the patient registry protocol and procedures, data sources, data collection systems and data definitions (with interpretation), accompanied by formal records of training and education [2].

For instance, if registry governance decides on applying standards, this does not all by itself necessarily lead to enhancing any of the registry quality dimensions. Such action also demands achieving a certain satisfactory level of education and training of registry staff, in order to ensure a straightforward implementation process of standards. For example, the ICD-10 terminology, depending on the purpose of use (cause of death, cancer, discharges, DRG, infectious diseases), requires appropriate levels of education and training fit for and according to purpose of use.

The registry governance should have a training plan through which refresher training is to be provided on an ongoing basis. Training content should also regularly be subjected to review and updates, following potential changes in legislation, and national and international standards [3].

Training is not only important for registry staff, but also for the staff of the healthcare unit which provides data for the registry, in order to increase data quality. Training includes various methods, from providing manuals for proper data collection and data dictionaries to organizing training sessions with data providers (clinicians etc.) as participants where e.g. data extraction guidelines are discussed and practised with patient cases [4].

4.1.3 Resource planning and financial sustainability

Since achieving objectives relies foremost on available resources (human, physical, financial), the managing organisation responsible for the registry should plan and manage its resources to ensure that they are used efficiently and effectively. Resources should be adequate to ensure the sustainability, continual relevance and maximum impact of the data for which the registry holders are responsible. Considering that budgets are limited, careful planning and management of the use of resources is crucial to ensure they are used in the most efficient, useful and effective manner. How resources are used very much influences the quality of the information provided and the future sustainability of the registry. The allocation of resources is therefore also a fundamental factor in the delivery of quality data [3]. One of the more promising ways to provide financial sustainability is collaboration amongst all the stakeholders involved in the registry, an approach which can reduce or avoid duplication of efforts, foster improved quality and robustness of data collected, and finally, in a positive way, sustain registries as long-term ventures [5].

4.1.4 Interoperability as a quality dimension

Interoperability can be viewed as a quality dimension under the governance group in regard to the following:

Impact on any particular registry quality dimension cannot be pursued only within the registry holder (e.g. institution), as it is also necessary to influence the business processes and modus operandi of other registry stakeholders (data sources, identified data users, health information authorities etc.). Ideally, interoperability should be established through a gradual process of connecting internal processes of the stated stakeholders, therefore transforming the business processes towards convergence and making them mutual and public. Interoperability concepts and issues as well as interoperability as an envisioned common goal for patient registries across Europe is discussed further in chapter 3.

4.1.5 Self-assessment

One of the registry governance roles should be to consider how to ensure overall quality to a level sufficient for the intended purposes; therefore registries must pay careful attention to quality assurance issues. Quality assurance is important for any registry to ensure that appropriate patients are being enrolled and that the data being collected are accurate. Quality assurance activities can help to identify data quality issues resulting from inadequate training, incomplete case identification or sampling, misunderstanding or misapplication of inclusion/exclusion criteria, or misinterpretation of data elements and hence improve the overall quality of registry data [2].

Self-assessment should perform quality control and serve to identify the sources of potential data quality issues and assess them by using indicators on data quality dimensions, developing measurements for evaluation, subsequently used to correct issues and track improvements. Use of quality assessments is also recommended to guide any decision on changing or modifying registry practices and procedures. Self-assessment can be an important registry governance feature, as it constitutes in fact a great self-propelling mechanism that ensures continual quality improvement.

Data quality improvements can be based on regular internal data quality audits including the quality of coding that incorporate clinician input (data source) as well as on external audits and external data quality reports. Self-assessment refers to periodically performing quality control through a data quality assurance programme and subsequent instituting of data quality improvements based on identified quality issues. However, self-assessment is here envisaged also as a governance responsibility, which should concern not only data quality checks but also overall registry functioning.

4.2 Data quality

In addition to a full understanding of study design and methodology, analysis of registry events and outputs requires an assessment of data quality. Requirements for data collection and quality assurance should be defined during the registry creation phase, and following the "collect once, use many" rule of data collection and management, it is paramount that the data are of sufficient quality, as the information and subsequent use for multiple potential purposes are all derived from that initial data. Data quality can be defined as the totality of features and characteristics of a data set that bear on its ability to satisfy the needs that result from the intended use of the data [6]. High-quality data are then data that are fit for use by data consumers, data that have sufficient usefulness and usability. This fact leads to viewing data quality as having many attributes, or in other words data quality is presented as a complex multidimensional concept.

4.2.1 Data quality dimensions and its assessment

Determining the quality of data is possible through data assessment against a list of dimensions which can be defined and measured. Data quality dimensions can be defined as a “set of data quality attributes that represent a single aspect or construct of data quality” [7]. The dimensions are organized in a data quality framework, which attempts to capture all aspects of data quality that are important to data consumers.

Deciding on a list of quality dimensions is mainly dependent on the patient registry context (nation and/or region specific provisions, legal obligations etc.), type and purpose. When defining a data quality framework, in order to ensure subsequent appropriate measurements of data quality, the developer should take care to include all the context relevant data quality dimensions.

A large number of distinct data quality attributes that might determine usability through literature review have been identified.[note 2] Most of the data quality dimensions were overlapping and had different interpretations, often with ambiguous definitions or completely lacking definitions, while the two most frequently cited were data „accuracy“ and „completeness“.

Trying to list all internationally used data quality dimensions and include their interpretations would prove a futile effort. Thus, the underlying principle for deciding on these dimensions and arranging them into a meaningful whole was providing comprehensive coverage while keeping dimensions organized in a collectively exhaustive way. Mutual exclusiveness was desired but is hardly achievable at the general level of description. Here is proposed a set of six data quality dimensions (Table 4.1).

Table 4.1 Data quality dimensions
Data quality dimension Description
Accuracy How well information in or derived from the data reflects the reality it was designed to measure [8]. It is usually characterized in terms of error in statistical estimates. It may also be described in terms of the major sources of error that potentially cause inaccuracy (e.g., coverage, sampling, non-response, response) [9].
  • How good are the data?
  • What is done with the data?
Completeness Extent to which all necessary data that could have registered have actually been registered [6].

It is usually described as a measure of the amount of available data from data collection compared to the amount that was expected to be obtained (e.g. coverage) [10].

  • Are all the appropriate data present?
Interpretability and Accessibility Ease with which data may be understood and accessed [8].

This includes the ease with which the existence of information can be ascertained, the suitability of the form or medium through which the information can be accessed, whether data are accompanied with appropriate metadata and whether information on their quality is also available (including limitation in use etc.) [9].

  • How readily accessible are the data?
  • How well documented are the data?
  • How easy is it to understand the data?
Relevance The degree to which data meets the current and potential needs of users.

The purpose is to assess how well data collection can adapt to change and whether it is perceived to be valuable [8].

  • Can user needs be anticipated and planned for?
  • How valuable are the data?
Timeliness Refers primarily to how current or up to date the data are at the time of release, by measuring the gap between the end of the reference period to which the data pertains and the date on which the data becomes available to users [8].

It is typically involved in a trade-off against accuracy. The timeliness of information will influence its relevance [9].

  • Are data made available in a reasonable amount of time?
  • Are key documents released on time?
Coherence Reflects the degree to which it can be successfully brought together with other statistical information within a broad analytic framework and over time. Coherence covers the internal consistency of data collection as well as its comparability both over time and with other data sources [11].

The use of standard concepts, classifications and target populations promotes coherence, as does the use of common methodology across surveys. Coherence does not necessarily imply full numerical consistency [9].

  • Does the database use standard definitions for data definitions?
  • Can common groupings be derived from the data?
  • Can databases be joined via a common data element?
  • Are data values being converted correctly?
  • Are data comparable with themselves over time?

The dimensions provided in the table are applicable to different registry types (and with different objectives), however not all may be equally important.

Assessing quality includes adequate management of each dimension, and additionally failure in one dimension can severely hinder the usefulness of the final registry report (i.e. when considering cancer registries insisting on the dimension of data completeness may ruin the demand for timely reporting). Likewise, each of the dimensions may possess equal importance, but also there may be instances where the relative importance of one dimension exceeds another. As stated previously, the importance of a particular quality dimension depends on the set objectives of the registry, its type, as well as its scope and methodology. Specifically, based on the definition of data quality provided above, the intended use of registry data actually determines the necessary properties and requirements of the data.

For instance, in a registry that is used to calculate incidence rates of diseases, it is essential to include all existing patient cases, therefore the completeness dimension is of critical importance.

Additionally, the need to explore different aspects of data quality is an accepted practice among patient-registries, and should be accentuated when not present.

For example, population-based cancer registries are considered particularly attentive to assessing data quality, as the value of the modern cancer registry and its ability to carry out cancer control activities rely heavily on the underlying quality of its data and the quality control procedures in place [12]. Data quality regarding cancer registries is usually assessed against the following three quality dimensions: comparability, validity, completeness, as well as sometimes timeliness as a fourth one. Factors influencing data quality and methods (both quantitative and qualitative) for measuring data quality within these dimensions have been devised and made available.[note 3]

Data quality dimensions are components that allow the user to quickly identify specific problematic aspects of data. Interrelatedness and overlapping are always necessarily present; the quality dimensions are not specific to quality measuring, and for that to be possible, as exemplified by cancer registries, decisions are needed to identify which methods and indicators are to be used in order to successfully measure registry data quality against dimensions. The data quality assessment programme should thus precisely define a data quality framework, preferably logically grouping what should be measured and how it should be measured and monitored in the data domain, thus making data dimensions more specific by creating data characteristics and criteria, along with a rating method. Such an example of a comprehensive method for assessing data quality is the Data Quality Framework [8], by the Canadian Institute for Health Information (CIHI) issued with the purpose of improving data quality of national health data collections. The Data Quality Framework is based on Statistics Canada guidelines and methods and information quality literature. It is a highly developed hierarchical framework model, with established criteria useful for systemic data quality assessments.

In summary, efforts should be made to create various relevant data quality dimension groups dependent on type and objectives of the registry, and devise methods and indicators for assessing data quality, so that a registry can use those methods to measure and gradually improve data quality.

4.2.2 Mode of data collection and impact on data quality

Considering data quality as part of a complex whole brings out another important and often neglected aspect which can influence data quality – the point where data are collected.[note 4]

The quality of initial data input from clinicians and health practitioners can vary. Quite frequently incorrect patients are registered or data items can be inaccurately recorded or not recorded at all.

A sustainable workflow model is an important element of a successful registry, a workflow that can be integrated into the everyday clinical practice of doctors, nurses, pharmacists, and patients (while respecting privacy legislation). Prior to the full launch of a registry, pilot testing can be organized to gather preliminary input from health care workers and others included in the data collection.

A decision should be made on the mode of data collection, as there are a few ways to collect data, where the primary difference is whether it is collected in its conventional paper form or the modern electronic form.[note 5]

4.2.3 Improving data quality

Since data quality is critical to any registry, a patient registry should seek to implement and maintain a high standard in all of the quality dimensions identified here of patient registries (governance, data quality, information quality, ethical issues, security and privacy). The governance dimension is crucial here (as discussed in subchapter 4.1.5.), as the initiative within an organisation to improve data quality is driven by managerial decisions, setting forth standards and channelling staff efforts. In this light, the Health Information and Quality Authority of Ireland (HIQA) describes “seven essentials for improving data quality”[note 6], which it is useful to consider in the context of a patient registry. These essential features are presented in the table below.

Table 4.2 Essentials for improving data quality
Leadership & Management
  • What: involves having in place executive-level responsibility, accountability and leadership.
  • Why: knowing who does what (e.g. the establishment of a governance committee that will ensure the registry is committed to data quality). Decision-wise, this includes the selection of only essential data elements when datasets are established.
Policies and procedures
  • What: developing and implementing clear policies and procedures on data quality for staff that are based on legislation and standards.
  • Why: can help ensure that a high level focus on data quality is translated into good practice amongst all those involved in data collection and handling within the registry.
  • What: ensuring that data are collected and processed in a standardised fashion (e.g. use of minimal datasets, data dictionaries and the creation of standard templates for data collection), designing the registry with respect to national and international standards.
  • Why: facilitates data interoperability and making data available. Also can improve consistency and reduce error.
Data quality dimensions
  • What: set of data quality attributes upon which data can be assessed, aligned with policies, procedures and training.
  • Why: measuring and monitoring level of data quality within a registry.
  • What: training of the staff in the requirements and importance of data quality.
  • Why: ensuring that policies and procedures adopted to generate high quality data are implemented and understood in practice.
Data quality audits
  • What: independent systematic examination of data (internal or external).
  • Why: providing feedback to all staff, indicating the areas for improvement, highlighting good practice in order to facilitate learning (e.g. automation of data collection over manual collection where possible will reduce error rate, however, this will not be verified without planned audits of data quality).
Make data available
  • What: availability of data when and where needed, in accordance with information governance safeguards (security, privacy).
  • Why: fulfilling the purpose for which the registry was created, increasing quality of registry data through its efficient utilization and dissemination.

4.3 Information quality

Information can be considered an output and the extension of the data collection process. Its quality is measured by the purpose of its use, which in the case of patient registries can be grouped into surveillance (including health statistics), outcomes, and scientific publication.

Scientific publication can be considered as a control for methodological prerequisites including sufficient level of data quality. Therefore, it can serve as an indirect information quality indicator. Levels of measuring can be publication amounts (total, yearly), subject relevance, up-to-date, impact factor, citation index.

Similarly, statistical data from registries focused on surveillance can be used as an indirect quality measure with regards to real-world decision making. Outcome based registries serve the same purpose in terms of indirect quality measurement albeit from a different viewpoint, i.e. using information from patient registries for influencing and improving treatment outcomes. Quality information gained from patient registries leads to informed healthcare management and better decision making.

4.4 Confidentiality, security, privacy, ethical issues, secondary use of information

This quality dimension is concerned with ethical issues and confidentiality and privacy regarding use of personal health information, as well as the need for proper patient registry data security and clear provisions regarding secondary use of information. Although actually concerning data and stemming from the wider dimension of (information) governance, it is here discussed separately as it involves privacy protection, a sensitive and seminal issue when discussing patient registries.[note 7]

Not meeting ethical and legal requirements concerning privacy renders the patient registry inoperable. Levels of data confidentiality, privacy and security also influence registry interoperability capability as well as information dissemination.

Creating a balance between respecting individual privacy and providing high quality personal health information can, although very important, also be a difficult task faced by patient registries as well as other healthcare related stakeholders. Striving for cross-border interconnecting and interoperability of patient registries is accompanied by emerging security risks concerning privacy, judging by the fact that health information systems present technical challenges to existing privacy protection legal frameworks.

In order to maintain the privacy of participants enrolled in a registry and the data confidentiality, security measures should be implemented. All security measures should be contained in a document that describes in detail the data security risks, policies, and procedures specific to that registry. Physical and technical safeguards should be incorporated in the collection, storage, transmission of and access to data. These include data encryption, restriction of data access, data back-ups, methods (software) for de-identification of local data during potential transmission and storage etc. Also, implementation of safeguards should not be done only once, but should undergo continuous review and revision.

Considering data usage, we can distinguish between two types: 1) primary purpose; 2) secondary use of data. This classification as primary or secondary is based on the relationship of the data to the registry purpose. Primary data sources include data collected and being kept by the registry holder (custodian) for direct purposes of the registry (i.e., primarily for the registry). The secondary use of health data considers uses for purposes other than those for which it was originally collected. Secondary uses include using information for (further) research, performance monitoring, service planning, audit and quality assurance purposes etc. When thinking about the secondary use of health data, it is necessary to carefully balance between the public interest and individual data subjects. Since secondary use of data may violate patient privacy, precautions should be taken and conditions must be satisfied if proposing to use information for secondary purposes. Clear definitions of the circumstances where data are to be used for secondary reasons should be developed.

Legislative provisions concerning the secondary use of data are typically contained within general privacy or data protection legislation, which can differ depending on the specific MS context.

The important things with secondary data use are that patients should be made aware that their information may be used for this purpose and have the benefits of the practice clearly explained to them. Likewise, consent must be obtained for the collection, use or disclosure of information for purposes outside the direct registry's data outline plan. Efforts should be performed to make data anonymous as well as using data sharing agreements which offer an additional safeguard against inappropriate use of information.

To repeat and to conclude the subchapter, researchers and other data users should disclose clearly how and why personal information is being collected, used, and secured, and should be subject to legally enforceable obligations to ensure that personally identifiable information is used appropriately and securely. In this manner, privacy protection will help not only to ensure research participation, public trust and confidence in medical research, but also prompt cross-border registry cooperation. If registry holders are confident that their information is being appropriately protected and have trust in the system, then they are more likely to share information, which leads to improved safety and quality of care at an individual level.

4.4.1 Privacy impact assessment (PIA) – a method to assess privacy

A privacy impact assessment (PIAs) is a tool, process or method to identify, assess, mitigate or avoid privacy risks [13]. PIAs are used internationally and across all sectors but are particularly useful to healthcare providers to identify potential risks around the collection and use of sensitive personal health information. PIAs can help respond to the new privacy challenges in the design of cross-border health information systems. The primary purpose of undertaking a PIA is to protect the rights of service users. The process involves the evaluation of broad privacy implications of projects and relevant legislative compliance, through describing how data are collected, processed, disseminated and published. Where potential privacy risks are identified, a search is undertaken, in consultation with stakeholders, for ways to avoid or mitigate these risks and to facilitate solutions which help safeguard privacy. As PIA considers the future privacy consequences of a proposed project that involves the collection and use of personal health information, it is most beneficial when conducted in the early stages of a project, and ideally at the planning stage [3].

Related with the goals of the PARENT project, a very useful PIA initiative has been identified with the EUBIROD project. EUBIROD explored privacy issues at the level of systems’ users, assessing the variability of data processing approaches in MS and their deviation from EU privacy standards and legislation, and by using the adapted version of the Canadian PIA Guidelines. Key elements of data protection (factors) were selected to ascertain the compliance/non-compliance with privacy principles/norms of data processing operations occurring in EUBIROD registries.

Registry privacy and data protection which should be investigated when conducting PIA are:

“accountability of personal information”; “collection of personal information”; “consent”; “use of personal information”; “disclosure and disposition of personal information”; “accuracy of personal information”; “safeguarding personal information”; “openness”; “individual access to personal information”; “challenging compliance”; “anonymisation process for secondary uses of health data” [14].


  1. Several other quality frameworks have been defined by different users’ groups. At this point, one should note the European Statistical System’s Quality Assessment Framework, available at http://ec.europa.eu/eurostat/documents/64157/4392716/ESS-QAF-V1-2final.pdf/bbf5970c-1adf-46c8-afc3-58ce177a0646.
  2. For a more detailed summary of the internationally commonly used data quality dimensions refer to a publication from HIQA: International Review of Data Quality. Dublin: HIQA, 2011. Available at: http://hiqa.ie/press-release/2011-04-28-international-review-data-quality
  3. Reviews of these methods are presented in more detail in [12] and [15].
  4. This issue has been briefly discussed in subchapter 4.1.2,, 6.2.4, 6.4 and 9.1
  5. Methods of data collecting (paper or electronic) are discussed in subchapter
  6. Health Information and Quality Authority. What you should know about Data Quality. Dublin, Ireland: HIQA, 2012.
  7. Privacy, confidentiality and security are mentioned in more detail in chapters 5 and 6.1.4.


  1. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Institute of Medicine, Washington: Committee on Quality of Health Care in America, 2000.
  2. 2.0 2.1 2.2 (AHRQ) Registries for Evaluating Patient Outcomes: A User’s Guide, 3Ed, Volume 2. In: Guide, editor, 2012.
  3. 3.0 3.1 3.2 3.3 (HIQA) Guiding Principles for National Health and Social Care Data Collections. Dublin: HIQA, 2013. Available at: http://www.hiqa.ie/publications/guiding-principles-national-health-and-social-care-data-collections
  4. Arts DGT,Bosman RJ ,de Jonge E, Joore JCA, de Keizer NF. Training in data definitions improves quality of intensive care data. Critical Care 2003;7:179-184.
  5. EURORDIS-NORD-CORD Joint Declaration of 10 Key Principles for Rare Disease Patient Registries. Available at: http://download.eurordis.org/documents/pdf/EURORDIS_NORD_CORD_JointDec_Registries_FINAL.pdf
  6. 6.0 6.1 Arts DGT, de Keizer NF, Scheffer G-J. Defining and Improving Data Quality in Medical Registries: A Literature Review, Case Study, and Generic Framework. Journal of the American Medical Informatics Association. 2002;9(6):600-611.
  7. Wang, RY, Strong DM. Beyond accuracy: What data quality means to data consumers. Journal of Management Information Systems. 1996;12(4):5-33.
  8. 8.0 8.1 8.2 8.3 8.4 Canadian Institute for Health Information, The CIHI Data Quality Framework, 2009 (Ottawa, Ont.: CIHI, 2009). Available at: http://www.cihi.ca/CIHI-ext-portal/pdf/internet/data_quality_framework_2009_en
  9. 9.0 9.1 9.2 9.3 Statistics Canada Quality Guidelines, Fourth Edition, October 2003. Available at: http://www.statcan.gc.ca/pub/12-539-x/12-539-x2003001-eng.pdf
  10. Ehling, Manfred, Körner, Thomas. (eds.) Handbook on Data Quality Assessment Methods and Tools Eurostat, European Commission, Wiesbaden, 2007. Available at: http://unstats.un.org/unsd/dnss/docs-nqaf/Eurostat-HANDBOOK%20ON%20DATA%20QUALITY%20ASSESSMENT%20METHODS%20AND%20TOOLS%20%20I.pdf
  11. Australian Bureau of Statistics. ABS Data Quality Framework [Online]. Available at: http://www.abs.gov.au/AusStats/ABS@.nsf/Latestproducts/5AFFD020BC4D1130CA25734700151AA5?opendocument
  12. Bray F, Parkin DM. Evaluation of data quality in the cancer registry: principles and methodsPart I: comparability, validity and timeliness. Eur J Cancer. 2009;45(5):747–755.
  13. (NHS)Privacy Impact Assessment: care.data. Chief Data Officer, NHS England, 2014. Available at: http://www.england.nhs.uk/wp-content/uploads/2014/04/cd-pia.pdf
  14. Privacy Impact Assessment Report. The EU.B.I.R.O.D. PIA Team, 2010. Available at: http://www.eubirod.eu/documents/downloads/D5_2_Privacy_Impact_Assessment.pdf
  1. Parkin DM, Bray F. Evaluation of data quality in the cancer registry: Principles and methods Part II. Completeness. Eur J Cancer 2009; 45:756–64.