If you are not a medical device company or do not have devices on the market in Europe, then comprehensive clinical evaluation reports may be a new concept. But make no mistake, the use of these powerful assessments extends beyond just Europe and just medical devices. As regulators, payers and the public increases emphasis on clinical data to support medical devices, this familiar format will be useful for other applications. Presenting clinical data in an unbiased format has been successful for biologic devices in other international geographies. Executive summaries of these reports may be leveraged to demonstrate the similarity of new devices to those currently with payer coverage to gain reimbursement.

The following article outlines the content and process for completing a clinical evaluation report (CER) in consideration of the new guide issued June 2016. In addition to overviewing the content and methodology for the CER, the article considers how the CER can be leveraged for 1. CPT® Coding Applications, 2. Dossier for other payers’ consideration of coverage in the U.S., and 3. Application for International Geography submissions.

INTRODUCTION:

1. A CER is a comprehensive, unbiased assessment of published literature and unpublished internal data. CER’s are required for all classifications of medical devices with CE Mark allowing for distribution in the European Union (EU). This evaluation is central to the medical device lifecycle management and risk management requirements defined by ISO 13485 “Medical devices – Quality Management Systems – Requirement for Regulatory Purposes” and ISO 14971 “Medical Devices – Application of Risk Management to Medical Devices.”

The CER analysis determines if the benefits of the device evidenced in performance outweigh the risks of the device’s actual or potential adverse effects. This benefit/risk evaluation determines if the device may immediately be placed on the market in Europe, if the device needs additional clinical data potentially in the form of a controlled clinical trial, or if the device should not be marketed in Europe. The benefit/risk conclusions that aid in European market entry are similar to considerations of medical devices by regulators and payers:

• If the device is placed in a patient, does the device perform as intended, extending life or improving the quality of life?
• Does the device perform as well as alternative standards of care for patients to justify the increased cost of healthcare?
• Does the device demonstrate substantial improvement over existing solutions?

COMPONENTS OF CLINICAL EVALUATION

The format of a CER is provided in MEDDEV 2.7.1 Revision 4 “Clinical Evaluation: A Guide for Manufacturers and Notified Bodies under Directives 93/42/EEC and 90/385/EEC”[1] that was recently updated as of June, 2016. Specifically, the CER is a tool used to demonstrate conformity to the following essential requirements defined in the Medical Device Directive (a.k.a., European device regulations):

• E1 General Requirements relating to reduction of risk of device,
• E3 General Requirement that the device perform as intended, and
• E6 General Requirement that undesirable side effects are acceptable when weighed against benefits of the device.

What’s new in Revision 4 of the guidance document?

• Analysis of pre-clinical verification and validation
• Stricter use of equivalent devices
• Closer consideration of pre-market clinical trials

The CER may include relevant scientific literature related to the device itself or equivalent device as well as all clinical investigations. For devices that are not CE Marked, clinical data for an equivalent device may be utilized. The CER includes the following stages:

• Stage 0: Define the scope
• Stage 1: Identify the pertinent data
• Stage 2: Appraisal of pertinent data
• Stage 3: Analysis of clinical data
• Stage 4: Finalize the Clinical Evaluation Report

STAGE 0: DEFINE THE SCOPE:

The scope of the CER (Stage 0) includes a device description as specific as part numbers to easily determine which products the report covers for future submissions and audits. The product design should be evaluated to determine if any special design features need special considerations (e.g., a medical device that includes an animal-derived component). The scope also includes an evaluation of the risk management documents that can include the risk analysis and risk management plan. The scope should define “state of the art” and information relating to current disease management.

STAGE 1: IDENTIFY THE PERTINENT DATA:

Stage 1 is the identification of pertinent data that the manufacturer generates and that is retrieved from literature. Data should include all favorable and unfavorable data related to the device. Specific to the new guidance document, preclinical testing used for verification and validation during the design process should be included in the CER. Preclinical testing can include bench testing and/or animal testing. The CER should consider if the device meets the acceptance criteria for design verification or validation. Any failures of the devices (e.g., mechanical failure of a device in fatigue testing prior to clinically-relevant runout) should be considered in the CER.

Manufacturer-generated clinical use data includes premarket clinical investigations, post market clinical data generated in geographies where the device is launched, complaint reports, explanted device evaluations, and field safety correction actions. Other unpublished data can include the instructions for use (IFU), surgical techniques and/or user manuals and may include claims related to special design features.

Literature searches are rigorously documented with reasons for inclusion/exclusion and involve not only U.S. databases like PUBMED, but also international medical device registry reports. As PUBMED does not contain many European journals, other sources like EMBASE, Cochrane and Google Scholar should be considered to ensure comprehensive data is identified. Internet searches can be helpful to identify relevant industry standards. But such searches often result in 1,000+ abstracts; therefore, a well-established protocol for assessing inclusion and exclusion criteria is important. The cross-functional design team can add in search terms including competitive devices that are similar to the subject device. Broad search terms should be limited to ensure that applicable articles are identified in the search process. Typically, appendices of the CER contain all search results with reasons for exclusion so that the reader can easily follow the authors’ assessment criteria. Tables documenting exact search terms and date limits should be included in the CER so that the search may be repeated at any time. Care should be taken for duplicate search results as well as multiple publications with the same patient data set.

EQUIVALENT DEVICES:

When the subject device has robust clinical data with long-term follow-up, equivalent devices may not be needed. In cases where manufacturers wish to leverage published clinical data, equivalent devices can be included to determine if sufficient data exists to support the performance and safety of the subject device. The new guide MEDDEV 2.7.1 provides detailed methodology to determine an equivalent device. Careful consideration should be made to the clinical, technical and biological characteristics of the equivalent device compared to the subject device. Clinical characteristics include severity and stage of disease, intended purpose, and patient population. Technical characteristics include design, specifications, and method of use. Biological characteristics identifies the material and contact with human tissue or body fluids. Previous guidelines allowed use of an equivalent device made of different materials provided its clinical performance was similar. For example, a clinical evaluation for an orthopedic plate composed of a novel material such as a resorbable material may have included equivalent devices composed of metallic materials (e.g., titanium alloy), as pre-clinical testing would demonstrate that both exceed acceptance criterion for clinical relevant loads and are biocompatible. Under the new guidance document, leveraging clinical data from devices composed of different materials may be more difficult.

STAGE 2: APPRAISAL OF PERTINENT DATA:

Appraisal of the pertinent data in Stage 2 is needed to ensure the quality of the data and its applicability to the indication, the target population and the intended users of the medical device. Appraisal includes a detailed review to assess relevance to the device followed by weighing the contribution of the data to the clinical evaluation. Previous versions of MEDDEV 2.7.1 provided an example of a detailed weighing of data. This guidance document omits a specific example, but includes that typical weighing provides the highest weight to randomized controlled clinical trials of the device followed by non-randomized trials of the device. Clinical data for the device is weighted higher than clinical data for an equivalent device. The evaluator should carefully document each piece of data reviewed and document the rationale for inclusion or exclusion in the CER. For devices with controlled, randomized clinical trials, the evaluator may choose to exclude case reports when evaluating performance, but then include case reports if adverse events are included in a safety evaluation.

To fully leverage the CER for other applications, consider aligning literature to CPT® Coding Applications. These applications are submitted to U.S. national payer, Medicare, for coding changes. CPT® Coding Applications increasingly rely on clinical evidence for changes in codes. The level of evidence for this application is as follows:

• Ia: Evidence obtained from meta-analysis of randomized trials
• Ib: Evidence obtained from at least one randomized trial
• IIa: Evidence obtained from at least one well-designed controlled study without randomization
• IIb: Evidence obtained from at least one other type of well-designed quasi-experimental study
• III: Evidence obtained from well-designed, nonexperimental descriptive studies, such as comparative studies, correlation studies and case controls
• IV: Evidence obtained from case reports or case series
• V: Evidence obtained from expert committee reports or opinions and/or clinical experience of respected authorities

In general this U.S. coding application requires that three (3) reports are of the U.S. population and at least one (1) meets the evidence of level III. Abstracts should be considered for negative safety data if reports of adverse events differ from those published in literature, but they should not be included as performance evidence in a CER. These adverse event reports should be considered since the abstract publication timeframes are shorter than journal article publications. As with a CER, coding applications should include the evaluation of negative publications.

Typically, only publications with the device or equivalent devices are included in the CER. Data may also be weighted depending on the device within the article (e.g., 1 if the subject device, 2 for equivalent device, and 3 for neither). Weight by device highlights those that could be leveraged for reimbursement. Clinical data on the subject device is preferential for payer coverage in the U.S. With clinical data tables in the CER organized by device, the process of identifying and pulling out publications for dossier can be completed quickly when requesting coverage.

Driven by the new guidance in detailed determination of equivalent devices, CERs should identify the relevant device studies for each publication. Historic publications may not identify the model and manufacturer of the subject device, and equivalent device leveraging will prove more difficult for manufacturers. Determining the relevance of these publications can be difficult for commodity devices. For example, common bone screws date back to at least the 1930’s.[2] Consider that a bone screw can have general indications:

The Bone Screw set is intended to be used for the fixation of bone fractures, fusion of joints or bone reconstruction.[3]

Small bone screws can be used in most bones of the body from the head for cranial flap fixation, in the hand for metacarpal fracture repair, and in the foot for fusion of the tarsometatarsal (TMT) joint. Larger screws of a system can be used in femur and tibial fracture repair. As the new guidance specifies that each indication should be adequately supported, mapping publications with specific uses to this general indication will be challenging. Additionally, publications for bone screws historically have not listed the device or manufacturer; therefore, the majority of the publications would be excluded. This consideration will continue to confound differences in indications for the U.S., the European Union and the rest of world.

STAGE 3: ANALYSIS OF CLINICAL DATA:

Methodology used to collect the clinical data in each publication should be considered. For example, case reports of a single patient are excluded as these lack sound scientific methodology. Data are appraised by considering the number of patients for statistical relevance, statistical methods, control (such as single-arm studies), or appropriate endpoint. Data are summarized typically in tabular format with the number of patients, follow-up time period, outcomes and any adverse events observed. Performance of devices is typically observed in outcomes. For example, cervical spine implant outcomes are typically measured with these parameters for the cervical spine: Neck Disability Index (NDI) score; radiographic fusion status; adverse events such as reoperations, removals, or pain; and quality of life measurements such as visual analog score or SF-36 questionnaire. Outcomes vary by medical device and should be described in the CER.

The analysis should evaluate the nuances for each publication considering the disease and patient population. Consider a single site publication that has negative outcomes such as high revision rates of a vascular graft to treat patient with peripheral artery disease (PAD) at a specific center. If this publication has a large number of patients, and the only other evidence is single site publications and case reports, the evaluator may determine that there is insufficient benefit to outweigh the risk of high revision. Yet, if the patient population are those that would receive amputation without the graft, the evaluator may determine that the risk of revision is minimal compared to the risks of the alternative of amputation. The analysis will vary greatly depending on the device, disease and intended patient population.

Safety data considers any adverse events, especially severity of death, allergic reaction, re-operation, and amputation. For certain populations, these severe events are expected based on the disease state and severity. The CER should consider expected rates of the events without the medical device versus those with the medical device to narrow the effect of the medical device on the patient populations.

The analysis should determine if additional clinical investigations or post-market clinical follow-up are needed. Post-market clinical follow-up is common for long-term implantable devices such as hip replacements with a lifetime expectancy of 10-20 years. Although national joint registries are providing some data related to survivorship, manufacturers are expected to determine survivorship of specific variants or designs within large product families.

STAGE 4: FINALIZE THE CLINICAL EVALUATION REPORT:

The CER weighs the risk/benefit ratio of the device by considering its performance compared to its adverse events to determine if the device should be marketed in the first place or remain on the market.

The analysis should tie back into the product development lifecycle. After the clinical data has been summarized, consider the following:

• Are there any additional risks that were identified in the clinical data appraisal that are not in the risk analysis? If so, the risk analysis should be updated to include the additional risk and risk mitigation activities should be conducted to reduce the risk as far as possible.
• Do adverse effects occur at a greater rate than the initial risk analysis predicts? Does the initial risk analysis predict occurrence as a greater risk than those for the equivalent devices? Update the risk analysis to consider true occurrence rates based on observation in published clinical data.
• Based on the clinical data, should any additional warnings be added to the instructions for use for the device? The IFU correctly identified the intended purpose and contains information to reduce the risk of user error; residual risks; and appropriate warnings, precautions and contraindications.

Post-market plans should also be updated to include any additional clinical investigations prior to CE mark of the device or post-market clinical follow-up in Europe. If the device is marketed outside the EU and insufficient data is available to CE mark the device, then the post-market plan should be updated to include investigations needed for CE Market. Alternatively, a pre-market clinical trial may be conducted in the EU to support CE marking.

Before finalizing the CER, ensure that all relevant information is included such that the document is independent of other design-related documentation. Add the background and resume of the clinical evaluator to demonstrate his or her qualifications to perform the evaluation.

In conclusion, the evaluator carefully considers the risks to ensure that they are minimized and acceptable when weighed against the benefits. The benefits should include a meaningful and measureable positive improvement on the health of an individual. As regulators for international regulatory submissions are increasingly requesting clinical data for the subject device, consider submitting the CER in its entirety to regulators around the world.

CONCLUSION:

Methodology developed to show compliance to European essential requirements has provided a powerful, unbiased methodology for identification, appraisal and reporting of unpublished and published data relating to medical devices. Overall, revision of the guidance document is a step towards increased regulation of medical devices in line with revision of ISO 13485 issued in 2016 for required compliance by 2019 and upcoming Medical Device Regulations requiring compliance in 2019. Increased regulation is also accompanied by an increased drive for clinical-based evidence to support coding and coverage globally. The CER can be leveraged in all these situations to present clinical data in an unbiased, comprehensive document.

---

[1] Clinical Evaluation: A Guide for Manufacturers and Notified Bodies under Directives 93/42/EEC and 90/385/EEC. MEDDEV 2.7.1 Rev 4 June, 2016

[2] Celeste M Abraham. A Brief Historical Perspective on Dental Implants, Their Surface Coatings and Treatments. Open Dent J. 2014; 8: 50–55. Published online 2014 May 16. doi: 10.2174/1874210601408010050. PMCID: PMC4040928. Suppl 1

[3] K 132248 “Flower Bone Screw Set” SE Oct. 30, 2013

Finis