This post is Part 5 of a 6 part series. Visit weekly or enter your email to subscribe.
FDA guidance suggests sponsors should include information about the intended use of computerized systems in their data management plan. Compliant with 21 CFR part 11, this data management plan should also describe the “security measures employed to protect the data and a description of the flow of electronic data”1. In the workflow demonstrated in the Figure below, note that a unique user name and password is required to access the system. Each user is associated with a user group and each user group is limited to specific roles/functions within the eSource. The user activity is date and time stamped in real time and an audit trail serves as the automated documentation of who did what, to whom and when.
eSource systems/vendors should:
support controlled and secure access typically by
a username/ ID and password,
data encryption, and
create and maintain an audit trail of data entry and data revisions including
the “old” value,
the “new” value,
date and time stamp,
and the user who entered/changed the data
reason for the data edit
have a provision for data transfer and storage for the time required by the regulations
This post is Part 4 of a 6 part series. Visit weekly or enter your email to subscribe.
Electronic diary solutions are typically customized software programs built with rules and edit checks that can track compliance, facilitate efficiencies and significantly reduce queries. Electronic boundary and edit checks should be used to minimize errors and omissions at the time of data entry. Additionally, an eDiary solution can include a subject dosing screen to capture required investigational product administration as well as software customization to prompt the subjects to complete missing data before allowing them to move from one question or screen to the next. Electronic diaries can include fields to “record storage temperatures of study medication that was stored in subjects’ refrigerators”1 in compliance with storage condition requirements specified by the protocol. Likewise, drug administration screens can be linked to drug inventory management systems to meet the FDA requirement to maintain adequate records of the disposition of the investigational drug (21 CFR §312.62(a)).
When implementing electronic edit checks, user acceptance testing should be a priority to ensure business logic and protocol driven rules fit the purpose, prevent duplicate entry, make sense to the user and do not conflict. For example, it would be confusing for a patient who identifies himself in the eDiary as male to be asked about his pregnancy status. Data elements that are captured more than once should auto populate when the data is static (i.e. date of birth), but require a new entry for data expected to change (i.e. weight). Electronic triggers should be tested to avoid conflict and verify missing or out of range data. Device logic that requires the patient to answer a question about climbing stairs before moving on to the next question should have an option to indicate a reason for leaving the question blank. In this example, the patient may be in a wheelchair, or may not have climbed any stairs in the applicable timeframe for the interval questionnaire. In another example, an electronic rule that requires the patient to complete a “visit 2” diary before allowing access to “visit 3” diary data entry could be problematic when the patient may have missed or skipped their second visit or the allowable time window to enter the visit 2 information. Sponsors should include information on electronic prompts, flags, and data quality checks that are designed to address data inconsistencies, missing data, and entries out of range in their data management plan so that it is available for FDA inspection.2
This post is Part 2 of a 6 part series. Visit weekly or enter your email to subscribe.
In the event of an inspection or an audit, sponsors may be asked to show evidence the eSource selected for a clinical trial has been vetted as fit for purpose. Specific to ePRO, the FDA intends to “examine the final version of an instrument in light of its development history, including documentation of the complete list of items generated and the reasons for deleting or modifying items…whether the PRO instrument’s final conceptual framework (e.g., the hypothesized relationships among items, domains, and concepts measured) is confirmed in the appropriate study population and is consistent with the endpoint model of the planned clinical trials.”1 Documentation of eSource solutions as compliant with FDA “fit for purpose” expectations would include evidence of:
software and system development according to protocol required specifications
sponsor user acceptance training as part of the system development life cycle
screen shot validation showing the electronic tool does not influence or alter the user ability to consistently use or gather the data because of variability in the user display
equivalency testing when the solution platform varies from user to user
consideration for cultural practices when studies are global (i.e. a VAS scale read from right to left in Hebrew instead of traditional left to right practice)
a “demo” device the inspector can use to evaluate the functionality
Sponsors should verify and validate they chose the right ePRO solution and it was built correctly. For example, a web-based ePRO application, dependent on varying web-browsers, may pose a challenge for data consistency across all modalities when there is variation in operating system and content format that may or may not require scrolling to see the entire question. “When a PRO instrument is modified, sponsors should provide evidence to confirm the new instrument’s adequacy. That is not to say that every small change in application or format necessitates extensive studies to document the final version’s measurement properties.”1 However, moderate and substantial modifications may require additional validation when the PRO instrument is altered in item content or format. ISPOR (International Society for Pharmacoeconomics and Outcomes Research) suggests examples of moderate and substantial modifications to a validated paper and pencil PRO as:
Moderate – changes in item wording, splitting a single item into multiple screens, significantly reducing font size so that a subject would need to scroll or change screens to see all the possible responses, significant changes that alter interpretability, changing the order of items
Substantial – removing items or making significant changes to item text to fit the PRO to an electronic screen2
According to ISPOR, validated ePRO questionnaires that have been adapted from paper do not require equivalence studies if the data produced is superior (higher reliability), has comparable psychometric properties, and there is no modification to content from the original paper questionnaire. Sponsors need to decide whether or not to invest in customized software to accommodate the chosen validated PRO instrument; or the potential need for equivalency analysis or full psychometric validation before implementing an electronic tool for widespread use.
Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report* Value in Health 12 (4). 2009.
This post is Part 1 of a 6 part series. Visit weekly or enter your email to subscribe.
Technology is important in the product development life cycle as well as the clinical trial workflow. The regulations are flexible in allowing sponsors and CROs to choose electronic solutions for data collection either home grown or vendor purchased. In any case, however, the electronic systems should meet the sponsor needs and the sponsor should vet and document the system capabilities and functionality. First let’s define the “e” terms.