Subject-Reported Diary Data: What’s the problem with paper?

Subject compliance in any clinical trial is always a concern, and may be a catalyst to use electronic diaries, or some modality other than paper.  This blog entry will discuss a few examples of when subject compliance is critical, data that exposes risks associated with paper, and what the FDA guidance suggests about unsupervised subject-reported data.

Below are a few examples of when subject compliance plays a crucial role in the overall success of a study.

  • Subject-Reported Diary data used to determine randomization eligibility.
    • Example:  Subject must be 80% compliant with placebo dosing during run-in period to be eligible for randomization.
  • Subject-Reported Diary data used to determine dosing escalation during study.
    • Example:  Subject’s daily symptoms are used to calculate dosing escalation throughout treatment period.
  • Subject-reported Diary data used to determine if subject withdrawn from study.
    • Example: Subject’s compliance with daily survey (90% compliance) is required for data to be considered in study results.

The examples above are taken from studies in which the Sponsor used electronic diaries to mitigate the risks associated with paper.

So what’s the problem with using paper to capture these critical data points? 

A study published in Volume 324 (18 May 2002) of the BMJ explains the risk introduced when using paper diaries to capture important data from subjects at home.  In their study, half the subjects were given paper diaries, while the other half were given electronic diaries.  What the paper-diary subjects did not know is their diaries came equipped with a photo-sensor on the inside cover.   Although 90% of the subjects reported completing their paper diary according to the protocol specified time periods, the embedded photo-sensor confirmed only 11% of the subjects actually completed the diary as instructed.  Instead, most often the subjects opened their diary in the parking lot either immediately after it was dispensed or just prior to their follow-up clinic visit.  The same study discovered over 94% of subjects were compliant when using their electronic diary.   You can read the BMJ article in its entirety by clicking here.

When discussing this study with Sponsors, their first question is typically – What does the FDA say?  An excerpt from FDA Guidance on Patient Reported Outcome Measures (December 2009) states the following:

“If a patient diary or some other form of unsupervised data entry is used, we plan to review the clinical trial protocol to determine what steps are taken to ensure that patients make entries according to the clinical trial design and not, for example, just before a clinic visit when their reports will be collected.”

It’s reasonable to deduce from this guidance, that the most effective way to satisfy this requirement is to use an electronic diary that keeps a real-time audit trail with a time and date stamp for each subject-reported entry.

You can read the FDA PRO Guidance document by clicking here.

Return to next week to read about techniques used in clinical trials to improve compliance.


Free Webinar: Patient Reported AEs using eDiary Technology: July 17th

Patient Reported Adverse Events are becoming vital to clinical trials as studies are collecting critical data from patients at home.

Join assisTek for a free webinar on  July 17, 2013 at 1:00PM EDT as we discuss Patient Reported AEs using eDiary Technology.


The Webinar will discuss the following, and will conclude with an interactive Question & Answer Session:

  • Research that suggests problems with the typical AE reporting process
  • Real World Examples of Patient Reported AE’s using eDiary Technology (Phase III Global Studies)
  • System Integration (eDiary& EDC)
  • What are the Concerns?
  • What are the Benefits?


Date and Time

Weds, July 17, 2013 at 1:00PM EDT

Click to Register:


Contact Evonne Roberts,, 480-874-9400 x. 336


Webinar: eSource in Clinical Trials, Pros/Cons & Lessons Learned from Phase III Clinical Trials

For those of you who missed the eSource Webinar hosted by assisTek on April 4, 2013, you can still watch the entire Webinar on video below!


This webinar will focus on Real-Life examples of ongoing Phase III global clinical trials using mobile Direct Clinical Data Capture (eSource) solutions. Topics discussed will include:
- Overcoming data quality and data access problems that plague studies using the traditional paper transcribed to EDC process.
- Supporting site workflow with eSource
- Cost Savings Model (Query Reduction & Elimination of SDV)
- Impact and Integration with external clinical databases (EDC, etc)

 Please contact Evonne Roberts, with any questions or to schedule an on-site demonstration.


Patient Reported AE’s using eDiary Technology

scdm logo


Recently assisTek authored an article published by the Society of Clinical Data Management’s “Data Basics”.

The article discusses the benefits of assessing AE’s in real time when the symptoms are experienced by the subject (versus during site visits only) and how the data can be integrated into existing clinical databases (including EDC).




eSolutions to Meet FDA Requirements to Maintain Adequate Records (Part 6 of 6)

This post is the final of a 6 part series. Visit weekly or enter your email to subscribe.

FDA “… recommends that clinical data be entered electronically by study site personnel at the time of the subject visit to avoid transcription from unnecessary paper records1. The acronym “ALCOA” is frequently used in reference to best practices related to source documentation. When eSource solutions are selected and developed appropriately, data is attributable to a unique user with a secure password and role-specific privileges; legible in a clean and easy to read standardized format; and contemporaneous with a date and time stamp for every entry/edit/modification. Original records initially captured electronically can include direct entry of clinical data into a study database, data transmitted from an electronic health record, and electronic diary data entered by a subject or transmitted from an automated instrument that captures a biological result. eSource provides the opportunity to engineer accurate data with boundary and edit checks for missing and/or out of range data devoid of contradicting information often represented by subjective reporting in an unstructured format.

To minimize risks related to FDA requirements to keep accurate records and protect human subjects, sponsors using electronic source solutions should keep records of:

  • system validation and maintenance,
  • system specifications showing customization to fit the eSource for the purpose of the study,
  • 21 CFR § 11 compliance and
  • data integrity practices.

Ultimately, the same system development collaboration and documentation intended to manage risk, will have the added benefits of quality design, validated performance and accurate data capture with electronic efficiencies that reduce data noise, facilitate analysis and expedite time to data lock…and that’s what is best for patients.

eSource Best Practices
1Guidance for Industry Electronic Source Documentation in Clinical Investigations Draft Guidance. U. S. Department of Health and Human Services, Food and Drug Administration, Office of the Commissioner. December 2010. Retrieved October 23, 2012 at…/Guidances/UCM239052.pdf


eSolutions to Meet FDA Requirements to Maintain Adequate Records (Part 5 of 6)

This post is Part 5 of a 6 part series. Visit weekly or enter your email to subscribe.

FDA guidance suggests sponsors should include information about the intended use of computerized systems in their data management plan. Compliant with 21 CFR part 11, this data management plan should also describe the “security measures employed to protect the data and a description of the flow of electronic data”1. In the workflow demonstrated in the Figure below, note that a unique user name and password is required to access the system. Each user is associated with a user group and each user group is limited to specific roles/functions within the eSource. The user activity is date and time stamped in real time and an audit trail serves as the automated documentation of who did what, to whom and when.

Figure 3 

eSource systems/vendors should:

  • support controlled and secure access typically by
    • a username/ ID and password,
    • user roles,
    • role-specific privileges,
    • data encryption, and
    • workflow
  • create and maintain an audit trail of data entry and data revisions including
    • the “old” value,
    • the “new” value,
    • date and time stamp,
    • and the user who entered/changed the data
    • reason for the data edit
  • have a provision for data transfer and storage for the time required by the regulations

to be continued…

1 Guidance for Industry Electronic Source Documentation in Clinical Investigations Draft Guidance. U. S. Department of Health and Human Services, Food and Drug Administration, Office of the Commissioner. November 2012. Retrieved December 18, 2012 at






eSolutions to Meet FDA Requirements to Maintain Adequate Records (Part 4 of 6)

This post is Part 4 of a 6 part series. Visit weekly or enter your email to subscribe.

Electronic diary solutions are typically customized software programs built with rules and edit checks that can track compliance, facilitate efficiencies and significantly reduce queries. Electronic boundary and edit checks should be used to minimize errors and omissions at the time of data entry. Additionally, an eDiary solution can include a subject dosing screen to capture required investigational product administration as well as software customization to prompt the subjects to complete missing data before allowing them to move from one question or screen to the next. Electronic diaries can include fields to “record storage temperatures of study medication that was stored in subjects’ refrigerators”1 in compliance with storage condition requirements specified by the protocol. Likewise, drug administration screens can be linked to drug inventory management systems to meet the FDA requirement to maintain adequate records of the disposition of the investigational drug (21 CFR §312.62(a)).

When implementing electronic edit checks, user acceptance testing should be a priority to ensure business logic and protocol driven rules fit the purpose, prevent duplicate entry, make sense to the user and do not conflict. For example, it would be confusing for a patient who identifies himself in the eDiary as male to be asked about his pregnancy status. Data elements that are captured more than once should auto populate when the data is static (i.e. date of birth), but require a new entry for data expected to change (i.e. weight). Electronic triggers should be tested to avoid conflict and verify missing or out of range data. Device logic that requires the patient to answer a question about climbing stairs before moving on to the next question should have an option to indicate a reason for leaving the question blank. In this example, the patient may be in a wheelchair, or may not have climbed any stairs in the applicable timeframe for the interval questionnaire. In another example, an electronic rule that requires the patient to complete a “visit 2” diary before allowing access to “visit 3” diary data entry could be problematic when the patient may have missed or skipped their second visit or the allowable time window to enter the visit 2 information. Sponsors should include information on electronic prompts, flags, and data quality checks that are designed to address data inconsistencies, missing data, and entries out of range in their data management plan so that it is available for FDA inspection.2

to be continued…

1 FDA Warning Letter Ref. No. 06-HFD-45-11. Retrieved October 25, 2012 at

2 Guidance for Industry Electronic Source Documentation in Clinical Investigations Draft Guidance. U. S. Department of Health and Human Services, Food and Drug Administration, Office of the Commissioner. November 2012. Retrieved December 18, 2012 at



eSolutions to Meet FDA Requirements to Maintain Adequate Records (Part 3 of 6)

This post is Part 3 of a 6 part series. Visit weekly or enter your email to subscribe.

eSource solutions must allow PI access to electronic data. Developing and planning eSource solutions should also address data transfer to the sponsor in an effort to protect the rights, safety, and welfare of subjects. Sean Kassim of the FDA recently pointed out that in the world of eDiary and DCDC, direct transfer of ePRO to the sponsor is NOT appropriate. As part of their oversight, the Principal Investigator (PI) needs to have access to this data typically via site access to an online portal (See Figure).


eSource Manuscript Figures 1_2

In addition, “To comply with the requirement to maintain accurate case histories (21 CFR 312.62(b)) and 812.140(a) (3)), investigators should review and electronically sign the eCRF for each subject before the data are archived” and released to the parties (IRB, Data Management Center, SMO/CRO, Regulatory authority).Data that goes directly to the sponsor may contain information wholly relevant to patient care and human subject protection (HSP). For example, PROs with multiple entries of shortness of breath should be reviewed by the PI to determine whether or not clinical intervention is indicated. Compliance with this requirement for PI oversight can be in the form of a formal process for review of the data or the electronic instrument can alert the PI to review certain criteria specified at the onset of the study – either way, the PI needs to have access to the data.

In the process of data management, data entered into the diary by the subject should be available primarily for subject safety, but also to review and query for accuracy. In the figure above, the electronic model indicates PRO questionnaires are accessible as view only data and not editable by the site staff while the PI/designee with a specific user role is able to modify concomitant meds (CM), infusion data, and untoward events in response to a query. Some have suggested only the originator, in this case the subject via the eDiary,  should be allowed to change or modify the “original source”. The most recent FDA draft guidance however, suggests someone other than the “originator” can modify original electronic source so long as the modifications meet the standards in the guidance:

  • Modified or corrected data carry a new data element identifier
  • Identifier includes date, time and originator of the change
  • Both the modified data element and the identifier is write-protected (read only)
  • The original data element with its original data element identifiers is preserved and available for FDA investigators

Additionally, when it makes sense to wipe out data to recycle the device, the local information including the audit trail of data modifications needs to be saved, and a data file (CD or pdf) serves as the certified copy representative of data collected in compliance with the standards in the guidance as well as  21 CFR§11.1.2

1 Guidance for Industry Electronic Source Documentation in Clinical Investigations Draft Guidance. U. S. Department of Health and Human Services, Food and Drug Administration, Office of the Commissioner. November 2012. Retrieved December 18, 2012 at

2 Helfgott, J., Kassim, S. 2012. Drug Information Association Clinical Management Teleconference “FDA on FDA Perspective on ePRO”. October 19, 2012.

to be continued…………..


eSolutions to Meet FDA Requirements to Maintain Adequate Records (Part 2 of 6)

This post is Part 2 of a 6 part series. Visit weekly or enter your email to subscribe.

In the event of an inspection or an audit, sponsors may be asked to show evidence the eSource selected for a clinical trial has been vetted as fit for purpose. Specific to ePRO, the FDA intends to “examine the final version of an instrument in light of its development history, including documentation of the complete list of items generated and the reasons for deleting or modifying items…whether the PRO instrument’s final conceptual framework (e.g., the hypothesized relationships among items, domains, and concepts measured) is confirmed in the appropriate study population and is consistent with the endpoint model of the planned clinical trials.”1 Documentation of eSource solutions as compliant with FDA “fit for purpose” expectations would include evidence of:

  • software and system development according to protocol required specifications
  • sponsor user acceptance training as part of the system development life cycle
  • screen shot validation showing the electronic tool does not influence or alter the user ability to consistently use or gather the data because of variability in the user display
  • equivalency testing when the solution platform varies from user to user
  • consideration for cultural practices when studies are global (i.e. a VAS scale read from right to left in Hebrew instead of traditional left to right practice)
  • a “demo” device the inspector can use to evaluate the functionality

Sponsors should verify and validate they chose the right ePRO solution and it was built correctly.   For example, a web-based ePRO application, dependent on varying web-browsers, may pose a challenge for data consistency across all modalities when there is variation in operating system and content format that may or may not require scrolling to see the entire question. “When a PRO instrument is modified, sponsors should provide evidence to confirm the new instrument’s adequacy. That is not to say that every small change in application or format necessitates extensive studies to document the final version’s measurement properties.”1 However, moderate and substantial modifications may require additional validation when the PRO instrument is altered in item content or format. ISPOR (International Society for Pharmacoeconomics and Outcomes Research) suggests examples of moderate and substantial modifications to a validated paper and pencil PRO as:

  • Moderate – changes in item wording, splitting a single item into multiple screens, significantly reducing font size so that a subject would need to scroll or change screens to see all the possible responses, significant changes that alter interpretability, changing the order of items
  • Substantial – removing items or making significant changes to item text to fit the PRO to an electronic screen2

According to ISPOR, validated ePRO questionnaires that have been adapted from paper do not require equivalence studies if the data produced is superior (higher reliability), has comparable psychometric properties, and there is no modification to content from the original paper questionnaire.  Sponsors need to decide whether or not to invest in customized software to accommodate the chosen validated PRO instrument; or the potential need for equivalency analysis or full psychometric validation before implementing an electronic tool for widespread use.

  1. Guidance for Industry Patient Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. December 2009. Retrieved October 25, 2012 at
  2. Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report* Value in Health 12 (4). 2009.

to be continued…………


eSolutions to Meet FDA Requirements to Maintain Adequate Records (Part 1 of 6)

This post is Part 1 of a 6 part series. Visit weekly or enter your email to subscribe. 

Technology is important in the product development life cycle as well as the clinical trial workflow. The regulations are flexible in allowing sponsors and CROs to choose electronic solutions for data collection either home grown or vendor purchased. In any case, however, the electronic systems should meet the sponsor needs and the sponsor should vet and document the system capabilities and functionality. First let’s define the “e” terms.

to be continued….

  1. Guidance for Industry Electronic Source Documentation in Clinical Investigations Draft Guidance. U. S. Department of Health and Human Services, Food and Drug Administration, Office of the Commissioner. November 2012. Retrieved December 18, 2012 at
  2. CDISC Clinical Research Glossary.  Applied Clinical Trials 20(12): 11-43. 2011.