A Clinical trial involves the testing of the drug(s) of interest on humans before making it available for commercial use. Clinical trial is a highly regulated process in which every small detail is documented and submitted for approval and audit.
One of the most important and obvious stages in a clinical trial is the analysis of the data that has been collected. Now, that the clinical trial process is highly regulated, it mandates to prepare a protocol as well as a Statistical Analysis Plan (SAP) before initiating the Clinical trial. These documents serve as the Standard Operating Procedures (SOP) for the Clinical trial processes.
There are many types of SAPs viz., Clinical Study SAP, Interim Study SAP, Data Monitoring Committee SAP, and Integrated SAP. These different types of SAP play a crucial role during different stages and for different departments in the clinical trial.
A statistician in combination with the investigator is present during the preparation of the SAP as well as while interpreting and gathering the requirements from it. This article is for those statisticians who are starting out new and have difficulties in understanding the SAP. This article focuses on addressing all the questions a statistician would have about the SAP.
According to the ICH E9 Statistical Principles for Clinical Trials, “A statistical analysis plan is a document that contains a more technical and detailed elaboration of the principal features of the analysis described in the protocol, and includes detailed procedures for executing the statistical analysis of the primary and secondary variables and other data.” The SAP serves as a guide or a compilation of preset rules that are to be followed during the Clinical trial data analysis. Every detail starting from the sample size calculation, cleaning the data, handling the bias, descriptive analysis, etc., to the inferential analysis and visualization if required is pre written in the SAP. It also contains the name of the software to be used for statistical analysis and the details of the deliverables that has to be submitted in the Clinical Study Report (CSR).
Now, as a statistician who is a beginner in this field, one would wonder, why this cumbersome process of preparing a 40 page SAP is mandatory and why only the Protocol isn’t enough. Well, any action could be performed efficiently when it is planned ahead of time. Especially in a highly audited clinical trial process, the prior planning is often required because any small deviation from the protocol may lead to the loss of integrity of the entire study. This not only affects the CROs and the trial sites but also the credibility of the sponsors. Thus, the Statistical Analysis Plan is very crucial and every step of the data analysis must be planned way ahead and should be strictly adhered to. The lack of reproducible results and the transparency in the methodology used during the data analysis, makes SAP highly mandatory. The other advantages of developing the SAP beforehand helps in the clear communication between the statistician and the investigator. Several research have proven that the SAP present at the time of the data analysis have improved its internal and external validity.
The Statistical Analysis Plan is always created at the beginning of the data analysis. It is also called the Data Analysis Plan (DAP) in some organizations. It has to be developed before the treatment un-blinding. This ensures to curtail the possibility of “data-dredging” or “p-hacking”. Every clinical trial will have a DSMB (Data Safety Monitoring Board) that constantly monitors the safety of the trial subjects. During the course of the study, interim analysis is performed at the prespecified time to monitor the adverse events among the trial subjects. The DSMB utilizes the SAP and the Protocol as a guide during this process to decide on whether to continue or stop the study, based on the severity of the adverse events.
SAP is a guide that is similar to the protocol and is prepared cautiously like that of the Informed consent form, the CRF, etc,. The SAP is stored along with other documents in the clinical trials and is made available to all the personnel. The investigator, the statistician, and the DSMB have a copy of the SAP. NIH-funded interventional studies are mandated to submit a copy of the SAP to ClinicalTrials.gov. The SAP is also attached to the Clinical Study Report (CSR) while submitting to the regulatory authority. SAP is mandatory for interventional studies, whereas some of the journals require an SAP for observational studies too.
SAP is prepared collaboratively by the statistician and the investigator. Sometimes the
statistician is also listed as the co-investigator. The investigator provides the study design,
desired variables, exposure and desired outcomes. The statistician is responsible for framing
hypotheses based on the study design and the outcomes, the sampling methods and the
sample size calculation, the method of data analysis and the presentation of the results.
Statisticians also include mock shells of Tables, Listings and Figures in the SAP, however it is
After the SAP is developed, it is reviewed by another senior statistician who was not involved in the development of the SAP. All of these have to be completed before the treatment or data un-blinding.
Once we understand what is an SAP and why do we need it, the question of how to prepare the SAP arises. In 2017, Gamble et al., provided an exhaustive list of all the components required in a SAP. These are divided into 6 major sections of:
- Administrative Information
- Study Methods
- Statistical Principles
- Trial Population
The entire list can be found in the Table below:
|Section 1: Administrative Information|
|Title and Trial Registration||A title that matches the protocol, the trial acronym and the trial registration number|
|SAP Version||Versioning of the SAP along with the dates|
|Protocol Version||The reference to the version of the protocol that is being used|
|SAP Revisions||The number of revisions of the SAP as and when applicable along with the time and the justification.|
|Roles and Responsibility||The names, affiliations and other details of the SAP contributors|
|Signatures of:||The statistician, the chief investigator, senior statistician and others as applicable|
|Section 2: Introduction|
|Background and Rationale||The general introduction about the trial and the rationale for pursuing it. The background of this and the previous similar studies.|
|Objectives||The Primary and Secondary objectives of the study|
|Section 3: Study Methods|
|Trial design||The type of study design used (parallel arm, single arm etc.,) as per the instructions from the lead investigator|
|Randomization||The method of randomization or stratification used and the justification|
|Sample size||The full sample size and power calculation
or reference to these calculation in the
(to avoid redundancy)
|Framework||Superiority, equivalence, or non inferiority hypothesis testing frameworks including which comparisons will be presented on this basis|
|Statistical interim analysis and stopping guidance||Details about what interim analysis will be carried out and when; The guidance on stopping the trial and its justification|
|Timing of final analysis||Timeline of the final analysis that is when all the outcomes are analyzed finally.|
|Timing of outcome assessments||The time points at which the outcomes are assessed along with the details of every visit window|
|Section 4: Statistical Principles|
|Confidence intervals and p-values||The preset levels of significance; The justification for any adjustments for any multiplicity and how type 1 error can be controlled|
|Adherence and Protocol Deviation||Description of the adherence to the intervention and the protocol deviation; and which deviations will be summarized.|
|Analysis Population||The intent to treat population as per the protocol|
|Section 5: Trial Population|
|Screening data||To represent the trial sample|
|Eligibility||The list of eligibility criteria for taking part in the trial|
|Recruitment||The recruitment guidelines to be listed in the CONSORT flow diagram|
|Withdrawal/follow-up||The guidelines to account for the withdrawn patients and also to follow up existing patients.|
|Baseline Patient characteristics||The baseline details of the parameters of the patients|
|Section 6: Analysis|
|Outcome definitions||Clear description of the outcome variables|
|Analysis methods||The methods used for the analysis of the data (descriptive and or inferential) depending on the type of data that will be collected|
|Missing data||Methods to handle the missing data|
|Additional analysis||Details of any of the additional analysis that is required|
|Harms||The details on safety, adverse events, causality and how the adverse event data will be analyzed and reported etc.,|
|Statistical software||The details of the statistical packages that will be used for the analysis|
|References||Reference to the protocol, trial master file, Data management plan, non-standard statistical methods and any other references to the standard operating procedures to be used in the trial.|
As a statistician, the Statistical Analysis Plan (SAP) is an important document that decides the integrity of the clinical trial. The SAP must contain the in-depth information of all the details needed for the data analysis. The statistician must possess comprehensive and critical evaluation skills in order to prepare as well as review the SAP without looking at the data. SAP is attached to the Clinical Study Report (CSR) and also submitted to the ClinicalTrials.gov, for interventional studies. SAP is highly important so as to avoid “data-fishing”.
- https://www.clinicalleader.com/doc/fairly-simple-statistical-analysis-plan-elements-to-impl ement-in-your-clinical-study-0001
- Pharmaceutical Programming: From CRFs to Tables, Listings and Graphs, a process overview with real world examples. Mark Penniston et al., Lax Jensen 2005.
- Statistical Analysis Plan – Clinical Programming Reviewers Guide. Xiaoyin (Sherry) Zhong et al., Lax Jensen 2018.
- ICH Topic E 9 Statistical Principles for Clinical Trials
- Gamble C, Krishan A, Stocken D, Lewis S, Juszczak E, Doré C, Williamson PR, Altman DG, Montgomery A, Lim P, Berlin J, Senn S, Day S, Barbachano Y, Loder E. Guidelines for the Content of Statistical Analysis Plans in Clinical Trials. JAMA. 2017 Dec 19;318(23):2337-2343. doi: 10.1001/jama.2017.18556. PMID: 29260229.