| Element
of Paper |
Critical
Issues |
| Title |
Does
the title reflect:
-The study design?
-The primary hypothesis?
-The population under study?
The title should - reflect study objectives. |
| Research
Question |
What
is the primary hypothesis?
- On what basis was the research question conceived?
- Has the question been previously studied?
- Does it address the question it sets out to answer?
|
| Study
Design |
Generally
should fit into two broad groups:
-Retrospective
Case report
Case series
Case-control studies
-Prospective
Cohort studies (non-interventional)
Randomised controlled trial (interventional) |
| Study
outcome(s) and appropriateness of outcome(s) to research
question |
Primary
Outcome
- Should be one main primary outcome
- Should be the most important and easily measured outcome
- Should be free from bias
- Should be appropriate to the research question
E.g. Death, survival, presence of disease, length of
hospital stay, number of re-operations (all very objective
parameters)
Secondary Outcomes
- Usually softer and more subjective end-points
- Usually added to the study to address secondary issuesE.g.
Cost-effectiveness, pain, quality of life etc. |
| Study
Population |
Look
at the inclusion and exclusion criteria carefully:
| - |
Broad
inclusion criteria means greater external validity
but also means presence of confounding factors |
| - |
Narrow
inclusion criteria mean less confounding and
an easily studied group but also means limited
external validity |
Check
if the study population is appropriate to the
study question at hand |
Check
if the study population is similar to your clinical
practice population |
|
| Sample
size considerations (especially in a clinical trial) |
Essential
that sample size be calculated a priori before commencing
any trial!Key components:
| -
|
Type
1 error rate (significance level, p):
Rejecting the null hypothesis when it should
not be rejected. Usually 2-tailed and set at
95% i.e. < 5% chance that results obtained
due to chance and results can be greater or
lesser than expected difference |
| - |
Power:
Probability of determining significance in the
study if a true difference exists. Reflection
of Type 2 error rate i.e. failing to reject
the null hypothesis when it should be (Power=
1 – type 2 error rate). Usually set at
80% |
| - |
Difference
in primary outcome between intervention
and control groups: Usually gleaned from previously
published or pilot studies. The larger the difference,
the smaller the sample size required! |
|
| Type
and appropriateness of statistical analysis |
This
requires some statistical knowledge. Information on
data handling and analyses should be clearly stated.
Be circumspect about analyses with multiple comparisons
as the level of statistical significance may be influenced.Most
tests should use a 2-tailed significance level set
a 95% i.e. p<0.05. However, be wary if the analyses
are disproportionate to the type and amount of data
available. |
| Strengths
and weaknesses of the paper |
Key
questions:
- Was the study well conceived?
- Are all patients accounted for?
- Is there room for improvement?
- Any ethical concerns?
- Are the results well presented?
- Discussion relevant? |
| Should
my clinical practice be altered? |
Key
question:
Is there external validity?
- Efficacy vs effectiveness study
- Look at Table 1 – Demographics of patients,
does your patient group resemble the study group
- Review the inclusion and exclusion criteria
- Search for confounders
- Any long term or safety data?
- Look for previously published studies examining similar
issues |
| Does
the study contribute to the existing body of literature? |
If
the study is just one in a series of many similar studies
but with slight variation in patient group or question,
then it may be quite pointless!
The key questions:
- Is the study pioneering?
- Does it contribute to resolving a key issue in clinical
practice?
- Will it influence clinical practice in the long and
short run?
E.g. The Women’s Health Initiative study has influenced
the practice of HRT use and has caused many clinicians
to re-examine their practice and many patients to be
more involved in their own care. |
