Study Set Content:
factors when assessing the quality of a body of evidence:
– Risk of bias – Precision – Consistency – Directness – Publication (or reporting) bias– Magnitude of effect size (or treatment effect) – Presence of confounders that would diminish an observed effect – Dose-response effect (or gradient)
refers to threats to internal validity, i.e., limitations in the design and implementation of studies that may cause some systematic deviation in an observation from the true nature of an event, such the deviation of an observed treatment effect from the true treatment effect.
Risk of bias
refers to the extent to which a measurement, such as the mean estimate of a treatment effect, is derived from a set of observations having small variation (i.e., are close in magnitude to each other).
precision
is inversely related to random error. • Small sample sizes and few observations generally widen the confidence interval around an estimate of an effect, decreasing the precision of that estimate and lowering any rating of the quality of the evidence.
Precision
refers to the extent that the results of studies in a body of evidence are in agreement
Consistency
can be assessed based on the direction of an effect, i.e., whether they are on the positive or negative side of no effect or the magnitudes of effect sizes across the studies are similar.
Consistency
refers to the proximity of comparison in studies, that is, whether the available evidence is based on a direct comparison of the intervention and comparator of interest, or whether it must rely on some other basis of comparison
Directness of comparison
where there is no direct evidence pertaining to intervention A vs. comparator B, evidence may be available for
intervention A vs. comparator C and of comparator B vs. comparator C
refers to how many bodies of evidence are required to link the use of an intervention to the impact on the outcome of interest
Directness of outcomes
determining whether a screening test has an impact on a health outcome, a single body of evidence that randomizes patients to the screening test and to no screening and follows both populations through any detection of a condition, treatment decisions, and outcomes
direct evidence
Requiring multiple bodies of evidence to show each of detection of the condition, impact of detection on a treatment decision, impact of treatment on an intermediate outcome, and then impact of the intermediate outcome on the outcome of interest
indirect evidence
refers to unrepresentative publication of research reports that is not due to the quality of the research but to other characteristics
Publication bias
includes tendencies of investigators and sponsors to submit, and publishers to accept, reports of studies with “positive” results, such as those that detect beneficial treatment effects of a new intervention, as opposed to those with “negative” results (no treatment effect or high adverse event rates)
Publication bias
managing publication bias
–Prospective registration of clinical trials (e.g., in ClinicalTrials.gov)
– adherence to guidelines for reporting research
– efforts to seek out relevant unpublished reports
can improve confidence in a body of evidence where the relevant studies report treatment effects that are large, consistent, and precise.
Magnitude of effect size
refers to instances in which plausible confounding factors for which the study design or analysis have not accounted would likely have diminished the observed effect size
Plausible confounding that would diminish observed effect
: a group of patients receiving the new treatment has greater disease () at baseline than the group of patients receiving standard care, yet the group receiving the new treatment has better outcomes, it is likely that the true treatment effect is even greater than its observed treatment effect
severity
refers to an association in an individual study or across a body of evidence, between the dose, adherence, or duration of an intervention and the observed effect size
Dose-response effect
Initial confidence in an estimate effect
Study design
High confidence
Randomized trials
