AMA Publishes SHAM Chelation Study

AMA PUBLISHES JUNK SCIENCE IN APPARENT ATTEMPT TO DISCREDIT CHELATION

Refer to a much more detailed analysis with references below

Click here for earlier critique of this study written prior to publication
In January, 2002

The American Medical Association published a deceptively worded and blatantly unscientific study that alleged to disprove benefit from EDTA chelation as a treatment for heart disease—the so-called Calgary PATCH study.(1) Nowhere do they actually claim to have disproven chelation, although that is implied and that is the way many readers will interpret it. They merely state that they found, “no evidence to support a beneficial effect.” In their final sentence they reiterate that conclusion: “Larger trials with a broader range of patients will be needed to assess the safety and impact of EDTA chelation therapy on clinical event rates.” That conclusion is not surprising, since a careful reading of the published report clearly shows that this study was too small, too flawed, and too poorly designed in many ways to produce anything of significance, beneficial or otherwise. It is puzzling that the American Medical Association, with its reputation for scientific integrity to uphold, would publish such pseudoscience in its flagship journal.

Careful analysis shows that the study seems designed to disprove chelation from the outset. Only one-fourth the number of patients needed for statistical significance was included. Patients most likely to benefit were selectively excluded. Most patients in the study had only minor symptoms, and 30% had no symptoms at all. It is not possible to study a treatment for angina in patients who do not have angina.

Twice as many patients were placed in the EDTA-treated group who had previously experienced myocardial infarctions.

The exercise protocol was bizarre. They failed to screen for reproducibility as a condition for entry. Accepted scientific guidelines were ignored. The primary endpoint was not clearly defined. The type of electrocardiographic ST-depression used as an endpoint is now considered non-specific and is no longer accepted as diagnostic for coronary disease.

Approximately twice as many patients in the misrepresented “placebo” group were given potent anti-anginal drugs. That would obscure comparative improvement in EDTA-treated patients, who received only half as much anti-anginal drug therapy. There was therefore no true “placebo” group. The F.D.A. in the United States. has never approved a new drug to treat angina without first requiring trials wherein all other anti-anginal medications had been discontinued.

Four patients in the placebo group and none in the EDTA-treated group underwent angioplasty during the one-year follow-up after chelation, suggesting that EDTA chelation reduces the need for invasive procedures. EDTA-treated patients also showed more improvement in maximal oxygen consumption. If the study had only included more subjects for statistical analysis, and produced these same results, it would have been a very favorable study showing a greatly reduced need for angioplasty or bypass following chelation therapy.

Below is Dr. Olmstead's detailed scientific analysis of the PATCH study with references.

1. Knudtson ML, Wyse DG, Galbraith PD, et al. Chelation therapy for ischemic heart disease. A randomized controlled trial. JAMA. 2002;287:481-6.

Critique of the PATCH Chelation Trial

by Stephen F. Olmstead, MD

INTRODUCTION

The PATCH trial was a very small randomized controlled clinical trial that attempted to evaluate the effect of multiple infusions of magnesium EDTA versus placebo on exercise-induced electrocardiographic (ECG) changes in patients with coronary heart disease. The PATCH investigators reported their results in the January 23/30, 2002, edition of the Journal of the American Medical Association.(1) Critics and detractors of EDTA chelation therapy have and shall continue to draw upon the publication of this study to spin out the tired untruth that EDTA chelation is ineffective therapy for cardiovascular diseases. A dispassionate scientific analysis in response to purported "scientific studies" of EDTA chelation is important. Such an analysis of the PATCH trial reveals it to be a poorly conceived and designed clinical study. The results of the study do not support the investigators’ reported conclusions that EDTA chelation does not increase ischemia threshold. On the contrary, the study, flawed as it is, suggests that EDTA chelation therapy may improve maximal oxygen consumption and reduce the need for percutaneous coronary interventions in a highly select group composed primarily of asymptomatic or minimally symptomatic patients with coronary artery disease.

POSSIBLE BIAS BY STUDY INVESTIGATORS & FUNDING ENTITIES

The principal investigators and authors were affiliated with the Division of Cardiology, University of Calgary and the Calgary Regional Health Authority, Calgary, Alberta. The PATCH investigators were not objective, unbiased observers. The PATCH investigators’ previous publications show that they believed EDTA chelation has already been proven to be ineffective and may deprive patients of the “well-established benefits” of other therapies.(2) The study was supported by the Alberta Health Services Research and Innovation Fund, Medical Services Incorporated Research Foundation, and the Calgary Regional Health Authority. Canada’s health organizations and authorities have a long history of opposition to EDTA chelation therapy. It is possible that the funding entities set out to support an objective fair study of EDTA chelation for therapy of chronic stable angina pectoris. However, the highly defective study design suggests that the funding entities goal was to underwrite a negative clinical trial that could be used in an attempt to discredit the practice of EDTA chelation therapy. A review of the study design reveals erroneous assumptions underlying the size of the study groups, a highly select study population, an unusual and ineffective exercise protocol, lack of concordant qualifying treadmill tests, problems with the primary study endpoint, uncontrolled confounding factors, and suboptimal statistical analysis.

ERRONEOUS ASSUMPTIONS FOR STUDY GROUP SIZE

The PATCH trial is seriously underpowered, rendering any conclusions based on the study outcomes tenuous at best. The investigators stated they designed a trial with a 90% power to detect a 60-second mean time in “exercise time from baseline to 27-week follow-up.” The authors do not clearly state what is meant by “exercise time.” Did they do the necessary calculations based on time to 1 mm ST segment depression? Or time to maximal oxygen consumption (VO2max)? Or time to anaerobic threshold? Or time to exercise termination? The basic primary endpoint is never precisely expressed in the assumptions underlying the entire study.

Putting aside the imprecise articulation of the hypothesis to be tested, the PATCH investigators made the assumption that the standard deviation of the “exercise time” would be 80 seconds. The standard deviation of the endpoint of interest in the population to be studied is critical to determine the number of subjects required to show a statistically significant difference between the means of the endpoint of interest in each group. The PATCH investigators' assumption is entirely arbitrary. Given that they used a rather unique exercise protocol, possibly peculiar to Calgary, one would think they had enough experience with the protocol to realistically estimate the standard deviation, rather than guess. This appears not to be the case. How accurate and appropriate was their assumption? This question is easily answered by looking at the measured standard deviation of time to ischemia on the baseline exercise tests. (Assuming this is the endpoint of interest for the PATCH investigators’ statistical assumptions.) The actual standard deviation for this endpoint was 172 seconds for the placebo group and 176 seconds for the chelation group. The PATCH investigations’ assumption was wrong by a factor exceeding 100%.

Given that the real standard deviation for time to ischemia on the Calgary exercise protocol in the study’s highly selected patient population was more than twice the assumed standard deviation, what can be said about the power of the study to detect a difference between the study groups? The PATCH study is grossly underpowered. The number of subjects that should have been required in each group at a power of 0.90 to detect a 60 second difference with a P value <0.05 is about 169 patients. The PATCH study randomized less than 25% of the patients truly required to test the null hypothesis that EDTA chelation does not increase the time to ischemia by more than 60 seconds. The PATCH trial is therefore totally impotent. No reasonable findings can be made from the study except that the concept and design were poor and the design flawed.

HIGHLY SELECT STUDY POPULATION

The PATCH trial randomized a highly select group of patients. Patients were selected from a minute section of the clinical spectrum of patients with coronary artery disease. This narrow selection makes it highly inappropriate and misleading to arrive at the extraordinarily broad conclusion that EDTA chelation does not have a beneficial effect "in patients with ischemic heart disease, stable angina pectoris, and a positive treadmill test for ischemia." The PATCH investigators have no idea whether this statement is true or false. Whatever the answer may be, their study provides no meaningful insight into the question because they performed a poorly designed trial using a population mostly consisting of patients with no symptoms or minimal symptoms.

Approximately 30% of both the treatment and control groups had no symptoms. They had no angina pectoris at all. They may never have had angina pectoris. This information is not presented. Although the PATCH investigators explicitly state that the presence of "stable angina pectoris while receiving optimal medical therapy" was an entry criterion for participation in the trial, they apparently ignored this key inclusion criterion nearly one-third of the time. Also, when 30% of the subjects in a very meager trial do not have angina, it is not scientifically honest to extrapolate findings in this small asymptomatic patient subset to the general population of people suffering from angina pectoris. Of the remaining study patients, 39.5% of the control group and 53.7% of the treatment group had mild angina pectoris. It seems obvious that it is a challenge to improve the status of patients without symptoms, or with only minimal symptoms. The design and study population of the PATCH trial would have made it difficult, if not impossible, to assess the possible favorable impact of any intervention in these patients with coronary artery disease.

Just how select the PATCH study population is, can be readily appreciated when it is realized that of the 3,140 patients screened, only a miniscule 171 or 5.45% were considered eligible for treadmill screening. Only half of the subjects who exercised met the treadmill criteria. Thus only 2.68% of the patients screened were randomized, a highly select population indeed. Nonsignificant disease was reported as the reason for ineligibility in 14.97%. Nonsignificant disease is never defined. Was it no coronary artery disease? Was it no lesion greater than 50% or 70%? A rather large percentage, 18.34%, was unable to perform the treadmill. Perhaps increasing the workload every 10 to 15 seconds was too rapid of an increase? A planned revascularization was the reason why 13.12% did not participate. That is ironic as it is this very population that may benefit the most from EDTA chelation therapy.(3)

A recurrent criticism of clinical trials is that they often have little relevance to the everyday practice of medicine. For a clinical trial to have any scientific, let alone practical, relevance to clinical medicine, the study population, conditions of treatment, and outcomes should mirror the patient population encountered in the clinic and the hospital. The PATCH investigators have managed to design and conduct a trial with no relevance at all to the care of patients with coronary heart disease (30% had no angina, another 40% or more had only minimal angina).

UNUSUAL AND INEFFECTIVE EXERCISE PROTOCOL

The exercise protocol employed by the PATCH investigators is unusual, even strange. The described protocol begins the treadmill at an exercise level of 2 metabolic equivalents (METS). A full 12-lead electrocardiogram (ECG) was recorded every 20 seconds. Workload was increased every 10 to 15 seconds until 13 METS was reached at 14 minutes. As a reference, 13 METS are attained when the first minute of Stage 4 of a standard Bruce protocol exercise test is completed. Sedentary healthy patients would not be expected on average to exercise to 13 METS. Such exercise capacity is generally encountered in healthy, physically active people with adjustment for age and gender. Increasing workload every 10 to 15 seconds is an unusually rapid rate of increase. Given the rapid rate of increase in workload and the high level of exercise expected, the PATCH investigators may wish to consider renaming their exercise protocol the “Calgary Stampede.”

The PATCH investigators are not forthcoming with basic information about the protocol they used. Was treadmill speed fixed and incline altered? Was incline fixed and speed altered? Were both speed and incline increased? How great was each increase? Was the magnitude of the increase constant or did it vary? If it varied, was the variance constant? We are given no real information about the protocol. The information supplied is highly suggestive that the PATCH investigators selected an exercise protocol unlikely to detect a true benefit of EDTA chelation therapy on time to ST-segment depression.

There are recognized guidelines for the use of exercise testing in clinical research. The PATCH investigators ignored generally accepted guidelines. Among these guidelines are those issued by the American Heart Association’s Assessment of Functional Capacity in Clinical and Research Applications.(4) Although commonly used, the Bruce protocol is generally considered a poor choice because of the high increment in workload from stage to stage. Protocols with modest increases in workload are preferred. When a "ramp" protocol is employed, which seems to be the case in the PATCH study, then workload should increase every 30 to 60 seconds and the interval between increments should be consistent.(4) This allows for accurate determination of onset of ischemia and symptoms. The selected protocol should yield a fatigue limited exercise duration of about 10 minutes. Durations above 12 minutes should be avoided. The PATCH protocol follows no recognized exercise guidelines.

The choice to increase exercise load every 10 to 15 seconds is particularly bizarre. Was it increased every 10 seconds? Every 12 seconds? Every 15 seconds? Was workload increased sometimes by 10 seconds and sometimes by 12 seconds? Sometimes was workload increased by an interval between 10 and 15 seconds? Was the interval 10 seconds in some patients during some tests and 15 seconds during other tests? The odds are, in the PATCH study, patients were not exercised using the exact same protocol. This introduces much bias. Most importantly, patient exercise workload could have been increased 3 or possibly more times between ECGs. Thus the PATCH protocol was highly insensitive to accurately detecting the onset of ST-segment depression. It is highly likely that ST-segment depression may have occurred at constant workloads, but the time to ST-depression varied purely as a function of inconsistent increases in exercise loads. The glaring difficulties with the PATCH trial exercise protocol make all the data on time to ST-depression meaningless.

The only data originating from the serial exercise tests with any possible meaning in this study are the VO2max and the anaerobic threshold (more commonly called the ventilatory threshold). But these data are also questionable. VO2max is the product of cardiac output and arteriovenous oxygen difference at exhaustion. It is measured in liters per minute. It is usually expressed in milliliters per kilogram body weight to facilitate intersubject comparison. Because it is unclear whether subjects exercised to exhaustion, it is unknown whether the PATCH investigators really measured VO2max in their subjects. Because the PATCH investigators terminated exercise testing at 14 minutes, clearly some subjects definitely did not have their real VO2max measured. Instead, one imagines a false VO2max was computed. When added to actual VO2 max values, if there were any, these computed values would lower the mean. In addition, failure to properly express the VO2 max, real or imputed, per kilogram body weight makes these data nearly useless for intersubject and intergroup comparison. Although the PATCH study showed that EDTA chelation improved VO2 max, a comparison between two group VO2 max means is not scientifically valid.

The aerobic or ventilatory threshold (VT) is defined as the exercise load at which ventilation begins to increase exponentially for a given increase in VO2. This increase in ventilation is caused by the need to eliminate excess CO2. VT is a commonly used submaximal index of aerobic capacity. VT is usually about 47% to 64% of VO2 max.4 There is no general agreement on methods for determining VT. The PATCH investigators provide no information about which method they used—assuming, of course, they used a method accepted outside of Calgary. They also provide none of the VT data to facilitate analysis. Why is that? All they provide is time to VT. Given that VT by definition is an exercise level, time to VT is meaningless when exercise levels are increased arbitrarily and inconsistently every 10 to 15 seconds.

The exercise methodology, failure to follow accepted guidelines, and undefined approaches to data collection and reporting render the exercise data virtually meaningless. Standards exist for exercise testing to evaluate antianginal pharmacologic interventions.(5) The PATCH investigators were either unaware of them or choose to ignore them. The exercise protocol employed in the PATCH trial would almost certainly never have been accepted by the US Food and Drug Administration for the evaluation of an anti-anginal drug. The likelihood is that the US National Heart Lung and Blood Institute would not have accepted this protocol in any study it funded. That this strange exercise protocol was employed at all is indicative that either the investigators and funding entity reviewers had no experience with clinical trials of anti-anginals or had a secondary agenda aside from science. There is only one legitimate scientific finding that can be made based on the PATCH exercise data. The data are of poor quality and permit no reasonable interpretations that would withstand scrutiny.

LACK OF CONCORDANT QUALIFYING EXERCISE TESTS

The PATCH investigators recognized that one explanation for the reported observed improvement in time to ST-segment depression in both the control and treatment groups may have been a "combination of placebo and training effects. Did the PATCH investigators only think about this after the study? Competent clinical researchers and funding entity reviewers would have been aware of this possibility while writing the protocol. While the presence and magnitude of a placebo effect during serial exercise testing is a subject of debate (6), there is evidence it does occur in some individuals.(7,8) In fact, the anti-anginal effect of beta-blocking agents has only been clearly shown in placebo-non-responders.(8) The training effect is very well known.(9) Both effects should have been known to the PATCH investigators prior to designing the protocol. Methodologies for minimizing the impact of these confounding effects have been around for years.(4,9) It is unknown why the PATCH investigators elected not to employ these well-established techniques in their trial.

Among the most commonly used methodologies is the requirement for concordant exercise test results before randomizing the subjects. Concordant exercise test results mean that the time to an exercise endpoint should be reproducible on serial screening exercise tests. Exercise duration should not vary by >60 seconds on repeated testing. Should disagreement >60 seconds occur between the first two qualifying tests, a third test is routinely accepted as a tie-breaker and the time concordant tests are accepted as the baseline test. The longer exercise time is used as the baseline exercise time. If reproducible exercise tests cannot be obtained, a patient is not a suitable subject for a trial using exercise-induced endpoints. Patients should be exercised at the same time of day to minimize the well-known diurnal variation in exercise performance and onset of ischemia.(4,9,10) The use of concordant qualifying exercise tests greatly helps minimize the placebo and training effects. In the PATCH trial, no concordant qualifying tests were required. It is not even clear if follow-up exercise tests were performed at the same time of day under similar conditions. At a minimum, interpretation of the PATCH exercise data is fraught with error from numerous potential confounding factors. At a maximum, the data are worthless.

PROBLEMS WITH THE PRIMARY ENDPOINT

The PATCH trial's primary endpoint was time to 1 mm of ST-segment depression. ST-segment depression is widely known among cardiologists to be a nonspecific finding. ST-segment depressions correlate poorly with other more sensitive tests of myocardial ischemia, such as nuclear medicine scans, especially in low risk populations such as the PATCH population.(11) Although the PATCH population reportedly all had coronary disease, it is entirely uncertain whether >1 mm of ST-segment depression in any given patient was really indicative of ischemia. Another imaging modality such as echocardiography or myocardial scintigraphy would have been of much greater use. Especially if a recognized exercise protocol with required concordant exercise tests had been used.

As it is, the PATCH investigators used a definition of significant ST-segment depression that made the finding as nonspecific as possible. Contrary to accepted practice, ST depression in only one lead was required. This greatly reduces the specificity of the finding. Most anti-anginal or anginal intervention protocols require ST-segment depression in at least two leads, usually in two contiguous leads. Most protocols also require two independent observers to agree on the presence and depth of ST-segment depression. The PATCH trial apparently did not require these simple measures, making any reported ST-segment changes more subjective than objective findings.

The selection of time to ST-segment depression as an endpoint is compromised by the lack of concordant baseline tests, the fact that a consistent exercise protocol was not employed, and actual exercise load may not have been the same at any given time. It would be very useful to know the heart rate (HR) and blood pressure (BP) at which ST-segment depression occurred. The double-product (HRxBP) at which ST-segment depression occurs is generally constant and is especially useful to determine when exercise time increases because of training effect. The omission of these data is a serious oversight; if it was an oversight.

The primary endpoint in the PATCH trial, time to ST-segment depression, is nonspecific and subjective. Given the major defects with the exercise protocol and the failure to use general accepted techniques to minimize well-known confounding factors, the reported times to ST-segment depression are highly suspect.

UNCONTROLLED CONFOUNDING FACTORS

The PATCH trial is replete with confounding factors. Chief among them are the placebo and training effects. No measures were implemented to decrease these effects. Thus it is uncertain why time to ST-segment depression or the exercise component of the Seattle Angina Questionnaire (SAQ) may have improved. In fact, it is interesting to speculate whether the SAQ can be appropriately used in patients without angina?

There are many other potential confounding factors. No information is given on the percentage of cigarette smokers in either group. Smoking is thought to significantly reduce the potential benefit of EDTA chelation. It is possible that patients with coronary artery disease in Alberta do not smoke, but that seems highly unlikely. Assuming some of the patients smoked, was the percentage of smokers similar in each group? All subjects were reportedly seen at the University of Calgary Cardiovascular Risk Reduction Clinic. One assumes smokers were counseled to stop smoking. How many stopped? Was the percentage who stopped similar in each group? Was any attempt made to ascertain whether patients who said they stopped really did? Smoking as a confounding factor in the PATCH trial appears to not only have been uncontrolled, but not considered or reported.

The standard for testing of anti-anginal therapies is to have the subjects discontinue other anti-anginal therapies during the trial.(5,12) Withholding standard anti-anginal therapy during trials is known to be safe.(12) The US Food and Drug Administration routinely requires that all other anti-anginal drugs be discontinued during clinical trials of new anti-anginal therapies. The investigators are remiss for not following this standard, as continued use of anti-anginal medications during the trial serves only to confuse and confound the results. Most, if not all, of the PATCH patients were on anti-anginal therapy. Although reportedly not statistically significant, almost twice the number of patients in the control group were on nitrates and more than twice the number were on "triple therapy" (nitrates, calcium channel blockers, and beta blockers combined), compared to the treatment group. The number on nitrates approaches a significant difference (p=0.06). The sample sizes are small, so the real significance of the difference is uncertain. Did the control group have more intensively treated anginal symptoms or require more intensive treatment to get them minimally or asymptomatic? The answers are unknown and the possibilities confound the outcome. More bothersome is that medical therapy was apparently not kept constant during the study period. This means the reported improvements may have had some relationship to altered medical therapy during the study. This is a significant confounding factor that further compromises an already fatally flawed study.

Nearly twice as many patients in the treatment group sustained a prior myocardial infarction compared to controls. This difference also approached statistical significance. Thus, it is possible, if not probable, that the two study groups were not comparable. These problems highlight the need for proper study design and statistical planning, which were absent in the PATCH study.

IMPROPER STATISTICAL ANALYSIS

Most people are familiar with the phrase, "Garbage in garbage out." This phrase aptly describes the statistical section of the paper by the PATCH investigators. When the data collected are defective and flawed, as in the PATCH trial, any analysis of such data is not going to be meaningful. Nevertheless, attempting to retrieve something, anything, publishable from a poor study is usual behavior in academic medicine and the biomedical industry. Therefore, a look at the statistical analysis for the PATCH data is instructive.

The PATCH investigators employed C2 or the Fisher exact test for categorical variables. Continuous variables were examined with paired and unpaired t-tests. Analyses of exercise and quality of time data were conducted using last-observation-carried forward. These tests should have been conducted as a secondary analyses, but a non-parametric analysis of covariance (ANCOVA) would have been a better choice.(13) ANCOVA is especially the analysis of choice because almost 10% of the control group dropped out of the study. This is because a two-sample t-test is not a full intention-to-treat analysis while ANCOVA is. These study drop-outs must be included in a proper statistical analysis or bias is introduced.

A nonparametric statistical approach is warranted because baseline treadmill test times are likely to be highly related to the final test times. In other words, patients who exercise to 3 minutes at baseline will probably exercise past 3 minutes to 5, 6, or 7 minutes on the final test while patients who exercise to 8 minutes on the baseline test would be expected to exercise past 8 minutes to 9, 10, or 11 minutes. One would not expect patients who exercise to 3 minutes to perform to 14 minutes or patients who exercise to 8 minutes to end up with a 3 minute exercise time. A gain in statistical power would be expected by controlling for baseline treadmill time in the analysis. Simply put, if everyone's baseline exercise time were transformed (coded) as a zero time to ST-segment depression, then the increment in exercise time could be evaluated in a rank based test. Had the PATCH data been reliable or valid, proper statistical analysis could have made the study more powerful.

CONCLUSIONS

Clinical trials of EDTA chelation have been characterized by particularly bad science. The possible efficacy of EDTA chelation for atherosclerotic vascular disease is worthy of serious, objective, competent study, if for no other reason than the large number of patients who seek this therapy out worldwide. The PATCH investigators have managed to squander an opportunity and have contributed one more sham study of chelation to the literature. The PATCH investigators conclude their study does not support the use of EDTA chelation to increase ischemic threshold and improve quality of life in patients with ischemic heart disease. Their study is so defective and flawed that this statement is true. It is no surprise that such a flawed study does not support the use of EDTA chelation in asymptomatic and minimally symptomatic patients to improve questionable and ill-defined endpoints. Equally true is that the PATCH trial provides no evidence whatsoever that EDTA chelation is not an effective therapy.

It is intriguing that 4 patients in the control group and none in the EDTA chelation group underwent angioplasty during the one year follow-up after the study, suggesting that EDTA chelation might reduce the need for revascularization? The most appropriate statement by the PATCH investigators is that "Larger trials with a broader range of patients" are needed. Amen.

REFERENCES

1) Knudtson ML, Wyse DG, Galbraith PD, et al. Chelation therapy for ischemic heart disease. A randomized controlled trial. JAMA. 2002;287:481-6.

2) Quan H, Ghali WA, Verhoef MJ, Norris CM, Galbraith PD, Knudtson ML. Use of chelation therapy after coronary arteriography. Am J Med. 2001;111:686-91.

3) Hancke C, Flytie K. Benefits of EDTA chelation therapy in arteriosclerosis: A retrospective study of 470 patients. J Adv Med. 1993;6:161-71.

4) Fleg JL. Pi¤a IL, Balady GJ, et al. Assessment of functional capacity in clinical and research applications. An advisory from the Committee on Exercise, Rehabilitation, and Prevention, Council on Clinical Cardiology, American Heart Association. Circulation. 2000;102:1591-7.

5) Draft guidelines for the clinical evaluation of anti-anginal drugs. FDA CDER Executive Secretariat Staff 1989.

6) Khurmi NS, Bowles MJ, Kohli RS, Raftery EB. Does placebo improve indexes of effort-induced myocardial ischemia? An objective study in 150 patients with chronic stable angina pectoris. Am J Cardiol 1986. 15;907-11.

7) Starling MR, Moody M, Crawford MH, Levi B, O'Roarke RA. Repeat treadmill exercise testing: variability of results in patients with angina pectoris. Am Heart J. 1984;107:298-303.

8) Folli G, Radice M, Beltrami A, Potenza S, Mariotti G. Placebo effect in the treatment of angina pectoris. Acta Cardiol 1978;33:231-9.

9) Myers J, Froelicher VF. Optimizing the exercise test for pharmacological investigations. Circulation. 1990;82:1839-46.

10) Garrard CS, Emmons C. The reproducibility of the respiratory responses to maximum exercise. Respiration. 1986;49:94-100.

11) Galassi AR, Azzarelli S, Lupo S, et al. Accuracy of exercise testing in the assessment of the severity of myocardial ischemia as determined by means of technetium-99m tetrofosmin SPECT scintigraphy. J Nucl Cardiol. 2000;7:575-83.

12) Glasser SP, Clark PI, Lipicky RJ, Hubbard JM, Yusuf S: Exposing patients with chronic stable exertional angina to placebo periods in drug trials. JAMA. 1991;265:1550-4.

13) Knoke JD. Nonparametric analysis of covariance for comparing change in randomized studies with baseline variables subject to error. Biometrics. 1991;47:523-33.