This is the original paper claiming to have resolved the trichotomy into human-chimp, with a large battery of highly replicable measurements of genome difference, delta-T₅₀H. It does not mention any numerical corrections, simply the calculation of the heteroduplex melting temperature and subtraction from the homoduplex. A simple statistical test showed that the results were significant.

This is the second paper, containing a larger array of data; it gives specific values for each experiment. It also doesn't mention the existence of numerical corrections. (Our inability to match the values given for 40% of their listed experiments is what led to the discovery of the omitted part of the data analysis.)

Here we outlined the potential problems with DNA hybridization, showed that the data do not in fact seem to resolve the trichotomy for they subsume a great deal more variability in measurement than Sibley and Ahlquist had reported, and inferred the existence of unreported data alterations.

Here we discussed the broader issues surrounding the manners in which DNA hybridization was not being adequately scrutinized. We called attention to the bulk of the Sibley-Ahlquist corpus on bird phylogeny. They had been justifying their cockamamie unreported corrections to the primate experiments by saying they knew it was okay because they had been doing it to the birds all along. Of course, that meant that the bird work was entirely compromised.

A second set of DNA hybridization experiments, also from Yale's biology department, claims to have resolved the trichotomy into human-chimp again, and to have matched the altered Sibley-Ahlquist numbers (in spite of having measured a different variable, delta-T_m). The results are attributable to their DNA fragment-length correction, which is impossibly precise.

The authors acknowledge the previously undisclosed data alterations and admit that the resolution of the trichotomy was predicated upon them. They also give their delta-T_ms, which is the variable measured by Caccone-Powell. Unsurprisingly, these neither resolve the trichotomy, nor match the Caccone-Powell numbers. This obviously raises questions about those numbers.

This study was the first to apply DNA hybridization to the problem of human phylogeny. They got a low value for human-chimp, but acknowledged poor reciprocity and concluded that they could not resolve the trichotomy.

This study also applied DNA hybridization to the apes. They found a 3-way split.

I'm not sure if this dataset is independent of the one just mentioned, but they also found an unresolvable trichotomy, although their tree actually links human-gorilla.

Here I explain just what Sibley-Ahlquist and Caccone-Powell did to their data. You can decide for your self whether it's legitimate. Since the best arguments they can come up with to validate the "corrections" are that they got the same answers, it is important to know whether the answers are in fact the same or not.

Science journalist sings DNA hybridization's praises.

Lewin, Roger (1988) DNA clock conflict continues. Science, 241:1756-1759.

While our two manuscripts were out for review, Lewin wrote up the controversy as news. He reports that a frightened post-doc named Fred Sheldon removed his name from the 1987 paper, and that although Sibley minimizes the number of experiments altered, Sheldon verifies it is about 40%. Lewin uses ad hominem counter-arguments, such as a callout from Roy Britten: "Those manuscripts are not scientific articles, they are weapons with political purposes." He illustrates the articles with figures from our paper without permission. Then he cites Caccone-Powell's unpublished work in support of Sibley-Ahlquist's undocumented work. Lewin provides the first forum for Sibley's defense: It was error, not fraud; the accusers are jerks; and it doesn't matter whether they falsified their data, because they got the right answer. Lewin's conclusion is classic: "There is no doubt that Sibley and Ahlquist were seriously in error in making substantial, unreported alterations to their data.... But the very combative and partisan tone with which the challenges have been made has not advanced Sarich and his colleagues' stated concern with scientific integrity." In other words, you should only accuse people of fraud nicely. (Remember, we still didn't know exactly what they had done; we tried to discourage him from writing the story until the papers were out; and we tried to keep the focus exclusively on the data.)

Even Scientific American wrote it up. You'd think that might have caught the eye of the relevant administrators interested in scientific integrity.

In his syndicated column, even Isaac Asimov is aware that there are suspicions of dishonesty about this work. It wasn't much of a secret. So why wasn't there an investigation?

A student-run science journal at Yale wrote up the controversy -- if you can call data falsification controversial. But there was never any investigation.

In this featured book review on scientific fraud, I related my experiences as a whistle-blower to the books. (Of course, I'm a molecular anthropologist, not an expert on fraud, presumably they asked me to write the review based on my experience...). When Sibley threatened to sue, the editor agreed to solicit and print six letters criticizing me in the next issue. (You might think this would have stimulated an investigation, but it didn't.)

What I wrote was:

The interesting issues in scientific fraud are thus less psychological than sociological. It is not the mind of the perpetrator that requires exploration, it is the ways in which the structure of the scientific community can be exploited to cover one's tracks after the dishonest deed has been committed. The interesting structures are those that exist to prevent fraud from being detected and prosecuted and the knowledge of it from being disseminated once discovered.

There are consequently two processes here: first, the exposure of the misconduct; second, the reaction to it. The exposure itself may be an insufficient action if the social, hierarchical and bureaucratic nature of the scientific community can be exploited in the perpetrator's defense.

Indeed, the next generation of books on this topic will probably feature the story of Charles Sibley, a researcher in molecular evolution who was arrested in the 1970s for smuggling the eggs of endangered bird species out of England and into his starch-gel apparatus. In the 1980s he resurfaced with a technique called DNA hybridization, which solved the problems of avian and primate phylogeny with fanfare enough for election into the National Academy. In 1990, however, he admitted that the phylogenetic resolution had been achieved through the use of data manipulations that had not been mentioned in any of his papers. These included substituting controls across experiments and moving correlated points on a scatter plot into the regression line describing them (Journal of Molecular Evolution, 30:202) - protocols that might well be sufficient to fail an undergraduate.

In spite of the fact that the language used in the published papers wildly misled reviewers and readers as to the nature of the data analysis and the robustness of the conclusions, we have not seen a retraction of the work, an inquiry by the National Academy into the nature of the research of its member, nor a public repudiation of the work by the senior community of molecular evolutionists. Rather, those who have spoken out have taken the position that the data alterations were a bad idea (although not bad enough to jeopardize his membership), their absence from the previously published work was a sad coincidence, the subsequent withholding of documentation was a crying shame, and the serendipitous discovery of the data alterations by others in 1988 was a thorough embarrassment. In other words, "questionable research practices."

Right.

The lesson is not of widespread corruption but simply of bureaucracy in action. Although we may lose a bit of our idealism by facing the reality of scientific misconduct, we can help the next generation not to lose theirs. It is not a matter of telling graduate students not to cheat (the equivalent of instructing inner-city youth to "Just Say No"); it is a matter of subjecting the scientific establishment to examination - showing our students how science works and how it doesn't - so that when they enter the community, they can improve it from within.

A couple of letters just sort of said I was a big jerk, and that I couldn't accept that so much genetic data linked human-chimp, just like Sibley said -- which might be interesting if it were either relevant or true. A few others made more fascinating points.

Sibley and Ahlquist's letter indignantly demands that (as I had suggested in my review) the National Academy of Sciences investigate the matter, knowing they wouldn't. Shortly thereafter Vince Sarich and I separately wrote to the NAS with charges and documentation. You might figure that if both sides clamor for an investigation, they might undertake it. But we received a brief letter in return, saying they wouldn't.

Britten's critical letter attempts to defend the data alterations. He tries to make it sound like they're okay in the passive voice: "Dr. Marks alleges that Dr. Sibley substituted controls across experiments; in fact, a correction was made by using a well-known value for the melting temperature of hybridized pieces of DNA when the individual runs showed erronious results." And, "Finally, Dr. Marks charges that Dr. Sibley moved 'correlated points into the regression line describing them.' This is untrue. The fact is that regression-line plots were used to detect outliers and make corrections on runs with large or small extents of hybridization." Sure glad he cleared that up... I wonder if he treats his own data that way?

According to Powell, "The Sibley work is good science inasmuch as it is repeatable and independently corroborated." Actually, good science is simply carried out competently and honestly. The fact that two people make the same claim doesn't mean the claim is right. In this case, Powell argues that it doesn't matter whether they falsified their data, because they got the right answer. A frightening thought, especially from someone charged with educating the next generation of biology majors at Yale.

These two argued that "their methods were logical, and made very little difference to inferences from the complete data." Well, I suppose their was logic to the thought, "Let's take the experiments that gave us numbers we don't like like, and pretend they gave us numbers we do like." But the second clause is simply false, by Sibley's own admission in their 1990 paper.

My response to these letters was edited beyond what I considered to be acceptable, and the editor published her version anyway. American Scientist accused me of making a "personal attack" on Sibley, when again, the issue is the work, not the person. Since my original review was vetted by two levels of editors to begin with, you might think they would have noticed any "personal attacks". There weren't any.  Everything I said was accurate and relevant to evaluating scientific research.

In a letter to Charles Sibley dated August 31, 1993, the editor informed him that "Jonathan Marks, by the way, will not be reviewing books for American Scientist". I might mention that American Scientist is published by the scientific society called Sigma Xi, who also publish tracts on scientific integrity.  Blacklisting is not discussed in any of their tracts.

Here is my full, unpublished response to the letters:

Readers of American Scientist may note that the current issue contains six letters critical of points I raised in a book review on scientific fraud in the last issue. The abbreviated response that appears after them was published without my approval, and against my wishes. I provide a full response below, for any interested readers.

Sibley and Ahlquist claimed in 1984 and 1987 to have proved genetically and statistically that humans, chimps, and gorillas, widely regarded as genetically equidistant from one another, were actually not. Rather, they asserted that humans and chimps were genetically more similar, and were each others' closest relatives. They were under the same obligations as any scientist: to prove that their conclusions were sustained by their data. In their 1990 paper (J. Mol. Evol., 30:202-236), Sibley et al. acknowledged that the conclusions of their earlier papers had been derived with the use of unreported analytic procedures: had it not been for these data alterations (in their own words) "... it is virtually certain that Sibley and Ahlquist would have concluded that Homo, Panand Gorilla form a trichotomy" (p. 225).

The alterations were made without consistent objective criteria, as I have exhaustively detailed (Am. J. Phys. Anth., 85:207-219, 1991). The one they used most extensively involved constructing a bivariate plot of their data points, deciding (based on unclear criteria) that 40% of their data constituted aberrant outliers, and (in their words) "moving the aberrant point to the linear regression ... and calculating the new ... value." (1990, p. 232). This is fundamentally illegitimate, especially since they treated their data points as statistically independent in their papers, and neglected to inform readers that they had performed these data transformations. It is difficult to imagine that the papers would have been published has these data analysis procedures been specified.

Roy Britten's circumlocutions do not make the alterations any more legitimate. Indeed, the reason the alterations came to light at all was that Britten, in possession of the only data provided to anyone by Sibley (who had steadfastly refused many requests by many scholars), passed them on without having noticed their problematic nature. The alterations and their illegitimacy, however, were recognized by the fearful post-doc who removed his name from the work (Science, 241:1758, 1988); and by Vincent Sarich, Carl Schmid, and myself (J. Hum. Evol., 17:769, 1988; Cladistics, 5:3, 1989). This leads to the conclusion that the Sibley-Ahlquist data did not actually resolve the trichotomy at all, and were misrepresented as having done so.

Now, was the misrepresentation deliberate? In the best case, the research would be characterized methodological poverty; the omission of the vital information would be coincidental; and the refusal of the authors to allow others to see their data would also be coincidental. In the worst case, they knew exactly what they were doing, the omission was required to conceal the illegitimate analysis; and the sequestering of information was required to conceal the omission of the alteration procedures from the published papers.

The best light in which their work could be cast would be that they "got the right answer", and their conclusions were simply a matter of overzealous prophecy, rather than simple charlatainerie. Even if this were true, it would be irrelevant to evaluating the nature of the data alterations. The larger problem is that this argument places a premium on the conclusion, rather than on the rigor of the data collection and analysis. It reaches its most extreme form in Jeffrey Powell's "replication" of their altered numbers, measuring a non-comparable variable.

One reason to be skeptical of this is Powell's assertion that the "particulars" of the data analysis by Sibley and Ahlquist are of little concern, since they were used in obtaining the correct conclusion. The nature of their data, the methods of analysis, and their subsequent representations in the literature are the central issues; and I am hopeful that few readers will concur that the end ever justifies the means in scientific research. Valid scientific work cannot be conclusion-driven, contrary to Powell's assertion. It carries obvious consequences for the quality of both research and mentorship. It stands to reason that if we need not worry about Sibley's methodology, we need not worry about Powell's methodology either. Unfortunately, some of us do. Quite possibly the decline in popularity of DNA hybridization among mainstream scientists, noted by Kirsch and Krajewski, reflects an appreciation that in this field, "anything goes" methodologically, in generating the "right" answer.

Ultimately the issues center around data falsification, its exposure, and its concealment. Sibley and Ahlquist used covert techniques to alter their data, which they and their supporters can only justify by recourse to the conclusions of other researchers. That is not, and cannot be, a validation of their methodology; it is a non-sequitur.

But is there indeed a genetic concordance favoring human-chimp, as Sibley and Ahlquist concluded, thought their data didn't actually show it? Obviously, if the split among humans, chimps, and gorillas is so close as to be effectively a three-way split, any specific genetic study may randomly appear to link any two of the three genera. There are two classes of data bearing on this problem: (1) those that do not permit any clear bifurcation to be drawn; and (2) those that clearly link two of the three as closest relatives, and contrast with other studies clearly linking a different two of the three. I recently reviewed this literature (Curr. Opin. Genet. Dev., 2:883-889, 1992), and interested readers can find comprehensive references therein. Czelusniak and Goodman cite only their own data and ignore the rest. This is not scholarly, but coming as it does in defense of the work of Sibley and Ahlquist, it sadly does not seem inappropriate.

The scientific issue is that reviewers and readers of Sibley and Ahlquist's papers were misled about the analysis of the data and consequent robustness of the conclusions. That has been public knowledge for several years. The social issues are less clear cut: what to do, who should do it, and to whom it should be done -- which is why I raised it in my review of the books on misconduct. For Sibley's "illegally importing bird parts taken abroad in violation of foreign wildlife laws", I refer readers to the published account (Sports Illustrated, 24 June 1974), which appears to have been of greater concern to falconers than to molecular evolutionists. If I overreached, I extend my apologies to the American Scientistand stand duly chastised.

As for motives and self-interests, I choose not to speculate on those of the correspondents, and I hope that unless The Amazing Kreskin writes in, no one will be distracted by their speculations about mine.

Ruvolo, Maryellen
 (1995) Seeing the forest through the trees: Replies to Marks; Rogers and Commuzzie; Green and Djian. American Journal of Physical Anthropology, 98:218-232.

Maryellen Ruvolo gave an extended defense of the work in the AJPA, in response to three criticisms of her own work. Ruvolo has done DNA sequencing on the COII gene of mitochondrial DNA, which shows human-chimp (whoop-de-doo!) and somehow proves that all other data are wrong and Sibley is right. Reasoning like that is for the schoolyard. At any rate, her response occupied more than twice as much space as the three criticisms combined, and again argues that (1) their data really showed human-chimp; (2) it doesn't matter if they falsified because they got the right answer; (3) Caccone-Powell renders the discussion moot. Vitually everything she says about the work is false, and easily demonstrated to be such. Here is a rebuttal.

(1) Ruvolo asserts that after all is said and done, the infamous Sibley data actually do associate humans and chimpanzees, to the exclusion of gorillas. Here, she has greater confidence than do the authors themselves, who wrote in 1990, "If the corrections [sic] had not been used, it is virtually certain that Sibley and Ahlquist would have concluded that Homo, Pan, and Gorilla form a trichotomy" (J. Mol. Evol., 30:225). The only relevant question, then, is the validity of the "corrections". These were (1) substituting controls across experiments; (2) moving correlated points in to the regression line describing them; and (3) making precise numerical changes on the basis of a variable (fragment length) which was not even measured. They also neglected to report these "corrections" until their existence was discovered and their nature inferred quite serendipitously by others. The "corrections" were finally reported in the 1990 JME paper cited above, and discussed at length by me in the 1991 AJPA paper, cited above.

At any rate, for anyone to argue that these data actually do associate human and chimpanzee, they must accept the data alterations as valid. That is, in my opinion, a frightening prospect, with far-reaching implications for general scientific data analysis.

(2) Ruvolo, referring to a table in the 1990 paper, says that "for the entire uncorrected data set" there is "a fairly clear separation of the human chimpanzee lineage," which "seems to have been widely overlooked or ignored" (by me). In fact, I discussed that very table in my 1991 paper, cited above. As I showed in Table 2 of that paper (p. 217), pooling the data poses a considerable problem, for the data exhibit very poor reciprocity specifically in the experiments using human DNA -- thus, human-chimp yields values half as large as chimp-human; human-gorilla is half as large as gorilla-human; and human-orang is half as large as orang-human and all the permutations of orang vs. chimp or gorilla. In other words, the experiments using human tracer DNA were somehow technically fouled up. The pooled result that Ruvolo reports is a consequently a simple artifact, and nothing more. Actually the chimp-human value is 1.9, while the corresponding chimp-gorilla and gorilla-chimp values are 1.8, which stands in strong contradiction to her point.

(3) Ruvolo calls the subsequent Caccone-Powell DNA hybridization study "the most sophisticated". In fact the major claim from this study was not so much wrong as impossible. The claim was stridently that they had matched Sibley's (altered) numbers, given as T₅₀H measurements, with their own numbers, given as Tm measurements. (T_mmeasures the temperature at which 50% of the DNA that formed hybrids is single-stranded; T₅₀H measures the point at which 50% of the total input DNA is single-stranded; they are only expected to be identical when hybridization is exactly 100%.)

However, when Powell's T_ms are compared to Sibley's unaltered T_ms (finally given in the 1990 JME paper, p. 221) they do not in fact match at all. Matching altered and non-comparable numbers frankly requires far more than just technical sophistication.

(4) Ruvolo claims that "the data quality are high" for the Powell corpus, but as has been noted since its publication in 1989, they did not publish any melting profiles, which was the basis of the original criticism of the Sibley work, and is the only basis on which one can actually judge the quality of the data. She further claims that "these data have been freely available on request from the authors," which is directly contrary to my experience. I asked Powell, in person and in writing, several times in 1989 and 1990 for a look at his melting curves or the data on which they were based, and never received any. Vince Sarich received three representative experiments, which he copied for me. They are in fact far cruder than the comparable DNA melting profiles from the Sibley series. They are not at all of high quality, and exactly how data of poorer quality can possibly provide more precise phylogenetic resolution is quite a mystery to me. Ruvolo provides no evidence to the contrary, nor has any such evidence ever seen the light of day.

(5) Ruvolo's criteria for judging the Powell corpus are the "reciprocals and the outgroup-ingroup homogeneity" -- in other words, the final numbers, rather than the basis on which each number was generated. Since each number in this work was generated from a DNA melting profile with the aid of a "correction" for DNA fragment length, it is possible to ask what effect this "correction" had on the resolution reported. The data given in the Caccone-Powell paper (Evolution, 43:942, 1989) actually permit this calculation. It turns out that that this "correction" (based on a measurement of mean DNA length in an agarose gel, to the nearest single nucleotide, of whole genomic DNA, after sonication, labelling, incubation, and S1 nuclease digestion) is entirely responsible for the reciprocity, the resolution, and the claimed match to the (published, altered, non-comparable) Sibley numbers.

Ruvolo asserts that I merely said this correction might affect the result, but in fact I made the calculations and published them in Table 2 of the very paper she cited. They do affect the result; not only that, but they determine the result; I said so, and I say so. Anyone is free to make the calculations themselves from the data in the appendix to the Caccone-Powell paper.

The numbers I get are:
 

	Before	After
Human-Chimp	2.4 +/- 1.1	1.6 +/- 0.2
Chimp-Gorilla	3.1 +/- 0.6	2.5 +/- 0.2
Human-Gorilla	4.1 +/- 1.1	2.5 +/- 0.1

Before "correction", there is no consistency to the numbers, no match to the Sibley numbers, and no no resolution; after correction, the numbers are perfectly consistent, they now match the altered/published delta-T50H values of Sibley/Ahlquist, and now resolve the trichotomy into human-chimp.  Ruvolo does not provide any evidence to support her contention that the length "correction" was not responsible for generating the results, but rather she cites only unpublished "work in preparation".

Given that the precision of the reported length estimates themselves is difficult to justify on strictly a priori grounds, and that those estimates appear to be responsible for both the claim of phylogenetic resolution, and the reported match to the altered non-comparable published numbers of Sibley, I think it is difficult to sustain much faith in the general integrity and scientific value of this work.

(6) It is quite sad that these still have the appearance of being open issues. They should have been formally investigated, adjudicated, and settled years ago, having been discussed in the pages of Science (241:1598, 1988; 241:1756, 1988) and Scientific American (March 1989: 24). Informally, the work has largely been exorcised from the primary literature, and there are hardly any people even using the technique any more, since it appears capable of solving phylogenetic questions only when considerable liberties are taken with the data analysis. (With such liberties in data analysis, after all, any technique will work!) Powell himself quietly abandoned it a few years ago. Ruvolo is nearly alone in clinging to the work, loyal (if uncritical) of it to the end. That she can make such egregious misstatements in print at this late date is not of great credit to anyone.

(7) Lastly, of particular significance is the fact that the senior author of the original work (Sibley) is a member of the National Academy of Sciences, and I think it is extraordinary that that body has never investigated the matter. I raised this publicly in the American Scientist (81: 382, 1993). In the following issue, Sibley indignantly responded that he would "suggest such an investigation to the National Academy of Sciences Home Secretary." Whether he actually did nor not, I cannot say. However, Vince Sarich and I immediately wrote separately to the Home Secretary (Peter Raven), detailing and documenting the accusations, which were already in the public record. An investigation would seem to have been called for, given the apparent agreement of both sides on the matter. It did not happen, however.

This is Raven's full response to me, dated October 25, 1993:

************
Dear Dr. Marks:

Thank you very much indeed for your letter and the enclosures. I was extremely interested in what you had to say in reading the enclosures. It is obviously a very complex case and, as I am sure you understand, the National Academy of Sciences would not undertake to conduct a formal review of the activities of its members as a matter of general principle, lacking the judicial machinery to do so properly. I would add, however, that no one is elected to the Academy for a single piece of work, and thus it is incorrect, as a matter of principle, to say that "this is the work that ultimately resulted in Sibley's election to the National Academy of Sciences.....". In summary, I was very interested in the material that you sent. We will be conducting no investigation.

Yours sincerely,
Peter H. Raven
Home Secretary

*****************

Not investigating, of course, is the ideal bureaucratic solution for making a potentially embarrassing and egregious case of data falsification resemble a "you say potayto, I say potahto" scientific controversy. In other words, calling it a draw -- which is a victory for the original investigators, permitting them to retain the status they attained through the work in question.