Summary
holding theory of firearm toolmark identification used by San Francisco Police Crime Lab was reliable under Daubert
Summary of this case from United States v. JamesOpinion
No. CR 05-00167 WHA.
February 12, 2007
ORDER GRANTING IN PART AND DENYING IN PART MOTIONS TO EXCLUDE FIREARM IDENTIFICATION EVIDENCE
INTRODUCTION
In this RICO gang prosecution, the immediate issue concerns firearms forensics and Daubert. Defendants have moved to exclude customary firearms evidence under Daubert and Federal Rule of Evidence 702. A four-day Daubert hearing was held. The witnesses were G. Andrew Smith, a firearms and toolmark examiner at the San Francisco Police Department Crime Lab, Ronald Nichols, a firearms examiner with the Bureau of Alcohol, Tobacco and Firearms, and Adina Schwartz, a professor at John Jay College of Criminal Justice. Also received were voluminous literature items.
After considering the evidentiary record and the parties' arguments, this order holds that the theory of firearm identification used by the SFPD Crime Lab is reliable under Daubert. While there is some subjectivity involved, it is the subjective judgment of trained professionals with a keen practiced eye for discerning the extent of matching patterns. The methods used are reliable. The record, however, does not support the conclusion that identifications can be made to the exclusion of all other firearms in the world. Thus, the examiners who testify in this case may only testify that a match has been made to a "reasonable degree of certainty in the ballistics field." Accordingly, defendants' motions are GRANTED IN PART and DENIED IN PART.
STATEMENT
SFPD criminalist Smith explained the mechanics and theory of firearms identification. Smith began his criminalistics career in Florida in 1999, where he was trained as a firearms and toolmark examiner. He is a member of the Association of Firearm and Toolmark Examiners, the premier organization dealing with firearm and toolmark identification. He coauthored two articles in the AFTE Journal and testified approximately forty times previously as a firearms-identification expert.According to Smith, a cartridge is made up of four main parts: the bullet, the case, the powder, and the primer. The case is the covering that holds all of the cartridge components together. The bullet itself is the projectile propelled from the weapon. The powder sits behind the bullet and is exploded during firing. The primer is the component at the rear of the case that starts the reaction when the cartridge is fired.
Graph
When a gun is fired, the inner barrel of the gun imparts "rifling" on the bullet. The barrel of a gun is manufactured to impart a twist on a bullet as it travels, to ensure firing accuracy. The inside of a gun barrel is imprinted with cuts running the length of the barrel. The cuts within the barrel are called "grooves" and the raised surfaces are called "lands." Those rifling characteristics create marks on the bullet as it travels down the barrel. The raised lands cut into the surface of the bullet. Likewise, the bullet surface expands to fill the recessed grooves. The corresponding impressions left on the bullet as it travels through the barrel are depressed "land impressions" and raised "groove impressions." The twist imparted on a bullet can be either left or right, depending on the direction of the lands and grooves.
Prior to firing, the base of the cartridge abuts the breech of the gun when the cartridge sits in the chamber. When the gun is fired, the cartridge slams into the breech, thereby leaving "breech face characteristics." Meanwhile, the firing pin in the gun hits the base of the cartridge, where the primer is, which starts the reaction that fires the bullet. The firing pin contact creates a "firing pin impression" on the base of the cartridge case.
* * *
There are three types of characteristics observed by examiners: class, subclass, and individual characteristics. Class characteristics on a spent bullet or cartridge case allow an examiner to narrow the firearm possibilities to certain types of guns made by certain manufacturers. For a spent bullet, the class characteristics are the weight or caliber of the bullet, the number of lands and grooves, the twist of the lands and grooves, and the width of the lands and grooves. For example, a .9mm caliber bullet can only be fired by a .9mm caliber firearm. Additionally, if the bullet has six land and groove impressions, it can only have been fired from a gun barrel that has six lands and grooves.
When looking at a spent cartridge case, the examiner looks for the class characteristics of caliber, type of breech face, and type of firing pin. Breech face characteristics can be parallel, arched, smooth, granular, or circular. There are three types of firing pin impressions: circular, rectangular, and elliptical.
Once the firearm possibilities are narrowed by class, the examiner looks for individual characteristics. The range of possibilities can be further narrowed by individual characteristics — microscopic, random imperfections in the barrel or firing mechanism created by the manufacturing process. These unintended characteristics are caused by changes in the tool as it makes each barrel on the production line.
Individual characteristics typically fall into two categories: (1) striated marks made by movement of the bullet within the gun (typically appearing as scratches), and (2) impressed marks that are pressed into a surface. A spent bullet usually has striated marks, created as it moves through the barrel of the gun. On the other hand, a spent cartridge case can have both impressed and striated marks. Prior to firing, simply feeding the cartridge into the chamber can create striated marks. Then, as discussed above, impressed marks are created on the cartridge case by the gun's firing pin and breech during firing. Finally, additional marks can be made as the case is expelled from the gun. For example, after the gun is fired, the gun's chamber needs to be cleared for a new cartridge. The spent cartridge is then pulled backwards by the "extractor," which can leave striated marks on the case. Then, the "ejector" kicks the case out of the gun, leaving an impressed mark.
A third type of characteristic straddles the line between class and individual characteristics. These are subclass characteristics. These characteristics can exist, for example, within a particular batch of a brand of firearm. They arise due to imperfections in the manufacturing tool that persist during the manufacture of multiple firearm components. They cannot be considered class characteristics because they are not common to all units of a particular make and model of firearm. Nor are they individual characteristics because they persist throughout a period of manufacturing. Smith testified that a trained, qualified examiner takes care not to confuse subclass characteristics with individual characteristics, because an identification should not be made based on subclass characteristics. Nichols also explained that trained examiners can account for subclass characteristics.
* * *
Firearm identification has been a forensic discipline since the 1930s. Firearms identification is a subset of the broader forensic discipline called toolmark identification. Toolmark examiners are trained to examine the marks left by tools on any variety of surfaces in an attempt to "match" a toolmark to the particular tool that made the mark. Firearms are simply a subset of tools that imparts marks on bullets and cartridge cases.
According to the theory of firearms identification, a qualified examiner can reliably determine whether two bullets or two cartridge cases were fired by the same gun. This can be achieved based on an examiner's expertise, experiments, and daily practice. A conclusion that two cartridge components have a "common origin" can be reached when the examiner concludes that sufficient similarity exists between the patterns on the components.
When determining whether "sufficient agreement" of toolmarks exists on a bullet, for example, the examiner will look for spacial relations of the striations, along with the depth and width of the striations. If there is significant similarity between those marks, the examiner can conclude that the bullets were fired by the same firearm. The marks need not be identical. There need only be "sufficient agreement" between the marks based on the examiner's training and experience. This inspection is done under a split-screen comparison microscope.
There are three conclusions an examiner can state. The examiner can make: (1) an "identification" of the components, concluding that they came from the same source; (2) an "elimination" of the components, concluding that they did not come from the same source; and (3) "inconclusive," meaning that there is not enough evidence to identify whether the components either do or do not come from the same source. In the parlance of firearm examiners, if there is sufficient agreement to make an identification, a firearm examiner often states that the chance that another firearm could have made the mark is a "practical impossibility."
* * *
At the hearing, there was some discussion about a developing process within the firearm-identification community called consecutive matching striae ("CMS"). CMS is an attempt to eliminate some of the subjectivity of firearms identification by using an "objective" consideration of "matching striae." This is in contrast to the traditional pattern matching that is used by virtually all practicing firearms examiners, including Smith and other examiners at the SFPD Crime Lab. In theory, CMS can add some quantification to an examination so that there is some numerical and statistical data to support an examiner's conclusion of an identification.
Under CMS, an examiner looks at the number of consecutive striae that match between the bullets being compared. A "run" of striae is essentially a group of closely-aligned striae on a bullet. According to CMS, correspondence between one six-line run of striae or two three-line runs are enough to made an identification. CMS only applies to striated marks rather than breech face or other impressed marks. For that reason, CMS is only used on spent bullets and not on cartridge cases.
Smith does not subscribe to the CMS theory. Professor Adina Schwartz, however, does. Schwartz, who testified for the defense, is involved in John Jay College of Criminal Justice's forensic science Ph.D. program. She authored an article on firearms identification and Daubert. She is not, however, a firearms examiner. Schwartz was offered as a literature expert who had comprehensively reviewed the development of intellectual thought of firearms identification. She testified that CMS could offer a more objective approach to firearms identification. The crux of her testimony was that because traditional pattern matching is inherently subjective, it is invalid.
In rebuttal, the government offered the testimony of Ronald Nichols. He is a distinguished member of the firearms-examiner community and had published numerous articles on the topic. Nichols' testimony was noteworthy because he is a proponent of CMS. He testified that CMS is only an attempt to quantify an examiner's reliable, pattern matching-based conclusions. According to Nichols, CMS and pattern matching are not mutually exclusive. Rather, CMS is a process in development that seeks to inject an element of objectivity into a subjective — but no less reliable — science.
ANALYSIS
1. DAUBERT AND RULE 702.
"Rule 702 assigns to the district court the role of gatekeeper and charges the court with assuring that expert testimony rests on a reliable foundation and is relevant to the task at hand." United States v. Hermanek, 289 F.3d 1076, 1093 (9th Cir. 2002) (internal quotations omitted). Rule 702 provides:
If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.
In Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 593-94 (1993), the Supreme Court created a flexible, factor-based approach to analyzing the reliability of expert testimony. These factors can include: (1) whether a method can or has been tested; (2) the known or potential rate of error; (3) whether the methods have been subjected to peer review; (4) whether there are standards controlling the technique's operation; and (5) the general acceptance of the method within the relevant community. See Daubert, 509 U.S. at 593-94; see also United States v. Prime, 431 F.3d 1147, 1152 (9th Cir. 2005). The Supreme Court has further explained that a district court has "considerable leeway in deciding in a particular case how to go about determining whether particular expert testimony is reliable." Kumho Tire v. Carmichael, 526 U.S. 137, 152 (1999). Although the circumstances of the particular case will determine the "applicability of the factors mentioned in Daubert," this order finds analysis of the Daubert factors useful in the firearms-identification context. Id. at 150.
Several courts have admitted firearms-identification testimony following Daubert and Kumho Tire. See, e.g., United States v. Hicks, 389 F.3d 514, 526 (5th Cir. 2004); United States v. Monteiro, 407 F. Supp. 2d 351, 359-64 (D. Mass. 2006); United States v. Green, 405 F. Supp. 2d 104, 124 (D. Mass. 2005). Indeed, it has been stated: "Expert testimony identifying a particular weapon as the one source of both a questioned (crime scene) bullet and known bullets (test firings) is admissible in every American jurisdiction. At least 37 jurisdictions have approved it by appellate opinion." David L. Faigman et al., Modern Scientific Evidence, at 396 (4th ed. 2005). No reported decision has ever excluded firearms-identification expert testimony under Daubert. See Green, 405 F. Supp. 2d 122-23.
2. ANALYSIS OF DAUBERT FACTORS.
A. Whether the Theory or Technique Can Be or Has Been Tested.
This order finds that the AFTE theory is testable and that this factor weighs in favor of admissibility. It is recognized that there is a problem of absolute testability in firearms identification. Because the accepted practice in the field is based on a subjective assessment, in actual case work it is impossible to conclusively state that an examiner's conclusion is correct or incorrect. (The critique that firearms identification is ultimately subjective is one on which defendants base many of their arguments.) That alone, however, is not enough to render the theory not "testable."
As discussed below, there have been a multitude of peer-reviewed studies wherein researchers have been able to compare and match cartridge cases and bullets to the guns from which they were fired. In those studies, it was known with absolute certainty where each of the test components came from. Moreover, examiners frequently took (and continue to take) proficiency tests where the true answers were known. The vast majority of the time, examiners were able, using the theories applied in actual casework, to reach correct conclusions based on the samples before them. According to Smith, trainees also perform similar studies during their training to understand what constitutes "sufficient agreement" to warrant an identification. The theory of firearms identification has been and continues to be tested.
Furthermore, practically all laboratories, including the SFPD Crime Lab, require examiners to thoroughly document their results and findings. Any identifications made must be photodocumented. Examiners must indicate the primary areas on which they base identifications. The industry standard also requires confirmation by at least one separate examiner when an identification is reached by the first examiner. This order agrees that "the existence of the requirements of peer review and documentation ensure sufficient testability and reproducibility to ensure that the results of the technique are reliable." Monteiro, 407 F. Supp. 2d at 369.
This order holds that the theory of firearms identification, though based on examiners' subjective assessment of individual characteristics, has been and can be tested. Importantly, the literature from the field demonstrates that the traditional pattern matching theory has been tested — and verified — for the decades that firearms examination has been in existence. This factor weighs in favor of admissibility.
B. Whether the Technique Has Been Subject to Peer Review and Publication.
AFTE, the principal professional organization for firearms and toolmark examiners, publishes a peer-reviewed journal, the AFTE Journal. This journal has "always had a peer review process." There is a formal process for submission to the journal, including assigning manuscripts to other experts in the scientific community for technical review and the requirement of a final review by the journal's editorial committee. There is also a post-publication peer-review process whereby interested persons may comment on published articles. This order concludes that the peer-reviewed literature factor supports admitting the testimony in this case (PX 25 at 212).
Throughout this order, "PX" refers to the government's exhibits and "DX" refers to defense exhibits.
Moreover, the peer-reviewed literature generally supports the AFTE theory of identification. It seems clear from the literature that spent cartridge cases can be identified by an experienced examiner as having come from a particular firearm regardless of how many times the firearm had been fired. For spent bullets, the literature indicates that individual characteristics can change after many firings but that the matching of spent bullets to a particular firearm is frequently done and well-accepted.
In "Comparison of 900 Consecutively Fired Bullets and Cartridge Cases from a 455 Caliber S W Revolver," published in the 1980 AFTE Journal, Shane Kirby attempted to determine the persistence of individual characteristics through a large number of test firings from one gun. Kirby fired 900 cartridges from a 455 caliber Smith Wesson revolver. The results were mixed. With respect to the cartridge-case characteristics, Kirby concluded that after 900 firings, a positive identification could be made between the first and nine hundredth cartridge cases. The conclusions were based on the breech face and firing pin impressions left by the revolver. In other words, even after a heavy amount of firing, the breech-face and firing-pin characteristics persisted throughout all the firings.
The same could not be said for bullets. Kirby concluded that the individual characteristics of fired bullets did not persist as they had for the cartridge cases. By the thirtieth bullet, changes had begun to develop in the individual characteristics. When the fiftieth bullet was compared to the first bullet, it could "be clearly seen that a completely new series of finer accidental characteristics [had] evolved." Comparing the seventieth bullet to the first bullet, an examiner could only state a "good probability" of identification (PX 14 at 115, 124).
Robert Shem and Peter Striupaitis conducted a similar study in "Comparison of 501 Consecutively Fired Bullets and Cartridge Cases from a 25 Caliber Raven Pistol." In this study, the researchers made 501 test firings from a 25 Caliber Raven Arms semi-automatic pistol. They tested to determine the longevity of a sufficient number of individual characteristics needed to positively match one bullet and one cartridge case to another bullet and cartridge case fired many shots later. Shem and Striupaitis concluded that it was possible "for sufficient individual characteristics to appear on a fired bullet to make a positive identification with a bullet fired 500 rounds previously." They did recognize, however, that individual characteristics "tend to erode and evolve with each succeeding bullet," indicating that after even more firings, an examiner might not be able to make a positive identification between two bullets. Finally, with respect to cartridge cases, the researchers "found that the individual characteristics found on a breech face tend to remain virtually unchanged after firing 501 rounds" (PX 15 at 110).
Another similar study was conducted by Yoshimitsu Ogihara in "Comparison of 5000 Consecutively Fired Bullets and Cartridge Cases from a 45 Caliber M1911A1 Pistol," which was presented to AFTE training seminars in 1977 and 1982. The study was republished later in the AFTE Journal. There, a 45 caliber Colt Pistol was fired 5000 times to determine what variations were produced from the first round through the five thousandth. With respect to the cartridge cases, positive identification could be obtained based on breech face marks between all 5000 cases. For the bullets, the examiners identified six land impressions that remained "stable through test firing of the five thousand bullets." For one of the impressions, positive identification was positive through all 5000 bullets. A different land impression was positively identified through 4000 bullets. Three of the land impression were identified through 3000 bullets. The final land impressions was identifiable through 2500 bullets (PX 16 at 127, 139-40).
In "Morphological Study of Class and Individual Characteristics Produced by Firing 2500 Cartridges in a .45 Caliber Semi-Automatic Pistol," the researchers examined cartridge cases. After firing 2500 rounds of ammunition from a .45 caliber HP semi-automatic Springfield Armory pistol, the researchers saw some differences in the characteristics. In spite of those differences, the researchers concluded that there were no "significant changes in the individual characteristics of the marks left by the mechanical parts of the weapon in the course of firing 2500 rounds sequentially." The cartridges all could be matched to the test firearm used (PX 19 at 368, 371).
In "Consecutively Manufactured Ruger P-89 Slides," Amy Coody examined ten consecutively-manufactured slides. (The slide is the portion of the gun that contains the breech.) Smith explained that consecutively-manufactured firearms were the most likely to produce similar individual characteristics on bullets or cartridge cases. As discussed above, individual characteristics imparted on a bullet, for example, are the result of anomalous imperfections that exist on the tool that created the barrel. Whatever imperfections exist on the tool are most likely to be imparted on consecutively-manufactured barrels. Thus, the possibility of a false-positive conclusion that two bullets came from the same firearm is highest with bullets that were fired from two different but consecutively-manufactured firearms. In her study, Coody examined the breech faces of the cartridges that were fired from those ten consecutive slides. She found that "although each of the slides exhibited similar class characteristics, tests from each of the slides were easily distinguishable and individually unique." Coody concluded that each cartridge case could be identified to its respective slide, based on the "randomness of the file marks produced on the breech face, as well as other imperfections that occurred during the manufacturing processes" (PX 18 at 157, 160).
The fact that articles submitted to the AFTE Journal are subject to peer review weighs strongly in favor of admission. Moreover, the conclusions reached by the peer-reviewed literature further demonstrate the reliability of the theory and process used by examiners in the field.
The government also submitted two articles as surveys of the peer-reviewed literature on firearms identification. These articles cited to many other articles that were published in peer-reviewed journals like the Journal of Forensic Sciences and the AFTE Journal (PX 23 at 474; PX 24 at 10).
C. Known or Potential Rate of Error.
The peer-reviewed literature and the three experts who testified conceded that it is not possible to calculate an absolute error rate for firearms identification. This is partly because the standards and criteria for traditional pattern matching are subjective. Ultimately, with actual casework the best that could be offered is the opinion of other experts on an examiner's conclusions. One would never get a "real" answer based only on other experts' opinions. Nevertheless, the government has provided enough data to show that the error rates among trained firearms examiners are sufficiently low to counsel in favor of admitting the evidence.
In one study submitted by the government, "Cartridge Case and Bullet Comparison Validation Study with Firearms Submitted in Casework," researcher Erich Smith attempted to force examiners to reach false-positive conclusions. He created eight packets of ballistic ammunition, which he gave to eight different ballistics examiners. Within each packet were forty-five bullets and forty-five cartridges. Each test packet contained only one true identification and forty-four true eliminations for both cartridge case and bullet comparisons. The goal of the experiment was to attempt to generate false positives by the "overwhelming number of non-matches." The eight participants ranged from the recently-trained to an examiner with twenty years experience.
Remarkably, there were no false-positive identifications and no false eliminations. For the bullets examined, six examiners made proper identifications while the other two reached inconclusive results for the true matches. For the cartridge cases, seven examiners made the proper identifications and only one reached an inconclusive for the true match. Researcher Smith concluded that "[t]he results indicate[d] that the participants' comparisons were precise, using pattern recognition to determine a common source." Furthermore, "the absence of false positives or false negatives indicate[d] that the theory of firearms identification, using pattern recognition, is an accurate and precise method for determining a common source for bullets and cartridge case for firearms collected from casework" (PX 20 at 130-32).
In "The Identification of Consecutively Rifled Gun Barrels" by David Brundage, further research was done suggesting low error rates among trained firearms examiners. Brundage used thirty examiners from nationally-accredited forensic laboratories in his study. He gave each examiner a group of fifteen unknown bullets and asked the examiners whether the bullets could be identified to ten consecutively-manufactured gun barrels. Out of the 450 examinations by the 30 examiners, there were no false positives. Brundage concluded that the study "demonstrate[d] that, on a national level, properly trained examiners [could] distinguish two or more bullets fired from [consecutively-rifled gun] barrels" (PX 30 at 443).
In another study submitted by the government, "Firearm/Toolmark Identification: Passing the Reliability Test Under Federal and State Evidentiary Standards," a team of researchers including Richard Grzybowski took a broad assessment of legal issues regarding the admissibility of firearms and toolmark identification in courts. At the time of publication in the AFTE Journal in 2003, Gryzbowski was chief of the Identification Section of the Bureau of Alcohol, Tobacco, Firearms and Explosives Forensic Science Laboratory in Walnut Creek, California.
Gryzbowski summarized other researchers' calculation of error rates. Researchers had calculated error-rate estimates based on Collaborative Testing Service data. CTS was the only source of proficiency testing results in the firearms and toolmark discipline. CTS would gather data from crime laboratories and maintain the data. Gryzbowski stated that a 1995 study had found an "alarming" 12% error rate for firearms and 26% error rate for toolmarks. There was a flaw with that study, however. Those results had included "inconclusive" responses as incorrect. But inconclusive responses were an accepted, correct response in the forensic community. Moreover, many laboratories were obligated by policy to report inconclusive results when there was no tool supplied. The 12% and 26% rates were thus considered artificially high. In 2003, the CTS data was recalculated to account only for false positives. The error rates for the CTS data from 1978 to 1997 revealed false-identification error rates to be 0.9% for firearms and 1.5% for toolmarks. From 1998 to 2002, the error rates were 1.0% for firearms and 1.2% for toolmarks (PX 25 at 216-18).
Nichols also testified that in a recent CTS proficiency test, there were four misidentifications out of 116 examinations. Thus, the estimated error rate based only on that test was approximately 3.4%.
No true error rate will ever be calculated so long as the firearm-examiner community continues to rely on the subjective traditional pattern matching method of identification. Nevertheless, based on the peer-reviewed literature studies, CTS proficiency testing, and testimony by Nichols and Smith, it is reasonable to infer that the error rates among trained examiners are quite low. This order concludes that the error rate for firearms identification among trained examiners is not a bar to admitting the testimony under Daubert.
D. The Existence and Maintenance of Standards Controlling the Technique's Operation.
The principal standard controlling the technique of firearms identification is embodied in the AFTE theory of identification. This order holds that the practiced eye of a trained firearms examiner can apply the AFTE theory reliably. The AFTE theory states (PX 25 at 212):
1. The theory of identification as it pertains to the comparison of toolmarks enables opinions of common origin to be made when the unique surface contours of two toolmarks are in "sufficient agreement."
2. This "sufficient agreement" is related to the significant duplication of random toolmarks as evidenced by the correspondence of a pattern or combination of patterns of surface contours. Significance is determined by the comparative examination of two or more sets of surface contour patterns comprised of individual peaks, ridges and furrows. Specifically, the relative height or depth, width, curvature and spatial relationship of the individual peaks, ridges and furrows within one set of surface contours are defined and compared to the corresponding features in the second set of surface contours. Agreement is significant when it exceeds the best agreement demonstrated between toolmarks known to have been produced by different tools and is consistent with agreement demonstrated by toolmarks known to have been produced by the same tool. The statement that "sufficient agreement" exists between two toolmarks means that the likelihood that another tool could have made the mark is so remote as to be considered a practical impossibility.
3. Currently the interpretation of individualization/identification is subjective in nature, founded on scientific principles and based on the examiners training and experience.
The SFPD Crime Lab firearms examiners followed the Crime Lab's own procedures manual. The procedures manual explained for "interpretation of results":
A sufficient correspondence of individual characteristics will lead the examiner to the conclusion that both times (evidence and tests) originated from the same source. An insufficient correspondence of individual characteristics but a correspondence of class characteristics will lead the examiner to the conclusion that no identification or elimination was made with respect to the items examined. A disagreement of class characteristics will lead the examiner to the conclusion that both items (evidence and tests) did not originate from the same source. A lack of suitable microscopic characteristics will lead the examiner to the conclusion that the items are not suitable for comparison.
(DX 2 Tab 2 at 1-2.2). This standard, essentially a restatement of the AFTE theory of identification, is sufficient to control the analyses of firearms examiners. Although the essential phrase — "sufficient correspondence" — could be construed as vague, this order finds it is not an unreasonable standard when used by a competent firearms examiner. To competent examiners, "sufficient correspondence" has a specific meaning, understood through their study of the peer-reviewed literature, training, and experience. Their expertise, applied to an "examination of two or more sets of surface contour patterns comprised of individual peaks, ridges and furrows," can reliably determine whether the same tool produced the same marks. This order holds that the SFPD Crime Lab standard and AFTE theory of identification are understood by the members of the field. They "control" the technique as Daubert contemplates.
At the hearing the defense emphasized that there appeared to be little literature discussing a technique for identifying subclass characteristics. Nichols explained, however, that it was wrong to assume that subclass characteristics were difficult to account for. He testified that competent firearms examiners could eliminate subclass characteristics through recognition based on an examiner's training and experience. An examiner would apply his or her knowledge of the manufacturing process for a firearm to determine whether there was a potential for subclass characteristics. Smith also testified that subclass characteristics could be accounted for by trained examiners.
During the relevant period, the SFPD Crime Lab also adhered to industry-wide requirements controlling firearm-identification procedures. For example, the laboratory required thorough documentation of the examinations by firearms examiners. For a spent cartridge case, the examiner documented the source of the case, any trace evidence on the case, caliber, brand, type of primer, type of breech face mark, type of firing pin impression, and any other observations. In addition, the examiner would note what other cartridge cases the case in question was compared against and whether the examiner had made a positive, negative, or inconclusive identification for a match. The examiner would then briefly give reasons for his or her conclusion. For example, the examiner could indicate "FPAS/BF," which meant that the main basis for the conclusion was significant correspondence between the firing pin aperture shear and breech face (PX 26). For a spent bullet, the examiner documented the source of the case, any trace evidence on the case, caliber, number of lands and grooves, direction of the twist, the type or composition of the bullet, cannelure type, weight, diameter, measurements of the land impressions and groove impressions, manufacturer, and suitability for examination. As with cartridge cases, the examiner would document the comparisons he or she made and the conclusions reached. The examiner would also document the basis for the conclusion by, for example, writing "LIMP areas, only at base, good agreement," which meant that there was agreement between the land impression areas between two compared bullets (PX 28).
Also, the SFPD Crime Lab required that any identifications be "verified by a second examiner and properly noted and photodocumented." This is the same procedure adhered to by most, if not all, other crime labs throughout the country. This order recognizes that "the maintenance of standards with respect to documentation and peer review weigh in favor of admissibility." Montiero, 407 F. Supp. 2d at 371.
Although the AFTE theory lacks an objective standard, competent firearms examiners operate under standards controlling their profession. The practiced eye of a firearms examiner can render reliable opinions based on an evaluation of the evidence. Moreover, the requirements of thorough documentation and peer review ensure that the standards are reliably applied.
E. General Acceptance.
The AFTE theory of firearms identification based on traditional pattern matching appears to have broad acceptance in the forensic community. There has been no critique sufficient to undermine the traditional examination method as it is performed by competent, trained examiners. The few critiques — such as the impossibility of calculating a true error rate and the fact that there can be no statistical, objective verification of an examiner's conclusions — do not represent the instability in the field that defendants make them out to be. "It is clear that the community of firearm and toolmark examiners accepts the current identification methodology as reliable." Monteiro, 407 F. Supp. 2d at 372. Even examiners who promote CMS use do not contest the validity of traditional pattern matching. This order holds that the theory and method of firearms identification applied by the SFPD Crime Lab examiners in this case are generally accepted by the firearms-examiner community.
* * *
Judge Patti Saris and Judge Nancy Gertner in the District of Massachusetts have recently addressed the admissibility of firearms identification under Daubert. See Monteiro, 407 F. Supp. 2d at 372-73; Green, 405 F. Supp. 2d at 124. In both instances, the government sought to have the firearms examiners testify that they were "100 percent sure of a match" or that a match could be made "to the exclusion of all other guns." Both courts allowed the firearms identification testimony. Neither court, however, allowed the examiners to testify as conclusively as the government wished.
Similarly, the evidence before this Court does not support the theory that firearms examiners can conclude that a bullet or casing was fired by a particular firearm to the exclusion of all other guns in the world. For example, Nichols testified that there are instances where the pattern match is sufficiently borderline where some examiners would conclude there is a match and others would find it to be inconclusive. This Court agrees with Judge Saris's assessment of firearms identification: "Because an examiner's bottom line opinion as to an identification is largely a subjective one, there is no reliable statistical or scientific methodology which will currently permit the expert to testify that it is a `match' to an absolute certainty, or to an arbitrary degree of statistical certainty." Monteiro, 407 F. Supp. 2d at 372. That is why one cannot say that a match was made to the exclusion of all others in the world. Accordingly, in the government's case in chief, the experts will be permitted to testify that cartridge cases or bullets were fired from a particular firearm "to a reasonable degree of ballistic certainty." The government has agreed to so limit the experts' testimony (8/30/06 Hr. Tr. 42).
2. CMS.
Defendants contend that traditional pattern matching is unreliable in light of the more recent development of CMS. According to defendants, because CMS is based on objective criteria and statistics, it is more reliable under Daubert. That, however, is not the question. The government is only required to use a method that passes under Daubert. It is not required to use the method that best passes Daubert.
Within the field, moreover, it must be said that CMS is not a predominate methodology, only a method in its infancy, a work still in progress. At the hearing, the defense offered the testimony of Adina Schwartz, a professor at John Jay College of Criminal Justice. Her testimony was relevant to the history of the development of CMS. Schwartz first discussed a 1955 article by Alfred Biasotti, which has since been described as the best statistical study ever done on firearms identification. Biasotti test-fired several guns in his study. He then counted the "matching" striae on the test fired bullets. According to Schwartz, Biasotti found that there was a 15-24% overlap in matching striae between bullets fired from different guns. For known matches, Biasotti found a 21-38% similarity in matching striae. Further studies were performed by other researchers to see if a false positive could be generated with the CMS theory. Using land and groove impressions as units and applying the CMS criteria, the studies generated many false negatives but no false positives.
Schwartz admitted that there were numerous critiques of the CMS theory. First, CMS tries to identify matches to the exclusion of all other firearms in the world. According to the critics, however, it is impossible to conclude with absolute certainty that no other firearm could create the same individual characteristics. Second, proponents of CMS hold out that theory as an objective method of identification. The critics note, however, that CMS requires an examiner to count points of identification and that different examiners might count points differently. To the critics, CMS necessarily involves some subjectivity and is a difficult method to standardize.
Nichols, himself a CMS proponent, disagreed with Schwartz's assessment that the field was "in turmoil" due to the development of CMS. Nichols explained that all firearms examiners start their process by looking for patterns that match between the two items under the comparison microscope. CMS merely quantified and described the pattern the examiner was observing. Thus, the field was not "divided" because CMS and traditional pattern matching were not mutually exclusive. Nichols was adamant that CMS was not a separate theory of identification. Rather, it was only a method to describe the striated patterns on which firearms examiners had been basing conclusions for decades.
This order finds that CMS is a school of thought among a small subclass of professionals in the firearm-identification field. It has not gelled into a recognized discipline. CMS may eventually develop into a functional and reliable science. Right now, however, the evidence in this record does not warrant dismissing traditional pattern matching in favor of CMS. Traditional pattern matching is reliable. As stated in the peer-reviewed literature: "It is enough to state that CMS is not a new technique, nor in conflict with the traditional pattern matching that has characterized the discipline from the earliest of times. It is simply an extension, a manner of describing the pattern that is believed to be more concise, more easily understood, and allows for its use by others" (PX 25 at 216).
* * *
There is a method and science behind firearm and toolmark identification. The essence of it is that the examiner looks for a high degree of congruence in patterns created. A high degree of agreement supports a match "to a reasonable degree of ballistic certainty." In some ways, firearms identification is analogous to eyewitness identification. In ordinary life, we look for features that match our memories. When we recognize someone on the street, there is always a possibility that it is not who we think it is. One can never be absolutely certain it is the other person because there is always a possibility of error. Yet we know when we are confident enough in the pattern to make an "identification." Likewise, the practiced eye of parents of identical twins can immediately tell the difference between their children. A zookeeper knows each animal by name, where any other person would not be able to tell the difference between two elephants. The practiced eye of the firearms examiner also looks for patterns. The record demonstrates that there is no such thing as an identical match. There will always be some differences and, in all likelihood, some similarities. It is a matter of judgment whether the patterns are sufficiently close to warrant an identification. A vital aspect of firearms identification is based on the experience these examiners have when they are doing identifications.
Furthermore, it is impractical, even at this stage in the history of the art, to place a probability of error on individual analysis. The patterns do not lend themselves to probabilistic calculations. It is like recognizing someone on the street. It would be difficult to do a strict numerical analysis to determine whether it is the person one believes it to be, or a different person who has some of the same features.
By arguing in favor of CMS, the defense wants something that is beyond the state of the art. The holy grail of some in the profession is to find some way to impose upon firearms identification a statistical model to present to juries. It would be an improvement if that could be done. But the fact that CMS — the closest the field has come to a probabilistic model — has not been accepted does not mean that the existing system is not reliable. No doubt, if the government were here trying to use CMS to provide a high probability, say 99%, to the jury, the defense would challenge its use as a work still in progress.
Finally, it is important to note that — at least according to this record — there has never been a single documented decision in the United States where an incorrect firearms identification was used to convict a defendant. This is not to say that examiners do not make mistakes. The record demonstrates that examiners make mistakes even on proficiency tests. But, in view of the thousands of criminal defendants who have had an incentive to challenge firearms examiners' conclusions, it is significant that defendants cite no false-positive identification used against a criminal defendant in any American jurisdiction.
CONCLUSION
For the foregoing reasons, the firearms-identification testimony offered by the government passes muster under Daubert. The experts may not, however, testify to their conclusions "to the exclusion of all other firearms in the world." They may only testify that a particular bullet or cartridge case was fired from a particular firearm to a "reasonable degree of certainty in the ballistics field." Defendants' motions to exclude the expert testimony under Daubert is accordingly GRANTED IN PART and DENIED IN PART.
IT IS SO ORDERED.