Not so golden? Microscopic agglutination test for diagnosis of leptospirosis

The microscopic agglutination test (MAT) is designated the "gold standard" for the laboratory diagnosis of leptospirosis, a spirochete disease that can cause severe illness if not promptly treated.  Although imperfect, MAT is used as the benchmark when the performance of another diagnostic test for leptospirosis is being assessed.  It is also used to determine the prevalence of leptospirosis in a population.  How imperfect is MAT?  A recent study by Limmathurotsakul and colleagues, published in the journal Clinical Infectious Diseases, claims that its performance is much worse than scientists previously thought.

MAT involves mixing serial dilutions of patient sera with live suspensions of Leptospira.  If agglutinating antibodies against Leptospira are present, the spirochetes will clump.  The clumps can be seen by darkfield microscopy.  Although the idea behind MAT is simple to understand, the technique itself is cumbersome.  Since agglutinating antibodies react best with the specific Leptospira serovar infecting the patient, cultures of at least one serovar from each of the ~20 major Leptospira serogroups must be maintained.  To perform the assay, each serum dilution is mixed individually with a suspension from each culture and examined by microscopy one at a time.  The assay is time consuming, laborious, and potentially hazardous to laboratory personnel.  For these reasons MAT is not routinely employed for diagnostic testing outside of the research setting.

The performance of a diagnostic test is judged by its sensitivity and specificity.  The problem with leptospirosis is figuring out how many actually have the disease so that the sensitivity can be calculated accurately.  Since the sensitivity of culture is poor, researchers rely on antibody tests such as MAT to identify leptospirosis cases.  This approach assumes that the sensitivity and specificity of MAT are 100%.

In general there are two problems with using antibody tests for diagnostics.  The first is that it takes time for the immune system to generate enough antibody that can be detected.  The second problem is that those with previous exposure to the pathogen will test positive even if they are not currently infected.  To minimize these problems, patients with the signs and symptoms of leptospirosis are deemed to have a positive MAT if they fulfill one of the following criteria.

• At least a four-fold increase in MAT titer between paired sera. 
• At least a 1:400 MAT titer when only a single specimen is available.  This cutoff is sometimes adjusted based on the prevalence of leptospirosis in the population being examined.

Since some false negative MAT cases can be identified by culture, one way to calculate the sensitivity of MAT is to add the number of MAT-positive and culture-positive (but MAT-negative) cases together to estimate the number of patients with leptospirosis and then calculate the percentage of MAT-positive cases among these patients.  Limmathurotsakul and colleagues performed these calculations with data from their four earlier studies conducted in Thailand.  A total of 413 patients tested positive by MAT or culture (or both).  They found that the sensitivity of MAT was 86%-96% across the four sets of data.  The remaining 4-14% were false negatives, having tested positive by culture but not by MAT.

The authors next calculated the true sensitivity and specificity of MAT with a statistical tool called latent class analysis, which does not assume any perfect gold standard.  Since there is no perfect test, the true disease status of each patient is the unknown or "latent" variable.  Results from multiple diagnostic tests are related to the latent variable using statistical models.  The calculations go beyond the scope of this blog post, but the bottom line is that the true sensitivity and specificity of each diagnostic test can be estimated with these models.  In addition to MAT and culture, the authors tested some of their patients with an immunofluorescence assay (IFA), lateral flow test (LF), and/or PCR. Latent class analysis is more powerful when the diagnostic tests being evaluated detect different features of the infection.  MAT, IFA, and LF are antibody tests, and culture and PCR detect the pathogen itself.

The sensitivity of MAT calculated by this method turned out to be only 49.8%, much lower than the 86%-96% calculated using the standard method that assumes a perfect gold standard.  The sensitivity of culture alone was 10.5%.  Combining culture with MAT did not help much; the sensitivity of the combined approach was only 55.5%.  The low sensitivity of "MAT plus culture" suggests that the specificities calculated for the alternative tests may be underestimated by the standard method.  This is because some of the many false-negative cases may be correctly identified as having leptospirosis by the alternative tests.  This turned out to be the case for two of the tests.   Specificities for all tests were over 95% by latent class analysis.  However, the specificities for PCR (82.5%) and the lateral flow test (70.5%) were lower when "MAT plus culture" was assumed to be the perfect gold standard.

You can see that the accuracy of alternative leptospirosis tests is underestimated when MAT (or MAT plus culture) is assumed to be the perfect reference test.  Another implication of the study is that the prevalence of leptospirosis has been underestimated, at least in Thailand.  The only other study to evaluate the performance of MAT by latent class analysis was conducted by the CDC here in the U.S almost a decade ago.  In contrast to the Limmathurostsakul study, the CDC study determined that the sensitivity of MAT was high, at 98.2%.  There were many differences between the two studies, including the patient population, the alternative tests evaluated, the time interval between collection of paired sera, and the number of serovars included for MAT.  The poor performance of MAT in the Thailand study may therefore not be a universal finding.

References

Limmathurotsakul D, Turner EL, Wuthiekanun V, Thaipadungpanit J, Suputtamongkol Y, Chierakul W, Smythe LD, Day NPJ, Cooper B, Peacock SJ (August 2012).  Fool's gold: why imperfect reference tests are undermining the evaluation of novel diagnostics: a reevaluation of 5 diagnostic tests for leptospirosis.  Clinical Infectious Diseases 55(3):322-331.  DOI: 10.1093/cid/cis403

Bajani MD, Ashford DA, Bragg SL, Woods CW, Aye T, Spiegel RA, Plikaytis BD, Perkins BA, Phelan M, Levett PN, and Weyant RS (February 2003).  Evaluation of four commercially available rapid serologic tests for diagnosis of leptospirosis.  Journal of Clinical Microbiology 41(2):803-809.  DOI:  10.1128/JCM.41.2.803-809.2003

qPCR testing for Leptospira infection reveals sex differences in bacterial load

The pitfall of serological testing is that patients test negative during the early days of infection because time is needed for the immune response to ramp up antibody production against the pathogen.  Unfortunately clinical laboratories are stuck with serological methods for detecting Leptospira since direct tests for the spirochete are not available.

Since Leptospira has to enter the bloodstream to spread throughout the body, PCR testing of blood is one tool that may aid prompt diagnosis of leptospirosis.  In a study published in Clinical Infectious Diseases, Agampodi and colleagues conducted quantitative real-time PCR (qPCR) on patient sera collected during a 2008 leptospirosis outbreak in Sri Lanka, where leptospirosis is endemic.  With a reported annual incidence of 5.4 cases per 100,000, Sri Lanka has the sixth highest incidence of leptospirosis in the world.

The acute-phase sera tested for the study came from patients with confirmed leptospirosis.  The patients had clinical evidence of leptospirosis, and a second blood sample drawn at least a week later indicated that they had rising antibody titers or had seroconverted.

The PCR primers targeted the DNA encoding the 16S rRNA of Leptospira.  The sensitivity of qPCR turned out to be 51% (25/49) with acute-phase sera collected during the first ten days of illness.  This doesn't sound impressive at all, but the sensitivity of the microscopic agglutination test (MAT) was much worse.  MAT testing is done by mixing dilutions of serum with suspensions of Leptospira.  If anti-Leptospira antibodies are present, they will cause the bacteria to clump.  The sensitivity of MAT with acute-phase sera collected up to 15 days after symptoms began was a miserable 18% (13/73).  Past studies conducted in Sri Lanka and elsewhere around the world have also demonstrated poor performance of serological testing for leptospirosis.  Although the specificity of qPCR wasn't determined, it's clear that qPCR is an improvement over serological testing for the prompt diagnosis of leptospirosis.

One interesting observation came out of the qPCR data.  Men who were qPCR positive had a higher bacterial load than women who tested positive (median of 15,640 bacteria/ml in men vs. 5,611 bacteria/ml in women, P = 0.022, Mann-Whitney U test).  Based on this observation, the authors raised the possibility that men are biologically more susceptible to leptospirosis than women.  On the other hand, the difference in bacterial load may have nothing to do with biology.  It may simply reflect differences in when men and women sought medical care, or it may mean men were exposed to environmental sources that were more heavily contaminated with Leptospira.

A 2007 study conducted in Germany revealed sex differences in leptospirosis severity.  Men diagnosed with leptospirosis were more likely than women to have hemorrhage, jaundice, and renal impairment, all signs of severe disease (see table below).   The difference in disease severity could not be accounted for by differences in exposure risk, the infecting serogroup, or interaction with the health care system.  Could the difference be due to biological variation between the sexes?

Table 1 from Jansen et al., 2007.  OR adjusted for age.

Note: Here's a recent post in the blog Camp Other that examined sex differences in another disease of spirochetes, Lyme disease.

Main references

Agampodi SB, Matthias MA, Moreno AC, and Vinetz JM (March 12, 2012).  Utility of quantitative polymerase chain reaction in leptospirosis diagnosis: association of level of leptospiremia and clinical manifestations in Sri Lanka.  Clinical Infectious Diseases (published online ahead of print February 21, 2012).  DOI: 10.1093/cid/cis035

Jansen A, Stark K, Schneider T, and Schöneberg I (May 1, 2007).  Sex differences in clinical leptospirosis in Germany: 1997-2005.  Clinical Infectious Diseases 44(9):e69-e72.  DOI: 10.1016/j.ijid.2007.09.011


Other references

Pappas G, Papadimitriou P, Siozopoulou V, Christou L, and Akritidis N (July 2008).  The globalization of leptospirosis: worldwide incidence trend.  International Journal of Infectious Diseases 12(4):351-357.  DOI: 10.1016/j.ijid.2007.09.011

Reller ME, Bodinayake C, Nagahawatte A, Devasiri V, Kodikara-Arachichi W, Strouse JJ, Flom JE, Dumier JS, and Woods CW (September 9, 2011).  Leptospirosis as frequent cause of acute febrile illness in southern Sri Lanka.  Emerging Infectious Diseases 17(9):1678-1684.  DOI: 10.3201/eid1709.100915

Serologic testing for syphilis: missing the point

You may have seen several news sources touting the recent CDC finding that nearly one in five positive reactions with a newer syphilis test are wrong.  These headlines may grab the reader's attention, but the press took the finding out of context and failed to deliver the real message that the CDC was trying to convey.  Worse, the press reports may needlessly confuse and worry those who are being treated for syphilis.

Serological tests for syphilis are grouped into two categories.  Nontreponemal tests such as the VDRL and RPR are based on antibody generated against the lipid cardiolipin.  Presumably cardiolipin is released from damaged tissue in syphilis patients and gets incorporated into the membrane of Treponema pallidum.  The reason that these tests are "nontreponemal" is that antibodies to cardiolipin accompany many other conditions.  On the other hand, treponemal tests use T. pallidum proteins or even the entire spirochete as antigen to detect antibodies against the spirochete.  Although the classic treponemal tests such as the FTA-ABS (fluorescent treponemal antibody-absorption) and TP-PA (Treponema pallidum particle agglutination) are still used, the newer automated EIA (enzyme immunoassay) and CIA (immunochemiluminescence) treponemal tests enable clinical laboratories to rapidly screen a large number of sera.

The traditional approach to syphilis testing is to first screen the patient's serum with a nontreponemal test.  Since nontreponemal tests can give false positive reactions, reactive sera are retested with one of the treponemal tests.  However, the low cost of executing the automated treponemal tests have led some high-volume clinical laboratories to reverse the order of the assays:  they screen with the EIA/CIA treponemal test and confirm positive results with a nontreponemal test.  The CDC report in the Morbidity and Mortality Weekly Report deals with this so-called "reverse sequence" testing.

So where did the "nearly one in five" figure come from?  From 2006 to 2010, five large clinical laboratories screened 140,176 sera specimens with the reverse sequence procedure.  Of the 4,834 reactive with the EIA/CIA treponemal test, 2,743 gave negative results with the nontreponemal RPR test.  When the samples that gave discrepant results were tested further with one of the classic treponemal tests, 866 of the 2,743 samples were negative.  Overall, among the 4,834 samples that were reactive with the newer treponemal test, 866 or 18% were nonreactive with two subsequent tests.  These 866 were assumed to be false positives.

The news media pounced on the 18% figure and declared that hundreds may have been given antibiotics to treat a disease that they didn't have.  But they ignored the fact that doctors don't diagnose syphilis on the basis of a single lab test.  It is standard practice to perform a second test when the first comes back positive and to do even a third one if warranted.  Doctors also take into account the physical exam and the sexual and medical history of the patient before making the decision to treat with antibiotics.

Here's how the CDC responded to the assertion that those among the 18% may have been falsely diagnosed and treated unnecessarily with antibiotics:

There are two problems with this assertion. First, the current report does not document whether or not treatment was provided. Second, in those cases where treatment was provided, it may have been justified based on sexual risk and findings on clinical evaluation. It is also important to note that syphilis is not diagnosed on the basis of a single blood test. Many labs routinely will do additional testing when the first test is positive, without notifying the patient. Doctors diagnose syphilis after considering at least two syphilis tests, the patient's history, the physical exam, and a review of past syphilis test results. The MMMR analysis, while important, does not allow us to conclude that the newer tests led to inaccurate syphilis diagnosis or inappropriate treatment.

So what was the message that the CDC was trying to communicate to readers of the MMMR report?  Their intention was to provide guidance in the management of cases for which the reverse sequence screening is performed instead of the traditional sequence, which is still recommended by the CDC.  Specifically, when conflicting results occur (positive with the treponemal test, negative with the nontreponemal test), a third test should be done with the TP-PA.  (The CDC does not recommend the FTA-ABS because it is less specific and probably less sensitive.)  A positive reaction with the TP-PA indicates past or present syphilis; a negative reaction indicates that syphilis is unlikely.  As always, the clinical observations and medical history of the patient should also be considered in making an informed treatment decision.


Reference

Centers for Disease Control and Prevention (February 11, 2011).  Discordant results from reverse sequence syphilis screening -- five laboratories, United States, 2006-2010.  MMMR. Morbidity and Mortality Weekly Report 60(5):133-137.  link