Amanda Knox and Statistical Nullification

In today’s New York Times, Leila Schneps and Coralie Colmez correctly warn that

… math can become a weapon that impedes justice and destroys innocent lives.

They discuss Lucia de Berk, and Sally Clark, two unfortunate people who were convicted of crimes based on bogus statistical arguments. Statistician Richard Gill helped get de Berk’s conviction overturned.

So why do Schneps and Colmez argue that statistics should be used to help re-try Amanda Knox in a what will be another Italian travesty of justice?

They criticize the judge for bad statistical reasoning. Referring to a new test of DNA on a knife, they say:

His reasoning? If the scientific community recognizes that a test on so small a sample cannot establish identity beyond a reasonable doubt, then neither could a second test test on an even smaller sample.

They go on to argue that basic statistics tells us that further testing would “tell us something about the likely accuracy of the first result. Getting the same result after a third test would give yet more credence to the original finding.”

Indeed. It would confirm that she had touched a knife in her own apartment. I guess you would find my DNA all over the cutlery in my house. I hope no one uses that as evidence in a murder trial.

The Amanda Knox trial was a joke; it never would have made it to court in the U.S.

All of which raises a question. Who would testify as a statistical expert in a court case without taking into account the context? Even if I thought Schneps and Colmez were correct in their statistical reasoning, I would not testify on the narrow question of whether a second test would strengthen that particular piece of evidence if I thought the trial was a scam.

Some time ago, I was asked to be an expert witness on a statistical issue regarding random drug testing of high school students in Pennsylvania. The attorney working for the tyrants (the school district) tried to convince me that he had a good statistical case for doing such testing. I told him that, despite the attractive fee and the correctness of his statistical arguments, I would not testify in a case that helped the government harass its own citizens.

What Schneps and Colmez failed to mention was context. I don’t care how sound the statistical argument is. If it is being used for unethical and unjust means, we should refuse to testify. Call it Statistical Nullification.

19 Comments

  1. Posted March 27, 2013 at 11:24 pm | Permalink

    as a phd in biol, I thought it was schneps who made the eror: you have a bad test, repeating it doesn’t get you anything , cause it ain’t independent

  2. Posted March 28, 2013 at 2:58 am | Permalink

    Amanda Knox knowingly accused an innocent black man expecting racism would get her scotch free, only for that she should still be in jail. Why didn’t she blame the other black man that actually raped Meredith and was also beknown to her? Obviously, because the real culprit would have pointed at her.

    In Europe this case is seen just like the European OJ Simpson; but when money talks bullshit walks.

    • Martin Azizyan
      Posted March 28, 2013 at 6:39 am | Permalink

      Whether Amanda Knox is guilty of murder or not isn’t the issue. And whether she gave false testimony and should be convicted for that isn’t the issue either. In the US and other countries with similar criminal justice systems (which I know Italy is not), the integrity of the judicial process is [supposed to be] deemed more important than the outcome of any single case, even when that case has been turned into such a horrendous media circus. I think Larry’s point is that the responsibility to protect that integrity lies not only with judges, attorneys, and jury members, but also every single citizen, including those asked to testify as witnesses (expert or otherwise).

      Really, what Larry is saying is an instance of a common legal (resp. ethical) concept — if you expect your words or actions to facilitate an illegal (resp. unethical) deed, you are partially responsible for the outcome.

      • Posted March 28, 2013 at 8:54 am | Permalink

        Whether Amanda Knox is guilty of murder or not isn’t the issue. And whether she gave false testimony and should be convicted for that isn’t the issue either.

        It is not for you.

        In the US and other countries with similar criminal justice systems (which I know Italy is not), the integrity of the judicial process is [supposed to be] deemed more important than the outcome of any single case, even when that case has been turned into such a horrendous media circus.

        The integrity of your beloved system has freed OJ Simpson in a double murder case, driven to suicide Aaron Swartz and threatens with a life sentence people like Bradley Manning.

        I think Larry’s point is that the responsibility to protect that integrity lies not only with judges, attorneys, and jury members, but also every single citizen, including those asked to testify as witnesses (expert or otherwise).

        Really, what Larry is saying is an instance of a common legal (resp. ethical) concept — if you expect your words or actions to facilitate an illegal (resp. unethical) deed, you are partially responsible for the outcome.

        Glad to know you are Larry’s spokesperson but it seems to me you worry too much about what is legal and too little about what is fair. If Larry was so concerned about the integrity of the system he would not affirm thing like “despite the attractive fee and the correctness of his statistical arguments, I would not testify in a case that helped the government harass its own citizens”. Seems to be like the guy actually did something illegal (your resp. unethical) and yet Larry did not play along because he consider it unfair, so well done.

        And are you equating what is legal with what is ethical? I could not disagree more.

      • Posted March 28, 2013 at 8:55 am | Permalink

        Whether Amanda Knox is guilty of murder or not isn’t the issue. And whether she gave false testimony and should be convicted for that isn’t the issue either.

        It is not for you.

        In the US and other countries with similar criminal justice systems (which I know Italy is not), the integrity of the judicial process is [supposed to be] deemed more important than the outcome of any single case, even when that case has been turned into such a horrendous media circus.

        The integrity of your beloved system has freed OJ Simpson in a double murder case, driven to suicide Aaron Swartz and threatens with a life sentence people like Bradley Manning.

        I think Larry’s point is that the responsibility to protect that integrity lies not only with judges, attorneys, and jury members, but also every single citizen, including those asked to testify as witnesses (expert or otherwise).

        Really, what Larry is saying is an instance of a common legal (resp. ethical) concept — if you expect your words or actions to facilitate an illegal (resp. unethical) deed, you are partially responsible for the outcome.

        Glad to know you are Larry’s spokesperson but it seems to me you worry too much about what is legal and too little about what is fair. If Larry was so concerned about the integrity of the system he would not affirm thing like “despite the attractive fee and the correctness of his statistical arguments, I would not testify in a case that helped the government harass its own citizens”. Seems to be like the guy actually did something illegal (your resp. unethical) and yet Larry did not play along because he consider it unfair, so well done.

        And are you equating what is legal with what is ethical? I could not disagree more.

      • Martin Azizyan
        Posted March 28, 2013 at 11:58 am | Permalink

        The flaws of the US justice system are irrelevant (though numerous). I was referring to a certain principle to explain why I, as you put it, “worry too much about what is legal and too little about what is fair”. Certain principles are sometimes deemed to important to be sacrificed for the sake of [what the court of public opinion has deemed to be] fairness in a single instance. I’m sure this is the case in Italy, just as much as in the US.

        As for Larry’s anecdote about the Pennsylvania case, I don’t know the details of the case but it seems as though some evidence was (or might have been, at some point in the future) acquired against high school students using random drug tests. Many people in this country would consider such a method of law enforcement illegal (unconstitutional), and more importantly an example of totalitarianism/tyranny. Evidence (no matter how damning) that is acquired by the state through illegal actions is usually considered inadmissible in court (I hope it is obvious why). However, due to what some people consider to be an erosion of constitutional limits on government power during the drug war in the US, similar methods of enforcing drug laws (including drunk driving laws) are commonly accepted by courts. In the US, there are multiple ways in which citizens get to exercise control over their government (voting in elections is just a small part of American democracy). One of those ways is jury nullification, in which a jury can choose to acquit the accused even though they believe sufficient evidence has been provided for a guilty verdict (for instance, because the jury believe the law under which a person is being prosecuted is unjust). Hence the amusing term “statistical nullification”. I believe the analogy of an expert witness refusing to testify due to the belief that his or her testimony will be used improperly (unethically, illegally, or in flawed argument such as a logical fallacy) to jury nullification is obvious.

        You may have noticed that I abused vague terms such as “sometimes”, “many people”, “some people consider”, etc. This is because I am trying to explain my understanding of a point of view which may clarify the reasoning behind Larry’s post and help you see a perspective you may not have considered, WITHOUT injecting my personal opinions into the conversation. It just so happens that I am not American myself (I’m much closer to being Italian, ironically), so it is not “my beloved system”. Perhaps I offended you somehow with my original reply and made you think that I was trying to initiate some kind of petty nationalistic bickering between us. I would not be interested in that type of exchange, even if I was American.

      • antonio
        Posted May 20, 2013 at 8:37 am | Permalink

        [citation]

        The Amanda Knox trial was a joke; it never would have made it to court in the U.S.

        In the US and other countries with similar criminal justice systems (which I know Italy is not), the integrity of the judicial process is [supposed to be] deemed more important than the outcome of any single case

        The flaws of the US justice system are irrelevant (though numerous)

        [/citation]

        It is not possible for me to reply to such arguments A, B and C who are stated with probability 1 and without any need to support them.
        So I avoid to tell what here in Italy we think about Amanda Knox trail, and what it would be happened if she was not an US citizen.

  3. oz
    Posted March 28, 2013 at 10:15 am | Permalink

    “The Amanda Knox trial was a joke; it never would have made it to court in the U.S.”
    I don’t know the details of Amanda Knox’s case, but criticizing the Italian system while giving the US system
    as a good example we should aspire to is ironic. There are many anecdotal examples (e.g. the Simpson trial).
    More importantly – US and Belarus are the only western countries still having capital punishment, and of all countries,
    the US is locking up the highest (by far) percentage of it’s population in (many of which for-profit) jails,

    • Martin Azizyan
      Posted March 28, 2013 at 11:16 am | Permalink

      Straw man arguments. Original post claimed superiority of US system in a specific instance, not overall.

      • antonio
        Posted May 20, 2013 at 8:40 am | Permalink

        Really? So is US system like the estimator \hat{\theta}=0.5 which is perfect in the case \theta=0.5 and terrible otherwise?
        Or is my reasoning another drawn from a straw man argument random sequence?

  4. Posted March 28, 2013 at 7:37 pm | Permalink

    I thought you couldn’t be prosecuted for murder once being acquitted, maybe in the U.S. But I’m wondering if Larry is saying Knox had nothing to do with the murder of her roommate. I barely remember the case, but I thought she admitted to forcing her into an orgy that turned violent. I would rather not look into the sordid details, but the post has me curious as to what Larry thinks.

    • Posted March 28, 2013 at 7:44 pm | Permalink

      There is no double jeopardy rule in Italy.
      The story you gave is the one presented by the prosecutor
      who is a known crackpot who, for example, believes that the
      devil himself runs an active conspiracy in Italy and he has actually
      prosecuted people on those grounds.

      • Posted March 28, 2013 at 7:57 pm | Permalink

        No I take it she admitted this, and then she tried to make it look like a burglary–why? There was some sort of serious conflict between the two that others were aware of, I can’t recall if it was something like her roommate being uncomfortable with Knox’s social life. But look,the woman was murdered! Are you saying Knox wasn’t involved? Maybe you’re saying its irrelevant if the prosecutor conducted an unsound trial, which I know nothing about.

      • Bill Jefferys
        Posted March 28, 2013 at 11:58 pm | Permalink

        I had understood the double jeopardy issue a bit differently (based on what I have read and heard since this all came out). Apparently under Italian law, the prosecutor can appeal an innocent verdict and have it reversed; but had the court that reversed that verdict and required a retrial not had done that, I understand that the whole issue would have been over.

        Under US law, the situation as I understand it is different. Under US law an acquittal by a JURY means that jeopardy attaches and the prosecution cannot appeal; this is generally but apparently not universally true in a bench trial as well. But Italy does not have the jury system, it is entirely in the hands of judges.

        http://en.wikipedia.org/wiki/Double_Jeopardy_Clause

        http://en.wikipedia.org/wiki/Italian_Code_of_Criminal_Procedure#Judges_of_the_trial

        I think that some of the repugnance that I and many Americans may feel about this is simply that we feel that double jeopardy should have applied in this case, even though Italian law is different from US law.

        The whole issue is probably moot; if Ms. Knox is tried again and convicted in absentia, my guess is that it’s not likely that the United States will agree to extradite her, unless any new evidence that is introduced is really compelling. It would, however, put quite a crimp in her future travel plans.

        I don’t know about Mayo’s comments about what Knox may or may not have admitted, so my comments are not directed towards her comments in any way.

    • Ken
      Posted March 30, 2013 at 11:51 pm | Permalink

      All information about Knox changing her story and confessions came from the Italian police and also involved an interpreter who it is alleged felt that they should be helping the police rather than just translating. Basically the whole case revolves around these “admissions”, and that there is some doubt about the defences explanation that a third person had broken in. However there is for him strong evidence that he was there and committed the crime. There is reasonable evidence that Knox or her boyfriend were there.The idea of finding trace DNA for people either sharing or visiting a household is absurd, it is almost nearly certain that you are going to find it.

      This case has confirmed what I have heard about Italy. A lot of government based jobs you simply move up through the ranks automatically, so many of the top jobs are held by people who have been promoted beyond their competency. It appears that their prosecutor, judges, police and forensics people all lacking.

      For an Australian perspective, I recommend the case involving the death of Azaria Chamberlain http://en.wikipedia.org/wiki/Death_of_Azaria_Chamberlain for what can happen when deaths are poorly investigated. This involved some forensics work where nobody considered the rate of false positives.

      • antonio
        Posted May 20, 2013 at 9:57 am | Permalink

        “A lot of government based jobs you simply move up through the ranks automatically”
        Really? What do you mean?

  5. Posted March 29, 2013 at 9:57 am | Permalink

    Schneps and Colmez’s article is obviously a plug for their recently published book , “Math on Trial: How Numbers get Abused in the Courtroom”.

  6. Jake
    Posted May 15, 2013 at 5:11 pm | Permalink

    You completely missed the point – the DNA on the knife blade was being compared to Meredith’s and this was inconclusive. Amanda’s DNA was certainly on the handle and the knife was in Raffaele’s apartment. Of course your DNA in on the cutlery in your house, but hopefully you could understand the police/courts being suspicious if they also found the DNA of a murder victim who didn’t live there alongside yours.

%d bloggers like this: