Michael Turner explains why it is instructive on the role of expert evidence oncomputer evidence, and on the thorny issue of Novel Science.
The incidents which form the basis of thisarticle occurred at the time that the Macpherson inquiry into the death ofStephen Lawrence was investigating allegations of racism in the MetropolitanPolice (MPS). As a result, this high profile case has had extensive coverage inthe media, including a series of features on BBC2 Newsnight, and in more thanten parliamentary questions.
The Racist Letters
In December 1997 racist letters were receivedin the internal mail by a number of ethnic minority officers, including PS Virdi,in the Ealing Division of the MPS. In January 1998 a further set of racistletters were received in the internal mail by a number of ethnic minoritycivilian staff in the Ealing Division. Both sets of racist letters wereindividually addressed and comprised a single side of A4 containing a graphicimage and offensive text ending with the letters ‘NF’.
A police investigation was launched. No DNA orfingerprint evidence was found. Instead the investigation concentrated primarilyon computer evidence. Back-up tapes and copies of log files were seized from theservers of the police OTIS computer system and analysed. PS Virdi’s house, car,drains, loft and outbuildings were searched for seven hours. Dozens of policeofficers were interviewed.
Criminal Proceedings
PS Virdi, who had a previously unblemisheddisciplinary record, was arrested and charged with criminal offences relating tothe production and dissemination of racist letters, and suspended from duty. Hevigorously denied being responsible for the racist letters and had an alibi forthe December 1997 documents.
When legal advice was taken the CPS declined toprosecute. The reasons for that decision have not been disclosed.
MPS Discipline Board Hearing
The same allegations formed the basis ofcharges brought against PS Virdi in a four-week MPS Discipline Board hearing inFebruary 2000. MPS and PS Virdi were represented by counsel. The Board heardevidence from 51 witnesses.
Expert evidence was given for MPS thatidentified how and when the racist letters were produced. The technique usedwas:
- to reconstruct the documents
- to match print runs in event logs against reconstructed documents
- to identify the logged on User IDs responsible for the print runs.
This technique was subsequently christened Document Reconstruction. The MPSDiscipline Board did not hear any expert evidence for PS Virdi.
The MPS Discipline Board unanimously found PS Virdi guilty beyond reasonabledoubt on 11 counts relating to the racist letters, including impersonating acolleague police officer by using her User ID and password on two occasions,using MPS computer systems to produce both sets of racist documents anddistributing racist documents in the Ealing Division of MPS. PS Virdi wassacked.
Employment Tribunal
PS Virdi had made a complaint of racial discrimination against the MPS on anumber of grounds under the Race Relations Act 1976. In the hearing of the firstEmployment Tribunal case1 in July and August 2000 the complaint abouthis treatment relating to the racist letters was considered.
The Employment Tribunal decided it should first hear the computer evidence asa discrete matter. Evidence was given by two experts for PS Virdi and by thesame two experts who had given evidence at the Discipline Board Hearing for MPS.In an Interim Decision dated 18 July 2000 the Tribunal found that on a balanceof probabilities the two print-runs identified by the MPS expert witnesses werethe racist letters.
As a result of that finding the Tribunal then heard the evidence from theinvestigating officers and some 30 police officer alibi witnesses. The Tribunalfound that none of these officers had any particular reason to recall thealleged events of the early morning of Christmas Eve 1997 without the promptingof printouts from computer systems. The tribunal also heard evidence of passwordabuse.
The Tribunal found a number of anomalies arising from the factual evidencethat threw into doubt its interim decision on the computer evidence. Inparticular, the evidence given on the internal mail system suggested that thesecond set of racist letters were posted two days before it was alleged thatthey had been printed. As a result, the assumption that the documents wereproduced in-house was seriously undermined. Although the Tribunal did not rejectthe theory of the Document Reconstruction technique, this new evidence did callit into question.
The Tribunal noted that the MPS investigation had not checked whether thealleged computer evidence fitted in with what had actually happened and that MPSappeared to have proceeded on the basis that the computer evidence wasunchallengeable.
In August 2000 the Tribunal found on the balance of probabilities that:
- there was no evidence that the racist letters were produced during the print runs identified by the MPS computer experts on 24 December 1997 and 18 January 1998
- there was no evidence that the racist letters distributed were produced by PS Virdi
- and held that MPS had discriminated against PS Virdi on the grounds of his race, by treating him differently and detrimentally to a white WPC suspect.
Following that decision, the Metropolitan Police Authority announced anindependent enquiry into the case. At a hearing of the MPS Discipline Board inNovember 2000 PS Virdi’s appeal against dismissal was not opposed by MPS. He wasreinstated on full pay and received a written apology.
At a Remedies hearing in December 2000 the Employment Tribunal awarded PSVirdi a record £150,000 compensation for injury to his feelings arising fromdiscrimination, including aggravated damages and interest. The award foraggravated damages was in respect of the high-handed treatment of PS Virdi byMPS, particularly in its failure to apologise to PS Virdi until the end ofNovember 2000. A second Employment Tribunal case relating to the conduct of theMPS Discipline Board Hearing and his dismissal is pending.
The Computer Evidence
The case for MPS rested on a series of interrelated assumptions. It wasassumed that there was only one way of creating a series of documents and thatboth sets of racist letters were:
- created and printed within the Ealing Division of MPS using MS Word
- printed as a consecutive sequence of unnamed MS Word documents.
All the evidence emanated from event logs on the MPS OTIS system servers. Itwas claimed that this system was secure; each user had a User ID and a passwordthat was not meant to be disclosed to other users. Passwords had to be changedevery 28 days and the same password could not be re-used within a 12-monthperiod.
The Document Reconstruction technique was invented by the Assistant SystemsAdministrator for the Ealing Division. He thought that the January 1998 racistletters may have been produced in-house and to test this theory he set out torecreate the document. He produced a reconstructed document and noted the sizeof the print file (NB this is not the same as the file size) when it wasprinted. He then searched the event logs for a sequence of printed documentsthat matched that print file size. He identified a sequence of documents printedat a specific time using a specific user ID. It was these event log entries thatformed the basis of MPS’ evidence.
Expert Evidence on Computer Evidence
At the Employment Tribunal hearing, it was agreed that in a criminalinvestigation it was standard procedure for computers likely to contain evidenceto be seized and copied using a non-destructive forensic-imaging process, sothat all investigations could be conducted on write-protected copies of theevidence. It was also agreed that that procedure had not been followed in thiscase.
Two expert witnesses gave evidence for MPS. Mr A adopted the DocumentReconstruction technique. He conducted his own Document Reconstructions andsearches of the event logs, which identified the same two sequences ofdocuments. His expert evidence was that if the racist letters had been producedin-house, then the event log evidence indicated the User IDs responsible forcreating the racist letters.
Mr A had cracked the Password file and identified a highly relevant apparentbreach of the 28-day password rule and/or of the rule that a password cannot bereused within 12 months.
MPS’ second expert, Mr B, was asked by the investigating team to scrutinise thecomputer evidence. He adopted the Document Reconstruction technique and adoptedthe results of Mr A’s reconstructions without doing his own. He analysed theevent logs, and again identified the same two sequences of documents. His expertevidence was that his analysis of the event log evidence indicated beyondreasonable doubt that each series of racist letters was printed at a particulartime and by a particular logged on User ID.
He also identified an anomaly in the relevant log evidence – the sequence ofdocuments alleged to be the January 1998 racist letters were printed at a timewhen there were two concurrent logons using the same User ID on differentworkstations.
Both of the MPS’ experts accepted that they had no prior or subsequentexperience of Document Reconstruction as a forensic technique. Neither hadconducted tests separate and apart from work on this case to establish thevalidity of the technique in its own right.
The expert evidence for PS Virdi identified anomalies and discrepancies inthe computer evidence, including discontinuities in the security event logs,contamination of the security event logs and inconsistencies between server logentries.
Expert evidence for PS Virdi was given that the failure to secure and imagethe relevant servers in a timely fashion and the direct examination byuninformed investigators of the original computer evidence had irretrievablylost or contaminated highly relevant evidence. As a result both hearings hadbeen deprived of evidence that could have helped determine who, if anybody,within the Ealing Division of MPS, created and printed the racist letters. Suchloss or contamination would undoubtedly prevent PS Virdi from receiving a fairtrial. There was no doubt that no criminal court would have allowed the caseagainst PS Virdi to proceed. MPS’ actions in failing to secure the evidence didnot comply with the principles and practice guidelines set out in MPS’ ownPrinciples of Computer Based Evidence.2
The first expert for PS Virdi, Mr C, also identified evidence of relevantpassword abuse and questioned the reliability of all the relevant server logtime-stamps on a number of grounds. In his opinion, all the crucial evidence wasof poor reliability and of questionable admissibility.
Mr C challenged the whole basis of the Document Reconstruction technique. Heidentified 27 theoretical determinants of the size of a Windows print job file;many of those attributes having a very large range of permissible values.Without specifying all these variables (and they would not be specified for anytest document reconstructed only from information available on the face of aprinted document) then it would not be possible to guess the size of a Windowsprint job file with any accuracy at all. Mr B’s evidence was that the value ofsome of those variables would be known if, as MPS alleged, the platform used toprint the documents was known.
In Mr C’s opinion the Document Reconstruction technique was untested,unverified, untestable, unverifiable, unaccepted and unscientific. It was basedon a false premise – that it was possible to precisely determine the size of aWindows print job file by examining only the information available on the faceof a printed document. In his opinion, the use of the Document Reconstructiontechnique was technical speculation and the results should be seen as thecreation of evidence.
It was common ground between all the experts that they had no prior orsubsequent experience of Document Reconstruction as a forensic technique. It wasalso agreed that there is no known formula for precisely determining the size ofa print job file from the information available on the face of a printeddocument.
Document Reconstruction – Novel or Junk Science?
Counsel for MPS contended that Document Reconstruction was an example of anovel forensic technique and that it was legitimate for investigators to developrelevant novel techniques. Counsel for PS Virdi contended that DocumentReconstruction was not a valid scientific method on the basis of any or all ofthe four tests set out in Daubert,3 the leading American case:
- whether the theory or technique can be or has been tested
- the error rate associated with the method
- publication in a peer-reviewed journal
- whether the technique has gained widespread acceptance.
In the USA such invalid forensic science techniques are known as JunkScience,4 and there is a substantial body of (largely pharmaceutical)case law on the admissibility of such expert evidence. The decision of the USSupreme Court in Kumho Tire5 extended this Gatekeeping role of thecourts to evidence from ‘engineers and other experts who are not scientists’.Yet in the present case neither counsel cited any relevant English case law.
Commentary
The real identity of the documents in the two sequences of printed documentsidentified by Document Reconstruction will never be known. In that sense, tryingto match an event log print file size to a reconstructed document was nevermeaningful. The analogy was made with a blood sample found at the scene of acrime. When the forensic scientist seeks to match that sample with anothersample of blood from a suspect, that is matching a piece of real evidence withan independent sample of real evidence. In this case, the investigators tried tomatch a trace of a document found on a computer with a “sample” thathad been constructed solely for the purposes of matching.
This case illustrates many of the dangers inherent in over-reliance oncomputer evidence. Without the computer evidence there was no case against PSVirdi. Familiar topics such as password abuse and the reliability of time-stampswere very much in issue.
The case clearly demonstrates the risks of over-reliance on expertinterpretation of computer evidence. Experts giving evidence on computerevidence have a special responsibility to distinguish clearly between facts andspeculation, assumption, inference and opinion. It appears that distinction wasnot always made in this case. On that basis, it is suggested that this case wasthe first miscarriage of justice that resulted directly from the interpretationof computer evidence by experts.
The reversal of the Employment Tribunal’s Interim Decision on the computerevidence is a welcome reminder to all experts that their opinion evidence playssecond fiddle to evidence of fact. The case also illustrates how risky it is torely on a single stream of computer evidence6 – all the computerevidence in this case came from event log files (two logs, but essentially onestream of evidence).
The handling of the computer evidence was inadequate. It is not known why therelevant servers were not seized and forensically imaged as soon as possibleafter the incidents.
Experts confronted with a novel forensic technique such as DocumentReconstruction are in desperate need of legal guidance as to the limits ofacceptable forensic science under English law.
Endnotes
1. Mr G S Virdi v The Commissioner of Police of the Metropolis, London NorthEmployment Tribunal, Case Number 2202774/98.
2. Unpublished. Believed to be a version of the Association of Chief PoliceOfficers (ACPO) Good Practice Guide for Computer Based Evidence, alsounpublished.
3. Daubert v Merrell Dow (1993) 509 U.S. 579.
4. See for exampe Galileo’s Revenge, Peter Huber, Basic Books, ISBN: 0465026249.
5. Kumho Tire Company, Ltd v Patrick B Carmichael, 119 S.Ct 1167, 97-1709Supreme Court Of The United States.
6. See 4.8 of Submission of The British Computer Society to Criminal CourtsReview at www.bcs.org.uk/lac/ccr.htm
Michael J L Turner is an independent computer evidence consultant who gaveexpert evidence for PS Virdi in the first Employment Tribunal case.expert@computerevidence.co.uk
