The Right of Oblivion, Data Culling and Conservation – Charting the Development of the Fifth Principle

Since the first proposal for harmonised data protection laws in Europe was made by the Council of Europe in 1973,[1] one of the fundamental principles protecting individual rights in the data protection field has been the principle of data retention or data conservation. In essence, this places a negative obligation on data users or controllers to keep data for a limited period of time only. Throughout this paper, the author uses the term “data culling” to describe the cumulative process of review, selection and deletion of data required to limit the amount of data retained.

This negative obligation is in contrast to recent legal proposals on data retention at a European and United Kingdom level. These have been promoted for national security and crime prevention and detection reasons as part of the wave of anti-terrorism legislation which has followed the events of September 11, 2001. At a European level, during the Danish presidency of the Council of the European Union in June, the Council published a number of recommendations on information technology-related measures and the investigation and prosecution of organised crime[2]. In this document, the Council urged that:

“within the very near future, binding rules should be established on the approximation of Member States’ rules on the obligation of telecommunications services providers to keep information concerning telecommunications in order to ensure that such information is available when it is of significance for a criminal investigation”.[3]

In the United Kingdom, the Home Office recently published a regulatory impact assessment on the effect on industry of requirements for businesses to retain communications data.[4] This followed government proposals for a voluntary code of practice for all telecommunications providers to keep communications data for a maximum recommended period of at most 12 months, although the detailed operation of the retention period had not been finalised. In the face of concerns raised by privacy lobbyists, the government has decided to re-assess the measures. However, the issue of obligatory data retention remains on the government’s legal roadmap and remains in contrast to current data protection requirements. With the growth and development of such anti-terrorism legislation throughout Europe and the rest of the world, the issues associated with data retention are, accordingly, becoming more complex.

The fifth Principle[5] receives its impetus from the view that data protection does not justify storage simply for the reason that you never know whether any data “might perhaps come in handy in any unforeseeable future”.[6] To better understand the contrast between the anti-terrorism proposals on data retention and the fifth Principle, this article investigates the origins of the prohibition on data retention and its subsequent development, examining the trends, disparities and impact of the prohibition as it has developed since 1968.

Piecemeal Origins and Resolution 73(22)

The data retention principle, or data conservation principle as it is sometimes known, has its origins in Recommendation 509 of the Council of Europe on Human Rights and Modern Scientific and Technological Developments adopted in 1968. This acknowledged “the serious dangers for the rights of the individual” which were, in its view, inherent in some aspects of modern technology. Soon after this early statement, in relation to local government data processing,[7] it was stated that there was a need for “data banks” to be restricted to the necessary minimum of information required for specific purposes such as taxation and benefits. Such vague and piecemeal coverage failed to address the reality of the data processing future.

The principle continued to be developed at European level in the early 1970s. In the Annex to Resolution 73(22),[8] one of the principles, Principle 4, which the Council of Europe intended to apply to the storage of personal information in electronic data banks was that “rules should be laid down to specify the periods beyond which certain categories of information should no longer be kept or used”.[9] The statement contained several fundamental flaws. First, no guidance was provided as to what form those “rules” should take, which was hardly a portent of harmonised European rules on the subject. Second, while hindsight makes this comment easier to make, it must have been clear even in 1973 that specific periods could not be laid down for the retention of different categories of data, in isolation, without taking account of the specific people or organisations holding that data and the reasons why that data was being held. In fact, this problem was acknowledged by the Committee of Ministers in the explanatory report attached to the Resolution.[10] Third, it was not clear from the language used whether the limit of time was to apply to passive storage of data or active use of data or both.

Although not specifically excluded in the context of the Resolution, the intention was that the prohibition on data retention would not apply to information which remained indefinitely valid, such as “names, birth dates, diplomas or other qualifications acquired by a person”. However, the Resolution suggested that the rule be implemented by data users by programming computers to erase the pertinent information automatically when the terminal date was reached. The Committee also categorically denied that it was recognising a formal “right of oblivion” for data but was seeking to protect individuals from unreasonably long retention of data that could be harmful.

Principle 7 supplemented Principle 4 insofar as it provided that data users had an obligation to “erase obsolete information”, meaning that data users would have to examine not only the age of data stored by them but also whether data stored by them was obsolete from the outset.

The Resolution applied to any information relating to living natural persons but only if it was stored in “electronic data processing systems used to handle and disseminate that information”. On that basis, the impact of the Resolution was limited, particularly in view of the fact that dissemination of computerised information was not commonplace in industry at the time.

To a large extent, Resolution 74(29)[11] mirrors these words in relation to processing in the public sector. The only difference is by way of clarification, the 1974 version specifying that the rules should specify “time limits” rather than “periods” beyond which information should not be kept or used.

The later Resolution also acknowledged that the public sector could have legitimate grounds for requiring to keep data beyond these time limits. Accordingly, the Council of Europe recognized exceptions where the information was used for statistical, scientific or historical purposes and such use required the data to be “conserved” for an indefinite duration, subject to precautions being taken to protect individual rights of privacy. These exceptions were specifically directed at public authorities and government who, as “have a special duty to preserve certain information for posterity”.

In relation to Principle 4, the guidelines offered a rational basis for the introduction of these data retention obligations. They provided that “individuals have a legitimate interest in seeing certain kinds of information concerning them, particularly that which is harmful to them, wiped off or rendered inoperative after a certain time has passed”.

Following these Resolutions, the Council of Europe noted in connection with the 1980 Convention discussed below that data protection laws had been introduced in Austria, Denmark, France, the Federal Republic of Germany, Luxembourg, Norway, Portugal and Sweden and in Belgium, Iceland, the Netherlands, Spain and Switzerland data protection laws were in an advanced state of preparation.

The foundation stone had been set for the principle of data retention. Not only was it clear by the mid-1970s that some form of prohibition on the retention, conservation or use of data was required, but the Council of Europe had also established its principal justifications for seeking harmonised rules on this: namely that individuals needed to be protected from the harm that could result from the retention of data for unreasonable length of time.

1980 Council of Europe Convention

The 1980 Convention[12] covered “automatic processing” of personal data, including automated data storage, computer processing, alteration, erasure, retrieval and dissemination of data, in both the private and the public sector. Article 5 of the Convention provided that personal data should be “preserved in a form which permits identification of the data subjects for no longer than is required for the purpose for which those data are stored”. This marked an important change in emphasis for the principle of data retention. The principle still required erasure or deletion of personal data but focussed on the form of the data being retained. Data which was held or “preserved” in a form which did not allow identification of data subjects could be retained indefinitely. Even if the data subjects could be identified, the time limit or period during which the data could be processed was as long as was required for the purpose for which the data was stored. This also marked another important development: a rule had been promulgated which provided organisations with a sound basis upon which to set retention periods.

In providing guidance on the terms of the Convention,[13] the Council of Europe noted that this did not mean that “data should after some time be irrevocably separated from the name of the person to whom they relate” but rather that “it should not be possible to link readily the data and the identifiers”. This statement went a long way towards undermining the effective prohibition on data retention that previous recommendations had been leading toward, because it suggested that data could be retained indefinitely as long as it was not immediately or “readily” possible to link individuals with the data held on them. It was also contradicted by future developments as discussed below.

European Developments in the Early 1980s

Of the Recommendations adopted by the Committee of Ministers immediately following the 1980 Convention, several introduced significant developments on the “data retention” theme. Recommendation No. R (81) 1[14] applied to the operation of medical data banks and included a requirement for the operators of such data banks to publicise and adhere to a set of regulations which should include a section on the “long-term conservation of data”. It also provided that, generally, “data relatable to an individual should be kept on record only during a period reasonably useful for reaching their main purpose(s)”, recognising the “threat to privacy if information relating to any individual is allowed to accumulate as the years go by”.[15] The exception provided by the Recommendation applied to the situation where, in the interests of public health or, medical science, or for historical or statistical purposes, it would be justifiable “to conserve medical data that have no longer any immediate use” before and after the death of the relevant data subject.

For the first time, it was recommended that the usefulness of data should be a factor to be considered in defining retention periods and information about the retention policy used by an organisation was to be made available. However, both developments would fail to become fully part of future thinking on data retention.

Data Protection Act 1984

Following its signing of the 1980 Convention, the UK implemented its first data protection legislation in 1984, the Data Protection Act 1984 (the 1984 Act). The equivalent requirement on data retention under the Data Protection Act 1984 to Principle 4 of the 1980 Convention came to be known as the sixth Principle. The sixth Principle provided that “personal data held for any purpose or purposes shall not be kept for longer than is necessary for that purpose or those purposes”.

No guidance to the interpretation of the wording was provided. However, the sixth Principle had been proposed in that form in the 1975 government White Paper[18] found that the “objective” that information should be kept only for as long as it was needed “was not controversial in principle”, but that data users had urged that it should be for them to decide how long they needed to retain data. However, no explicit wording on the sixth Principle was included as part of the Report’s principal legislative recommendations.

This wording marked a significant departure from the direction which the post-1980 developments had been taking and followed a notably different form of wording, as well as challenging some of the themes, proposed by the Council of Europe. First, the movement away from sector specific rules on data retention started by the 1980 Convention was cemented by the 1984 Act. A rule was established across the board for the retention of personal data. Second, the wording used in the 1984 Act avoided the problems associated with whether data permitted the identification of an individual or whether an individual could be readily identified from data in relation to the sixth Principle by introducing the concept of “personal data” which applied throughout the Act. This simplified the sixth Principle. Third, the 1984 Act highlighted the necessity of further processing taking place to justify retention. The developments since 1980 had been moving towards processing being “reasonably useful” to justify retention and the 1984 Act set the standard which would be followed in future developments.

European Developments After 1984

The principle of data retention continued, after 1984, to be developed in a piecemeal fashion at European level. Recommendations were made by the Committee of Ministers of the Council of Europe covering areas as specific and diverse as social security,[19] police records,[20] employment,[21] credit card processing,[22] telecommunications[23] and medical records.[24] Throughout these Recommendations, there were a number of recurring themes.

First, the connection (established by the 1980 Convention and cemented by the 1984 Act) was maintained between the reason for the data being processed and the permitted retention period for that data. A clear link was established during this period between the purposes behind the processing which was taking place and the length of time which that data could lawfully be retained. For example, Recommendation No. R (86) 1 looked at the length of time that was justified for the accomplishment of the relevant task or as being in the interests of the individual data subject. In a similar manner, Recommendation No. R (87) 15 stated that the data would be deleted if no longer necessary for the purposes for which they were stored, for example taking account of the need to retain data in the light of the conclusion of an inquiry into a particular criminal case. Recommendation No. R (89) 2 proposed limiting the storage of personal data by an employer to a period no longer than justified for the recruitment of employees, fulfilment of the contract of employment or management, planning and organisation of work or as was required in the interests of a present or former employee: Recommendations No. R (90) 19 and No. R (97) 5 continued in a similar vein. Recommendation No. R (95) 4 stated that data needed for billing purposes, such as itemised billing data, should not be stored for any longer than strictly necessary for settling an account with the data subject, although this could be extended where such data needed to be kept for a reasonable period to deal with billing complaints or to satisfy legal obligations or the operator or service provider.

During the period after the 1980 Convention and 1984 Act, the trend was very much one of linking the retention period more and more closely with the underlying processing justifications. The developments also continued to move towards there being a need or requirement for the data to be kept rather than it being kept out of simple convenience or usefulness. In each case, taken to the extreme, this indicated that future rules on data retention would became increasingly focussed, discrete and discordant. A particularly dangerous development was the hint that there could be situations where data could lawfully be retained beyond that necessary for the relevant processing purposes where there was an additional interest in doing so, such as service provider interests or where it was adjudged to be in the individual’s interests, to do so.

Second, linked to this, it became clear that fixed retention periods would need to be designated by each individual or type of data user. For example, Recommendation No. R (86) 1 specifically required storage periods to be specified for each category of benefit, while Recommendation No. R (89)2 directed storage periods to be fixed for different categories of data depending on the data in question[25] and Recommendation No. R (90) 19 suggested that further consideration was required in relation to the benefits of setting specific time limits for the retention or conservation of data.[26] In an interesting development, while looking at data kept on the basis that it is required to defend potential legal proceedings such as to prove that the applicant was not rejected on grounds of sex, race or religion, or that correct recruitment and interview procedures were followed,[27] Recommendation No. R (87) 15 provided that such data should only be kept for a reasonable period of time.

These first two developmental trends are linked. Not only were data retention rules to be dependant on the purposes of the processing in question, but also any fixed retention periods which were established following these rules would need to depend on the particular type of data in question. Thrown into the proverbial hat was the additional complication that there could be circumstances in which the retention period established for any given data could be a “reasonable” period of time. By its nature, such a retention period would depend on the particular data to which it related and the other circumstances surrounding the relevant retention.

Third, there was the suggestion that organisations and local authorities should establish review procedures in line with their fixed retention periods. In particular, the Evaluation of the relevance of Recommendation No. R (87) 15 adopted in 1999[28] found that “the law should be explicit about the duration of storage of criminal intelligence”, and proposed a fixed number of years after the last addition of data to that record followed by a periodic review and deletion, where appropriate or a stricter system of obligatory deletion after a certain lapse of time. For the first time, the rules being proposed at a European level were raising awareness of the need for a systematic approach to compliance with the prohibition on data retention – the groundwork for a typical “data culling” process.

Fourth, Recommendation No. R (95) 4 proposed that information on the period over which communications data would be stored should be provided to data subjects, and that data subjects should have the right to require data to be deleted where it has been retained for an excessive period of time. This final development furthered the concept of user empowerment in the debate about data retention, a matter which was more fully extrapolated in the EU Data Protection Directive.

The EU Data Protection Directive

Following discussion in the context of the European Commission about the perceived lack of harmonized laws on data protection within the EU, based partly on the fact that relatively few Member States had signed up to the 1980 Convention, the European Commission proposed and the European Parliament, eventually, adopted the EU Data Protection Directive.[29]

Article 6 of the EU Data Protection Directive required EU Member States to introduce provisions into their national laws to the effect that personal data should be kept in a form which permits identification of data subjects for no longer than is necessary for the purposes for which the data were collected or for which they were further processed. Appropriate safeguards were to be laid down for personal data stored for longer periods for historical, statistical or scientific use.

Before the Directive, the direction which the prohibition on data retention or conservation had taken was divided. The earliest developments aimed for a body of rules aimed at specific types of personal data processing. The first attempt to outline a general rule came with the 1980 Convention. The premise of a general rule on data retention based on the purposes of processing, necessity of storage and retention periods, linked somehow to the requirement for the data subjects to be identifiable from that data, continued throughout the 1980s and early 1990s and culminated with the requirements of the Directive. The idea of user empowerment by means of information given to individual data subjects about retention policies was also introduced. However, during this period several additional themes were championed.

It was suggested that the retention of data had to be justified on the basis of usefulness or that the retention periods fixed by organisations should be reasonable. Data retention became more closely associated with specific processing industries or organisations rather than focussing on the purposes behind each type of processing, with different rules on retention applying to different types of processing. It was recommended that any data retention periods established by data users should depend on the category of data being processed. Lastly, the requirement for organisations to implement retention, review and deletion policies was introduced to the debate. Although not explicitly proposed in terms of the Directive, these themes have continued to have an influence on modern thinking about data retention.

Data Protection Act 1998 – The Current Position

The Data Protection Act 1998 (“the 1998 Act”) was introduced into United Kingdom law on March 1, 2000, in implementation of the Directive. Data retention became the fifth Principle under the 1998 legislation, set out in Paragraph 5 of Part I of Schedule 1 to the 1998 Act. It provides that “personal data processed for any purpose or purposes shall not be kept for longer than is necessary for that purpose or those purposes”. No guidance to the interpretation of the fifth Principle is provided in the Act itself. As with the Data Protection Act 1984, the 1998 Act utilises a defined term for “personal data” to simplify the fifth Principle.

It is clear that the wording from the 1998 Act follows the wording of the 1984 Act very closely. This means that several of themes present in 1980, such as the theme of “personal data” being identifiable with an individual, retention periods based on specific processing purposes and of any retention being “necessary”, have themselves been retained through the EU Data Protection Directive in the 1998 United Kingdom legislation. The legal guidance published by the Information Commissioner in relation to the 1998 Act also makes it clear that, in line with European and UK developments since 1980, the data retention periods which apply in any given circumstance will depend on the category or type of data being processed and the person controlling that processing. In particular, this provides that: (1) data controllers should review personal data regularly and delete information which is no longer required; (2) that statutes may make specific provision for retention of specific data and that guidance was given elsewhere by the Information Commissioner on, for example, retention of CCTV-recorded material; and (3) that the need to keep data should be reviewed when the relationship with the data subject has ended. Elsewhere in the 1998 Act, the concept of user empowerment which was furthered by the EU Data Protection Directive has also been developed.

However, there were a number of areas which have not been expressly clarified in the 1998 Act or the guidance notes on its interpretation. First, the question which was highlighted in the Annex to Resolution 73(22),[30] namely whether specific retention periods need to be applied to both the period of time that data can be retained and the period of time during which data can be used by the data controller. The 1998 Act applies a time limit to the period during which data can lawfully be kept, and the same was true of the EU Data Protection Directive and the 1984 Act. Similarly, the 1980 Convention focussed on the period of time that the relevant data could be “preserved” and European developments in the 1980s looked at the “storage” of data. However, early statements of the principle looked at the length of time that data could be kept or used. It is to be implied that the fifth Principle is intended to apply not only to the storage or retention of data in the widest sense but also active use, though this is not explicit in the 1998 Act or the guidance rendered under that legislation.

Second, in the Explanatory Report on the 1980 Convention,[31] it was proposed that data could be retained indefinitely as long as it was not immediately or “readily” possible to link individuals with the data held on them. The 1998 Act contains detailed guidelines on what constitutes “personal data” for the purposes of the legislation. In particular, data will be capable of identifying a living individual if the data controller can, from that data or from other data in its possession or likely to come into its possession, identify a living individual. The Information Commissioner, in guidance issued under the 1998 Act, has indicated that this goes as far as to say that simple Internet Protocol numbers, gathered by Internet software and which identify the computer which the Internet user is using, may be “personal data” as they identify the individual in cyberspace. There is a clear contrast between this approach and that adopted in 1980. In fact, the author would submit that the concept of identification has been broadened to an indefensible level, increasing the scope of 1998 Act beyond the processing towards which the 1980 Convention and the Directive were addressed. However, that debate is outwith the subject matter of this paper.

Third, in the early 1980s it was proposed that, in certain circumstances, data could be retained provided that it was reasonably useful to do so for certain processing purposes. The 1984 Act moved away from this and introduced, following the 1980 Convention, the necessity for data to be retained pursuant to those processing purposes. Subsequent developments have been broadly aligned with that principle. The wording of the 1998 Act permits no right of retention where mere usefulness, reasonableness or convenience is the only justification for keeping data. However, neither the legislation or subsequent guidance issued by the Information Commissioner has clarified the concept of “necessity”, which remains vague in view of the background to the 1998 Act.

Conclusion

In determining whether retention or conservation is necessary, little guidance is offered by the UK‘s Information Commissioner on the factors which should be taken into consideration by data controllers. It is submitted that, looking at a historical perspective on the fifth Principle, the principle of data conservation, the prohibition on data retention, there are a number of factors which can assist in this respect. These include the nature or type of data being kept, the circumstances of the data controller in question, the reasons for keeping the data, justifications or realistic business needs for retention, specific retention periods set by the data controller, the regularity that the data was reviewed by the data controller, the reasonableness of the retention period in question, the usefulness of the data and potential for harm to result from an excessive retention time.

It is clear from the development of the fifth Principle in the United Kingdom and its equivalent in Europe that two distinct approaches have in the past been taken to how this is implemented in practice. There was a focus, at one point in time, on fixing retention periods for data by industry or organisation type, whereas the initial and most recent developments in this area have focussed on a general rule based on the necessity of retention in view of the processing purposes in question. It is submitted that now is a convenient time for the UK Information Commissioner to issue guidance on the interpretation of the fifth Principle under the Data Protection Act 1998 to add meat to the splintered skeleton created by a long line of European developments since the early 1970s. This guidance should specify whether factors such as the “reasonableness” and “usefulness” of retention have a part to play in the consideration of the 1998 Act and create one singular approach to interpretation of one of the most understated obligations incumbent on a data controller under the Data Protection Act 1998.

Jeremy Warner, Lecturer in IT Law, University of Strathclyde