The “open data” movement is radically transforming policy-making. In the name of transparency and openness the UK, US and other governments are releasing large amounts of records. It is a way to hold the government to account: in UK for example, all lobbying efforts in the form of meetings with senior officers are now publicly released. Data also enable the public to make more informed decisions: for example, using apps from public transport services to plan their journeys, or tracking indicators of, say, crime or air pollution levels in their area to decide where to buy property. Data are provided as a free resource for all, and businesses may use them for profit.
The open data movement is not limited to the censuses and surveys produced by National Statistical Institutes (NSIs), the public-sector bodies traditionally in charge of collecting, storing and analyzing data for policy purposes. It extends to other administrations such as the Department for Work and Pensions or the Department for Education in the UK, which also gather and process data, though usually through a different process, not using questionnaires but rather registers.
Some of these data, especially those from registers, are difficult to publish as open, because they contain personal information about individuals, and these individuals may be re-identified by data recipients: This would violate the confidentiality pledge that public bodies are legally bound to offer. Imagine how you would feel if your health records, or your tax returns, were there for all to see!
Confidentiality protection thus seems (and to an extent, is) at odds with the open data principle. How to reconcile the two is a major conundrum for today’s public-sector data producers.
The search for a possible solution could draw on existing experience in access to individual data: Indeed legislation already permits access, notably for scientific and statistical research purposes. Of course, access cannot be uncontrolled and since the mid-1990s, advanced IT-based solutions have been developed to provide research access while minimizing the risk of re-identification of individuals.
Today, just as open data are driving fast-paced change in availability, content, and format of data release for the general public, there is no less pressure on NSIs to enhance, modernize and expand their arrangements for scientific access! While this may seem a challenge for NSIs, the experience already gained in this area at least in some countries is a valuable resource that can provide guidance for the future.
In Europe, examples of well-established solutions using state-of-the-art technology are the research data services of countries like France, Germany and UK, but also those of some smaller states like Slovenia. They can set the example for countries where data openness has been more limited so far. Their experience in data release may also help public-sector agencies gain self-confidence and take on board the transparency agenda more broadly. That would be a first step towards the goal of enlarging the horizon – finding ways to build trust among the general public, not just tiny groups of highly-trained specialists, while still sticking to strict confidentiality protection rules.
In sum, research access is no longer a service restricted to a particular category of citizens – researchers – but may serve as a basis for something larger and more ambitious, in pursuit of the public good. An important reason why research access should be high on the data policy agenda today – more than ever before.
Data Big and Small 