A social scientist's venture into big data, while still learning much from surveys and fieldwork
In the midst of the chaos and sadness of the past week, a more leisurely note: the first of a new “Databeers” series of events in London yesterday evening, following a format that has been experiencing a huge success in Spain, Italy and other countries. The event is very informal, and getting to know other data enthusiasts is the main goal. There are a few flash talks with free beers and networking time.
The next Data Beers London event is on 25 February 2016.
Yesterday at Biblithèque Nationale de France, I took part in a panel discussion on longue durée in history, organised by the Revue Annales – Histoire et Sciences Sociales. Of course I am not a historian, and I wouldn’t be able to tell whether one interpretation of longue durée is better than another. But historians are now raising questions that are common to the social sciences and humanities more generally: how to benefit from big data and how to re-think the political engagement of the researcher. So I was there to talk about big data and how they change not just research practices and methods, but also researchers’ position relative to power, politics, and industry. This questions cross disciplinary boundaries, and all may benefit from dialogue.
What ignited the historians’ debate was an attempt by two leading scholars, David Armitage and Jo Guldi, to restore history’s place as a critical social science, based on (among other things) increased availability of large amounts of historical data and the digital tools necessary to analyze them. Before their article in Annales, they published a full book in open access, the History Manifesto, where they develop their argument in more detail. Their writing is deliberately provocative, and indeed triggered strong (and sometimes very negative) reactions. Yet the sheer fact that so many people took the trouble to reply, proves that they stroke a chord.
What do they say about big data? They highlight the opportunity of accessing large and rich archives and to expand research beyond any previous limitations. Their enthusiasm may seem excessive but it is entirely understandable insofar as their goal is to shake up their colleagues. My approach was to take their suggestion seriously and ask: what opportunities and challenges do data bring about? How would they affect research, especially for historians?
The recent VW emissions scandal says it all: even a large company can’t get away with behaviours that disrespect key societal values. Protection of the environment is among these values today, so much so that not only public authorities step in to defend it, but even markets punish the transgressors.
Data protection is not (yet) such a value. Admittedly, some associations, individuals, and government officials fight for it, but the larger public is still unsure. It’s not that people don’t care, but that uncertainty as to what data are actually collected, for what usages, and by whom, is overwhelming; and it becomes difficult to identify the best course of action.
In this context, a new initiative is most welcome: an open letter on “Data for Humanity“, initiated by two scholars of the University of Frankfurt, pleads for a more responsible use of data. The message is simple: Do no harm. And if you can, on top of it, do something good. It’s so simple, and so necessary.
Sure, the world won’t change after this letter, but it will be a first step. Even the promotion of environmental protection started with simple, basic declarations, 30-40 years ago; and it was by insisting and perseverating, that it finally gained the conscience of everybody.
National Statistical Institutes (NSIs) have long been the recognised repositories of all socio-economic information, mandated by governments to collect and analyse data on their behalf. The development of big data is shaking this world. New actors are coming in and commercially-oriented, privately-produced information challenges the monopoly of NSIs. At the same time, NSIs themselves can tap into digital technologies and produce “big” data. More generally, these new sources offer a range of opportunities, challenges and risks to the work of NSIs.
The Statistical Journal of the IAOS, the flagship journal of the International Association for Official Statistics, has published a special section on big data – of particular interest to the extent that it is free of charge!
Fride Eeg-Henriksen and Peter Hackl introduce this special section by defining big data and emphasising its interest for official statistics. But it is crucial, albeit admittedly not easy, to separate the hype around big data from its actual importance.
The other papers are concrete examples of how big data may be integrated into official statistics:
Continue reading “New publications on big data and official statistics”
The rise of digital data, particularly data from the internet, is to be understood in social relational perspective. Online interactions – from email exchanges to use of VOIP services and participation in social media such as Facebook, Twitter and LinkedIn – make people’s social connections explicit and visible. The “social network”, once a metaphor used only in a small sub-field within sociology, is now familiar to everybody as the archetype of computer-mediated social interaction. Digital devices systematically record network structures, so that social ties become an essential part of every individual profile, and users are more and more aware of them.
One consequence of this is the booming popularity of network analysis concepts, which support the algorithms that handle digital data: for example, centrality measures are at the heart of search engine functionalities, and transitivity measures found “friend-of-a-friend” algorithms in social media. In passing, social network analysis itself which had been originally developed for small-sized, non-digital datasets (like surveys about friendship in schools) has undergone a major upgrade to account for social data from the web.
More importantly, the relational nature of digital data and the underlying possibilities to use social network analysis, open up new avenues for data collection. If user B publishes a post on, say, their Facebook wall, comments and “likes” received from their friends A, D and E will be connected to the profile of B, accessible and visible from it; in other words, it is possible to retrieve information on A, D or E through the profile of just B. In general social networks, a friend of my friend is my friend; in digital networks, the data of my friends are my data.
The “Impact of Social Science” blog of the London School of Economics has, in the past few weeks, published a series on “Philosophy of data science“. Each installment is an interview conducted by sociologist Mark Carrigan with a key contributor to the social science reflection on data.
I gave a presentation on the topic of “Data and social networks: empowerment and new uncertainties” at the Better Decisions Forum on Big Data and Open Data that took place in Rome on 12 November 2014. The event brought together six speakers from different backgrounds on a variety of topics related to data, and participants were businesspeople, public administration managers, journalists, data and computer scientists.
Here is a video of my talk:
Unfortunately as you will have noticed, the slides are not always very clearly visible, so it’s better to download them from their original source:
My interview before my talk:
See? I am trying to stick to my 1st-January commitment of blogging more this year…
I was back last week from the annual conference of British Sociological Association (BSA) in Leeds, and as usual, I try to put down my impressions as long as they’re still fresh in my mind. I wasn’t very quick, though, and the BSA’s members newsletter has already come out with comments and short reports about the plenaries, the prizes awarded, and the conference overall. While the conference is described as having been “very vibrant and sociable”, with “exciting conversations” during the breaks and a “diverse mixture of topics” that “reflected the breadth of interests”. My own feelings, I confess, are a bit more mixed.
In 2012, the BSA conference was followed by a lively debate after an article, by Aditya Chakrabortty on the Guardian, where he complained about the discipline’s lack of engagement with the financial crisis. He pointed to the BSA press releases featuring research on “older bodybuilders”, and to time devoted to the “holistic massage industry” at the conference, as evidence of what he saw as a retreat from public space. The BSA took the criticism very seriously and, apart from responding to the Guardian, put in place a massive effort to encourage public engagement. The 2013 conference was entitled “Engaging Sociology” and many sessions were dedicated to showing that the profession means it. Confrontation and comparison with economics was open and clear. A major project on social class was presented with all honours. The Sociology journal released a call for papers for a special issue to “Sociology and the Global Economic Crisis”.
This year, the “Changing Society” title aimed to stress continuity with last year’s efforts; yet it seems to me that we are back to business as usual. I had the impression that many paper presentations were on topics similar to the body builders and massage that Chakrabortty talked about. That’s why, as I said, my feelings are mixed.
On Friday last week, the British Sociological Association (BSA) held an event on “The Challenge of Big Data” at the British Library. It was interesting, stimulating and relevant – I was particularly impressed by the involvement of participants and the very intense live-tweeting, never so lively at a BSA event! And people were particularly friendly and talkative both on their keyboards and at the coffee tables… so in honour of all this, I am choosing the hashtag of the day #bigdataBL as title here.
(Visualisation: http://www.digitalcoeliac.com/)
Some highlights:
Data are not a new ingredient of socio-economic research. Surveys have served the social sciences for long; some of them like the European Social Survey, are (relatively) large-scale initiatives, with multiple waves of observation in several countries; others are much smaller. Some of the data collected were quantitative, other qualitative, or mixed-methods. Data from official and governmental statistics (censuses, surveys, registers) have also been used a lot in social research, owing to their large coverage and good quality. These data are ever more in demand today.
Now, big data are shaking this world. The digital traces of our activities can be retrieved, saved, coded and processed much faster, much more easily and in much larger amounts than surveys and questionnaires. Big data are primarily a business phenomenon, and the hype is about the potential gains they offer to companies (and allegedly to society as a whole). But, as researcher Emma Uprichard says very rightly in a recent post, big data are essentially social data. They are about people, what they do, how they interact together, how they form part of groups and social circles. A social scientist, she says, must necessarily feel concerned.
It is good, for example, that the British Sociological Association is organizing a one-day event on The Challenge of Big Data. It is a good point that members must engage with it. This challenge goes beyond the traditional qualitative/quantitative divide and the underrepresentation of the latter in British sociology. Big data, and the techniques to handle them, are not statistics, and professional statisticians have trouble with it too. (The figure below is just anecdotal, but clearly suggests how a simple search on the Internet identifies Statistics and Big Data as unconnected sets of actors and ties). The challenge has more to do with the a-theoretical stance that big data seem to involve.
Data Big and Small 