I had the privilege and pleasure to visit Madagascar in the last two weeks. I had an invitation from Institut Français where I participated in a very interesting panel on “How can Madagascar help us rethink artificial intelligence more ethically?”, with Antonio A. Casilli, Jeremy Ranjatoelina et Manovosoa Rakotovao. I also conducted exploratory fieldwork by visiting a sample of technology companies, as well as journalists and associations interested in the topic.
A former French colony, Madagascar participates in the global trend toward outsourcing / offshoring which has shaped the world economy in the past two decades. The country harnesses its cultural and linguistic heritage (about one quarter of the population still speak French, often as a second language) to develop services for clients mostly based in France. In particular, it is a net exporter of computing services – still a small-sized sector, but with growing economic value.
Last year, a team of colleagues has already conducted extensive research with Madagascan companies that provide micro-work and data annotation services for French producers of artificial intelligence (and of other digital services). Some interesting results of their research are available here. This time, we are trying to take a broader look at the sector and include a wider variety of computing services, also trying to trace higher-value-added activities (like computer programming, website design, and even AI development).
It is too early to present any results, but the big question so far is the sustainability of this model and the extent to which it can push Madagascar higher up in the global technology value chain. Annotation and other lower-level services create much-needed jobs in a sluggish economy with widespread poverty and a lot of informality; however, these jobs attract low recognition and comparatively low pay, and have failed so far to offer bridges toward more stable or rewarding career paths. More qualified computing jobs are better paid and protected, but turnover is high and (national and international) competition is tough.
At policy level, more attention should be brought to the quality of these jobs and their longer-term stability, while client tech companies in France and other Global North countries should take more responsibility over working conditions throughout their international supply chains.
What shapes differences in how people get paid, are deemed productive, or receive respect? Alongside traditional explanations of social inequalities such as class, gender, age, disability, race, migration status, rural vs. urban residence, and others, a recent literature highlights the effects of digital divides. The digitally resourced have more opportunities across all life spheres, from consumption to education, work, and health. Ironically, though, digital technologies also generate new vulnerabilities by generalizing low-paid and contingent work. Digital labour platforms like Uber, Deliveroo and Upwork use data and algorithms to match clients with workers, construed as independent contractors, for one-off ‘gigs’ without any long-term commitment. These workers are largely exposed to the vagaries of the market and have limited or no social protection, although increasing efforts aim to bring labour law to bear on platforms.
Growing concerns that platform workers compare unfavourably to conventional employees have already attracted significant research and policy attention. But more remains to be done to fully understand how the recent rise of labour platforms has undermined the relationship between digitization and inequalities, adding a layer of complexity. Scattered, but growing evidence indeed suggests that platforms may be accelerating transmission to digital worlds of ’legacy’ inequalities for example vis-à-vis race and gender, while also fostering the proliferation of ’emerging’ inequalities that diminish users’ agency and augment the power of technology creators and big-tech multinationals. Especially platforms for remote online-only labour change the geographical scale at which these questions arise, projecting workers toward a competitive planetary market that relentlessly selects winners and losers.
To tackle these questions, I’m happy and honoured to announce that I have just been awarded a major grant (almost 570k euros, at marginal cost) by the French National Agency for Research (ANR) for a new 4-year study called VOLI: Voices from Online Labour. As a team effort that builds on a solid record of interdisciplinary collaborations, VOLI innovatively combines hypotheses and methods from sociology and neighbouring disciplines, notably large-scale corpus linguistics (I’ll explain why below), and relies on speech technology and artificial intelligence to tackle the rising economic risks that coalesce around the nexus between online platform labour, digitization, and social inequalities. The project leverages the power and potential of the very digital tools whose societal effects it studies, to develop an original and potentially transferable methodology.
The innovative idea that underpins the project is to tackle the problem through language, benefiting from recent advances in linguistics research and its capacity to recast methods and tools from artificial intelligence in a broad sense – including speech and language technology and machine learning techniques – to capture features and processes that used to escape its traditional methods. Despite the importance of linguistic tasks (such as translation, transcription, writing, and editing) in online labour platforms, linguistic methods have never been applied to the study of these workers before, and thus are best positioned to bring fresh insight. To this end, we have assembled a team composed of speech technology scientists, computational linguists specialized in multilingual and large-scale corpora analysis, and computational, digital, and labour sociologists. Expected results sustain our ambition to devise policy solutions to mitigate the effects of inequalities, and to support the individuals and groups that accumulate multiple sources of disadvantage.
To harness our previous research experience and ensure continuity, we focus on so-called ’micro-work’, the necessary but inconspicuous contribution of low-paid masses who annotate, tag, label, correct and sort data to fuel the digital economy, especially artificial intelligence. Because it is performed remotely and can be allocated to providers worldwide, micro-work differs from location-based platform ’gigs’ such as delivery and transport. It also differs from online-only jobs for freelancers, for example in computer programming and design, insofar as its extreme segmentation and standardization allow dispersing tasks to an undefined crowd instead of a selected individual (whence the alternative denomination of ’crowdwork’). Micro-tasks include, for example, recording one’s voice while reading aloud a sentence, labelling files, translating short bits of text, classifying contents in an image or webpage. They perform essential functions in the development of machine learning and artificial intelligence, from data generation and enrichment to quality controls of automated outputs. We give voice to these workers, often invisibilized by the automation narratives popular in the technology industry, in that we interview them about their lived experience, aspirations, motivations and perhaps regrets; and we rely on their voices as data for the simultaneous development of sociology, linguistics, and artificial intelligence (specifically, speech recognition) itself.
Indeed while bringing to the next level our sociological knowledge of the linkages between micro-work and digital inequalities, the methods that will be developed within this highly interdisciplinary project advance the study of the factors driving speech variation within the discipline of linguistics, augmenting language corpora with rich sets of metadata from sociological surveys, while also building and testing new and improved tools for automated transcription, with potential commercial applications.
I am the PI of the VOLI project which involves four research centres within France:
AI is not just a Silicon Valley dream. It relies among other things, on inputs from human workers who generate and annotate data for machine learning. They record their voice to augment speech datasets, transcribe receipts to provide examples to OCR software, tag objects in photographs to train computer vision algorithms, and so on. They also check algorithmic outputs, for example, by noting whether the outputs of a search engine meet users’ queries. Occasionally, they take the place of failing automation, for example when content moderation software is not subtle enough to distinguish whether some image or video is appropriate. AI producers outsource these so-called “micro-tasks” via international digital labor platforms, who often recruit workers in Global-South countries, where labor costs are lower. Pay is by piecework, without any no long-term commitment and without any social-security scheme or labor protection.
In a just-published report co-authored with Matheus Viana Braz and Antonio A. Casilli, as part of the research program DiPlab, we lifted the curtain on micro-workers in Brazil, a country with a huge, growing, and yet largely unexplored reservoir of AI workers.
We found among other things that:
Three out of five Brazilian data workers are women, while in most other previously-surveyed countries, women are a minority (one in three or less in ILO data).
9 reais (1.73 euros) per hour is the average amount earned on platforms.
There are at least 54 micro-working platforms operating in Brazil.
One third of Brazilian micro-workers have no other source of income, and depend on microworking platforms for subsistence.
Two out of five Brazilian data workers are (apart from this activity) unemployed, without professional activity, or in informality. In Brazil, platform microwork arises out of widespread unemployment and informalization of work.
Three out of five of data workers have completed undergraduate education, although they mostly do repetitive and unchallenging online data tasks, suggesting some form of skill mismatch.
The worst microtasks involve moderation of violent and pornographic contents on social media, as well as data training in tasks that workers may find uncomfortable or weird, such as taking pictures of dog poop in domestic environments to train data for “vacuuming robots”.
Workers’ main grievances are linked to uncertainty, lack of transparency, job insecurity, fatigue and lack of social interaction on platforms.
We examine the implications of the use of digital micro-working platforms for scientific research. Although these platforms offer ways to make a living or to earn extra income, micro-workers lack fundamental labour rights and ‘decent’ working conditions, especially in the Global South. We argue that scientific research currently fails to treat micro-workers in the same way as in-person human participants, producing de facto a double morality: one applied to people with rights acknowledged by states and international bodies (e.g. Helsinki Declaration), the other to ‘guest workers of digital autocracies’ who have almost no rights at all.
Today’s artificial intelligence, largely based on data-intensive machine learning algorithms, relies heavily on the digital labour of invisibilized and precarized humans-in-the-loop who perform multiple functions of data preparation, verification of results, and even impersonation when algorithms fail. This form of work contributes to the erosion of the salary institution in multiple ways. One is commodification of labour, with very little shielding from market fluctuations via regulative institutions, exclusion from organizational resources through outsourcing, and transfer of social reproduction costs to local communities to reduce work-related risks. Another is heteromation, the extraction of economic value from low-cost labour in computer-mediated networks, as a new logic of capital accumulation. Heteromation occurs as platforms’ technical infrastructures handle worker management problems as if they were computational problems, thereby concealing the employment nature of the relationship, and ultimately disguising human presence. My just-published paper highlights a third channel through which the salary institution is threatened, namely misrecognition of micro-workers’ skills, competencies and learning. Broadly speaking, salary can be seen as the framework within which the employment relationship is negotiated and resources are allocated, balancing the claims of workers and employers. In general, the most basic claims revolve around skill, and in today’s ‘society of performance’ where value is increasingly extracted from intangible resources and competencies, unskilled workers are substitutable and therefore highly vulnerable. In human-in-the-loop data annotation, tight breakdown of tasks, algorithmic control, and arm’s-length transactions obfuscate the competence of workers and discursively undermine their deservingness, shifting power away from them and voiding the equilibrating role of the salary institution.
Following Honneth, I define misrecognition as the attitudes and practices that result in people not receiving due acknowledgement for their value and contribution to society, in this case in terms of their education, skills, and skill development. Platform organization construes work as having little value, and creates disincentives for micro-workers to engage in more complex tasks, weakening their status and their capacity to be perceived as competent. Misrecognition is endemic in these settings and undermines workers’ potential for self-realization, negotiation and professional development.
My argument is based on original empirical data from a mixed-method survey of human-in-the-loop workers in two previously under-researched settings, namely Spain and Spanish-speaking Latin America.
I’m glad to announce the publication of new research, as a chapter in the fabulous Digital Work in the Planetary Market, a volume edited by Mark Graham and Fabian Ferrari and published in open access by MIT Press.
The chapter, co-authored with Antonio A. Casilli, starts by recalling how In spring 2019, public outcry followed media revelations that major producers of voice assistants recruit human operators to transcribe and label users’ conversations. These high-profile cases uncovered the paradoxically labor-intensive nature of automation, ultimate cause of the highly criticized privacy violations.
The development of smart solutions requires large amounts of human work. Sub-contracted on demand through digital platforms and usually paid by piecework, myriad online “micro-workers” annotate, tag, and sort the data used to prepare and calibrate algorithms. These humans are also needed to check outputs – such as automated transcriptions of users’ conversations with their virtual assistant – and to make corrections if needed, sometimes in real time. The data that they process include personal information, of which voice is an example.
We show that the platform system exposes both consumers and micro-workers to high risks. Because producers of smart devices conceal the role of humans behind automation, users underestimate the degree to which their privacy is challenged. As a result, they might unwittingly let their virtual assistant capture children’s voices, friends’ names and addresses, or details of their intimate life. Conversely, the micro-workers who hear or transcribe this information face the moral challenge of taking the role of intruders, and bear the burden of maintaining confidentiality. Through outsourcing, platforms often leave them without sufficient safeguards and guidelines, and may even shift onto them the responsibility to protect the personal data they happen to handle.
Besides, micro-workers themselves release their personal data to platforms. The tasks they do include, for example, recording utterances for the needs of virtual assistants that need large sets of, say, ways to ask about the weather to “learn” to recognize such requests. Workers’ voices, identities and profiles are personal data that clients and platforms collect, store and re-use. With many actors in the loop, privacy safeguards are looser and transparency is harder to ensure. Lack of visibility, not to mention of collective organization, prevents workers from taking action.
Note: Description of one labor-intensive data supply chain. A producer of smart speakers located in the US outsources AI verification to a Chinese platform (1) that relies on a Japanese online service (2) and a Spanish sub-contractor (3) to recruit workers in France (4). Workers are supervised by an Italian company (5), and sign up to a microtask platform managed by the lead firm in the US (6). Source: Authors’ elaboration.
These issues become more severe when micro-tasks are subcontracted to countries where labor costs are low. Globalization enables international platforms to allocate tasks for European and North American clients to workers in Southeast Asia, Africa, and Latin America. This global labor arbitrage goes hand in hand with a global privacy one, as data are channeled to countries where privacy and data protection laws provide uneven levels of protection. Thus, we conclude that any solution must be dual – protecting workers to protect users.
There are differences of course: theirs is a large multi-country study while we focused on one national setting (France). Also: Kässi et al. consider online labour in general, while we looked specifically at micro-work.
Nevertheless, there are striking similarities. Both studies included larger as well as smaller and more peripheral platforms, often left aside in previous research. Both started from the numbers of registered users declared by the platforms in scope, although this is likely an upper bound. Indeed registering may not mean using, and for example researchers (like ourselves) and journalists would register only to observe, especially when registration is open and easy.
Also, both studies used web traffic analysis data but for different purposes. We used them as an estimate of minimally active users – those who connect at least monthly, as per the definition given by the providers of these data. For the platforms we observed, these numbers tend to be lower than registrations.
Instead, Kässi et al. have used these data to assess registration numbers for the platforms that do not report them. My first reaction would be to think their estimates are likely a lower bound. But presumably their use of a mix of sources, and the seriousness and caution with which they have conducted their estimate, provide enough correction.
Finally, both studies attempted to correct estimates downward by taking into account multi-homing – the tendency of users to rely on multiple platforms. The coefficient of Kässi et al. is 1.83, ours was 1.27. The gap is due to the fact that we focused only on micro-work: if we had counted participation across all types of online labour platforms, our coefficient would be just below 2 – not far from theirs! Kässi et al. also correct for the possibility of multiple workers using a single account, which we did not observe in our French sample. One might imagine other corrections depending on observed usages. For example my ongoing Latin American study of micro-workers suggests that there are unofficial sales and purchases of highly rated platform accounts, more likely to access better-paying tasks – again, something we did not observe in France. Kässi et al. rightly note that all these corrections come from ad hoc surveys and should be interpreted with caution.
Overall, I would say that both studies point to the need to put in place new and creative methods to account for these new forms of labour that traditional statistical studies fail to capture well. The price to pay, as both studies stress, is a high degree of uncertainty. I also dare suggest that both are mixed-method studies: while the design is essentially quantitative, input from smaller and even qualitative research is crucial – for example to get insight into multi-homing and multi-working.
Before concluding, let us recall the key results. Kässi et al. reckon that there are 163 million freelancer profiles registered on online labour platforms globally, of whom approximately 19 million have worked at least once, and 5 million work more intensely. We estimated that approximately 260,000 French residents are registered with micro-work platforms, of whom some 50,000 are ‘regular’ workers who do micro-tasks at least monthly, and a more restrictive measure of ‘very active’ workers would decrease this figure to 15,000.
Are these numbers large or small? Curiously, our French study attracted both criticisms: some worried that we might be overstating the importance of micro-work, others wondered why we bothered for such a tiny part of national GDP. It is not easy to answer this question, as the answer depends on the perspective taken and the goals – the same numbers would mean different things to policymakers and researchers, for example. Nevertheless, I think that the point that is important to all, is to say that this population exists and needs attention – despite its limited visibility and the fuzzy boundaries that make it so difficult to assess its size.
The research project TRIA (from its French title “Le TRavail de l’IA: éthique et gouvernance de l’automation”) is a study of the production systems of artificial intelligence. We investigate “micro-work” platforms, which allocate small standardized tasks to crowds of providers, and use the outputs of their work to prepare and annotate data for machine learning algorithms. We study the ramifications of this phenomenon in Spanish-speaking countries, which have remained under-researched so far despite their strong participation. With data from an empirical survey already started in 2020, and to be analyzed through mixed methods (including advanced NLP techniques), we will address important issues related to digital platform governance, online work ethics, and consequences (e.g. in terms of bias) of the use of these humans in the production of artificial intelligence. Funded by the French National Center of Scientific Research (CNRS), the TRIA project resembles research teams in the Paris and Rennes regions in France, as well as partners in Spain (Barcelona and Valencia) and Canada (Toronto).
We are currently looking for a student intern to help us set up a survey targeting micro-workers in Spain and Spanish-speaking Latin American countries. He/she will help us to :
update an inventory of micro-work platforms operating in Spanish-speaking countries, a first version of which was created in 2020;
launch a replication of the online questionnaire, already fielded on Microworkers.com, on another micro-work platform;
to liaise and ensure communication between the project teams.
The applicant should :
be enrolled in the first or second year of a master’s degree in social science (like sociology, political science, management or economics) ;
have skills in the design and/or execution of questionnaire surveys;
have some prior knowledge of, or at least interest in, the transformations of work and/or the societal effects of digital technology;
be able to work independently, with advanced relational skills;
have a fairly good command of French or English, and at least a basic knowledge of Spanish.
More information is available in the enclosed job description.
With Antonio A. Casilli, I will be presenting a paper tomorrow at the Crowdworking Symposium organized by the University of Paderborn, Germany. Unfortunately, we will participate only online because of the health situation.
Our mini-paper (3 pages), entitled ‘Portraits of micro-workers: The real people behind AI in France’, is available here.
What is it about? ‘Micro-work’ consists of fragmented data tasks that myriad providers execute on online platforms. While crucial to the development of data-based technologies, this little visible and geographically spread activity is particularly difficult to measure. To fill this gap, we combined qualitative and quantitative methods (online surveys, in-depth interviews, capture-recapture techniques, and web traffic analytics) to count micro-workers in a single country, France. On the basis of this analysis, we estimate that approximately 260,000 people are registered with micro-work platforms. Of these some 50,000 are ‘regular’ workers who do micro-tasks at least monthly, and we speculate that using a more restrictive measure of ‘very active’ workers decreases this figure to 15,000. This analysis is important to better understand platform labour and the labour in the digital economy that lies behind artificial intelligence.
Data Big and Small 