Invisible Gigs: Researching Microwork in Canada

Invisible Gigs: Researching Microwork in Canada

A review of what we know

“It’s like playing Etch A Sketch or a video game where you colour in certain dots.”

That’s how Vancouver microworker Harry K. describes searching medical images for breast cancer cells, then manually tagging them. As he explains to a Wired writer, it’s “tedious and detailed.”

The task’s especially slow for a nonspecialist like Harry: he works full-time at a large packaging company. “A couple of training screens” and a quiz were the only preparation required. 

Harry found this work on what’s now called Figure Eight. Like Amazon Mechanical Turk, the platform helps easily outsource low-skill digital tasks.

The image-tagging “requester” here was a Harvard medical professor who co-founded PathAI. (The cancer diagnosis startup has since raised over $90m in funding.) Crowdworkers like Harry then competed to snatch up the tasks, as they would any appealing work on the platform.

Why microwork matters for Canada

The global market for machine-learning related data annotation grew 66% to $500 million in 2018 and is set to more than double by 2023.

— Cate Cadell, Faces for Cookware

Microwork quietly keeps the digital economy going. It’s vital for everything from training AI to bridging “automation’s last mile,” as when Uber uses humans to confirm driver identity. But many people don’t even know the workers handling such tasks exist.

This trend seems particularly significant for Canada. Employers here are hiring ever-more freelance digital workers, according to the Online Labour Index. It measures postings on five major freelance platforms to track broader trends. From 2016 to 2018, the indicator showed a 30% increase in global demand for online labour.

In this same period, though, postings from Canadian requesters actually doubled. And the sort of research fuelling Toronto’s AI boom has long relied on microwork. 

Economic opportunity or threat?

It can nevertheless be difficult to evaluate the possibility of Canadians increasingly hiring or working through microwork platforms. Even Harry’s situation can quickly start to feel ambiguous.

His pay for the cancer image processing was incredibly low. A pathologist in the requester’s home country might make $80 USD per hour for such tagging. But Harry earned a cent per minute. Even outside Vancouver, the hourly equivalent doesn’t come close to covering cost of living anywhere in Canada.

But Harry also explains he quickly abandoned the cell-tagging task for other work. So we can’t tell how much it represents his broader microwork experience — or what the realistic alternatives might be.

He clearly got enough out of it previously to continue. Harry actually estimates he’s completed 25,000 other tasks. The earnings flexibly supplement his wages, helping pay legal bills and child-support from a bad divorce years earlier.

“If I had the opportunity to not do my day job and do crowdworking instead,” he ultimately says, “I would.”

Such tensions can make microwork difficult to interpret. Is it a creative hustle that can “provide a steady income,” or “a new kind of poorly paid hell”? Or something else entirely? 

The bigger picture

To properly evaluate any answer, we first need some basic facts about microworkers. How many are there, for example? What are they paid? And who are they? 

Microwork markets are complex: highly varied, rapidly evolving and often opaque. So without such baseline information, discussion collapses easily into people talking past one another.

That’s partly because it’s easy to find support in such markets for a wide range of sweeping judgements about them.

This isn’t just a matter of individual anecdotes like Harry’s. As discussed later, whole microworker subgroups and task types can have very different features. So it’s easy to stumble by accurately describing one facet of the market, then simply assuming others are similar.

We wouldn’t make sweeping generalizations about 1980s “telephone workers” based on just the experiences of administrative assistants, hotline psychics or commodities traders. But similar mistakes regarding today’s microworkers can be tempting.

Research on the economics of crowdsourcing has been, so far, remarkably thin…considering the growing size of the platform economy.

— Michele Cantarella & Chiara Strozzi, Workers in the Crowd

So we began this Toronto Workforce Innovation Group project by gathering all the available relevant research.

We soon realized there simply aren’t any studies on microwork in the GTA. This broadened our focus to the national level. And still, the basic fact is this: no one knows much. New research is bringing key points into clearer focus, though.

So this post reviews what combining all available sources can tell us about Canadian microwork. Along the way, it also highlights some key research findings (and challenges) about these new markets more generally.

How many microworkers are there in Canada?

There’s almost no direct, large-scale research about microwork in this country. This makes our most promising option the Canadian Survey of Consumer Expectations.

This Bank of Canada (BoC) questionnaire goes out quarterly to the heads of 2,000 households, selected to represent adults 18 and older in this country. For 2018, the surveys also included an informal work supplement. Like recent U.S. efforts, these extra questions focused on “gigs” or “side jobs” that otherwise don’t register in traditional employment statistics.

The supplement covered pay for offline services, from house-painting to eldercare. But it also asked about online earnings. Options here included driving for services like Uber, creating content (e.g. YouTube videos), and “getting paid to complete tasks online through websites such as Amazon Mechanical Turk, Fiverr or similar sites.”

This question makes the survey one of very few sources that specifically investigates Canadian microwork. And on average, 4% of respondents said they’d completed tasks online for money.

Talking clearly about digital work

If nationally representative, the Canadian Survey of Consumer Expectations responses suggest just over 1,195,000 people earned microwork income in 2018.

More details would obviously be necessary to make proper sense of that basic number. But complications set in even earlier.

People often don’t understand themselves in the language of labour analysts. Pew found that by the end of 2015, e.g., 89% of U.S. adults still weren’t familiar with the term “gig economy.” And the intersection of informal and digital work is complex enough to trip up even specialists.  

All this can make interpreting survey results challenging.

The U.S. Bureau of Labour Statistics sought information similar to the BoC survey with their 2017 Contingent Worker Supplement, for instance. But many respondents clearly misunderstood questions about digital platforms and “electronically mediated” work. Ultimately, Bureau employees had to manually check and recode responses.

The definition of microwork is also notoriously blurry, making this problem worse.

One basic challenge is that many different kinds of tasks can qualify. That’s why the recent International Labour Organization (ILO) report on microwork uses a 10-category classification. But looser, less standardized classifications remain common. 

A 2019 Boston Consulting Group survey on the gig economy shows the research challenges this can cause. Their report doesn’t distinguish microwork from other “low-skill” platform-based services, such as house cleaning. So it ultimately can’t tell us much about either type of work.

Small differences with big implications

Such blurring is also a risk for the BoC’s microwork question.

Among informal income options, it lists “completing tasks online” separately from “responding to surveys.” And BoC staff excluded the latter from their analysis of informal work.

But the ILO report just mentioned notes that 65% of microworkers earn income by taking surveys. This makes it the most common sort of work by a full 19% on platforms such as Amazon Mechanical Turk — also known as “AMT.”

So what might including online survey-taking do to this estimate of Canadian microworkers? Based on the BoC publication, it’s quite hard to know.

(Partly, that’s because the question actually combines completing surveys on- and offline. So even full access to the response data wouldn’t easily resolve things.)

But let’s say 0.05% of respondents picked the survey-specific option to represent their digital work, while not selecting “completed tasks online.” On average, this would take just one person per questionnaire they sent.

We might then increase our estimated population of Canadian microworkers by the same 0.05%. This would translate nationally into nearly 15,000 people — higher than the total 2016 census population of Fort Eerie, Ontario.

The Canadian Internet Use Survey

Such attempts to glean microwork insights from broader surveys also raise a more basic issue.

Even with many respondents, it can be hard to know they truly represent Canadians as a whole. For instance, the BoC survey’s online-only format might well leave platform workers over-represented.

On average, respondents each survey included in this analysis would have averaged 80 Canadian microworkers. But that number’s far too small to support generalizations about the microworkforce of nearly 1.2 million the BoC survey answers imply. We certainly can’t know anything about the socio-demographic details that matter most for TWIG’s project.

The challenge of representativeness is even clearer in our only other direct source for national numbers on microwork: Statistics Canada’s latest Canadian Internet Use Survey (CIUS).

The 2018 CIUS draws on responses from just over 14,400 people, secured by mail and phone. The survey had also just been updated for currency and clarity.

It directly asked participants if they’d earned money in the last year from “crowd-based microwork (e.g., Amazon Mechanical Turk, Cloudflower [sic]).” The survey also clearly distinguished this option from earnings through “online freelancing (e.g., Upwork, Freelancer, Catalant, Proz, Fiverr).”

Despite including the question, Statistics Canada didn’t release any data on microwork with the main CIUS results. And when asked, they explained the responses received didn’t allow enough certainty for publication. Any conclusions would simply be too unreliable.

More recent publications suggest Statistics Canada may ultimately be able to provide far richer insights. (See the last sections of this post for more on that.) But right now, we simply have no reliable information about how many microworkers there are in Canada — let alone Toronto.

How much does microwork pay?

Even without knowing their exact number, we hoped estimating Canadian microworkers’ earnings from international sources might still be possible. But this also quickly gets murky.

It’s occasionally simpler to treat using some platform as synonymous with doing a specific sort of task. But some platforms offering microwork also let people take on larger, higher-skill “macrowork” projects. And this opens up room for error.

The BoC’s informal work survey reflects the challenges here. As examples of sites respondents might use to “complete tasks online,” it offers both AMT and Fiverr. But the second is a far more general freelance marketplace. Work on offer there ranges from place-based street postering to mobile app development.

Workers on AMT might earn a cent per basic image-recognition task. But Fiverr’s highest-paid projects include complex video production, for up to $18,000 USD.

For those interested specifically in microwork, things then get even more ambiguous.

The survey’s definition of “tasks online” goes far beyond simple work like rating pictures. Examples provided include reviewing resumés, editing documents and doing graphic design. The questionnaire then gives both freelance computer programming and graphic/web design as entirely separate income options. Respondents could also select multiple options.

This isn’t just academic quibbling. It’s unclear how Canadian microworkers would interpret these options, and the result could be significant miscounting. Then this likewise undermines hope for any further insight from the survey responses, on topics such as average microworker payment.

Microworkers of the world

Such concerns make the 2018 International Labour Organization report mentioned earlier invaluable. Digital labour platforms and the future of work focuses specifically on microworkers. So it paints a much richer picture of how and why people around the world use these platforms.

The study draws on several rounds of surveys and interviews. Researchers collected data from an international group of 2350 microworkers, reached through five leading platforms. These included AMT, CrowdFlower and Prolific. (This last specializes in recruiting participants for higher-paying research surveys and experiments.) The authors then used this data to study more general features of platform microwork and microworkers.

Throughout, they focus on exactly the sort of compensation and demographic details relevant to TWIG’s work.

The report unfortunately doesn’t provide any Canada-specific analysis. Respondents nevertheless included Canadian microworkers. And their earnings presumably inform the authors’ broader conclusion that North Americans using these platforms make an average of $4.70 USD per hour.

Even factoring in the exchange rate, this obviously falls well below Ontario’s minimum wage.

But as with the BoC results earlier, survey representativeness matters. In total, the ILO study only actually draws on responses from 41 Canadian microworkers. And no more than 13 come from any given platform.

That said, the ILO’s North American average seems likely closer for Canada than Mexico. Socio-economic and other disparities are certainly potentially greater there. But the report also includes only 13 Mexican microworkers, from among the country’s 129m citizens.

The variation behind averages

Arriving at that $4.70 North American average required the authors make various choices about expressing and prioritizing underlying complexities. And they’re very transparent about this.

Beyond the hours microworkers devote to directly completing tasks, e.g., most spend considerable unpaid time on platforms looking for work. So the $4.70 compensation average very reasonably factors in that extra time.

But when we look at a blended hourly payment figure like this, it’s easy to forget one of crowdwork’s defining features: flexibility. This is particularly extreme in microwork, which leads to huge variation in how and why people do it.

Such short, purely digital tasks can certainly be completed in the eight-hour blocks of a traditional work day. But other workers just occasionally use them fill commercial breaks at home. Some ILO respondents even report microworking exclusively while at other jobs.  

Such variation is a major theme of Mary L. Gray and Siddharth Suri’s recent book, Ghost Work.

In traditional jobs, most colleagues have relatively similar hours. Gray and Suri argue that microworkers’ instead look more like a “power law” distribution. Essentially, relatively few do most of the work.

20 per cent of [microworkers] doing 80 per cent of the work guarantee that the work gets done, and the remaining 80 per cent of workers doing 20 per cent of the work fill in the gaps.

– Mary L. Gray and Siddharth Suri, Ghost Work

Similar patterns repeated across all the platforms studied. On that basis they divide microworkers into three groups: the “experimentalists, regulars, and always-on.” The key difference is time devoted to microtasking — which usually increases with financial dependence on platform income.

“Always-on” and “experimentalist” microworkers often engaged in quite different work, as we’ll discuss later. But it’s also worth nothing that there can certainly still be overlap at the level of particular tasks they undertake. Especially since many of us share a basic intuition that workers handling the same job will tend to resemble one another. Which just adds further opportunity for confusion about microwork.

The ILO report highlights another long-tail distribution, this time in hourly compensation. Accounting for unpaid work, e.g., the median hourly pay across all their respondents is $2.16 USD. But a tiny minority make almost $20 per hour.

In part, this reflects differences between platforms themselves. The researchers found that labour on Microworkers paid a median hourly wage of $1.01, including unpaid time; Prolific, however, pays $3.56.

National differences in “borderless” work

Such variations often reflect a key factor: where the workers live. National context turns out to be crucial in not only earning patterns, but many other aspects of microwork.

Amazon’s policies, e.g., long discouraged workers outside the U.S. or India from using AMT. And platform demographics today still reflect this history.

So on matters from worker compensation to family makeup, the ILO researchers report separately for each country. This decision to effectively present AMT as two separate platforms highlights the importance of distinctively national trends.

The ILO report describe AMT as a clearly “dual-banded” labour market, for instance. Experienced Americans focus on pursuing tasks that pay at or above their federal minimum wage. This leaves lower-paid tasks to inexperienced or foreign workers.

Including unpaid time, AMT’s average US hourly rate for 2017 is $6.54. Indian workers received just $2.53. Local purchasing power is also key in understanding earnings’ actual value to microworkers, however. And accounting for that, Indian workers actually earn 30% more than their U.S. counterparts.

Crowdwork became a lifeline for hundreds of thousands of Venezuelans. They became online migrant workers.

– Florian Alexander Schmidt, Crowdsourced Production of AI Training Data

Further national differences likewise emerge across platforms.

Prolific’s crowd is almost exclusively Europe- and North America-based, for example. And Venezuelan microworkers are particularly prominent in several recent studies. The country’s national oil industry collapse and hyperinflation left increasing numbers of citizens trying to earn foreign currency in microwork markets. So compared to other national groups described in the ILO report, they were unusually poor and survival-oriented.

Why do people microwork?

All this highlights an obvious-but-important fact. Within each country studied, microworkers have widely varying economic and intrinsic motivations for taking on similar platform tasks. Quasi-recreational workers, e.g., can end up competing for tasks with people trying to pay for basic necessities.

But even when microwork covers the essentials, its significance often varies widely.

If a household’s sole earner depends entirely on microwork for income, a sudden compensation dip or account lockout might be catastrophic. But for a stay-at-home parent in the same country and income quintile, even low and inconsistent pay might border on transformative.

And at least these same general dynamics seems likely to be true for Canada.

Who’s microtasking?

Microworkers’ offline social, political and economic context are (unsurprisingly) key to understanding how and why they use these platforms.

In the ILO report, almost half of respondents indicate microwork is their primary income source. In many cases, it’s the only way they can earn income while caring for children. Or perhaps they’re among the 9% whose health prevents conventional work outside the home. And many respondents also use these platforms to supplement other work income.

But when describing their motivations, many respondents selected options related to fun or leisure. Such responses are actually more common than those highlighting lack of better-paying alternatives. And most workers across all platforms reported positive satisfaction with their crowdwork.

The majority of the crowdworkers stated that they were satisfied or very satisfied with crowdwork…Overall, only 6 per cent were dissatisfied and 1 per cent very dissatisfied.

— International Labour Office, Digital labour platforms and the future of work

Country-to-country comparisons

Such aggregate statistics also conceal broad variation, of course. And this is particularly clear in national-level comparisons released since the ILO report.  

Lisa Posch and her collaborators, for example, have begun publishing results from a recent survey of almost 12,000 Figure Eight microworkers. And they’ve used the data to compare workers on the platform by country.

This reveals significant differences in motivation. But they’ve also measured international demographic variations. These range from the fact U.S. workers on the platform have incomes higher than the national average to Russia’s seemingly older-than-average microworkforce.

Then even more significantly, the EU’s Collaborative Economy project has released several pieces of analysis relevant to microwork since 2018.

Based on their extensive COLLEEM survey, these findings draw on data from 32,409 platform workers. This allows direct comparison of the digital gig economies in 14 European countries. Each is essentially represented by as many respondents as the entire ILO microwork survey, whose respondents span 50 countries.

COLLEEM’s added power and comparative approach produces several findings useful as context when considering Canadian microwork.

Maybe most significantly, researchers find clear variation in the amount of microwork the countries’ residents undertake. Levels differ by over 20%. This also isn’t straightforwardly a matter of national income or education levels. Microwork is highest in France and lowest in Germany, with Slovakia sitting roughly in the middle.

Further study may uncover reliable patterns here. And that could allow reasonable inferences about Canadian microwork levels — or even those of the GTA itself — without direct measurement. But in the meantime, such relatively wide variation reinforces reservations about assuming simple patterns recur internationally.

New modes of microwork

Finally, it’s worth noting that some key recent developments seem underrepresented or totally absent from the studies discussed above.

AI-related tasks have long helped drive demand for microwork, for instance, and the field continues expanding. But only 8.2% of the ILO study’s respondents worked on such tasks. Then just 7.9% indicated work on content moderation, a field already estimated in 2017 to be employing 150,000 people.

This seems to reflect the ongoing shifts that can make these markets so difficult to track. New microwork is happening, just not on more familiar (and easily studied) open platforms like AMT.

Sources such as Ghost Work emphasize the variety of these alternative arrangements. Early examples here include internal corporate microtask platforms, such as Microsoft’s Universal Human Relevance System or Google’s EWOQ/Raterhub. Companies now often also outsource development or staffing of such services to third-party “vendor management systems.”

This makes microwork still harder to track — even as it’s potentially more present in North America. Long used by companies here to offshore operations, for example, India’s iMerit recently opened its own New Orleans office.

Such developments tie into the emergence of more specialized microwork services. These use tailor-made tools and handpicked crowds to meet increasing demand for more accurate, confidential work. Florian Schmidt documents exactly this sort of shift in the auto industry’s increasing engagement of companies such as Mighty AI, Hive.ai and Scale.ai.

30 million Chinese crowd workers [serve] more than 190,000 enterprises and individuals worldwide. This generates a total business turnover of CNY 5 billion (approx. $900M USD).

— Yihong Wang et al, Crowdsourcing in China

There’s also a large and growing ecosystem of microwork platforms/providers operating in languages besides English. Often serving non-western requesters, these largely fall totally outside the literature discussed so far.

China’s massive market offers particularly striking examples. Microwork options there range from task distribution over chat services to emerging “crowdfarms” and specialized data-labelling services.

It seems unlikely Canadians are currently microworking for such services in significant numbers — at least at the national level. But this could change, if the global microwork market keeps growing. Such emerging businesses might then provide increasing numbers of Canadians with income. And that’s particularly true of diverse, multilingual cities like Toronto.

Promising research directions

Reviewing the sources above obviously still left us with many open questions. It nevertheless helped us more clearly identify key sources of uncertainty. And the process has already pointed us towards promising possibilities for future insights into Canadian microwork.

One key avenue here is the richer data increasingly available about the broader Canadian gig economy.

We originally hoped research on the topic might provide indirect insight into microwork. But there was virtually no scholarly research specifically on gig work in Canada. In fact, a systematic literature review ending in mid-2017 found nothing peer-reviewed on the subject.

The Canadian Centre for Policy Alternatives has actually studied the GTA gig economy. But their research focuses exclusively on in-person service delivery. This leaves little to work with for anyone specifically interested in microtasking.

Statistics Canada surveys the gig economy

Since then, the topic has thankfully benefited from greater research interest.

In particular, Statistics Canada recently published “Measuring the Gig Economy in Canada Using Administrative Data.” This groundbreaking study is a major contribution to increasingly widespread attempts at quantifying national gig economies.

(The OECD has published a helpful summary, for those curious about work in other countries.)

The research draws on a random sample of linked 2016 tax filing and census data, covering just over 4,780,000 individual citizens. This approach allows remarkably rich detail and accuracy. It also lets researchers explore demographic, compensation and other employment questions particularly relevant for labour force planners.

This new paper remains preliminary. But it charts the broad patterns of independent work in Canada, laying the groundwork for future studies.

Statistics Canada concludes that gig workers made up over 8.2% of all working adults in 2016 — more than 1,674,000 Canadians.

The authors first establish a definition of “gig workers” in the Canadian context. This extends far beyond those making their living with new technologies, to the sort of musicians and actors who coined the term. It also includes all other “unincorporated self-employed freelancers, day labourers, or on-demand platform workers.”

The authors then go on to provide a wide range of fundamental details about this broad gig-worker population, including their income and age distributions. They also sometimes dive deeper, with observations like the fact recent male immigrants work gigs almost twice as often as men born in this country.

How geographically concentrated are Canadian microworkers?

The paper doesn’t explicitly address microwork, however. And there’s no clear way to make reliable specific inferences from its broader gig analysis.

Consider the basic question of location. Statistics Canada’s analysis shows that the country’s gig workers are clustered in metro centres such as Montreal, Toronto and Vancouver. So can we safely assume microworkers are similarly concentrated?

Other sources suggest not.

The intuitive parallel doesn’t square, for instance, with work by the JPMorgan Chase Institute (JPMCI). Their team has access to anonymized data from 39 million bank accounts. This allows economic studies without the survey pitfalls already discussed. And they’ve used this very detailed, accurate personal data to produce a range of influential research on the U.S. gig economy.

Like the Statistics Canada paper, for instance, a 2019 JPMCI report found that platform economy participation varied widely between cities. But it also take a closer historical look at gig work by type, across 27 U.S. urban centres. This shows that transport roles such as driving or delivery overwhelmingly explain differences in cities’ number of gig workers. And as such work increases, they likewise see no evidence people take up other kinds of gigs.

More generally, microwork would fall under the “non-transport platform labour” tracked in JPMCI’s analyses. And they likewise report that earnings for this category are extremely consistent across the 23 states tracked, both big and small.

Gray and Suri’s research provides a further reason for caution about clustering. The American microworkers they studied for Ghost Work are “distributed throughout the United States in both highly and sparsely populated regions.”

This geographic issue is obviously in itself a narrow point. But it hopefully illustrates why even high-quality findings about the broader gig economy might not translate directly to microwork.

Labelling microwork

The recent Statistics Canada gig paper still offers at least one valuable perspective on microwork. By solidly counting gig workers nationally, it provides added context that can help interpret research such as the BoC’s informal work survey.  

This begins with the breakdown in “Measuring the Gig Economy” of how many gig workers are active in each of Canada’s economic sectors. But these proportional numbers alone can’t tell us anything directly about microwork.

That’s primarily because they’re extremely broad. To define work sectors, the authors use categories equivalent to the first two digits of the North American Industry Classification System (NAICS) codes for Canada. Sector 51, for instance, covers “information and cultural industries.” This includes businesses from book and newspaper publishing to film and much of tech.

It’s theoretically possible to identify narrower industries and sub-industries through longer NAICS codes. But even at the 5- and 6- digit levels, there’s nothing yet that specifically captures microwork.

Testing microwork estimates against gig data

But we might still use such data to estimate an upper limit on Canadian microworkers.

Let’s assume that they made up every gig worker in both NAICS sectors (#51 & #54) that include the codes used by major microwork platforms themselves. Then let’s include the entire further sector (#56) covering outsourced administrative and clerical tasks.

Based on the Statistics Canada analysis, these three hired a combined total of just under 549,000 gig workers in 2016. Which therefore seems like a more than reasonable upper limit for Canadian microworkers.

Many admittedly might not declare platform earnings, especially if making relatively little. And this could mean they wouldn’t register in the count, since the researchers depend partly on CRA filing for their data.

But simply adding sector gig worker totals together also entails significant overcounting, which should offset concerns on that front. Each sector covers many areas, the majority far removed from microwork. NAICS 56, for instance, includes all janitorial services.

Some people also work gigs in multiple sectors. The Statistics Canada authors count these individuals under the percentage of workers active for each sector. That’s why cross-sector totals add up to more than 100%. And it means the proposed ceiling certainly double- or even triple-counts some gig workers.

All that makes 549,000 seem a reasonable high-end estimate for Canadian microworkers. But even this ballpark estimate from administrative data sharply contrasts with implications of the BoC informal work survey. Extrapolating from those responses, we end up with a projected 1,195,000 Canadian microworkers.

I won’t speculate further here on how best to explain or reconcile the difference. But these are certainly the sorts of questions we hope new research will soon clarify.

Resource: Microwork library

Researchers scanned over 500 microwork resources from Reddit feeds to authoritative reports to research articles. We’ve put the top 100 in the TWIG microwork library.

---

Works cited in this article

There are over 35 works cited in this literature review. The selected citations linked below will open in this tab, so simply use the back button to return to this list.

1. Al Jazeera. ‘Scrubbing the Net: The Content Moderators. Accessed 3 March 2020. https://www.aljazeera.com/programmes/listeningpost/2017/05/scrubbing-net-content-moderators-170527124251892.html.
2. Bajwa, Uttam, Lilian Knorr, Erica Di Ruggiero, Denise Gastaldo, and Adam Zendel. ‘Towards an Understanding of Workers’ Experiences in the Global Gig Economy’, n.d., 42.
3. Bank of Canada. ‘Canadian Survey of Consumer Expectations—Overview’. Accessed 3 March 2020. https://www.bankofcanada.ca/publications/canadian-survey-of-consumer-expectations/canadian-survey-of-consumer-expectations-overview/.
4. Berger, Sarah. ‘The 8 Highest-Paying Side Hustles on Fiverr’. CNBC, 30 April 2018. https://www.cnbc.com/2018/04/30/the-8-highest-paying-side-hustles-on-fiverr.html.
5. Block, Sheila. ‘Sharing Economy’ or on-Demand Service Economy? Canadian Centre for Policy Alternatives, 2017. http://deslibris.ca/ID/10089542.
6. Board of Governors of the Federal Reserve System. ‘The Fed – Employment’. Accessed 3 March 2020. https://www.federalreserve.gov/publications/2019-economic-well-being-of-us-households-in-2018-employment.htm.
7. Cadell, Cate. ‘Faces for Cookware: Data Collection Industry Flourishes as China Pursues AI Ambitions’. Reuters, 28 June 2019. https://www.reuters.com/article/us-china-ai-data-insight-idUSKCN1TS3EA.
8. Crunchbase. ‘PathAI. Accessed 3 March 2020. https://www.crunchbase.com/organization/pathai#section-overview.
9. Farrell, Diana, Fiona Greig, and Amar Hamoudi. ‘The Online Platform Economy in 27 Metro Areas: The Experience of Drivers and Lessors’. SSRN Electronic Journal, 2019. https://doi.org/10.2139/ssrn.3391549.
10. Financial Times. ‘AI’s new workforce: the data-labelling industry spreads globally’. (subscription required). Accessed 3 March 2020. https://www.ft.com/content/56dde36c-aa40-11e9-984c-fac8325aaa04.
11. FPFIS. ‘New Evidence on Platform Workers in Europe’. Text. EU Science Hub – European Commission, 18 February 2020. https://ec.europa.eu/jrc/en/publication/eur-scientific-and-technical-research-reports/new-evidence-platform-workers-europe.
12. Fucci, Massimiliano. ‘COLLEEM (Collaborative Economy) Research Project’. Text. EU Science Hub – European Commission, 22 February 2017. https://ec.europa.eu/jrc/en/colleem.
13. Gallo Montero, Valeria. ‘Microtasking as a Quick Fix’. TWIG Microtasking Project (blog), 18 November 2019. https://www.microtasking.online/microtasking-as-a-quick-fix/.
14. Government of Canada, Statistics Canada. ‘Canadian Internet Use Survey (CIUS)’, 7 September 2018. https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=4432.
15. Government of Canada, Statistics Canada. ‘Measuring the Gig Economy in Canada Using Administrative Data’, 16 December 2019. https://www150.statcan.gc.ca/n1/pub/11f0019m/11f0019m2019025-eng.htm.
16. Gray, Mary L, Siddharth Suri, Syed Shoaib Ali, and Deepti Kulkarni. ‘The Crowd Is a Collaborative Network’. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing – CSCW ’16, 134–47. San Francisco, California, USA: ACM Press, 2016. https://doi.org/10.1145/2818048.2819942.
17. Gray, Mary L. and Siddharth Suri. Ghost Work. ‘Ghost Work’. Accessed 3 March 2020. https://ghostwork.info/.
18. Gray, Mary L. and Suri, Siddharth. ‘The Humans Working Behind the AI Curtain’. Accessed 3 March 2020. https://hbr.org/2017/01/the-humans-working-behind-the-ai-curtain.
19. Hans Böckler Stiftung. ‘Suchergebnis – Veröffentlichungen HBS’. Accessed 3 March 2020. https://www.boeckler.de/6299.htm.
20. Hansen, Jacqueline. ‘Online Gig Economy Is Growing, but No One Knows by How Much | CBC News’. Accessed 3 March 2020. https://www.cbc.ca/news/business/canada-s-digital-workers-1.4889691.
21. Hara, Kotaro, Abigail  Adams, Kristy  Milland, Saiph Savage, Chris Callison-Burch and Jeffrey P Bigham. A Data-Driven Analysis of Workers’ Earnings on Amazon Mechanical Turk | Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems’. Accessed 3 March 2020. https://dl.acm.org/doi/10.1145/3173574.3174023.
22. Harris, Mark. ‘It’s Not Just Robots: Skilled Jobs Are Going to “Meatware” | WIRED’. Accessed 3 March 2020. https://www.wired.com/2016/06/its-not-just-robots-skilled-jobs-are-going-to-meatware/.
23. Janine Berg and Marianne Furrer. ‘Digital Labour Platforms and the Future of Work: Towards Decent Work in the Online World’. Report, 20 September 2018. http://www.ilo.org/global/publications/books/WCMS_645337/lang–en/index.htm.
24. Kostyshyna, Olena, and Luu, Corinne. ‘The Size and Characteristics of Informal (“Gig”) Work in Canada’. Bank of Canada, 21 February 2019. https://www.bankofcanada.ca/2019/02/staff-analytical-note-2019-6/.
25. Metz, Cade. ‘A.I. Is Learning From Humans. Many Humans. – The New York Times’. Accessed 3 March 2020. https://www.nytimes.com/2019/08/16/technology/ai-humans.html.
26. Posch, Lisa and Bleier, Arnim. ‘Measuring Motivations of Crowdworkers: The Multidimensional Crowdworker Motivation Scale: ACM Transactions on Social Computing: Vol 2, No 2’. Accessed 3 March 2020. https://dl.acm.org/doi/abs/10.1145/3335081.
27. Posch, Lisa, Arnim Bleier, Fabian Flöck, and Markus Strohmaier. ‘Characterizing the Global Crowd Workforce: A Cross-Country Comparison of Crowdworker Demographics’. ArXiv:1812.05948 [Cs], 14 December 2018. http://arxiv.org/abs/1812.05948.
28. Poseda, Julian. ‘Microwork: An Introduction’. TWIG Microtasking Project (blog), 23 February 2020. https://www.microtasking.online/microwork-introduction/.
29. Savvy New Canadians. ‘17 Real Ways To Make Extra Money From Home and Online in Canada’, 22 April 2019. https://www.savvynewcanadians.com/how-to-make-money-home-online-canada/.
30. Schmidt, Florian, A. ‘Crowdsourced Production of AI Training Data’. Accessed 3 March 2020. https://www.boeckler.de/pdf/p_fofoe_WP_155_2019.pdf.
31. Semuels, Alana. ‘The Internet Is Enabling a New Kind of Poorly Paid Hell’. The Atlantic, 23 January 2018. https://www.theatlantic.com/business/archive/2018/01/amazon-mechanical-turk/551192/.
32. Smith, Aaron. ‘The New Digital Economy: Shared, Collaborative and On Demand’. Pew Research Center: Internet, Science & Tech (blog), 19 May 2016. https://www.pewresearch.org/internet/2016/05/19/the-new-digital-economy/.
33. U.S. Government. ‘Electronically Mediated Work: New Questions in the Contingent Worker Supplement : Monthly Labor Review: U.S. Bureau of Labor Statistics’. Accessed 3 March 2020. https://www.bls.gov/opub/mlr/2018/article/electronically-mediated-work-new-questions-in-the-contingent-worker-supplement.htm.
34. Wallenstein, J., de Chalendar, A., and Reeves, M. ‘The New Freelancers: Tapping Talent in the Gig Economy’. Accessed 3 March 2020. https://www.bcg.com/en-ca/publications/2019/new-freelancers-tapping-talent-gig-economy.aspx.
35. Wang, Yihong & Papangelis, Konstantinos & Khan, Vassilis-Javed & Lykourentzou, I. & Chamberlain, Alan and Michael Saker. ‘Crowdsourcing in China: Exploring the Work Experiences of Solo Crowdworkers and Crowdfarm Workers’. Accessed 3 March 2020 .https://www.researchgate.net/publication/338765020.