Category Archives: labor markets

AI, Labor, and the Parable of the Horse

Today I attended the 5th year anniversary celebration for MSR NYC. There was a great group of speakers and panelists—I’m super impressed by what MSR has accomplished. One topic that came up at several points during the day was the labor market effects of technological developments—particularly that powerful AI might displace many workers.

Economists have traditionally been sanguine about the effects of technological change on the labor market, viewing widespread technological unemployment as unlikely. This perspective is based on the historical experience of substantial technological change not having persistent disemployment effects. However, it has been pointed out that we have one vivid example where this optimism has not been warranted—what I call the parable of the horse.

The story is that the internal combustion engine came along and horses saw their marginal product decline below the cost of their feed and so horses disappeared, at least in the “labor” market. This is undoubtedly true—the figure below shows the number of horses (and mules) in the US (from The Demographics of the U.S. Equine Population). The implication is that today’s “horses”—low skilled labor or maybe even labor in general—will disappear as AI can do more and more tasks.

I find the horse parable interesting, but unpersuasive—at least with respect to how it is likely to affect relatively low-skilled workers—because I think the analogy misses the reason why horses fared so poorly. The problem was not that they were “low skilled”  but rather their extreme specialization. Horses did one job and one job only—exerting physical force, which could be used to pull or push things. That they could be almost entirely displaced by a superior pushing/pulling technology is, in some sense, not surprising. But what I think is important in the human labor market is that being able to do one thing—and one thing only—is typically a characteristic of high skilled labor, not low skilled labor.

Most low-skilled labor is no longer like horse labor, in that the low-skilled jobs that exist now are those that require some mix of physical, intellectual and even “emotional” skills. This mix makes full automation challenging. But even when some specific job does “fall” to automation, there is a still a very large pool of remaining jobs that could still need to be done and require relatively little skill or new training, by definition of being low-skilled. In short, one advantage of the low skilled labor market is that there are lots of jobs you qualify for. The downside, of course, is that precisely because lots of people can qualify for those jobs and so wages are low. The workers that are vulnerable to technical change—in the sense that they are likely to experience large declines in income—are those workers with highly specialized skills.

Truck driving as a low-skilled example

To give a more concrete example, consider the job of truck driver, which might be on the automation chopping block. First, many people with the job “truck driver” are actually some combination of sales representative, inventory-taker, first line mechanic, warehouse worker, forklift operator and so on. As such, it is far from obvious that even if substantial amounts of driving end up being done through automation that labor demand for “truck driver” would fall.

Second, even if the truck driver occupation sees a large negative demand shock, what other jobs could a truck driver do that pay about the same? Well, let’s look at the BLS  occupational data. The table below shows the most recent BLS occupational data, w/ US employment totals and average hourly wages, sorted by average hourly wage. I restricted the list to occupations with more than 500K employees. We can see that being a light truck driver (i.e., not a heavy truck transporting freight or heavy equipment) pays about $16.50/hour, which is below the median wage in the US but still substantially higher than the minimum wage. 

Retail Salespersons 4,612,510 12.67
Nursing Assistants 1,420,570 12.89
Landscaping and Groundskeeping Workers 895,600 13.20
Laborers and Freight, Stock, and Material Movers, Hand 2,487,680 13.39
Receptionists and Information Clerks 975,890 13.67
Security Guards 1,097,660 13.68
Substitute Teachers 626,750 14.25
Bus Drivers, School or Special Client 505,560 14.70
Team Assemblers 1,115,510 15.17
Office Clerks, General 2,944,420 15.33
Medical Assistants 601,240 15.34
Shipping, Receiving, and Traffic Clerks 674,820 15.55
First-Line Supervisors of Food Preparation and Serving Workers 884,090 16.02
Light Truck or Delivery Services Drivers 826,510 16.38
Industrial Truck and Tractor Operators 539,810 16.39
Medical Secretaries 530,360 16.50
Customer Service Representatives 2,595,990 16.62
Secretaries and Administrative Assistants, Except Legal, Medical, and Executive 2,281,120 16.92
Construction Laborers 887,580 17.57
Maintenance and Repair Workers, General 1,314,560 18.73
Bookkeeping, Accounting, and Auditing Clerks 1,580,220 18.74
Inspectors, Testers, Sorters, Samplers, and Weighers 508,590 18.95
Automotive Service Technicians and Mechanics 638,080 19.58
Heavy and Tractor-Trailer Truck Drivers 1,678,280 20.43

We can see that both above and below, there are a number of jobs that are plausible substitute occupations for a displaced truck driver. For example, of those below, most require no formal education or certification, perhaps except for substitute teachers or medical assistants. If we go higher, we start to see jobs that require more skills or that are more physically taxing or dangerous (e.g., construction laborer), but are still reasonable substitutes. For example, many truck drivers are also decent mechanics and perhaps with some more training, could find work as “Maintenance and Repair Workers, General.”

Not only do displaced truck drivers have lots of “nearby” occupations that pay about the same with little additional human capital requirements, the displaced drivers are not likely to drive down wages very much in their new occupations. There are, of course, lots of truck drivers, but if they split into a reasonably large chunk of other occupations, the new entrants would not be much of a supply shock.

A specialized occupation example

Now we’ll look at a more specialized occupation. Let’s consider accountants and auditors (even this one still seems far away from being even remotely automatable). It pays quite nicely and requires substantial specialized skill. If we look at nearby jobs, very few would be open to a displaced accountant without substantial re-training.

Lawyers 609,930 65.51
Financial Managers 531,120 64.58
General and Operations Managers 2,145,140 57.44
Software Developers, Applications 747,730 49.12
Management Analysts 614,110 44.12
Computer Systems Analysts 556,660 43.36
Accountants and Auditors 1,226,910 36.19
Business Operations Specialists, All Other 926,610 35.33
Registered Nurses 2,745,910 34.14
Market Research Analysts and Marketing Specialists 506,420 33.67
First-Line Supervisors of Construction Trades and Extraction Workers 517,560 32.13
Sales Representatives, Wholesale and Manufacturing, Except Technical and Scientific Products 1,409,550 32.11
Sales Representatives, Services, All Other 886,580 29.98
Police and Sheriff’s Patrol Officers 653,740 29.45
Secondary School Teachers, Except Special and Career/Technical Education 962,820 *

Accountants are the “horses” here—the ones that are vulnerable to large drop offs in earnings because of their specialization. Fortunately, if we care about inequality, this is the “right” group to be affected. Because of their existing financial wealth and their general human capital, they are likely better able to deal with the disequilibria created by technological change.

Would a job by any other name pay as much?

I’m working on a project where it would be useful to know what an oDesk job is likely to pay at the time it is posted. Although there are plenty of structured predictors available (e.g., the category, skills, estimated duration etc.), presumably the job description and the job title contain lots of wage-relevant information. The title in particular is likely to identify the main skill needed, the task to be done and perhaps the quality of the person the would-be employer is looking for (e.g., “beginner”, or “senior”).

Unfortunately, I haven’t done any natural language processing before, so I’m a bit out of my element. However, there are good tutorials online as well as R packages that can guide you through the rough parts. I thought writing up my explorations might be useful to others that want to get started with this approach. The gist of the code I wrote is available at here.

What I did:


I took 20K recent hourly oDesk jobs that where the freelancer worked at least 5 hours. I calculated the log wage over the course of the contract. Incidentally, oDesk wages—like real wages—are pretty well approximated by a normal distribution. 
2) I used the RTextTools  package to create a document term matrix from the job titles (this is just a matrix of 1 & 0 where the rows are jobs and the columns are relatively frequent words that are not common English words—if the job title contained that word, it gets a 1, otherwise a 0).

3) I fit a linear model using the lasso for regularization (using the glmnet package). I used cross validation to select the best lambda. A linear model probably isn’t ideal for this, but at least it gives nicely interpretable coefficients.

So, how does it do?  Here are a sample of the coefficients that didn’t get set to zero by the lasso, ordered by magnitude (point sizes are scaled by the log number of times that word appears in the 10K training sample): 

The coefficients can be interpreted as % changes from the mean wage in the sample when that corresponding word (or word fragment) is present in the title. Nothing too surprising I think: at the extremes, SEO is a very low paying job, whereas developing true applications is high paying.

In terms of out of sample prediction, the R-squared was a little over 0.30. I’ll have to see how much of an improvement can be obtained from using some of the structured data available, but explaining 30% of the variation just using the titles is a higher than I would have expected before fitting the model.


Bigger Big Mac by Simon Miller, on Flickr
Creative Commons Attribution-Noncommercial-Share Alike 2.0 Generic License Image by Simon Miller 

A standard pricing strategy in many industries is bundling goods, e.g., productivity “suites” like Microsoft Office, value meals at fast food restaurants, hotel and flight combos, etc. In the labor market, we also see a kind of bundling, though not by design: each worker is a collection of skills and attributes that can’t be broken apart and purchased separately by the firm. For example, by hiring me, my company gets my writing, meeting attendance, programming, etc.; they can’t choose to not buy my low-quality expense-report-filing service.

Good mangers deal with this bundling by keeping workers engaged at their highest value activity. However, every activity has decreasing marginal returns, so even activities that start out as high-value eventually reach the “flat of the curve” where the marginal benefit of more of X gets pretty small. This phenomena gives large firms an advantage, in that their (generally) larger problems give workers more runway to ply their best skills (by the same token, small firms have to worry much more about “fit” within their existing team).

While pervasive, this flat-of-the curve dynamic and the resulting small-firm handicap is not a fundamental feature of organizations or labor markets–it springs from the binary nature of employment. It goes away or it least is diminished if a worker can instead being partly employed (i.e., freelance) at a number of firms, each paying the worker to do what they do best. To date, the stated value proposition of most freelancing sites has been that they allow for global wage arbitrage. Obviously that’s important, but I suspect this “unbundling” efficiency gain will, in the long term, have a more profound effect on how firms organize and how labor markets function.