Sections

Cathy O’Neil: Unmasking Unconscious Bias in Algorithms

Fawn Fitter

In the wake of the 2008 banking crisis, Cathy O’Neil, a former Barnard College math professor turned hedge fund data scientist, realized that the algorithms she once believed would solve complex problems with pure logic were instead creating them at great speed and scale. Now O’Neil—who goes by mathbabe on her popular blog and 11,000-follower Twitter account—works at bringing to light the dark side of Big Data: mathematical models that operate without transparency, without regulation, and—worst of all—without recourse if they’re wrong. She’s the founder of the Lede Program for Data Journalism at Columbia University, and her bestselling book, Weapons of Math Destruction (Crown, 2016), was long-listed for the 2016 National Book Award.

We asked O’Neil about creating accountability for mathematical models that businesses use to make critical decisions.

Q. If an algorithm applies rules equally across the board, how can the results be biased?

Cathy O’Neil: Algorithms aren’t inherently fair or trustworthy just because they’re mathematical. “Garbage in, garbage out” still holds.

There are many examples: On Wall Street, the mortgage-backed security algorithms failed because they were simply a lie. A program designed to assess teacher performance based only on test results fails because it’s just bad statistics; moreover, there’s much more to learning than testing. A tailored advertising startup I worked for created a system that served ads for things users wanted, but for-profit colleges used that same infrastructure to identify and prey on low-income single mothers who could ill afford useless degrees. Models in the justice system that recommend sentences and predict recidivism tend to be based on terribly biased policing data, particularly arrest records, so their predictions are often racially skewed.

Q. Does bias have to be introduced deliberately for an algorithm to make skewed predictions?

O’Neil: No! Imagine that a company with a history of discriminating against women wants to get more women into the management pipeline and chooses to use a machine-learning algorithm to select potential hires more objectively. They train that algorithm with historical data about successful hires from the last 20 years, and they define successful hires as people they retained for 5 years and promoted at least twice.

They have great intentions. They aren’t trying to be biased; they’re trying to mitigate bias. But if they’re training the algorithm with past data from a time when they treated their female hires in ways that made it impossible for them to meet that specific definition of success, the algorithm will learn to filter women out of the current application pool, which is exactly what they didn’t want.

I’m not criticizing the concept of Big Data. I’m simply cautioning everyone to beware of oversized claims about and blind trust in mathematical models.

Q. What safety nets can business leaders set up to counter bias that might be harmful to their business?

O’Neil: They need to ask questions about, and support processes for, evaluating the algorithms they plan to deploy. As a start, they should demand evidence that an algorithm works as they want it to, and if that evidence isn’t available, they shouldn’t deploy it. Otherwise they’re just automating their problems.

Once an algorithm is in place, organizations need to test whether their data models look fair in real life. For example, the company I mentioned earlier that wants to hire more women into its management pipeline could look at the proportion of women applying for a job before and after deploying the algorithm. If applications drop from 50% women to 25% women, that simple measurement is a sign something might be wrong and requires further checking.

Very few organizations build in processes to assess and improve their algorithms. One that does is Amazon: Every single step of its checkout experience is optimized, and if it suggests a product that I and people like me don’t like, the algorithm notices and stops showing it. It’s a productive feedback loop because Amazon pays attention to whether customers are actually taking the algorithm’s suggestions.

Q. You repeatedly warn about the dangers of using machine learning to codify past mistakes, essentially, “If you do what you’ve always done, you’ll get what you’ve always gotten.” What is the greatest risk companies take when trusting their decision making to data models?

O’Neil: The greatest risk is to trust the data model itself not to expose you to risk, particularly legally actionable risk. Any time you’re considering using an algorithm under regulated conditions, like hiring, promotion, or surveillance, you absolutely must audit it for legality. This seems completely obvious; if it’s illegal to discriminate against people based on certain criteria, for example, you shouldn’t use an algorithm that does so! And yet companies often use discriminatory algorithms because it doesn’t occur to them to ask about it, or they don’t know the right questions to ask, or the vendor or developer hasn’t provided enough visibility into the algorithm for the question to be easily answered.

Q. What are the ramifications for businesses if they persist in believing that data is neutral?

O’Neil: As more evidence comes out that poorly designed algorithms cause problems, I think that people who use them are going to be held accountable for bad outcomes. The era of plausible deniability for the results of using Big Data—that ability to say they were generated without your knowledge—is coming to an end. Right now, algorithm-based decision making is a few miles ahead of lawyers and regulations, but I don’t think that’s going to last. Regulators are already taking steps toward auditing algorithms for illegal properties.

Whenever you use an automated system, it generates a history of its use. If you use an algorithm that’s illegally biased, the evidence will be there in the form of an audit trail. This is a permanent record, and we need to think about our responsibility to ensure it’s working well. D!

Comments

Fawn Fitter

About Fawn Fitter

Ever since discovering the fledgling Internet in the early 1990s, Fawn Fitter has been fascinated by the places where business and technology intersect. She’s spent 15 years in San Francisco, watching the ebbs and flows of the digital economy and writing for magazines, including Entrepreneur and Fortune Small Business.

Fast – But Not Too Fast – Wins The Race In Supply Chain Management

Richard Howells

Bonnie D. Graham, host of The Digital Transformation of Your Supply Chain with Game-Changers podcast, opened her recent show by recounting the tale of “The Tortoise and the Hare.”

You know the story: A slow but steady turtle beats a swift yet arrogant rabbit in a footrace.

Bonnie used the Aesop fable to illustrate two points about supply chain management:

  1. If you respond too slowly to your customers’ needs, you risk being left behind.
  1. If you respond too quickly, you risk acting without the proper insight.

The truth is, you need to strike the perfect balance, responding in a timely fashion with sound information that adequately supports your response.

One of the panelists on the show, Eric Simonson, director of solution management at SAP, likened this to another well-known tale: “Goldilocks and the Three Bears.”

Your supply chain organization, he suggests, needs to respond to its consumers just right.

Don’t just respond – predict

Responding to the needs of your supply chain customers is one thing. Anticipating consumers’ needs is something else altogether.

“Basically, if we look back into what we’re trying to do traditionally in supply chain management and supply chain planning, specifically,” said guest Jeroen Kusters, senior manager of supply chain management at Deloitte, “is we’re trying to predict the future.”

He admits, however, that “we’re always a little bit wrong.”

How can we change this?

The key is gaining an optimal view of the information you have at your disposal. In addition to taking a deeper dive into your own data, it’s important to have some insight into your supply chain partners’ information. This will enable your company to respond earlier – with greater accuracy – and even help you predict future demand.

Supply chain in the year 2020 and beyond

At one point during the podcast, Bonnie asked her panel of experts what they think the future holds for supply chain management.

Jeroen envisions organizations better integrating their planning, response management, and other operations across the entire supply chain. This will allow companies and their partners to more easily share – and capitalize on – customer insight and other key data.

Eric foresees a world where digital collaboration is much more prominent.

“[M]aybe it’ll start with the supplier side of things,” he says, “and then, eventually … we can get to some of the customer collaboration type, too, to get some better demand visibility.”

With a more holistic view into what’s happening across the entire supply chain, your business is primed to provide well-informed, timely responses to ever-evolving consumer demands.

Srini Bangalore, managing director at Deloitte, believes the immediate future of supply chain revolves around digitalization. But his long-term outlook is more focused on cognitive intelligence.

“I look at cognitive supply chain as a 20-year journey,” he says, “where your machines and computing systems that you use within your supply chain have machine-based intelligence. They can learn, they can problem solve, and they can make decisions on your behalf in the processes in your extended supply chain. The role of a human being is to actually augment the machines.”

A first-rate lesson in digital response and supply chain management

As discussed on the show, predicting the future isn’t easy. But if you’re going to listen to anybody about what direction supply chain is headed in, it ought to be industry thought leaders like Eric Simonson, Jeroen Kusters, and Srini Bangalore.

Check out the entire episode of The Digital Transformation of Your Supply Chain with Game-Changers to hear more expert opinions on digital response and supply chain management from Bonnie and her panel of guests.

Comments

Richard Howells

About Richard Howells

Richard Howells is a Vice President at SAP responsible for the positioning, messaging, AR , PR and go-to market activities for the SAP Supply Chain solutions.

Will Technology Make Custom Clothing More Affordable?

Simon Davies

From tech startups to e-commerce titans, the quest is on to bring custom-made clothing to a new generation whose income would usually prevent them from being able to afford it. Bespoke clothing tends to be exclusive to high-end stores and is often seen as a rare luxury to all but the wealthiest of people.

The problem is that most of us are walking around with comparatively ill-fitting clothes; however, with the introduction of custom-sizing tech, not only could the fashion industry be transformed, your wardrobe could be too.

The problem with the size of our clothes

Despite the fact they are one of the most commonly worn items of clothing in history, we have come to begrudgingly accept that our t-shirts won’t quite fit most of the time. A so-called medium in one store may be too large and too small in the next. The problem dates back to the 1800s, when advances in technology meant that clothing could start to be mass-produced. Prior to this, as all clothing was hand-tailored, customized sizes were the norm and more affordable for the masses, who would simply own far less clothing.

Even t-shirt manufacturers have come to accept that there’s a problem, with one suggesting three primary reasons why t-shirt sizes vary so significantly. First, there is no restriction on standard clothing sizes, so although agencies such as the International Organization for Standardization (ISO) may have certain expectations, physically enforcing clothing sizes is technically impossible. Sizing varies from manufacturer to manufacturer, which brings us to our second reason: vanity sizing.

Vanity sizing is the process of stores labeling clothing in a smaller size than it actually is. The theory is, not unsurprisingly, that people will be happier buying clothes that they feel thinner in. If that makes you feel slightly disturbed, the third reason that the size of clothes differ significantly from store to store is because some businesses do the opposite. In trendier stores, sizes labeled large are anything but. In 2006, the CEO of Abercrombie and Fitch came under fire for eliminating its plus-size clothing, saying that “we don’t market to anyone other than…cool, good-looking people.”

Of course, there’s another reason why machine technology has been inferior to the human hand when producing bespoke clothing: the simple fact that bodies come in a wide range of different sizes. There is no one small, medium, or large, and that is a problem machine production hasn’t circumvented…until now.

The rise of bespoke clothing technology

Custom clothing has been described as the future of fashion, and, through the use of automation, that future may be very likely. Amazon’s patent for an “on-demand” apparel manufacturing system can quickly fill online orders for suits and dresses. This would, in theory, be able to make mass-produced, custom-made clothing, and in turn make custom clothing more affordable.

It works like this: The customer enters their exact measurements and other information like style, color, etc. Various computer-driven systems then produce the clothing. First, the aptly named cut engine cuts out pieces of fabric. Then a robotic arm places the fabric into a conveyer belt, which delivers the pieces to a sewing station, where an automated sewing machine (basically a robot) stitches them together. The entire process is monitored via cameras to ensure quality control.

However, it isn’t just Amazon that’s looking at ways to produce machine-made, custom clothing. In a piece in Apparel News, Andrew Asch talked to the inventors and entrepreneurs behind Susarel, a California company that aims to build a fully integrated vertical factory with an automated sewing component that will eventually produce custom clothing.

The plant will be able to produce all sorts of clothing – not just high-end suits – including t-shirts, yoga pants, leggings, board shorts, and hoodies. Susarel’s project will focus on energy-efficiency and being eco-friendly. However, the important thing for many consumers is whether or not it will actually make custom clothing any cheaper.

Will this service eventually become truly affordable?

Susarel seems to think so. Because this tech will allow it to produce clothes domestically in the United States, it believes it will benefit from tariff-free trade. Tom Keefer, one of the entrepreneurs behind the business, reasons that “[although] the cost of the factory build-out will be higher in order to incorporate these technologies, the resulting efficiencies will makes us cost-competitive with offshore manufacturers who handle the lowest-cost labor pool.”

Susarel is planning a soft launch next year, while Amazon’s patent is still just that: a patent. So all this talk of cheap, robot-made, custom clothing remains hypothetical. However, one business is already using a combination of tech and the human hand to bring genuinely cheaper custom clothes.

Copenhagen-based startup Son Of A Tailor offers t-shirts created to fit the wearer perfectly by producing them via a specially developed online algorithm. It works by asking the user five questions (height, weight, age, jeans, and shoe size), then using what it calls its Ideal Size Algorithm to calculate a perfect-sized tee. The pattern is then laser-cut before humans take over, stitching the garment by hand.

Although Son Of A Tailor only offers t-shirts, this combination of machine and human may be the best way to get affordable, custom clothing today.

For more on technological change in manufacturing, learn 6 Surprising Ways 3D Printing Will Disrupt Manufacturing.

Comments

Simon Davies

About Simon Davies

Simon Davies is a London-based freelance writer with an interest in startup culture, issues, and solutions. He works explores new markets and disruptive technologies and communicates those recent developments to a wide, public audience. Simon is also a contributor at socialbarrel.com, socialnomics.net, and tech.co. Follow Simon @simontheodavies on Twitter.

Taking Learning Back to School

Dan Wellers

 

Denmark spends most GDP on labor market programs at 3.3%.
The U.S. spends only 0.1% of it’s GDP on adult education and workforce retraining.
The number of post-secondary vocational and training institutions in China more than doubled from 2000 to 2014.
47% of U.S. jobs are at risk for automation.

Our overarching approach to education is top down, inflexible, and front loaded in life, and does not encourage collaboration.

Smartphone apps that gamify learning or deliver lessons in small bits of free time can be effective tools for teaching. However, they don’t address the more pressing issue that the future is digital and those whose skills are outmoded will be left behind.

Many companies have a history of effective partnerships with local schools to expand their talent pool, but these efforts are not designed to change overall systems of learning.


The Question We Must Answer

What will we do when digitization, automation, and artificial intelligence eject vast numbers of people from their current jobs, and they lack the skills needed to find new ones?

Solutions could include:

  • National and multinational adult education programs
  • Greater investment in technical and vocational schools
  • Increased emphasis on apprenticeships
  • Tax incentives for initiatives proven to close skills gaps

We need a broad, systemic approach that breaks businesses, schools, governments, and other organizations that target adult learners out of their silos so they can work together. Chief learning officers (CLOs) can spearhead this approach by working together to create goals, benchmarks, and strategy.

Advancing the field of learning will help every business compete in an increasingly global economy with a tight market for skills. More than this, it will mitigate the workplace risks and challenges inherent in the digital economy, thus positively influencing the future of business itself.


Download the executive brief Taking Learning Back to School.


Read the full article The Future of Learning – Keeping up With The Digital Economy

Comments

Dan Wellers

About Dan Wellers

Dan Wellers is the Global Lead of Digital Futures at SAP, which explores how organizations can anticipate the future impact of exponential technologies. Dan has extensive experience in technology marketing and business strategy, plus management, consulting, and sales.

Tags:

Why Millennials Quit: Understanding A New Workforce

Shelly Kramer

Millennials are like mobile devices: they’re everywhere. You can’t visit a coffee shop without encountering both in large numbers. But after all, who doesn’t like a little caffeine with their connectivity? The point is that you should be paying attention to millennials now more than ever because they have surpassed Boomers and Gen-Xers as the largest generation.

Unfortunately for the workforce, they’re also the generation most likely to quit. Let’s examine a new report that sheds some light on exactly why that is—and what you can do to keep millennial employees working for you longer.

New workforce, new values

Deloitte found that two out of three millennials are expected to leave their current jobs by 2020. The survey also found that a staggering one in four would probably move on in the next year alone.

If you’re a business owner, consider putting four of your millennial employees in a room. Take a look around—one of them will be gone next year. Besides their skills and contributions, you’ve also lost time and resources spent by onboarding and training those employees—a very costly process. According to a new report from XYZ University, turnover costs U.S. companies a whopping $30.5 billion annually.

Let’s take a step back and look at this new workforce with new priorities and values.

Everything about millennials is different, from how to market to them as consumers to how you treat them as employees. The catalyst for this shift is the difference in what they value most. Millennials grew up with technology at their fingertips and are the most highly educated generation to date. Many have delayed marriage and/or parenthood in favor of pursuing their careers, which aren’t always about having a great paycheck (although that helps). Instead, it may be more that the core values of your business (like sustainability, for example) or its mission are the reasons that millennials stick around at the same job or look for opportunities elsewhere. Consider this: How invested are they in their work? Are they bored? What does their work/life balance look like? Do they have advancement opportunities?

Ping-pong tables and bringing your dog to work might be trendy, but they aren’t the solution to retaining a millennial workforce. So why exactly are they quitting? Let’s take a look at the data.

Millennials’ common reasons for quitting

In order to gain more insight into the problem of millennial turnover, XYZ University surveyed more than 500 respondents between the ages of 21 and 34 years old. There was a good mix of men and women, college grads versus high school grads, and entry-level employees versus managers. We’re all dying to know: Why did they quit? Here are the most popular reasons, some in their own words:

  • Millennials are risk-takers. XYZ University attributes this affection for risk taking with the fact that millennials essentially came of age during the recession. Surveyed millennials reported this experience made them wary of spending decades working at one company only to be potentially laid off.
  • They are focused on education. More than one-third of millennials hold college degrees. Those seeking advanced degrees can find themselves struggling to finish school while holding down a job, necessitating odd hours or more than one part-time gig. As a whole, this generation is entering the job market later, with higher degrees and higher debt.
  • They don’t want just any job—they want one that fits. In an age where both startups and seasoned companies are enjoying success, there is no shortage of job opportunities. As such, they’re often looking for one that suits their identity and their goals, not just the one that comes up first in an online search. Interestingly, job fit is often prioritized over job pay for millennials. Don’t forget, if they have to start their own company, they will—the average age for millennial entrepreneurs is 27.
  • They want skills that make them competitive. Many millennials enjoy the challenge that accompanies competition, so wearing many hats at a position is actually a good thing. One millennial journalist who used to work at Forbes reported that millennials want to learn by “being in the trenches, and doing it alongside the people who do it best.”
  • They want to do something that matters. Millennials have grown up with change, both good and bad, so they’re unafraid of making changes in their own lives to pursue careers that align with their desire to make a difference.
  • They prefer flexibility. Technology today means it’s possible to work from essentially anywhere that has an Internet connection, so many millennials expect at least some level of flexibility when it comes to their employer. Working remotely all of the time isn’t feasible for every situation, of course, but millennials expect companies to be flexible enough to allow them to occasionally dictate their own schedules. If they have no say in their workday, that’s a red flag.
  • They’ve got skills—and they want to use them. In the words of a 24-year-old designer, millennials “don’t need to print copies all day.” Many have paid (or are in the midst of paying) for their own education, and they’re ready and willing to put it to work. Most would prefer you leave the smaller tasks to the interns.
  • They got a better offer. Thirty-five percent of respondents to XYZ’s survey said they quit a previous job because they received a better opportunity. That makes sense, especially as recruiting is made simpler by technology. (Hello, LinkedIn.)
  • They seek mentors. Millennials are used to being supervised, as many were raised by what have been dubbed as “helicopter parents.” Receiving support from those in charge is the norm, not the anomaly, for this generation, and they expect that in the workplace, too.

Note that it’s not just XYZ University making this final point about the importance of mentoring. Consider Figures 1 and 2 from Deloitte, proving that millennials with worthwhile mentors report high satisfaction rates in other areas, such as personal development. As you can see, this can trickle down into employee satisfaction and ultimately result in higher retention numbers.

Millennials and Mentors
Figure 1. Source: Deloitte


Figure 2. Source: Deloitte

Failure to . . .

No, not communicate—I would say “engage.” On second thought, communication plays a role in that, too. (Who would have thought “Cool Hand Luke” would be applicable to this conversation?)

Data from a recent Gallup poll reiterates that millennials are “job-hoppers,” also pointing out that most of them—71 percent, to be exact—are either not engaged in or are actively disengaged from the workplace. That’s a striking number, but businesses aren’t without hope. That same Gallup poll found that millennials who reported they are engaged at work were 26 percent less likely than their disengaged counterparts to consider switching jobs, even with a raise of up to 20 percent. That’s huge. Furthermore, if the market improves in the next year, those engaged millennial employees are 64 percent less likely to job-hop than those who report feeling actively disengaged.

What’s next?

I’ve covered a lot in this discussion, but here’s what I hope you will take away: Millennials comprise a majority of the workforce, but they’re changing how you should look at hiring, recruiting, and retention as a whole. What matters to millennials matters to your other generations of employees, too. Mentoring, compensation, flexibility, and engagement have always been important, but thanks to the vocal millennial generation, we’re just now learning exactly how much.

What has been your experience with millennials and turnover? Are you a millennial who has recently left a job or are currently looking for a new position? If so, what are you missing from your current employer, and what are you looking for in a prospective one? Alternatively, if you’re reading this from a company perspective, how do you think your organization stacks up in the hearts and minds of your millennial employees? Do you have plans to do anything differently? I’d love to hear your thoughts.

For more insight on millennials and the workforce, see Multigenerational Workforce? Collaboration Tech Is The Key To Success.

Comments