Bias in algorithms, patients' data and zombie capital

November 17, 2019: A roundup of news and perspective on disruptive technology. In this issue: Apple Card and bias, Google and health data, and do VCs matter

N S Ramnath

[Image by mohamed Hassan from Pixabay]

Note: This Week in Disruptive Tech brings to you interesting stories that highlight a new development or offer an interesting perspective on technology and society. Plus, a curated set of links to understand how technology is shaping the future, here in India and across the world. If you want to get it delivered to your inbox every week, subscribe here.

Why the Goldman Sachs - Apple Card bias allegations matter to India

When former Goldman Sachs director and Mckinsey chief Rajat Gupta was in Bangalore last weekend to speak about his book at the Bangalore Literature Festival, he repeated his claim to Bloomberg Quint that he didn’t commit any crime, and that he was made a scapegoat for the financial crisis. When a tweet went viral on how a tech entrepreneur got 20X the credit limit than his wife did from Apple Card, Goldman Sachs, the bank behind the credit card, faced all the heat. The New York State Department of Financial Services launched a probe on the credit practices of the bank. US presidential candidate Elizabeth Warren declared, “Let’s just tell every woman in America, ‘You might have been discriminated against, on an unknown algorithm, it’s on you to telephone Goldman Sachs and tell them to straighten it out’.” 

Goldman Sachs says it has done no wrong, its algorithms are not biased, it doesn't even collect gender or marital status for the card, and that it’s not a black box, and the bank can trace the decision back to specific factors from specific credit reports. Some pointed out it’s not even in Goldman Sachs’s interest to be biased against gender. All it would want is to minimise defaults. Yet, few will deny that algorithmic bias is a real and serious issue. 

So, is the outrage against Goldman Sachs merely an attempt to make a scapegoat out of a well known, perhaps innocent, entity for a more widespread problem? We won’t know till there is more information. The whole episode, however, has some important lessons for India even as Indians become data-rich—and soon we will have technology infrastructure and laws to share specific data for specific benefits. Two of them stand out. 

  • One: it's important to distinguish between honest (inevitable) mistakes, mistakes that result from individual biases, institutional biases and structural biases. It's not even clear if we are aware of all these in India. 
  • Two: Be extremely wary of black boxes. Be in a position to explain why you took a certain decision that has impacted your customer or a partner. 

Does permission matter?

For all the technological progress made in the last several decades, we still haven’t solved some of our most pressing healthcare problems. Crunching data can help. But, there is a huge resistance to let tech meet data, especially in the healthcare sector. Health data is sensitive, and privacy matters. So, whenever there’s news about tech companies accessing health data, it triggers paranoia. Earlier this week, The Wall Street Journal reported that Google is accessing a wide range of patient data, including names and diagnoses from Ascension, a non-profit hospital system that spans 2,600 hospitals in 21 states in the US. Doctors and patients weren’t informed. Google has apparently done nothing illegal. Nor is Google alone in doing this. “All of Google’s work with Ascension adheres to industry-wide regulations (including HIPAA—Health Insurance Portability and Accountability Act) regarding patient data, and come with strict guidance on data privacy, security and usage,” Google said in a note. 

Yet, why did it set off so much fear? Part of the reason is Google’s business model. It’s about harvesting data and making money through ads. It also has to do with Google’s ‘easier to say sorry, than ask for permission’ culture. And more importantly, it’s happening now—when people are much more aware of the dangers of data leaks and misuse, and the starry-eyed optimism about tech companies is long gone

Are VCs useful?

Toronto-based password manager 1Password raised $200 million in Series A from venture capitalists led by Accel. It caught everyone’s attention not only because it was a huge amount for a Series A, but also because the company went for venture capital funding a full 14 years after it was founded. The reason, according to one of the VCs, is growth: “We realised that if the company could get to this sort of scale organically, we can’t imagine what could happen if we pushed deeper into enterprise.” It also kicked off discussions about why didn’t the company go straight for an IPO. Do VCs matter?  

Vinod Khosla doesn’t have a high regard for his fellow VCs. “Ninety percent really add no value, and I truly believe 70% of them reduce the potential of a company,” he says. Sridhar Vembu, who built Zoho without relying on VC money, told Forbes India, “A VC is not really a capitalist in the true sense—‘money broker’ would be a far more accurate term; after all, a real capitalist is not going to collect an annual commission on the money he or she invests…that very idea would be absurd.” A friend recently shared a chart which showed how top unicorns lost value after they got listed. Another friend said, forget entrepreneurs, most VCs don’t even add value to their limited partners: “50% of VC funds lose value. Only 5% return more than 3X.” 

@dhh whose tweet kicked off the debate on Goldman Sachs’s gender bias started yet another debate on VCs, tweeting: “Venture capital, like private equity, is money without a soul. Zombie capital that scours the land to eat the brains of entrepreneurs.” Responses show that the issue is far from settled.

Explore

  • How big tech is dragging us towards the next financial crash | The Guardian 
  • How to Invest in medicine’s future, biotech roundtable | Barron’s
  • As SpaceX launches 60 Starlink satellites, scientists see threat to ‘astronomy itself’ | NYTimes
  • The digital roadmap: How developing countries can get ahead | Pathways for Prosperity

About the author

N S Ramnath
N S Ramnath

Senior Editor

Founding Fuel

NS Ramnath is a member of the founding team & Lead - Newsroom Innovation at Founding Fuel, and co-author of the book, The Aadhaar Effect. His main interests lie in technology, business, society, and how they interact and influence each other. He writes a regular column on disruptive technologies, and takes regular stock of key news and perspectives from across the world. 

Ram, as everybody calls him, experiments with newer story-telling formats, tailored for the smartphone and social media as well, the outcomes of which he shares with everybody on the team. It then becomes part of a knowledge repository at Founding Fuel and is continuously used to implement and experiment with content formats across all platforms. 

He is also involved with data analysis and visualisation at a startup, How India Lives.

Prior to Founding Fuel, Ramnath was with Forbes India and Economic Times as a business journalist. He has also written for The Hindu, Quartz and Scroll. He has degrees in economics and financial management from Sri Sathya Sai Institute of Higher Learning.

He tweets at @rmnth and spends his spare time reading on philosophy.