Home / Blog / Humans have bias, machines too

Humans have bias, machines too

The shining Apple credit card hides an algorithm that decides how much trustworthy (or risky) you might be. It is not alone. Image credit Apple.

As 2019 wraps up I took a look at what went wrong with technologies this year. Unfortunately there is plenty, and you can Google for tech failures 2019 and spend some time browsing several of them.

I am not surprised at all. We have had, in 2019 as in the previous years, plenty of technology advances, plenty of new products released to the market. To get a broad figure, over 700,000 products have been launched at CES in the last 50 years, that means that just in the electronic consumer market we have had some 14,000 products per year (you can take a look at a few of the products announced at CES 2019 here). Overall, the estimate is over 35,000 new products were launched worldwide in 2019 and the vast majority of them failed. (these figures do not take into account the number of apps released, just on the Android store there have been over 6,000 apps released every day this year,  the figure includes update to existing apps, meaning oover 2 million releases in 2019!).

The estimates on products failures range from 70 to 95%. Of course we would have to set a metrics first, i.e. to agree when a product is considered a failure. One metrics could be products that do not generate sufficient revenue to cover the cost (over what period of time?) but one could claim that a product may not have generated revenues but served to create brand awareness, to prepare the market for a revenue.successful one… There are also products that are released for free, so you cannot apply a revenue based metrics to them. It is tricky. We can anyhow agree to the statement that a large percentage of products fail on the market.

There are a number of reasons for this abundance, both in launch and failure and you can find plenty of discussion on the web on this topic. To me the main ones are:

  1. the possibility offered by the new production ecosystem to launch new products with very limited (upfront) capital
  2. the low transaction cost in the whole value chain (it cost very little to market niches (use social media, release ads on YouTube…) and in the logistic distribution chain that has become a pay-per-use service -e.g. you can piggy back on Amazon-, softwarization of products is slashing down manufacturing cost,…
  3. copycats are skyrocketing resulting in continuous linear innovation that in turns kills just released products
  4. the tendency to do market study by just releasing the product and see how the market reacts (this includes having the users debugging the products…)

New products (and services) are also generating new issues. The existence of bias is the one that prompted me to write this post. That each of us, humans, has a bias is no surprise, We are strongly influenced by our environment and our cultural roots. Even though we would like to think we are “objective” … we are not. We may try our best to overcome bias but in many cases we are not aware of our own biases. Hence the impossibility to correct them. Participating in a rich context full of diversity surely helps since others may point out our biases (or just looking at others may make us perceived our biases).

What has been arising in 2019 is that also machines can be biased. The way machines are learning starting from data in thee cyberspace has been shown to lead to biases. This is an issue particularly for AI based systems and the problem is that we may not realise their level of bias.

An example is the algorithm used to evaluate the risk, and trustworthiness of credit card clients. This evaluation used to be done by some (trained) humans, now more and more it is done by AI based algorithms. The 2019 release of the Apple credit card, apparently more client friendly in terms of accepting new subscriptions, has been shown to have a gender bias. Males are more likely to get a higher credit line limit than females. An entrepreneur has got a spending limit 10 times higher than the one his wife got, even though they have a common account.

Of course, you can say that machines may be biased because we are transferring our biases to them. However this is not completely the case. We have created algorithms that support the learning but we are not controlling them, we actually know less and less about the machine “knowledge” and the machine reasoning. There are now researches aiming at addressing this problem, trying to find ways for us, humans, to understand how a machine “thinks”! I guess the next decade will be partly devoted in solving a problem that we managed to create in this one.

About Roberto Saracco

Roberto Saracco fell in love with technology and its implications long time ago. His background is in math and computer science. Until April 2017 he led the EIT Digital Italian Node and then was head of the Industrial Doctoral School of EIT Digital up to September 2018. Previously, up to December 2011 he was the Director of the Telecom Italia Future Centre in Venice, looking at the interplay of technology evolution, economics and society. At the turn of the century he led a World Bank-Infodev project to stimulate entrepreneurship in Latin America. He is a senior member of IEEE where he leads the Industry Advisory Board within the Future Directions Committee and co-chairs the Digital Reality Initiative. He teaches a Master course on Technology Forecasting and Market impact at the University of Trento. He has published over 100 papers in journals and magazines and 14 books.