algorithms – FemLab.Co

[By Samarth Gupta]

The 21^st century has seen the rapid rise of algorithms to control, manage, analyze, and perform any number of operations on data and information. Some believe that decision-making processes can be significantly improved by delegating tasks to algorithms, thereby minimizing human cognitive biases. As Tobias Baer argues:

“The rise of algorithms is motivated by both operational cost savings and higher quality decisions […]. To achieve this efficiency and consistency, algorithms are designed to remove many human cognitive biases, such as confirmation bias, overconfidence, anchoring on irrelevant reference points (which mislead our conscious reasoning), and social and interest biases (which cause us to override proper reasoning due to competing personal interests).”

However, there are increasing concerns that algorithmic decisions may be as least as biased and flawed as human decisions. One worry is that, they may exhibit ‘race’ or ‘gender’ bias as Amazon’s Artificial Intelligence (AI) based recruitment tool demonstrated, compelling the company to scrap it altogether. According to a Reuters article, this was because “Amazon’s computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry.”

With the boom in AI, algorithms are being viewed with renewed scrutiny given its sweeping impact on our everyday lives, including how it will shape the future of work. As a student of computer science, I have been actively coding since the last 5 years. I have been involved in various open source communities where programmers develop, test and debug software issues. Moreover, having done internships at different institutions across India, I have closely interacted with programmers from different geographies and socio-economic backgrounds, learning about their perspectives. Reading the ongoing news on algorithmic bias, ethics, and how these systems can discriminate, got me thinking. I wanted to understand how these biases find their way into algorithms. Are coders even aware of the possibility of bias in programming? What are the different (technical as well as non-technical) approaches used by them to eliminate these biases? Do they receive any kind of ethical/value-based training from the company they work for? Did they face dilemmas of programming something they didn’t want to? If so, then what was their approach to solving the issue?

To explore these questions, I did a pilot study by interviewing software engineers working in different tech corporations across India. The participants were recruited from my school alumni group as well as members from different programming groups where I have contributed/contribute in the software development cycle. The purpose was to explore the notions of ethics and human values coded into the algorithms based on a programmer’s perspective. The interviewees were selected keeping certain diversity parameters in mind – level in the company, kind of work they do and their years of experience. The companies too ranged in size from start-ups to software giants.

Image credit: Pixnio

The responses were revealing. Firstly, they suggest that none of the organizations conducts any kind of ethical or moral training sessions for the software engineers. In the words of one of the participants:

“No bro…there weren’t any such trainings […]. There were classes on corporate communication and email etiquettes. But majority of the training was focussed on Java and Kubernetes (Computer Software) […]. The manager and other teammates are always there for help in case of any difficulty.”

It was also noted that most of these companies are dominated by male workers, as has been highlighted in the media. According to a diversity report by Google, women make up only about 20% of the engineering staff. This lack of representation inevitably reflects stereotypes about women in the software.

Tech companies often develop innovative products that cater to extremely large audiences. For instance, WhatsApp has close to two billion active users across 180 countries. Developing a product of such a massive scale depends on complex databases, data structures and algorithms. Even basic software programmes are technically intricate, and involve teams of designers, managers, and engineers with different specializations. However, very often, the code is subdivided into smaller modules and specialized teams work on those aspects independently of other teams. In the words of one of the interviewees:

“Actually, everything happens in a hierarchical manner… first the senior management decides what all features should be there in the product…after that the product managers refine it […] finally the software engineers are asked to code for the features. For example, if I am developer engineer I will only code for the GPS route optimization… testing people will do different types of testing for checking if the code is stable […] designers will redefine and redesign the UI/UX […] and so on.”

This subdivision of the tasks according to the hyper-specialization of the employees prevents them from getting to know the broader context as they are limited to their own ‘tiny’ specialized job. This micro-division of work according to the hyper-specialization has two major consequences:

Employees often feel alienated from the bigger picture as they are confined to their specialized task – debugging the code, testing it on different pre-decided parameters or developing a module of software. This alienation may reflect in the software they develop and thereby affect the users. For instance, a programmer who is hesitant to communicate with others may favour developing an AI-powered chatbot for implementing help functionality in his/her software instead of developing an algorithm that works by asking for help from other users.
It becomes extremely hard to implement ethics in this institutional system. The engineers most often have no contextual awareness of the product they are developing. They just focus on the efficiency of their code, its scalability and efficient design without any regard for ethics or human values.

Most software companies and start-ups proudly claim that they have a casual working environment, provide different kinds of perks, flexibility, recreation facilities and many other benefits to the employees. The organization might have a culture of empathy, diversity and collaboration. They may be empathetic while caring for the employee’s family, inclusive with recruitments specifically for women, and LGBTQ groups, for instance. In the words of one of our interviewees:

“The company has a very sweet, family-like atmosphere […]. There are lots of fun activities, no restrictions on the dress code or fixed work timings…to the extent that on Friday evenings, the whole team goes out for some fun activity…But friend, there is no value of our suggestions…See, what happens is that in the weekly meetings we are asked to give our suggestions on the product, but you understand the working of corporate […] it’s just for formality so that we don’t feel excluded…strict top down orders as given by the senior management are followed […] we are just for coding their desirable features.”

This shows that when it comes to business, companies may become strictly hierarchical, with the junior employees having no say in any of the products that they code for. It clearly reflects the contradiction between the culture that a company envisages and what the employees actually experience on the job.

Another dilemma that programmers face is conflict of interest. Even if they encounter unfair work practices, they must choose between raising their voices against it and risking their job or remain on the safer side and just keep on moving things forward. As one interviewee remarked, when asked whether (s)he had faced any such dilemma:

“Yes…this happened while I was working as an intern with [name of organization]. My task was to analyse the data of the financial transactions…I found out later that the analysis that I was doing was being used to gauge the consumer’s behaviour, but users were not informed…I felt like raising my voice…talked to my friends there [who were also working as interns]…finally decided not to do anything and continue with my project…as nothing was going to change with my objection and I would also lose my PPO [Pre placement offer]…company’s CTC [Cost to company] was very good…why should I interfere with it?”

The final dilemma that emerged in the interviews had to do with designing ethical AI algorithms. Programmers are careful to not explicitly insert any bias in their algorithms. For instance, no programmer would insert a line like “if the gender is female then assign a lower score to the resume”. The bias creeps in at the time of ‘nurturing’ the algorithm – i.e. training it on the data. Real-world data reflects human limitations and judgment, and this invariably gets inscribed into the algorithm when it is trained on real world data.

Hence, the need of the hour is to sensitise the data annotators and collectors about this problem, diversify data to become more representational and subject it to rigorous testing to eliminate stereotypes and biases. The platform-based gig economy has been rising consistently, as reported by the Economic Times and hence will tremendously affect the future of work. Algorithmic bias plays an important role here. At Uber Eats, for instance, gig workers blame the algorithmic system for the unexplained changes that led to slashing their jobs and salaries. Many women are excluded from the gig work force due to the lack of value-based design for women in digital platforms, as reported by the Observer Research Foundation, an Indian think tank.

Algorithms (and code in general) are increasingly impacting our lives in this age of the platform society. If these tools are to work for everyone, in a fair and just manner, values of fairness and justice should be part of their framework. It is not enough to appeal to programmers through measures such as ethical guidelines. Algorithmic biases need to be met in the same complex and wide way they affect us – from individuals to organizations, regulators, and society by and large.

[By Pallavi Bansal]

I remember getting stranded in the middle of the road a few years ago when an Ola cab driver remarked that my trip had stopped abruptly and he could not take me to my destination. Frantic, I still requested him to drop me home, but he refused saying he cannot complete the ride since the app stopped working. On another unfortunate day, I was unable to find a cab back home as the drivers kept refusing to take up what they saw as a long ride. When I eventually found a cab, the driver continuously complained about how multiple short rides benefit him more. I tried to tip him after he finished the ride, but instead he requested me to book the same cab again, for a few kilometres, as that would reap more rewards. While I wanted to oblige, I couldn’t find the same driver, even though he had parked his car right outside my house. In yet another incident, I spent the entire night at the airport as I was terrified to book a cab at that late hour. I regretted not checking the flight timings before confirming the booking, having overlooked the fact that women need to be cautious about these things.

Image credit: Pixabay / Pexels

Although my first response was to blame the cab drivers for what I saw as an unprofessional attitude, it slowly dawned on me that they have their own constraints. In the first scenario, the app had actually stopped working, so he couldn’t complete the ride due to the fear of getting penalized, which also resulted in a bad rating by me. In the second situation, I wondered why the algorithms reward shorter rides rather than longer ones. Moreover, how do they assign drivers if proximity isn’t the only factor and why was my driver not aware of that? In the third instance, why couldn’t I be assigned a woman driver to make me feel safer when traveling late at night?

I spoke to a few senior managers and executives working at popular ride-sharing apps in India to find the answers.

Constant tracking

A senior manager of a well-known ride-sharing platform explained their tracking practices on condition of anonymity:

“The location of driver-partners is tracked every two-three seconds and if they deviate from their assigned destination, our system detects it immediately. Besides ensuring safety, this is done so that the drivers do not spoof their locations. It has been noticed that some drivers use counterfeit location technology to give fake information about their location – they could be sitting at their homes and their location would be miles away. If the system identifies anomalies in their geo-ping, we block the payment of the drivers.”

While this appears to be a legitimate strategy to address fraud, there is no clarity on how a driver can generate evidence when there is an actual GPS malfunction. Another interviewee, a person in a top management position of a ride-sharing company, said, “it is difficult to establish trust between platform companies and driver-partners, especially when we hear about drivers coming up with new strategies to outwit the system every second day.” For instance, some of the drivers had a technical hacker on board to ensure that booking could be made via a computer rather than a smartphone or artificially surging the price by collaborating with other drivers and turning their apps off and on again simultaneously.

Though the ‘frauds’ committed by the drivers are out in the public domain, it is seldom discussed how constant surveillance reduces productivity and amplifies frustration resulting in ‘clever ways’ to fight it. The drivers are continuously tracked by ride-sharing apps and if they fail to follow any of the instructions provided by these apps, they either get penalized or banned from the platform. This technology-mediated attention can intensify drivers’ negativity and can have adverse effects on their mental health and psychological well-being.

Algorithmic-management

Algorithms control several aspects of the job for the drivers – from allocating rides to tracking workers’ behaviour and evaluating their performance. This lack of personal contact with the supervisors and other colleagues can be dehumanizing and disempowering and can result in the weakening of worker solidarities.

When asked if the algorithms can adjust the route for the drivers, especially for women, if they need to use the restroom, a platform executive said, “They always have the option not to accept the ride if there is a need to use the washroom. The customers cannot wait if the driver stops the car for restroom break and at the same time, who will pay for the waiting time?”

While this makes sense at first glance, in reality, algorithms of a few ride-sharing platforms like Lyft penalize drivers in such cases by lowering their assignment acceptance rate (number of ride requests accepted by the driver divided by the total number of requests received). Lee and team, HCI (Human Computer Interaction) scholars from Carnegie Mellon University explored the impact of algorithmic-management on human workers in context of ride-sharing platforms and found:

“The regulation of the acceptance rate threshold encouraged drivers to accept most requests, enabling more passengers to get rides. Keeping the assignment acceptance rate high was important, placing pressure on drivers. For example, P13 [one of the drivers] stated in response to why he accepted a particular request: ‘Because my acceptance rating has to be really high, and there’s lots of pressure to do that. […] I had no reason not to accept it, so […] I did. Because if, you know, you miss those pings, it kind of really affects that rating and Lyft doesn’t like that.’”

Uber no longer displays the assignment acceptance rate in the app and states that it does not have an impact on drivers’ promotions. Ola India’s terms and conditions state “the driver has sole and complete discretion to accept or reject each request for Service” without mentioning about the acceptance rate. However, Ola Australia indicate the following on their website: “Build your acceptance rate quickly to get prioritised for booking! The sooner and more often you accept rides (as soon as you are on-boarded), the greater the priority and access to MORE ride bookings!”

The lack of information coupled with ambiguity complicates the situation for drivers, who would try not to reject the rides under any circumstances. Moreover, the algorithms are designed to create persistent pressure on the drivers by using psychological tricks as pointed out by Noam Scheiber in an article for The New York Times:

“To keep drivers on the road, the company has exploited some people’s tendency to set earnings goals — alerting them that they are ever so close to hitting a precious target when they try to log off. It has even concocted an algorithm similar to a Netflix feature that automatically loads the next program, which many experts believe encourages binge-watching. In Uber’s case, this means sending drivers their next fare opportunity before their current ride is even over.”

The algorithmic decision-making also directs our attention to how the rides are allocated. The product manager of a popular ride-sharing app said:

“Apart from proximity, the algorithms keep in mind various parameters for assigning rides, such as past performance of the drivers, their loyalty towards the platform, feedback from the customers, if the drivers made enough money during the day etc. The weightage of these parameters keep changing and hence cannot be revealed.”

All the four people interviewed said that number of women driving professionally is considerably low. This makes it difficult for the algorithms to match women passengers with women drivers. Secondly, this may delay ride allocation for women passengers as the algorithms will first try to locate women drivers.

A lack of understanding of how algorithms assign tasks makes it difficult to hold these systems accountable. Consequently, a group of UK Uber drivers have decided to launch a legal bid to uncover how the app’s algorithms work – how the rides are allocated, who gets the short rides or who gets the nice rides. In a piece in The Guardian, the drivers’ claim says:

“Uber uses tags on drivers’ profiles, for example ‘inappropriate behaviour’ or simply ‘police tag’. Reports relate to ‘navigation – late arrival / missed ETA’ and ‘professionalism – cancelled on rider, inappropriate behaviour, attitude’. The drivers complain they were not being provided with this data or information on the underlying logic of how it was used. They want to [know] how that processing affects them, including on their driver score.”

The fact is that multiple, conflicting algorithms impact the driver’s trust in algorithms as elaborated in an ongoing study of ‘human-algorithm’ relationships. The research scholars discovered that Uber’s algorithms often conflict with each other while assigning tasks, such as, drivers were expected to cover the airport area but at the same time, they received requests from a 20-mile radius. “The algorithm that emphasizes the driver’s role to cover the airport was at odds with the algorithm that emphasizes the driver’s duty to help all customers, resulting in a tug o’ war shuffling drivers back and forth.” Similarly, conflict is often created when drivers are in the surge area and they get pings to serve customers somewhere out of the way.

Ultimately, we need to shift from self-optimization as the end goal for workers to that of humane algorithms – that which centres workers’ pressures, stress, and concerns in this gig economy. This would also change the attitudes of the passengers, who need to see platform drivers as human drivers, facing challenges at work, like the rest of us.