[By Pallavi Bansal]
Imagine a scenario in which you do not get shortlisted for a job interview – not because you are underqualified – but because the algorithms were trained on data sets that excluded or underrepresented your gender for that particular position. Similarly, you found out that you are consistently paid less than your colleagues in a sales job – not because of your inability to fetch clients or customers for the company – but because the rewarding algorithms favoured clients belonging to a certain religion, race or ethnicity. Further, you are asked to leave the company immediately without any notice or opportunity to interact with your manager – not because you committed a mistake – but because the clients rated you low based on prejudice.
While these biases, favouritism and discrimination could soon become a reality in mainstream workplaces due the exponential growth of decision-making algorithms, it is already causing disruption in the online gig economy. Recently, researchers at George Washington University found social bias in the algorithms related to dynamic pricing used by ride-hailing platforms Uber, Lyft and Via in Chicago, US. The study found “fares increased for neighbourhoods with a lower percentage of people above 40, a lower percentage of below median house price homes, a lower percentage of individuals with a high-school diploma or less, or a higher percentage of non-white individuals.” The authors of this paper Akshat Pandey and Aylin Caliskan told the American technology website, VentureBeat, “when machine learning is applied to social data, the algorithms learn the statistical regularities of the historical injustices and social biases embedded in these data sets.”
These irregularities are also spotted in relation to gender. A study conducted by Stanford researchers documented a 7% pay gap in favour of men, using a database of a million drivers on Uber in the United States. However, this study highlighted an even bigger problem that the researchers attributed to the following factors – differences in experience on the platform, constraints over where to work (drive), and preference for driving speed. A Cambridge University researcher Jennifer Cobbe told Forbes, “rather than showing that the pay gap is a natural consequence of our gendered differences, they have actually shown that systems designed to insistently ignore differences tend to become normed to the preferences of those who create them.” She said the researchers shifted the blame to women drivers for not driving fast enough and ignored why the performance is evaluated on the basis of speed and not other parameters such as safety. Further, in context to women workers in the Indian gig economy, it is imperative to understand whether these biases are socially inherent. For instance, if certain platform companies segregate occupations based on gender, then the resulting pool will inherently lack gender variation. This also compels us to ponder whether the concentration of female labour in beauty and wellness services, cleaning or formalised care work is a result of an inherent social bias or technical bias.
To make sense of all of this and understand how we can improve the design of these digital labour platforms, I spoke to Uday Keith, a Senior AI Developer with Wipro Digital in Bengaluru. His responses drew my attention towards Informatics scholar Bardzell’s feminist human-computer interaction design paradigm, which I use to contextualize them.
PB: How can we overcome biases in algorithms?
UK: First of all, algorithms are not biased, it is the datasets which are biased. The imbalances in the datasets can be corrected via a method known as SMOTE (Synthetic Minority Over-sampling Technique) where the researchers recommend over-sampling the minority and under-sampling the majority class. In order to achieve this, we need to bring diversity to our training datasets and identify all the missing demographic categories. If any category is underrepresented, then the models developed with this data will fail to scale properly. At the same time, it is essential for the AI developers to continuously monitor and flag these issues as the population demographics are dynamic in nature.
This points us toward the two core qualities proposed by Bardzell – Pluralism and Ecology. According to her, it is important to investigate and nurture the marginal while resisting a universal or totalizing viewpoint. She stresses to consider the cultural, social, regional, and national differences in order to develop technology. The quality of ecology further urges designers to consider the broadest contexts of design artifacts while having an awareness of the widest range of stakeholders. This means AI developers cannot afford to leave out any stakeholder in the design process and should also consider if their algorithms would reproduce any social bias.
PB: Can there be a substitute for the gamification model?
UK: To simplify the process and ensure equity in the gig economy, platform companies can advise AI developers to introduce a “rule”. This would mean fixing the number of minimum rides or tasks a platform worker gets in a day, which can also help in ensuring a minimum wage to them and provide a certain level of income security. The introduction of a fixed rule can even eliminate social biases as this would not result in a particular gender or social group getting less work. Further, the reward system can undergo a major overhaul. For instance, rather than incentivizing them to drive more and indulge in compulsive game-playing, platform companies can build algorithms that provide financial rewards when the drivers follow traffic rules and regulations, drive within permissible speed limits, and ensure a safe riding experience. In fact, we can even provide options to the customers where they could be given discount coupons if they allow drivers to take short breaks.
Elaborating on participation, Bardzell suggests ongoing dialogue between designers and users to explore understanding of work practices that could inform design. This also means if the platform companies and AI developers are oblivious to the needs and concerns of labour, they may end up designing technology that could unintentionally sabotage users. Secondly, an advocacy position should be taken up carefully. In the earlier example, “driving fast” was considered as a performance evaluator and not “safety”, which usually happens because the designers run the risk of imposing their own “male-oriented” values on users.
PB: How work allocation can be more transparent?
UK: Well, deep learning algorithms used by various companies have a “black box” property attached to them to a certain extent. These algorithms are dynamic in nature as they keep learning from new data during use. One can only make sense of this by continuously recording the weightage assigned to the pre-decided variables.
The quality of self-disclosure recommended by Bardzell calls for users’ awareness of how they are being computed by the system. The design should make visible the ways in which it affects people as subjects. For instance, platform companies can display the variables and the corresponding algorithmic weightage per task assigned on the smartphone screen of the workers. So, if a platform driver has not been allocated a certain ride due to his past behaviour, then the technology should be transparent to reveal that information to him. Uncovering the weightage given to various decision-making algorithms will enable the platform worker to reform their behaviour and gives them a chance to communicate back to the companies in case of discrepancies or issues.
PB: How can we improve the rating systems?
UK: The platform companies have started using qualitative labels that could help users to rate the workers better. However, we do need to see whether sufficient options are listed and suggest changes accordingly. Moreover, if we want to completely avoid the numerical rating system, we can ask the users to always describe their feedback by writing a sentence or two. This can be analysed using Natural Language Processing (NLP), a subfield of Artificial Intelligence that helps in understanding human language and derive meaning.
Bardzell writes about the quality of embodiment in respect to meaningful interactions with the technology and acknowledging the whole humanness of individuals to create products that do not discriminate based on gender, religion, race, age, physical ability, or other human features. This concept should also be applied in relation to how users rate workers and whether they discriminate on the basis of appearances or other factors. Hence, there is a strong need to include the qualitative rating systems along with the quantitative ones.
Additionally, Uday Keith recommends defining “ethics” and frequently conducting ethics-based training sessions since a diverse set of people form the team of data scientists, which comprises of roughly 10% of women in an urban Indian city Bengaluru. He concluded by remarking that the issues in the platform economy are more of a system design fault than that of an algorithmic design – the companies consciously want to operate in a certain way and hence do not adopt the above recommendations.
These pointers make the case for the adoption of a feminist design framework that could bring about inclusive labour reforms in the platform economy. As Bardzell says, “feminism has far more to offer than pointing out instances of sexism,” because it is committed to issues such as agency, fulfilment, identity, equity, empowerment, and social justice.