Ravi is a senior police officer with vast experience in riot control and cyber-policing. Since one year, he has been the Superintendent of Police (SP) of a district with a history of frequent rioting.
Last year, Ravi had sought installation of an AI enabled software for predictive policing. This system has been operational for approximately six months. This new system employs advanced algorithms for capturing the biometric data of persons in a crowd and swiftly relating it to a data library. This has enabled the police to identify the persons involved in various crimes.
The system has identified an immigrant and low-income neighbourhood as a centre for gang violence and drug trafficking. Aided by this AI analysis, the local police has focused its patrolling, preventive detentions and establishing checkposts. Consequently, public order and law enforcement has visibly improved.
Last week, some community leaders, civil rights lawyers and human rights activists visited Ravi’s office. They submitted a memorandum that the new system is faulty as it is based on incorrect historical data caused by social biases and discriminatory policing. The memorandum also alleges that the increased surveillance has created a climate of tension amongst residents. This feeling is aggravated by the fact that the residents are not aware of the data noted against their names.
(a) What are the ethical issues including biases involved in the use of AI in data-driven policing?
(b) Place yourself in Ravi’s role and discuss the alternatives available. Justify the action that optimises compliance with ethics.

Answer outline

Introduction

Predictive policing uses past crime data to forecast where crime will occur or who may commit it. Its accuracy is only as good as its data, and in this district the data may reflect who was policed rather than who offended.

(a) Ethical issues and biases

• Historical data bias: if past patrols and arrests were concentrated on poor and migrant areas, the algorithm learns that pattern and labels the same people as risky.
• Feedback loop: more patrols in the flagged area produce more recorded offences there, which the system reads as proof that it was right.
• Discrimination: targeting a neighbourhood by income and migrant status raises equality concerns under Articles 14 and 15; it can turn into collective suspicion.
• Privacy and proportionality: crowd biometric capture without a specific law, notice or limits fails the legality, legitimate aim and proportionality test laid down in Puttaswamy (2017).
• Liberty: preventive detentions guided by an opaque score weaken the safeguards of Articles 21 and 22.
• Transparency and accountability: residents do not know what data is held against them and cannot correct it; officers may defer to the machine (automation bias).
• Chilling effect and loss of trust: heavy surveillance creates tension, which can undermine the cooperation policing needs.

(b) Alternatives before Ravi

• Continue as is, citing improved order: ignores credible complaints of bias and illegality; short term gains, long term loss of legitimacy.
• Scrap the system entirely: removes the harm but throws away a useful tool and may be seen as yielding to pressure without evidence.
• Calibrated course: keep the tool for decision support, suspend its use as the sole basis for preventive detention and checkposts in the flagged area, and order an independent audit.

Justification of the calibrated course

• Commission an external audit of training data, error rates and disparate impact; fix or retrain the model on verified data.
• Require a human officer to record reasons, based on objective facts, before any coercive action; no detention on a risk score alone.
• Data governance: purpose limitation, retention limits and access logs, aligned with the Digital Personal Data Protection Act, 2023; inform residents and create a grievance and correction channel.
• Engage the community: meet the memorandum signatories, hold beat meetings and share results of the review; add social measures (youth programmes, de-addiction) to pure enforcement.
• This balances public order with fairness and rule of law, and follows principles of transparency, accountability and non-discrimination.

Conclusion

Technology should serve, not replace, a fair police officer's judgment. By auditing the tool, limiting its coercive use and opening it to scrutiny, Ravi protects both public order and the rights and trust of the very residents he is meant to protect.