Harnessing Big Data to Revolutionise Public Policy

Big Data has been the buzzword of the century, grabbing headlines for good and bad reasons. For some, the term triggers anxiety due to concerns about privacy invasion. For others, it is a powerful tool with the potential to solve some of the world’s toughest problems. Whether one is worried about its implications or excited about its possibilities, big data is undeniably a major force shaping our future.

Before delving deep into the topic of how harnessing big data can revolutionise public policy, it becomes important to understand what exactly big data is. 

In their work “Big Data in the Public Sector: Lessons for Practitioners and Scholars,” Desouza and Jacob define “big data” as massive collections of information from various sources like sensors, human input, and monitoring systems (Desouza & Jacob, 2017). Beyond its sheer size, big data is significant due to its potential to transform lives by enabling more effective and efficient policies. Let us now delve into how big data can revolutionise public policy.

Importance of big data in revolutionising public policy

• Traditionally, public policy decision-making relies on outdated information and lengthy decision-making. Big data has changed the game altogether; it has enabled real-time decision-making. Governments can now make more informed decisions and react swiftly to emerging issues. Policies can continuously be refined based on real-time updates due to continuous evaluation of policies. 
• Big data has enabled anomaly detection. It has enabled authorities to detect behavioural anomalies and take preemptive actions. Governments create digital models of irregularities and apply them to massive datasets to identify potential abnormalities. For example, HM Revenue and Customs used the British Connect System to detect tax evasion. Through analysis of extensive datasets, authorities could detect irregularities with up to 95% accuracy, leading to effective enforcement actions.
• Big data has the potential to revolutionise public service delivery. For example, in Singapore, big data has been used to improve the experience for millions of daily commuters. Real-time data monitoring has been used to manage traffic and public transportation effectively. Singapore uses advanced analytics to monitor over 12 million transactions daily, improving the overall efficiency of the transport system. 
• Big data has proven to be helpful in gauging public sentiment. With the help of sentiment analysis tools, governments can gauge public opinion on public policies in real-time. It can help formulate policies that are more aligned with public expectations. For example, the Vizie tool developed by CSIRO can be used to monitor social media and other online platforms to understand public grievances and capture public sentiment to take action before issues escalate.

CHALLENGES

• Challenges related to data integrity have often come to the forefront. The Facebook- Cambridge Analytica scandal of 2018 led to criticism of Facebook’s data handling practices, which eventually led to facebook having to pay a $5 billion fine to the Federal Trade Commission and face ongoing legal and ethical challenges related to data privacy and trust. 

• Over-reliance on big data is also a cause of concern, as it can sometimes lead to misleading conclusions due to biases or inaccuracies in data. Predictive policing systems use data gathered in the past to forecast crime hotspots and allocate police resources accordingly. However, if historical data is biased then predictions generated by the system can perpetuate and even exacerbate over time. It can lead to higher predictions of crime in that area, leading to over policing rather than addressing the root cause of crime.
• The successful implementation of big data analytics would require substantial investment in both infrastructure and skilled personnel. Governments of developing countries often face resource constraints, which limits their ability to make full use of big data. 

WAY FORWARD

• Governments need to adopt comprehensive data strategies that encompass data collection, management, and analysis. At Amazon Web Services big data session at Public Service Data Live in London in 2023, Deepak Shukla highlighted that integrating modern data platforms with advancements in AI and generative AI can enhance decision-making processes and improve public services. 
• Deepak Shukla also calls for the need to educate citizens on how their data is being used by the government and how it helps the public sector deliver better and superior services to them. This will help build trust and boost confidence among citizens. According to him, becoming data-driven is more of a change programme than a data transformation programme. Having the right mechanism to bring people, processes, and cultural change is important in maximising return on investment in big data and AI technologies.
• There is a need for a dedicated group or body that manages data sharing between different government departments. This will help in avoiding problems related to data management and ensure that information flows smoothly across agencies. Secure data exchange can help break down data silos and promote collaboration.

CONCLUSION

The role of big data in public policy and governance will likely expand in the future. While there are challenges, the benefits of data-driven policymaking outweigh the challenges. Improved transparency, better-targeted interventions, and enhanced efficiency make it an attractive route for governments. Since the benefits outweigh the challenges, the focus now must be on addressing the challenges and ensuring the ethical use of data. By harnessing the power of big data, governments can pave the way for a more informed approach to addressing societal challenges and the formulation of effective public policies.

Register your Interest to Study at ISPP

References

1. Azzone, G. (2018). Big data and public policies: Opportunities and challenges. Statistics & Probability Letters, 136, 116–120. https://doi.org/10.1016/j.spl.2018.02.022
2. Content, P. (2024, January 23). Big but not scary: how to use big data to shape government policy and delivery. Global Government Forum. https://www.globalgovernmentforum.com/big-but-not-scary-how-to-use-big-data-to-shape-government-policy-and-delivery/
3. Marr, B. (2017, January 17). How big data helps to tackle the No 1 cause of accidental death in the U.S. Forbes. https://www.forbes.com/sites/bernardmarr/2017/01/16/how-big-data-helps-to-tackle-the-no-1-cause-of-accidental-death-in-the-u-s/?sh=67de616d39ca
4. Anticipatory government. (n.d.). Deloitte Insights. https://www2.deloitte.com/us/en/insights/industry/public-sector/government-trends/2020/predictive-analytics-in-government.html
5. Content, P. (2024b, January 23). Big but not scary: how to use big data to shape government policy and delivery. Global Government Forum. https://www.globalgovernmentforum.com/big-but-not-scary-how-to-use-big-data-to-shape-government-policy-and-delivery/#:~:text=Here%2C%20Deepak%20Shukla%20shares%20his,take%20advantage%20of%20generative%20AI.

Diksha Agrawal
Programme Advisor – Admissions

Diksha Agrawal, a Sociology graduate from Miranda House and postgraduate from Delhi School of Economics, embarked on her professional journey as a research fellow for Prashant Kishore’s Jan Suraaj initiative. Travelling across various districts of Bihar, she gained profound insights into the diverse needs of communities, igniting a deep interest in the field of public policy.

Furthering her engagement in politics and governance, Diksha served as a consultant for Inclusive Minds, political consultancy working for Indian National Congress during the Chhattisgarh elections. This role provided her with invaluable insights into the socio-political landscape of the state, reinforcing her determination to drive positive change through effective policy formulation and implementation.