Decision-making in a time of big data

Big data has changed the world forever – just ask Netflix. But, argues Professor Ulrike Rivett in her keynote address at eResearch Africa 2019, when it comes to society, public sector and government big data is perhaps not the silver bullet we imagine.

“It is worth reflecting on how big data is used, by whom and for what,” says Rivett, director of the University of Cape Town’s (UCT) School of Information Technology. “If you look at the top businesses in the world right now, the biggest transport company doesn’t own a single vehicle, the biggest property manager doesn’t own a single property, the biggest entertainment house doesn’t pay actors and the biggest shop doesn’t employ a single cashier. These businesses – Uber, Airbnb, Netflix and Amazon– are all digital businesses set up entirely on data systems.”

Big data in the private sector

Netflix makes a wonderful case study of how a private company has used big data to disrupt the television industry and set a trend. Today they have more than 86 million members, support more than a 1 000 different types of devices and everyday over 125 million hours of Netflix videos are livestreamed.

“How have they been so successful? Big Data, with capital letters,” says Rivett. Netflix is always capturing data to monitor how you watch, when you watch and what you watch, all to allow them to respond perfectly to your exact needs.  And it has paid off.

If the use of big data to guide decision-making has been so successful in the private sector, can we do the same in the public sector? Unfortunately, the same rules simply don’t apply, argues Rivett.

Collecting data: by whom, for whom?

There is a huge difference between how data is collected in the private sector versus the public. Reflecting on her own experiences both in the health and water and sanitation sectors, Rivett stresses the incentives of the person on the ground you are asking to collect this data for you. What is this data being used for?

In one study Rivett was involved in, the funder requested that 180 data points be collected on every patient in the clinic. This, she says, was far beyond the normal scope of a nurse or doctor’s work.  Nor is there any benefit to the staff or the patients in a public clinic to develop massive automated systems for patient data records.

“The nurse won’t become a better nurse because she collected the 180 data points for somebody located on another continent,” says Rivett. “This data is not going to help her care better for patients – and that is what government clinics in developing countries are all about, caring for the public – not collecting data points.”

Data, decision-making and behavioral change in the public sector

Rivett worked very closely with the City of Cape Town during the water crisis of 2017/18 and says it offered fascinating insights into how far data can take you, and how data is used in the context of government decision-making.

One of the things the City used as an incentive to citizens was to make water-usage across the City transparent, to encourage Cape Town residents – particularly those in the wealthier areas where the most water was being used – to save more water.

“The response to the publication of the data was immediate public outcry that the City had manipulated the data. People could not believe it was them wasting the water.”

“The perception of many of those in the suburbs was that the water wastage must be happening in the informal settlements.”

In the end, attitudes and behaviour changed with the announcement of Day Zero: the day the taps would run dry in Cape Town.

“The moment Day Zero was announced everybody went into Armageddon mode,” says Rivett. People began to buy bottled water for household use and the social condemnation for water-wasting behaviour became a driving factor for change. Everybody was seeing who could save the most water. And that strategy worked. There was no big data and there was not even a good model for estimating the occurrence of Day Zero.

“In the end what had the biggest impact, what turned Cape Town into an amazing example of catastrophe averted, did not need to be driven by big data,” says Rivett. “It was driven by an understanding of human nature and behaviour.”

“Big data will influence us, it will empower us to make decisions – but not all decisions using big data will result in the outcomes we are looking for. We need to rely on our experience, our insight and understanding, as well as the data.”

Image by skeeze from Pixabay