When AI Needs To Work Smarter, Not harder

It has been suggested recently that we are living in a ‘quantocracy’ – a society defined by the idea that everything must be counted and judged based on how many of something we have accumulated. And, perhaps unsurprisingly, more is usually deemed to be better.

This school of thought has quickly infiltrated every aspect of our lives, including conversations around data. Today it is broadly assumed and accepted that the more data we have at our disposal, the more useful – and financially beneficial – it is.

Ethical concerns aside, such an assumption sounds reasonable. The potent mix of unlimited data, powerful processing power and deep pockets should, in theory, enable us to understand every challenge facing society, and this would make the world an easier place to navigate. In practice, though, things look very different.

International Data Corporation has predicted that: “Humans and machines will create more data in the next three years than the previous 30. This explosion of data is… a key factor for why machine learning has advanced so much in the last 10 years.” A sharp resurgence in neural networks and increased computational power has enabled us to throw huge amounts of data at an algorithm in the hope of obtaining valuable insights. But at what cost?

Put simply, algorithms learn by finding patterns in data. The more data we throw at an algorithm, the more complex it must be to consider all of the possible outcomes. This additional complexity not only makes it more difficult for humans to understand what is happening and how decisions are being made, but also results in a significantly disproportionate use of resources – both financially and environmentally.

As climate change continues to dominate the headlines, the environmental impact of the technologies we use is attracting more attention than ever before. The examples are numerous – from the European Union warning that the proportion of greenhouse gas emissions attributable to the IT industry could rise sevenfold within the next 20 years, to the high-profile case of one AI model, which enables computers to make sense of text, recording the same carbon footprint as 125 round trips between New York and Beijing.

Another key risk of too much data, together with little or no human input, is that AI systems can begin to draw spurious connections and identify patterns that may be nothing more than coincidences. For example, as Tyler Vigen points out in his book ‘Spurious Correlations’, changing per capita consumption of mozzarella cheese in the US between 2000 and 2009 was highly correlated with the number of civil engineering doctorates awarded each year over the same period. With no additional context, an algorithm might think that one could predict the other, where in actual fact, there is likely no predictive power between delicious cheese and bridge-building.

Though some spurious correlations like this one can be spotted immediately by any human with common sense, not all are quite so obvious. For example, at Mind Foundry we worked with an insurance company that suspected care workers were in breach of their policies, because it thought they were using their cars for ride-hailing, where in actual fact they were simply moving between care appointments. AI models that proactively involve humans can help to identify such inaccuracies – in this case, the collaborative relationship enabled the insurance company to correct course and offer more appropriate coverage.

AI learns from existing datasets created and informed by human behaviour, but this can be particularly dangerous when it comes to certain applications and recruitment is a good example. For instance, if the AI recognises a pattern whereby a recruiting organisation can earn its fees by pairing candidates within existing frameworks of bias, such as education background or socioeconomic status, then that is exactly what will happen. This is where human/AI collaboration is critical as it can help mitigate the bias perpetuated by the system, which would otherwise systematically exclude huge swathes of the population.

It could be argued that for large organisations with deep pockets and limitless resources, perhaps it makes sense to run all of the available data through the most complex models to surface a few useful insights and gain a competitive advantage. However, it is not always necessary to have huge amounts of data to make good decisions – especially when you take a ‘problem-first’ approach.

The alternative to churning endless data is to develop a deep understanding of the problem from the outset and encode that information into the AI. In other words, having less but more relevant and better data is the AI equivalent of the old adage ‘work smart, not harder’. This is also where collaboration between humans and AI is crucial. Keeping a human – or domain expert – involved in an AI’s decision-making process provides a lot more control and minimises the bias being introduced into the system.

The unavoidable truth is that AI systems built for a specific purpose better serve us as a society. We need to focus on creating intuitive AI models that are not only capable of taking on repetitive tasks, but can also recognise when they require human input. AI models built with these principles will both free up human time to focus on more creative and interesting tasks and – crucially – raise the overall intelligence of an organisation.

AI systems are complex and there is no one size fits all solution. While vocal advocates for deep learning and neural networks stand firm in their belief that complex models are always the best approach, sometimes taking a step back and looking at the problem more holistically can reveal another truth. Complex models of machine learning do have their place and can be useful – even necessary – for applications including language translation, image recognition and computer vision, for example.

However, they can come at a high cost – from a loss in transparency to our responsibility for the environment. In high-stakes applications where transparency and understanding are so important, problems should be assessed from multiple perspectives. We need to understand how the different components fit together, and only then decide how to design a solution – in other words, complexity shouldn’t be the starting point.

Often, an incremental improvement in problem design, feature selection and data quality will yield huge improvements, all while preserving transparency, accountability and sustainability. These are the parts of our AI models that we need to prioritise if humans and AI are to work in harmony to create a world that works for the benefit of everyone.

The bottom line is, whether we have lots of data or limited data, it is how it’s being used that matters.

Source: E&T