Grow VC Group
  • Home
  • Group
  • Team
  • FAQ
  • Join Us
  • Trainee Program
  • Contact
  • News

Data is the basis for most digital decision-making but how reliable is data?

8/8/2021

Comments

 
Data is the basis for many operations, but it doesn’t mean data is always reliable. Things can get complicated when you don’t know which data source is reliable and which is not. But we must use data all the time. Sometimes it is possible to increase the accuracy, but the more meaningful solution is to build a software layer to correct data before using it.

I earlier wrote about known and unknown things and data points. The reality is even more complex. We know some data is relevant, and it is available, but we don’t always know how reliable it is. We all know about opinion polls and their error margins. It is just one example, but uncertainty is linked to all data sources and models that utilize data.

In aeroplanes or nuclear power stations, the core systems do not necessarily trust individual sensors or data sources. There can be many reasons why a particular sensor gives incorrect data. For example, a pitot tube that measures an aircraft’s airspeed can transmit incorrect information if frozen, which has caused several plane crashes. Today, a plane typically has several pitot tubes, and the software tries to draw conclusions and give pilots warnings if one or more give inconsistent readings.

Sometimes the situation is more demanding when it is difficult, even impossible, to know if data sources and sensors give accurate data and how large the error margin is. Examples of this are wearable devices. They can measure your exercise patterns, sleep, and body functions like heart rate, temperature or blood pressure. These devices are calibrated using higher accuracy devices during development. But it is still hard to say how accurate they are for different people in different situations. For example, even with top-level research instruments, it is not easy to measure how much light sleep, REM, and deep sleep a person has at night.

We might also have a situation where we have many sensors, but some data might be missing. It is a complex task to combine data from different sources, and it is also tricky to know if available data makes any sense combined. This can occur when having many IoT sensors or an organization’s internal data from multiple sources to measure processes or even financials.

It is often said that intelligence makes up only 20% of AI implementations, and the rest is getting data, combining it and correcting errors. This layer is often underestimated. I have seen projects where 95% of the data is inaccurate, incorrect, or missing data points.

There are several ways to increase the accuracy of data, for example:
  1. If we get the same data from several sources, we can have a ‘voting’ model to determine the ‘correct’ data from most sources. The pitot tube system often works like this.
  2. We can learn from different sources’ accuracy and take their error margins into account to correct data. Opinion poll models often have correction factors.
  3. More complex solutions combine several data sources and make conclusions about what the data can indicate and how well data fits this. For example, if a person is running, this can be concluded from a combination of several data points from wearable devices, e.g. motion sensors, speed, heart rate and oxygen. Another example is that the software in a phone camera system tries to make a photo better based on each camera’s pixel data by correcting individual pixels alone and how the pixels fit together.
  4. Sometimes it is possible to have a feedback loop to know how accurate some data is and then have machine learning type models to develop correction factors and models to use data.

These layers combine, correct and smartly use data and become more important as we get more data sources. One could even say it is pretty simple to create AI models if someone has developed this layer to make reliable data available. It is often said that IoT business is not really to sell sensor hardware but to manage data, but what is ignored many times is the critical question of getting reliable data.

It is not easy to make these layers that combine data because each source is different, and it can also require an understanding of the data to be able to analyze and integrate data sources. It is possible to make general models and tools for this, but they often need tailoring for the different data sources and combinations of data sources.

With AI’s hands, these smart data combining models and layers become a vital part of the data and AI business. Data is valuable only if it is reliable. We can trust AI only if it can use correct data. The reality is that no data source is 100% reliable, so we need intelligence, how to correctly and optimally use data sources.

The article was originally published on Disruptive.Asia. 
Picture
Photo Source: Wikipedia, automatic landing system.
Comments

    About

    Est. 2009 Grow VC Group is building truly global digital businesses. The focus is especially on digitization, data and fintech services. We have very hands-on approach to build businesses and we always want to make them global, scale-up and have the real entrepreneurial spirit.​

    Read the latest Grow VC Group  FinTech, distributed and crypto finance, and blockchain report
    Read the AI, Asia and FinTech report - including comments about potential trade wars.
    Download

    Research Report 1/2018: Distributed Technologies - Changing Finance and the Internet 


    ​Research Report 1/2017: Machines, Asia And Fintech:
    Rise of Globalization and
    Protectionism as a
    Consequence


    Fintech Hybrid Finance Whitepaper

    ​Fintech And Digital Finance Insight & Vision Whitepaper


    Learn More About Our Companies:
    • Difitek
    • Prifina​
    • RE Bearing
    • Token Index Fund
    • Startup Commons
    • Lost in Translations
    • Robocorp
    • Nodi Liber​

    Archives

    January 2023
    August 2022
    July 2022
    June 2022
    May 2022
    April 2022
    February 2022
    January 2022
    December 2021
    November 2021
    October 2021
    September 2021
    August 2021
    May 2021
    April 2021
    March 2021
    February 2021
    January 2021
    December 2020
    October 2020
    September 2020
    July 2020
    May 2020
    April 2020
    March 2020
    February 2020
    January 2020
    December 2019
    November 2019
    October 2019
    September 2019
    August 2019
    July 2019
    June 2019
    May 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    September 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    December 2017
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    May 2017
    April 2017
    March 2017
    February 2017
    January 2017
    December 2016
    November 2016
    October 2016
    September 2016
    August 2016
    July 2016
    June 2016

    Categories

    All
    Difitek
    Grow VC Group
    Robocorp

    RSS Feed

Digital Intelligence Globally
Picture
© 2009-2023 Grow VC Operations Ltd. All Rights Reserved.
  • Home
  • Group
  • Team
  • FAQ
  • Join Us
  • Trainee Program
  • Contact
  • News