Vicious cycle of data platform

Data is new fuel” , “ Data Driven Enterprise” — this has been the common adage in the new digital and information age, everyone is leveraging the data to create competitive advantage and grow their business. But reality is far from the truth.

  • Millions are spent on analytics but businesses are still not making return on the investment.
  • Users are still scrambling for the right data to support their daily activities
  • Execs are inundated with reports and are not getting clear concise view

what are the traits ? can this be changed ? where is your organisation?

Too much data no insight

Stage 1: Too much data no insight — “Give me all the data”

Depending on the business processes, for every system and digital touch point there are can be exponential amount of data that is generated. In a typical small to medium size business that can be upto to 10 systems interacting with customer for any simple transaction. Once this data is generate every organisation attempts to collect and process the data to produce insight. Thanks to cloud service providers, this process has relatively become effortless by using SaaS solutions like Google Cloud Storage, AWS lake formation or Azure Synapse.

Challenges:

  • Ability to procure data from source
  • Lack of understanding of data in business context limiting capability to model and transform the data.
  • Uncovering technical Data Governance issues: Consistency, Completeness and Timeliness

Rationale : Data collection is often conceived as technical task rather than business activity. Focus on business outcome : engage users and source system SMEs to ensure data brought across is right and fit for business need.

Stage 2: Too much insight no value — “ Let produce insights for execs”

Once all you data is collected in the right format and quality, organisations starts their journey of enabling insights and serving business users with required data. This is usually done in layer within the lake of termed as raw, curated and modelled data, which is further hooked into the serving/presentation layer through reporting tools like Tableau/Qlik/ PowerBI/data mining tools.

Challenges:

Users are now exposed with all sort of data ( realtime, batch etc), there is continuous flow new technology and process. This results in unnecessary reports, insights and tremendous amount of money being spent in the interpretation of data, diminishing return on investment.

Rationale

While the project’s key objectives are well defined and understood, users are presented with new technology stack and now have access to the new data with improved performance and business agility. In this continuous process exploration and learning, organisations ends up with hundreds if not thousands of insights that was never meant to be produced or required.

Setting right direction : “Guiding light..

Irrespective of the challenges and stage of the data journey you are in, the return on investment can be maximised by keeping the basic alignment of objective and purpose to project activities in following ways

  • Define the business rationale for data pipeline: Collect, Process & Serve
  • Don’t over engineer: set the right level of automation
  • Setup metric on user adoption and reporting to value drive consumption

This will ensure the investment are returning value to the business.

Traits for success

  • Shorter time to release the data to the business users
  • Increased adoption on new revamped data platform
  • Optimised reporting and insight focused on the business objectives
  • For every $ of investment return on investment can be measured ( refer to next blog on how to implement this )

Data and Analytics expert with focus on the delivering results to business with accelerate framework