Why 2021 is the year we spend less time crunching
numbers, and more time telling stories.
Data automation: definition
Traditional data processing and analysis can be broadly broken into 3 tasks:
- ➡️ Extraction: pulling data from a source
- 🔃 Transformation: converting raw data into a format that you can mine for business intelligence
- ⬅️ Loading: The data has to live somewhere - data loading gets information into your system of choice
What data automation does is simple: it performs data extraction, transformation and loading without any need for manual support or supervision.
Understanding what data automation is capable of is a little more complicated (but a lot more exciting, too).
Benefits of data automation
The biggest benefit of data automation is the same as any automation, winning back time for the parts of work you actually enjoy.
Manual Data Extraction, Transformation and Loading (ETL) can force you to spend more time crunching data than analysing it.
And the more systems you have to extract data from, the longer the process. Generally, customers with legacy systems tend to struggle with this the most.
Data Automation cuts that time down substantially.
That isn't just valuable because it saves time and money (although it does both).
Data automation is valuable because it frees up more time for analysts to do what they love - uncovering brilliant insights in data. They get to spend more time being motivated by the most creative and rewarding aspects of their job, and less time just moving data around.
Spend less time crunching numbers, and more time telling stories
We're going to try and live up to the point we're making. Throughout this article, we'll spend less time talking process and more time telling stories.
Case study: Amazon
The story: Discovering that people are surprisingly predictable when it comes to buying products online
Someone will load the page, click 6 times, look at 5 pages and then leave. That's it.
The interaction is only different 1.8% of the time - when a user will make a purchase or contact the brand.
Now ask yourself - what could you learn about a customer if you knew exactly what pages they looked at during that session, or how far they scrolled, and where they lingered? Probably a lot.
On it’s own, that’s not so useful - we’re a world of individuals and who’s to say if the next visitor wants something completely different.
Fortunately, the average business websites gets 694 visits per month. That’s 51 hours of non-stop website interaction. What could you learn about your customers over a year if you looked at that information closely?
What could you do with that knowledge? The short answer is lots. Amazon asked themselves this same question 20 years ago, and it helped them become the 4th largest company in the world.
Depending on your personality type, Andreas' weekly meeting either sounds like an interesting night (the author is that type of person) or a really dry way to spend your time (we get that too). But there’s no doubt that it’s valuable.
Amazon’s focus on data gave rise to:
- 📳 Personalisation
- 🎁 Targeted product recommendations
- 👨💻 Customised front pages for every customer
Any Amazon customer knows about those front-end uses of data automation, what might surprise you is what happens in the back-end.
As soon as you start browsing a new type of product, Amazon starts automatically forecasting the inventory they’ll need in 3-6 months. The reason? Because they’ve looked at millions of transactions and worked out exactly how long you’re likely to wait until you ‘unexpectedly’ decide to buy that product.
The truth with data automation is that there’s no reason why that same process shouldn’t take a fraction of the time today.
Our favourite data automation tools
Excel and Microsoft 365 for personal data automation
Story: Disrupt your industry, not your business
Case study: CorporateHealth
We’ll admit, there’s a lot of buzz around automation as a modern, innovative technology, so Excel might seem odd to mention.
That said, so much more data automation is possible through Excel than people realise - particularly when used with other apps in the Microsoft 365 ecosystem.
Take the international medtech company CorporateHealth as an example - healthcare is highly regulated, so CorporateHealth relies on crystal-clear processes for quality research and clinical excellence.
As a medtech innovator, CorporateHealth needs the freedom to adjust the way it works quickly and easily.
By adding an inexpensive Visio license to their Office 365 subscription and making use of the Visio data visualiser add-in on Excel, any team member can automatically create professional-looking process maps from a list of excel bullet points.
No design skills required. Staff can stay in the platform they’re most comfortable in, Excel, and automatically convert data into something visual for colleagues to follow.
The benefits? Regulatory compliance, less time wasted on complex process map design, shared insights and business-wide transparency.
See? Not bad for Excel. That’s data automation you can’t do with a macro, but has clear benefits across a business.
Microsoft Power Automate: for company-wide data management
Story: Winning back time for customer care
Case Study: Thirteen Group
The beauty of Power Automate
What if the data you need to extract isn't digital at all? Suppose your organisation still collects data with pen and paper forms. How could you possibly use data automation?
Three words - Optical. Character. Recognition (OCR). Using OCR, Power Automate can automatically detect and extract data from scanned documents.
Here's a quick overview of how OCR-based data extraction can work as part of an automated process:
Power Automate in practice
In 2020, we helped Thirteen Group start their digital transformation with these two technologies. They manage 34,000+ properties as a landlord and housing developer - each one with dozens of associated documents.
Thirteen's agents need fast access to those documents to deliver outstanding customer services, so efficient storage and document management is essential.
Thirteen's property management process was mostly analogue, relying on agents processing paper documents and manually importing each into an SQL database.
Slow and boring, staff often lost concentration during the process and made mistakes. These mistakes made it difficult for other agents to find data quickly when they needed it to help customers.
As a Gold Microsoft Partner, we knew that Microsoft SharePoint and Power Automate could help Thirteen transform this process. Combining the two with OCR, we built Thirteen a system that automatically:
- ✔️ Extracts written or printed text from a scanned document
- ✔️ Transforms the data into a machine-readable format, and
- ✔️ Automatically load that data into a customer folder on SharePoint
With it, Thirteen's agents won back roughly 250 days of working time a year from manual data management. Less time hunting data, more time helping customers.