5 Steps to Making your Big Data Useful

This is the second in our Big Data series. To read more, see our first article, You got it or not? How to tell if Big Data is something you need to worry about in your application

The promises for big data in manufacturing are tantalizing… improved accuracy and efficiency in your production processes, better forecasting of product demand, quicker, more agile decision making. But realizing these benefits is not as commonplace as you might think, with many mid-market manufacturers collecting data without being able to use it to grow their business or meet their other business goals. How much potentially business-changing data is collecting dust on your servers right now? And how can you begin to use that data?

There are 5 steps to making the most of your big data:

Step 1. Identify the problem and/or goals

Before you even begin to look at the data, you need understand what problems you have that might be solved with a better understanding of your data. Maybe it’s a material bottleneck, an increase in production time, or an increase in “near-miss” safety incidents. For this post, let’s use an example of a manager at a mid-sized sensor manufacturer who has noticed a higher-than-usual number of gas sensors needing to be serviced lately and wants to know what’s causing the problem. Regardless of your goals, knowing what you’re looking for from the get-go is critical to your big data success.

Step 2. Ask smart questions

An often overlooked aspect of big data analysis is quality of the inquiries. While many analytical tools can help you spot anomalies in data, you need specific ideas for what to look for. In our sensor example, some specific questions might be:

• When did the increase in sensors needing service begin?
• Are any of our other products seeing a higher-than-average return rate?
• Has an increase like this happened before? When?
• Is there a correlation between new employees onboarding and broken sensors?
• Is there a correlation with the weather?

Collect as much data & fields as you can to build your cubes. The more granular the cubes are, the more flexible your reporting will be.

Step 3. Assess your data

Now is the time to look at what data you currently have, what other data sources you might have access to, and evaluate their accuracy, relevance, and timeliness. In our manufacturing example, available data sources might include their ERP system, suppliers, machines, HR system or outside sources such as weather or social media.

For each source, consider its relevance and based on the 4 V’s - Volume (the amount of data), Variety (its type and structure), Velocity (how quickly the new data is coming in and needing to be used), and Veracity (how accurate or trustworthy the data is).

Step 4. Collect / Ingest:

One key difference between “small” and “big” data has to do with its collection and ingestion. At some point, you hit the limitations of your ERP system’s native reporting functionality – standard ERP reports don’t give you enough flexibility to look at the data the way you need to, the processing of huge reports can slow down performance for the rest of your team, and you can’t incorporate external data sources. The process of ETL (extract, transform, load) allows you to get all data from multiple sources, including your ERP, transform it into a format specifically designed for big queries, then move it into an analysis tool.

PRO TIP: Think big. Collect as much data & fields as you can to build your cubes. The more granular the cubes are, the more flexible your reporting will be. In other words, do not limit yourself to what you want to filter by today, think about what other filters might come in handy tomorrow. Building a cube is very time consuming and can cost a lot of human as well as machine time, this is why preparing a data set as big as you can to begin with will save you time down the road.

abas ERP makes ETL easy with the Unity server, detailed here: https://abas-erp.com/en/news/unity-server

Step 5. Analyze

This is where Business Intelligence (BI) and advanced analytic tools shine with visual representations of complex data sets and other ways to find answers to your question.

These tools can help you account for every piece of your business’s data, and consolidate it to deliver accurate information to decision makers in a timely manner and support effective decision making. This is done by setting specific, measurable goals, coming up with a course of action to work towards them, and finding a means to measure if that course of action is resulting in progress towards those goals. For more on BI in ERP, see this article: https://abas-erp.com/en/news/going-global-business-intelligence-bi

In our example with the sensor manufacturer, our manager is now looking at all of the data in one place. He can see that there have been similar increases in service orders for the gas sensors in the past, and that the increases have typically happened in late March and early April. There doesn’t seem to be any correlation between new employees onboarding and the increase in broken parts, because the new employees are hired throughout the year. The insight comes when the manager looks at the combination of the weather, manufacturing location, and repair dates – the sensors that are breaking seem to be the ones that were manufactured in the company’s cold-climate Canadian warehouses, then shipped to locations in their warm, wet spring seasons. That’s actionable information that he can use to improve their product or distribution processes moving forward, for example, by shipping sensors to warm wet climates from their facility in Mexico, rather than the one in Canada.

How big is your data?

When analyzing data, what’s the difference between big and small data? The line can get blurred, but you need to make sure that your analytic tools and infrastructure can support the type of data analysis you need to do.

Stay tuned for our next topic in the Big Data series: using meta data effectively.