What to do when the data becomes too large?

Diminishing returns - the point of diminishing performance in Big Data data is now quite possible. So here are tips on how to avoid getting bogged down on unnecessary redundant information.

In the 1950s, scientists have identified that the human brain can only remember about seven objects. The capacity of the human brain has not changed much - but the amount of data we are receiving is clearly increasing.

"We've got all the data we need, but the missing ones are business insiders who turn their learning into analytics," said Don Sullivan, Product Line Manager for VMware. Data into breakthrough opportunities for businesses. "

I attended the recent Microsoft PASS Summit with Sullivan, a conference for DBAs and analysts using Microsoft and related data platforms and analytics tools.

At the conference, I saw many excellent database automation tools that could be used for a variety of data sets and performed database, data and application database shuffling - but besides, The conference also lacks serious discussions on how to use these valuable data sources and turn them into valuable and useful business values.

I wonder: How much data is enough?

In the monitoring and production system, we have machines that can communicate with each other on the production platform and the endpoints communicate with each other in the enterprise network. Bodies and endpoints will collect and transmit valuable pieces of information - but these will also be assigned as worthless machine gibberish. Do network administrators really need it? 

Smart bar codes now can hold up to 7000 data characters for a given item. For example, a bar code on a T-shirt can indicate how many stitches a shirt has. But do you really need this information when your job is to ensure that the product is removed from production on time and will be available at the warehouse or retailer that is right for you? for the holidays or not?

In other words, whether we talk about network outputs, clothing or TV video signals, there seems to be a diminishing return. That's where the value you get from your data starts to drop.

There are two key points of the gradual decrease in data values:

• Data begins to be given without a business case to produce it
• The data is so complex that users simply do not know what to do with them

The following are suggestions for you to be able to neutralize the key:

Solution for data without business case

• Always identify the business case along with the expected results (ie, reduce operating costs in production) before designing the data center, plugging in the IoT, etc. You focus more on your goals. , the more likely your employees will be to work.
• Weekly, check the data analysis for the "transfer" project. In other words, is the project moving away from the business case that needs to be addressed? If you find the project is starting to go wrong, adjust it to the business case
• Never assign an analysis project to only one technician. Without the knowledge of the business, the project would be a technical masterpiece, but a business disaster. To avoid this pitfall, a business-savvy user or IT business analyst should be the lead for ensuring compatibility between the project and business goals.

The solution to the data is too complex, or too much

• Understand what end users need before you start designing an analytics application. If users are working outdoors in a cold and rainy field, they do not need to struggle with the pull down menu layers on a mobile device. A single image showing the location of the trailers and emphasizing key issues may be enough.
• Pursue to the end of your business case. If the goal is to see how many flu cases in different districts of the city, go with it to the end. Project tomorrow may be to add demographic characteristics such as factors affecting the unemployment rate, etc., but it is not the work of today.