DecisionMaking in Times of Big Data

We have all heard the catchphrase “drowning in data, starving for information”.  10 years ago, we thought it was so relevant.  Little did we know how far that journey would take us. In the world we live in today we are surrounded by and drowning in data in quantities most of us could never have imagined.  There are some modern solutions which have yet to be adapted

Here at Pelz Consulting, we are currently working on the problem of making the best use of data from specialist sensors supporting security operations.  Specifically, we are networking sensors that detect chemical, biological and radiological threats as well as those able to observe other environmental changes.  In the world of security, when challenged to make split-second decisions in complex and life-threatening environments, the abundance of data can be more of a curse than a blessing. Being overwhelmed by data is only adding to decision fatigue.  We all know the feeling of uncertainty as you press “buy” about whether you have really made the best choice or whether you should have considered more possibilities.  There is so much data pouring in that you can no longer process it in a reasonable amount of time.  Imagine how much worse that feeling can be when you have seconds to save lives.  We really have to learn to do better.  At the same time, we need to learn the lessons already being learned in manufacturing, that the vast stores of data now available to us can have huge value and should be intelligently used.

With all that in mind, what is the remedy; how to get out of this cul de sac?

Well, there is, of course, more than one answer. Whatever you end up doing, you need to get an overview of what data you have, sort the good from the bad and once stored, find them back when you need them. Then comes the not trivial task to present them in a way that makes sense to decisionmakers.

Modern networking capabilities enabling to connect data sources with mass data storage in conjunction with Machine Learning and Artificial Intelligence came a long way and now present opportunities we did not think of a few years ago.

Recently we started to look into networking our CBRN data sources in a different way. Previously we were determined to connect detection instruments to software and process the data for further consumption. This was more of a one-off integration of a data source to make use of its detection capabilities. It was only after we looked for a solution to a logistic problem that it dawned on us that we had to take a different approach.

Get the data all into a data lake, then sort them into a data warehouse for subsequent use and consumption. Once you have all the data collected in your warehouse you now can do a variety of things. One huge advantage is that you will have all the data in a common language available to you. Now you can make maximum use of the vast amount of data by taking out what you need to fulfill your task. If you need position data together with status information, you have it all stored. If you require a dashboard showing the results of what the sensor detected

linked to what the CCTV camera saw in that area, you have it in your data warehouse.

Now back to what I mentioned in the beginning. How does that help the decision-maker?  Because you have access to all possible data sets you can design the most effective support tool for the poor individual challenged to make the right decision. You provide him with the right information at the right time because you have it in your data warehouse. Just because you have that treasure trove does not mean that all data you can collect are needed. Irrespective of a need today it is probably a good idea to store the data anyhow. You might run into an opportunity to make use of it. You can only turn data into information and into money if you have it.

If you want to know more about what we do, contact us at lpelz@pelzconsulting.com or JHall@pelzconsulting.com .