The Risks Associated with Predictive Models – What We All Need to Remember: We Can’t Predict Everything!

Kirk Borne is the Principal Data Scientist at Booz Allen and a top Big Data, Data Science and AI influencer. He recently pointed out on Twitter “The Risks Associated with Predictive Models – What We All Need to Remember: We Can’t Predict Everything!”.

He referenced an article from Data Science Central, “Predictability of Life Outcomes – Guess We Can’t Predict Everything.”

Summary from the above article: Whether trying to predict the life outcomes of disadvantaged kids or to model where ventilators will be most needed, a little humility is in order. As this study shows, the best data and the broadest teams failed at critical predictions. Getting the model wrong, or more importantly using it in the wrong way can hurt all of us.

Scenario 4 (attribution xbcd.com)

Scenario 4 (attribution xbcd.com)

 

This equally applies to industrial process chemistry. We have to be very careful about so called black box predictive models being promoted by various vendors for controlling the process chemistry. Sometimes these proprietary systems are promoted by suppliers, who want to use it as a way to tie your industrial site to their chemicals or equipment.

If you are considering the installation of such a system, PCA can help you to evaluate it on your process.

For now, we reference an article from Dan Steinberg’s Blog, “Black Boxes and Data Mining Systems”. Dan Steinberg, President and Founder of Salford Systems, is a well respected member of the statistics and econometrics communities. Here is a quote:

“In the world of data mining and predictive modelling, the concept of the black box often comes up in the context of proprietary prediction systems in which the vendor does not disclose details of the algorithm by which the predictions are being made….

In the 1990’s, many financial institutions paid hefty fees to use a proprietary system for predicting interest rates; the vendor was successful in persuading banks that the predictions were accurate enough to warrant subscribing to the service even though the banks did not know how the predictions were generated.

Today, in the field of data mining and predictive modeling software, there are new black box vendors who prefer to offer the most minimal descriptions of their algorithms. Instead of describing their own algorithms in detail, they offer general discussions of data mining principles and pepper their white papers with formulas for well-known procedures such as logistic regression and ROC calculation.

The topic addressed in this blog is: Should you seriously consider such a black box system? In general, we think not for the following reasons:….”

 

About This Site

PCA is the independent technology service assisting industry to improve process, chemistry, productivity, sustainability. For Chemicals, Mining, Paper, Metals, Alumina, Food, Utilities, Retail and other Industries.

With PCA’s Core Trust Design, processes, chemistry, equipment and control strategies are assessed in an impartial, objective manner using industrial process and chemistry knowledge, the industrial site’s own data and “statistical thinking”.

The founder of PCA holds discussions with the client to define the objectives and scope of work. With decades of experience in process and chemistry technical consulting for multinationals in the Chemical and other industries, such as Mining, Paper & Board, Metals, Alumina, Food & Beverage, Utilities. Work directly with the founder of PCA, not an intermediary. Trust in a positive outcome.

PCA is the originator of the PCA Trust System – A PCA Innovation. Using advanced cryptography and public blockchain, Trust Statements can be produced for any supplier, manufacturer and industry that wants to build trust in their process, product or service marketing.

PCA Trust Statements are unique and the most secure form of statement. The client can use them to make verifiable statements important for the marketing of their business.