Why Proxy Model Explanations of Neural Networks Inevitably Fail 

For financial service providers, credit risk modeling is all about identifying opportunities and predicting the likelihood of payment default. But, it’s also about industry compliance and treating consumers fairly. By providing them with accurate reasons (aka reason codes) when they’re denied credit, they can take steps to improve their credit.

Here we examine how one credit risk modeling strategy simply doesn't work within the financial service industry. We use logistic regression proxy models to explain advanced neural network models. And, we illustrate how this failure can impact consumers and the businesses that seek to serve them. 

3 Consumers, 3 Mismatched Reason Code Lists  

Consumer 1:

The table below reflects three consumers exhibiting different problems with reason codes returned from the proxy model. We’ll start with Consumer 1 and the first set of four reason codes in the left column. The largest points drop in the neural network model, labeled as NeuroDecision, is caused by the Number of Collections. In the right column, the logistic regression proxy model returns Bankcard Utilization as the reason for the largest points drop. Bankcard Utilization does not appear in the top four reasons for points lost in the NeuroDecision model. It follows that a consumer acting logically and rationally on the information provided—namely, to decrease her bankcard utilization—will have no idea that this behavior may fail to improve her score in the model that actually generates the score.

Consumer 2:

Consumer 2 shares two reason codes across the NeuroDecision and proxy model. But neither code is the primary reason for points lost in the NeuroDecision model. The main reason this consumer’s score is not higher is that he has too many Accounts With Balances. The logical reaction to this information would be to reduce one or more account balances to zero. However, the proxy model informs him that the Percent Of Accounts That Are Credit Union is causing the largest points drop. If the consumer reacts rationally to the proxy model reason codes, the score may improve. But it may not improve as much as it could. In addition, reacting rationally to the next two reason codes returned by the proxy model may have no discernable impact on his credit score. Furthermore, there is no way for the consumer to know this.  

Consumer 3:

Finally, Consumer 3 receives distinct reason codes from each model. The logistic regression fails to agree on any of the four most important reasons the consumer lost points. Therefore, the explanation returned for points lost on her score completely misleads the consumer. This is not her fault, and she has no way to correct it. Based on our internal examination, these rearrangements in reason codes between the model generating the score and the proxy model occur a lot. Specifically, 95 percent of the scored population fails to meet the regulatory requirement. This occurs due to:

• reporting the reason codes out of order compared to the scoring model
• having one or more reason codes that do not appear in the scoring model

In credit scoring systems, reason codes are returned in the order of their importance. Mismatch in the order or the reason code itself between the neural network and the proxy explanation model indicate the proxy model is a poor substitute for the model that generates the score.   

Model Differentiators

The NeuroDecision® neural networks used in this study comply with existing U.S. regulations by providing users with reason codes for disclosure to consumers in the event credit is denied or other adverse action is taken. These reason codes emerge directly from the model that generates the credit score, as required. And, they serve the purpose of explaining to consumers why they are denied credit. 

However, another method for explaining neural networks is to use a proxy model as we illustrated in this study with the logistic regression proxy model. The ideal and regulatory compliant proxy model would generate the exact same score and reason codes as the neural network. After comparing the patterns of reason codes returned directly from the neural network model to those from a logistic regression proxy model, we see why proxy model explanations fail to comply with existing regulations for credit scoring systems and as model explanations.

The Takeaway? Proxy Models can Harm Consumers and Businesses 

One thing is clear: proxy models fail to return logical and actionable information to the consumer and should never be used for credit risk applications. In the sample study used for this article, mismatches in order and actual reason codes occur for the overwhelming majority of consumers evaluated. Alone, this leads astray consumers who may be attempting to improve their credit score and can aggregate to have detrimental effects on credit portfolios as consumers react rationally to inaccurate information.  

 

Take a tour of a sample XAI model to learn how regulatory compliant neural networking technology is powering the next generation of credit risk modeling.