Analysis

How to measure quality of underwriting: the Gini coefficient

In the quest for developing the best underwriting model possible we believe that there is a need for an objective and quantifiable way to measure with a number the quality of a model. Such a metric exists. Many people are familiar with the Gini coefficient for inequality. It can also be used to measure the quality of an underwriting model.

The Gini coefficient is most widely used by economists as a measure of statistical dispersion intended to represent the income distribution of a nation’s residents. It is the most commonly used measure of inequality.

A Gini coefficient of zero expresses perfect equality, where all values are the same (for example, where everyone has the same income). A Gini coefficient of one (or 100%) expresses maximal inequality among values (for example, where only one person has all the income or consumption, and all others have none).

Another use for Gini Coefficients, and more relevant to our readers here at Lending Times, is to evaluate the predictive power of credit scoring models. While calculating a Gini for a lender is fairly complex, understanding the Gini is somewhat simpler.

The GINI co-efficient for measuring credit models also has values between 0 and 1. A higher value means that a particular credit model can better discriminate among good and risky borrowers. A value of 1 means that the model predicts perfectly, and with certainty, which borrowers will repay and which borrowers will default. A value of 0 means that the model is completely random, or in other words, it is the statistical equivalent of a coin toss, resulting in a 50/50 probability of repayment or default for each applicant.

A Gini Coefficient can help a lender (or investor) understand how good the lender’s credit model is at predicting who will repay and who will default on a loan.
The GINI co-efficient compares the “Lorenz” curve (the cumulative distribution) with the line of perfect randomness. Graphically illustrated, the Gini is the ratio of the area under the curve (A) but above the line of perfect randomness to the entire area above the line of perfect randomness (A+B). See graphic below.

gini1

Why do underwriting models matter ?

Underwriting models are the key to the lending business for many obvious reasons. Beyond that, with many originators in the small business and consumer loan space, and even more new originators in the pipeline, it is becoming increasingly hard to differentiate among lenders. One clear way to differentiate among originators is building strong, proven, and accurate credit models. And this is where the Gini becomes very useful.

While it is hard to actually measure and only the most sophisticated investors would have the resources to calculate a Gini for a particular originator, there is plenty of data available for the retail investor to analyze.

An example

Let’s look at Lendingclub for one example. In a very simple way of analyzing the credit scoring model, lets look at Lendingclub’s default rates by loan grade. Lendingclub grades its loans on a letter scale of A through G, and each letter has a subscale of 1 through 5 (A1 being the least risky and G5 being the most risky). We would think that the proportion of defaults would increase as the buckets increase in risk. We can easily illustrate this by plotting the percentage of loan defaults (measured by charged-off loans) for each bucket. Using Lendingclub’s loan data from 2009 through 2013, we can see that this generally holds true. Overall, for each year, each bucket of increasing risk has generally resulted in a slightly larger percentage of charged-off loans.

gini 2

Lendingclub’s risk model appears to be pretty good at measuring risk. We would assume the Gini coefficient to be appropriately high for Lendingclub.

Limitations

First, it is fairly hard to calculate the Gini coefficient. There no single polynomial formula to do so. It requires more calculations.

Second, the Gini coefficient does have its limitations. The only real way to compare two underwriting models is to run both models on exactly the same data and see how the results compare. In other words the exact way to compare Gini coefficients is to calculate the Gini coefficient on 2 different models using the same training and test data. While this is a limitation of any model comparison we believe that in first approximation it is not a major limitation of the usage of the Gini coefficient.

Future developments

Lending Times would like to encourage originators to publish the Gini coefficient of their underwriting. We would be ready to publish such information and present it in an organized fashion.

By publishing and comparing the Gini coefficients of different companies each originator will get a better idea how they compare to the industry average. This will allow them to improve or identify what changes are needed. The entire industry will gain is the overall quality of the underwriting will improve. It will be easier to attract funding and investors from major established institutions and regulators will be more comfortable that the industry is stable, reliable and unlike what Lord Turner claimed, marketplace, p2p and online alternative lending will be here to stay for long time.

 

Authors: Mark Smith and George Popescu

About the author

George Popescu

Serial entrepreneur.

George sold and exited his most successful company, Boston Technologies (BT) group, in 2014. BT was a technology, market maker, high-frequency trading and inter-broker broker-dealer in the FX Spot, precious metals and CFDs space company. George was the Founder and CEO and he boot-strapped from $0 to a $20+ million in revenue without any equity investment. BT has been #1 fastest growing company in Boston in 2011 according to the Boston Business Journal and the only company being in top 10 fastest in 2012-13 as it was #5 in 2012. BT has been on the Inc. 500/5000 list of fastest growing companies in the US for 4 years in a row ( #143, #373, #897 and #1270). After the company sale in July 2014 until February 2015 George was Head-of-Strategy for Currency Mountain ( www.currencymountain.com ), a USD 100 million+ holding company focused on retail and medium institutional currencies, precious metals, stocks, fixed income and commodities businesses.

• Over the last 10 years, George founded 10 companies in online lending, craft beer brewery, exotic sports car rental space, hedge funds, peer-reviewed scientific journal ( Journal of Cellular and Molecular medicine…) and more. George advised 30+ early stage start-ups in different fields. George was also a mentor at MIT’s Venture Mentoring Services and Techstar Fintech in NY.

• Previously George obtained 3 Master's Degrees: a Master's of Science from MIT working on 3D printing, a Master’s in Electrical Engineering and Computer Science from Supelec, France and a Master's in Nanosciences from Paris XI University. Previously he worked as a visiting scientist at MIT in Bio-engineering for 2 years. George had 3 undergrad majors: Maths, Physics and Chemistry. His scientific career led to about 10 publications and patents.

• On the business side, Boston Business Journal has named me in the top 40 under 40 in 2012 in recognition of his business achievements.

• George is originally from Romania and grew up in Paris, France.

1 Comment

Click here to post a comment

Your email address will not be published. Required fields are marked *

Testimonials

default image

"Your daily letter is great!" , Ram , Founder and CEO, PeerIQ

default image

"Hi George - just want to tell you that you are doing a great work with Lending Times;-) Brgds, Kasper" , Kasper, Partner and Co-founder at Dansk Faktura Børs A/S

default image

"I've been following your newsletter for some time now and have been very impressed with the content." Charlie,Co-Founder | Bolstr

default image

"Hey George, I must say I really enjoy your site. It has inspired me to do some changes at our platform and we are the biggest consumer lender in Sweden." , Ludwig, CEO @ Savelend Sweden AB

default image

"Your daily email is very useful. It gives quick update on what's going in the market. Thank you very much for all that info." Yann Murciano, Head of Base Metals Trading at Morgan Stanley

Our daily p2p news digest

Daily News Summary Digest Sent Daily To Your Inbox