Are there any federal or state laws in the US regulating which models can banks and money lenders use to decide for/against lending money to an individual? I am interested on this topic in the context of mechanisms to avoid discrimination (for example, how do you ensure that your AI model doesn't have a variable that indirectly or directly takes into account ethnicity when making a decision).
4
-
When it is natural to base money lending decisions on wealth and income factors and these factors happen to be highly correlated, the same AI that can make good predictions about money lending can certainly also make better-than-average predictions of the applicant's ethnicity. How would you decide if implicit criteria (perhaps automatically learned from "experience") such as "add 10 points if applicant practices as a dentist" or "deduct 10 points if given name begins with 'Yam'" are deeply hidden in a neural netwrok?– Hagen von EitzenCommented Dec 28, 2020 at 11:16
-
1The classic method of demonstrating discrimination is not by examining the formula, but by presentating sets of matched applicants who differ in ethnicity but are very comparable in wealth and other relevant indicators. If the model reliably favors the same ethnicity from such matched pairs, it is discriminatory.– David SiegelCommented Dec 28, 2020 at 21:01
-
@HagenvonEitzen I concur this is an interesting, and that is precisely what I am trying to understand further by going to the exact laws regulating it. How are implicit criteria defined? Or is it up to a judge to determine what is or what is not from intuition– A. FenzryCommented Dec 29, 2020 at 0:49
-
Thanks @DavidSiegel. Do you happen to have references to a trial or public case where this practice occurred?– A. FenzryCommented Dec 29, 2020 at 0:49
Add a comment
|
1 Answer
Are there any federal or state laws in the US regulating which models can banks and money lenders use to decide for/against lending money to an individual?
There aren't really safe harbors regarding what lenders can use. There are only laws that prohibit certain practices like discrimination in lending. Everything else is permitted.
ow do you ensure that your AI model doesn't have a variable that indirectly or directly takes into account ethnicity when making a decision).
Using a "black box" machine learning or AI model is inherently risky legally because it isn't transparent and there is no good way to demonstrate that discriminatory effects aren't due to to discriminatory factors as opposed to neutral factors.
AI models tend to pick up on impermissible factors by attaching weight to factors that aren't substantively important, but are correlated strongly with impermissible factors, for example, assigning different weights to different surnames.
On the other hand, if you can spell out the precise formula by which your model is constructed (in a closed court proceeding if necessary to preserve trade secrets) and can demonstrate that no impermissible factors are considered, directly or indirectly, then your mathematical model is very likely to be upheld as legal and permissible.
answered Dec 28, 2020 at 9:20
-
Can you give specific references to the laws prohibiting discrimination in lending and what methods does law enforcement use to ensure their compliance? @ohwilleke Commented Dec 28, 2020 at 10:09
-
The federal reserve summarizes the requirements here: federalreserve.gov/boarddocs/supmanual/cch/fair_lend_over.pdf enforcement is mostly through private civil actions and by complaints to regulatory agencies (of which there are many, divided by type of lender) Commented Dec 28, 2020 at 21:36