Massachusetts slapped student loan company Earnest Operations LLC with a $2.5 million settlement over allegations that its AI lending models discriminated against minority borrowers. The investigation, led by Attorney General Andrea Joy Campbell, uncovered what officials called a textbook case of algorithmic bias gone unchecked. Turns out, robots can be racist too.

The company's AI models allegedly produced disparate impacts on Black and Hispanic loan applicants. No surprise there. Earnest employed a so-called "Knockout Rule" that automatically denied applicants based on immigration status—without even bothering to tell them why they'd been rejected. Real classy. These unexplainable decisions highlight how AI systems often operate as complex black boxes, making accountability difficult to establish.

State officials found the lender used cohort default rates as a factor in refinancing decisions, a practice that raised serious red flags. The company also failed to adequately test its algorithms for bias before deploying them. Because why bother checking if your fancy AI might discriminate? That would be too much work.

Massachusetts isn't waiting for federal regulators to wake up. States are increasingly filling the void left by federal inaction on AI lending practices. Campbell's office demanded sweeping reforms to Earnest's practices, not just a check.

The settlement requires Earnest to overhaul its lending practices from top to bottom. Every automated stage of the lending process now faces scrutiny. The company must test AI models for bias, comply with both state and federal fair lending laws, and improve disclosure requirements for applicants. This agreement establishes an important precedent for AI use in financial services across the industry.

This case highlights how AI is reshaping lending decisions—determining eligibility, terms, pricing, and risk scores. Algorithms now control prescreen processes and quick decline decisions. Machine learning has infiltrated every corner of underwriting.

For consumers, the message is clear: AI-driven lending isn't the objective, bias-free utopia tech companies promised. The Massachusetts Assurance of Discontinuance signed on July 10, 2025 represents a significant regulatory action addressing AI in consumer finance. Massachusetts has shown it won't tolerate violations of unfair and deceptive acts or practices laws. Other states are likely to follow suit.

The robots might be running the show, but regulators are watching.