The promise and risk of AI in mortgage lending and secondary markets - HousingWire

Artificial intelligence is creating opportunities to automate labor-intensive mortgage processes, analyze more data and reduce the amount of manual work required across the loan lifecycle. But in an industry where inaccurate information can create consequences for borrowers, lenders and secondary-market investors, implementation requires more than simply choosing an AI model and putting it to work.

Julia Curran, senior managing director of residential AI products at SitusAMC, has spent more than 40 years in mortgage banking across origination, secondary markets, servicing and technology. Curran discusses why mortgage companies need to understand the differences between AI models, how rigorous mortgage AI testing should work and why subject-matter expertise and human judgment remain critical as adoption expands.

Not all mortgage AI is built the same

HousingWire: AI is often discussed as a single technology, but you’ve emphasized that not all AI is the same. What should mortgage leaders understand when evaluating it?

Julia Curran: Our market is very niche, and what we need AI to do is complex. It’s not just extracting data or summarizing servicing comments. Leaders must be sure to include their subject matter experts when choosing LLMs or vendors. We tested proof-of-concept cases with 27 different vendors, and 26 couldn’t perform the tasks necessary to really make AI work for secondary market reviews.

Every AI model has its strengths and weaknesses. Whether it’s identifying documentation, summarizing things or using tables, your model selection should be specific to the use case, and every model that’s out there has different costs. If you’re trying to optimize your cost and accuracy, you have to understand how to manage the whole process, model strengths and weaknesses and credit usage.

You also need subject-matter experts. To automate something accurately, you have to understand exactly how someone performs that task today. AI might extract information correctly, but the real question is whether it extracted the right information for the situation. That logic comes from people who do the work every day.

Accuracy is only the first test

HW: What does rigorous testing need to account for beyond basic accuracy?

JC: You have to test against the widest possible variety of scenarios. We have a ground truth database with hundreds of loans that have already been reviewed, so we can test different loan types, borrowers and documentation.

Think of income. One borrower may have a pay stub and W-2, while another has six income sources, tax returns and K-1s. You need to know that the AI handles all of those variations correctly and consistently produces the same answers an experienced person would.

Mortgage AI testing also doesn’t stop when something goes live. Models change, so you need regression testing to make sure your prompts and agents still produce the expected results after a model changes. Bias testing is critical too. AI can introduce bias based on information you may not expect it to use, so companies need controls around what the model should and should not consider.

Efficiency has limits when judgment is involved

HW: Where can AI meaningfully reduce manual work today, and where should human judgment remain?

JC: Any input going into an LOS, POS or servicing system AI can help. We shouldn’t necessarily be manually keying information that can be accurately extracted or fed from somewhere else. But I don’t believe AI should make the final credit decision.

Credit decisions aren’t always based only on numbers. Guidelines vary, there can be compensating factors and there are circumstances that require human judgment. AI can support that decision, but I believe the final decision should remain with a person.

AI could expose what loan sampling misses

HW: How could AI reduce some of the risk created by traditional loan sampling in due diligence?

JC: IIn most securitizations, the client reviews the loan before they know the exit strategy. If securitization is the exit, loans are selected from a pool of loans; however, only the highest-rated loans are usually selected, with loans that received C or D grades being excluded.

For example, if you had 1200 loans, a sample of 600 loans could have been reviewed, with 400 receiving A or B grades and 200 receiving C or D grades, which means the C or D rate was 33%. An issuer creating a securitization may take the 600 nonreviewed loans along with the 400 that received an A or B grade to create a 1000 loan deal. As the rating agency is only seeing the grades for the reviewed loans, they would see 400 A and B loans and could move to rating the overall pool of 1000 loans based on that information, given the appearance of a 40% sample. This could understate the risk, as the actual C or D rate was 33% of the original 600 loans reviewed before an exit strategy was selected. The solution to this problem can come through AI, as it gives us an opportunity to look at all loans across a securitization through a combination of traditional full third-party reviewer (TPR) reviews and AI-driven reviews.

For example, take compliance or credit testing. For loans that were not in the full TPR review population, you have an opportunity to extract the appropriate data, feed it through the compliance engine or credit calculation engine and identify potential issues. If any issues are flagged, a person can examine the loans with concerns in additional detail.

HW: How do you think lenders determine when an AI capability is ready to move from testing into a live workflow?

JC: It comes down to the testing, controls and checks performed on the results. Did you test every type of borrower and loan you originate? Is there a logic check against the AI? Are you continually regression testing?

We won’t release anything below a 95% accuracy level, and reaching that point can take months. We test against live loans, run processes in parallel and soft launch before broader use. The best approach is still to have a human in the loop. Even after implementation, companies should manually review a selection of loans periodically and confirm that they are getting the same answers as the AI.

Why quality matters more than speed or price

HW: What will separate firms that create lasting value from AI from those that introduce new risks?

JC: What scares me most is that there’s no industry-wide way to determine whose AI models or data sets are actually better. We have a very rigorous process for putting AI into the market, but someone else can release a model much faster and claim it can extract the same information. Without another check, it’s difficult to know which answers are more accurate.

That matters across the entire mortgage cycle. In the secondary market, for example, AI could help determine whether a loan is good or bad. If the model isn’t well trained or accurate, a loan could be viewed as stronger than it really is, rated accordingly and ultimately sold to investors. The reliance on data is huge. Accuracy matters for the borrower, the lender, rating agencies and investors. If we aren’t careful about AI accuracy, it can affect every part of the mortgage cycle.