NEW YORK ( TheStreet) -- An estimated 9 million Americans remain eligible to refinance their homes at today's historically low interest rates, according to the April Mortgage Monitor report from Lender Processing Services.

These are borrowers who have more than 20% equity in their homes, have credit scores higher than 720 and still pay interest rates greater than 4.5% on their mortgage loans, making them well qualified to refinance.

Refinancing has been a steady source of fee income for banks over the last few years amid a low interest rate environment. Banks including Wells Fargo ( WFC - Get Report), JPMorgan Chase ( JPM - Get Report), US Bancorp ( USB) and Fifth Third ( FITB - Get Report) are among those that have profited from the spike in refinancing.

But with interest rates climbing from the bottom, refinancing activity has dropped sharply in recent weeks. The Mortgage Bankers Association reported that its Refinance Index declined 15% last week and is at its lowest level since November 2011.

Still, analysts including FBR Capital's Paul Miller have argued that the market for refinancing still remains large. For specialty servicers such as Nationstar ( NSM) an d Walter Investment ( WAC), the average coupon on the servicing portfolio is more than 5%, which still provides an incentive to refinance, given the 30-year mortgage rates is currently at 4%.

The government has also extended its Home Affordable Refinance Program, which allows borrowers who owe substantially more than their homes are worth to refinance, till 2015. A proposal to expand the eligibility requirements of the program to include more borrowers is still being debated.

So while refinancing activity might be ebbing, it is likely to stay elevated for a while unless rates move significantly higher from here.

-- Written by Shanthi Bharatwaj in New York.

>Contact by Email.

Disclosure: TheStreet's editorial policy prohibits staff editors and reporters from holding positions in any individual stocks.