NEW YORK (

TheStreet

) -- Mortgage applications decreased 4.6% from a week earlier, according to data from the Mortgage Bankers Association's weekly survey.

The decline in the week ending Aug. 16 was driven by a drop in refinance activity as rates moved higher. The Refinance Index decreased 8% from the previous week, just as the rate on the 30-year fixed-rate conforming mortgage moved up from to 4.68% from 4.56%.

The Refinance Index is down 62% from its recent peak reached in early May, when the 30-year rate was as low as 3.5%.

The seasonally adjusted Purchase Index rose 1% from a week earlier. Unadjusted, the index dipped 0.4% compared with the previous week and was 5% higher from a year ago.

Refinance as a share of total mortgage activity decreased to 62% from 63% a week earlier. The adjustable-rate mortgage share of activity increased to 6% of total applications.

The drop in refinancing activity could hurt mortgage banking revenues for the big banks. Unusually high refinance activity had buoyed profits for banks such as

Wells Fargo

(WFC) - Get Report

,

U.S. Bancorp

(USB) - Get Report

and

Fifth Third

(FITB) - Get Report

over the past few years.

According to a recent report by LPS, with the rise in interest rates, the pool of borrowers who can still refinance shrunk 33% from 8.9 million in March to 5.9 million in June, representing about 12% of all active loans that can still be refinanced. LPS determines the pool of eligible borrowers by including those who have at least 20% equity in their home, have a credit score of greater than 720 and pay an interest rate of more than 5% on their mortgage.

Banks have however been prepared for the decline in refinance activity.

JPMorgan Chase

(JPM) - Get Report

CFO Marianne Lake warned last quarter that the refinance market could drop 30% to 40% if rates stayed at these levels or moved higher.

-- Written by Shanthi Bharatwaj in New York.

>Contact by

Email

.

Follow @shavenk

Disclosure: TheStreet's editorial policy prohibits staff editors and reporters from holding positions in any individual stocks.