NEW YORK (TheStreet) -- Today's headlines on Bank of America's (BAC) earnings report dwell on the $4 billion in legal costs that made second-quarter results miss analyst estimates.

But the real problem for the stock is Bank of America's business isn't growing the way it should in an improving economy.

At around $15.60, shares are down almost today as analyst opinion is all over the place on whether the Charlotte-based bank beat the numbers or missed them after adjustments made for litigation expenses and other charges.

But the core of the problem is two-fold: The bank hasn't gotten the improvements in loan volume that it should get as economic prospects improve, and the delay in the anticipated rise of interest rates is keeping B of A's net interest margin lower than it was in a more torpid 2013 economy.

B of A reported that average loan balances in its key consumer and business banking unit shrank over the last year, dropping $3.3 billion to $160.2 billion. Worse, net interest margins for the whole company -- the spread between what it pays for deposits and charges for loans -- actually declined, to 2.22% from 2.26% in the first quarter, Keefe Bruyette & Woods analyst Chris Mutascio says.

>>Intel Soars: What Wall Street Thinks

>>Why Sports May Be the Key to Rupert Murdoch’s Bid for Time Warner

The bull case for banks is that a recovery makes people borrow more, and rising interest rates that accompany expansion boost the spread between deposit and loan rates. Neither one happened for B of A this time around.

The weakness is fairly broadly spread across the bank's business. The mortgage unit, as expected, is suffering from fewer refinancing applications. The global banking business that includes the Merrill Lynch investment bank grew just 1% on the top line. Loans grew 6% in this business, but return on capital dropped sharply as rates stayed low even for loans to real-estate developers and larger businesses. B of A invested heavily in this business during the quarter but hasn't yet seen a return.

B of A's report is pretty similar to one last week by rival Wells Fargo (WFC). Wells also saw a dip in net interest margin, but from a much higher level than Bank of America's as loans grew 4%. Its shares also fell on the day the San Francisco-based bank reported.

If you liked this article you might like

How to Get Rich Using Warren Buffett's Favorite Stock Market Indicators

With the Fed, It's Different This Time

How to Live Just Like Billionaire Warren Buffett

Why Hurricanes Won't Force the Fed to Ditch a December Rate Hike

How to Make a Deal Like Billionaire Investor Warren Buffett