NEW YORK (TheStreet -- The collapse of the housing bubble is still a painful memory for many investors. Even as the economic data points to a recovering housing market and homebuilder stocks have been rising, they are having a hard time believing in this real estate rally. They have good reason to be worried.

On a positive note, the

Federal Reserve

has reported that aggregate household net worth at the end of 2012 was $66.1 trillion, nearly back to its pre-crisis peak of $67.4 trillion, reached at the end of the third quarter of 2007. Home prices have been steadily moving higher with January levels hitting the best point in six years. Existing home sales have risen and new home sales are at impressive levels. This is delivering tremendous performance for homebuilder stocks. Even delinquency and foreclosure rates are down. This week the market will receive more housing data and several key homebuilders like


(LEN) - Get Report


KB Homes

(KBH) - Get Report

will report their earnings.

J.P. Morgan analyst Michael Rehaut has a positive outlook on the housing sector and believes that housing industry fundamentals will continue to improve over at least the next 18 months. He points to lower inventories of homes and increasing sales. He also notes that most homebuilders on their recent first-quarter conference calls suggested solid demand trends in April. However, the recovery of wealth amongst individuals has not been uniform, causing the hesitation in believing in this real estate rally.

The wrench in the machine, though, is the potential for rates to go up now that Federal Reserve Chairman Ben Bernanke has signaled that he'd like to tap the brakes on bond buying. Even though the Fed has been buying bonds like a drunken sailor, banks have been careful about who gets a mortgage. You needed a near-perfect score to get a mortgage, which has kept the market in check. It is also the same thing that has some worried that if rates go up, this pool of buyers could shrink.

Chris Whalen of Carrington Investment Services believes the "recovery" in housing is not normal and he writes that it's not likely to endure unless job creation and income growth follow. He thinks the increase in home prices that has fueled the real estate rally is related to a decrease in the flow of distressed loans and foreclosures. He also feels that the refinance boom has peaked and bank lending is already showing signs of declining. Whalen is also concerned about the costs for banks associated with mortgages like agency loan guarantee fees. He recently wrote, "The huge legal and operational cost of dealing with legacy mortgage issues is going to drive most commercial banks out of the residential-mortgage business entirely."

Jay Brinkman, chief economist at the Mortgage Bankers Association, agreed during a recent speech at the American Enterprise Institute . He said that mortgage origination costs are going up considerably and questioned whether it's worth the litigation and headline risk. The cost of a successful lawsuit against a lender is $180,000 to $200,000 and an unsuccessful lawsuit costs in the range of $40,000. The profits on these loans? Some $2,000, he said. The fixed cost on a mortgage loan is now roughly $5,600, according to Brinkman.

Clearly, the math doesn't work and now Washington is putting even more pressure on the banks to lower standards and steer high risk borrowers to FHA loans. These loans are insured against defaults, so the banks are not on the hook, just the taxpayers.

Some believe that the pendulum has swung too far for potential home buyers, cutting out many worthy homeowners, while others worry that loosening up standards again will create another housing bubble. Those that believe the housing recovery is sustainable point to the recent past action but don't really look ahead. The housing recovery bears seem to have more arguments to justify their view, namely higher costs for banks, higher rates for customers and a decline that has already started showing itself.

The Federal Reserve Bank of Chicago noted that, in May, consumption and housing was flat and housing permits decreased. Housing starts were up, but permits were down. May is prime home season. This can be seen as early signs that this housing recovery is not sustainable.

If Washington steps in to prop up the housing market, it may boost it short term, but it won't be a long term solution. Investors should be prepared to see these trading and investing strategies related to the housing market reverse course. The headwinds against this recovery are too great for even Washington to sail through.

--Written by Debra Borchardt in New York.

>To contact the writer of this article, click here:

Debra Borchardt


Follow @WallandBroad

Disclosure: TheStreet's editorial policy prohibits staff editors, reporters and analysts from holding positions in any individual stocks.