NEW YORK ( TheStreet) -- Some of the largest Wall Street investment managers are preparing to take advantage of the government's abandonment of large mortgages, called "jumbos," and have set up investment vehicles to cherry-pick the best loans.

A report in Wednesday's New York Times says that the real estate market is beginning to shudder at the prospect that government-backed mortgage provider Fannie Mae ( FNM.F) will lower its high-cost area loan limits on Sept. 30, effectively backing away from guaranteeing mortgages on higher-priced homes.

The lower limits, which could drop to as little as $650,000 in states like New York, California and New Jersey, are stoking worries that housing prices for the most expensive areas may collapse as Uncle Sam retreats from the market.

But some of the largest investment firms on Wall Street are betting on a profitable, private jumbo loan market and have set up vehicles to fund them.

"We look forward to the potential roll-back at the end of September 2011 of the temporary increase in the conforming loan limits applicable to Fannie Mae and Freddie Mac, which would increase the share of the mortgage market for the private sector and reduce future risk to taxpayers, " said Martin Hughes, president of California-based REIT Redwood Trust ( RWT), when it announced the closing of a $290 million prime jumbo residential mortgage loan securitization in March.

California-based PennyMac ( PMT) set up a fund to underwrite jumbo residential loans late last year, according to a report in Reuters. "I feel that real estate is nationally at a bottom," Penny Mac's chief investment officer David Spector told the news service. "From an economic point of view, we're in the camp that things get better in 2011."

Wall Street investment titan BlackRock ( BLK) set up its $1 billion BlackRock Mortgage Investors Fund last year that will invest in jumbo loans.

"The securitization outlets have been severely reduced for a whole host of reasons -- regulatory, ratings agency, investor, the list goes on and on," said BlackRock's Randy Robertson in an interview with TheStreet, when the fund was announced in 2010. "For another host of reasons, banks haven't been taking on additional real-estate exposure either. The combination of those two and Freddie and Fannie not accepting larger balance loans, the market's underserved. We want to serve that market."