TheStreet.com TV Recap: Why Investors Fear the Fed

The financial stocks are weak because of Fed chatter that the latest job number could be a fluke and that people should not worry about it, Jim Cramer said on TheStreet.com TV's Wall Street Confidential Web video Monday.

"The idea that you could base anything on one number, they are saying, is not prudent," he said.

However, Cramer said he's concerned that although it is certain there could be revisions and that one number in the scheme of many data points may not be something people want to act on, the market is very uncertain here.

The fact that the Fed is not including the employment number as part of the overall mosaic of what's going on in the market is frightening to a lot of investors, he explained.

"A lot of what's happened in the mortgage market ... was based on the idea that housing prices have come down so people are not as certain about the home they bought," Cramer said. "Maybe they'll walk away from their down payment -- we're seeing a lot of that."

But at no time, he said, have people ever worried about the other part of the model. Most of the models -- such as those from Countrywide (CFC Quote), Accredited Home Lenders (LEND Quote) and Washington Mutual (WM Quote) -- are predicated on the idea that as long as there is employment growth, there's not going to be a problem, Cramer said.

"The idea that that could suddenly be off the table and we should be worried about employment means that all these companies' mortgage models are wrong and far too bullish," he said. The Fed is advising people not to panic, but what it may not understand is that "if you get a couple of numbers like this, then all of the models are wrong and we're going to get a lot of defaults," Cramer continued.

Therefore, he said, he found the Fed's reassurances to be precisely what the market didn't want to hear.

Concerning the next Federal Open Market Committee meeting slated to take place Sept. 18, Cramer said he expects it will do a quarter point rate cut.

"There are people that think they should do nothing and there are people who think they should do a quarter point," he said. "That's really the substance. I hope the people who say they do nothing don't have sway."