NEW YORK ( TheStreet) -- JPMorgan Chase ( JPM) will have to be very careful if it admits wrongdoing as part of an expected $4 billion settlement with the Federal Housing Finance Agency, to avoid opening up major exposure in private mortgage putback lawsuits. As part of the large settlement expected between the nation's largest bank, the Department of Justice, federal regulators and states' attorneys general, JPMorgan Chase is working on a $4 billion settlement with the FHFA, which regulates Fannie Mae ( FNMA) and Freddie Mac ( FMCC). Fannie and Freddie were taken under government conservatorship in September 2008. JPMorgan Chase, Bank of America ( BAC) and other major mortgage lenders have already entered into large direct settlements with Fannie and Freddie, over the losses on loans sold directly to the agencies. Those loans were made under Fannie Mae and Freddie Mac guidelines, and the agencies had all the relevant data to make strong cases for settlements. The FHFA settlement is completely different, because it will cover the losses taken by Fannie Mae and Freddie Mac -- together known as the government sponsored enterprises, or GSEs -- on private label mortgage-backed securities sold to them by JPMorgan. The GSEs bought a huge amount of private label paper during the housing boom. This means that they were buying the same MBS that other institutional investors bought from JPMorgan. "The Department of Justice now has this mandate that any settlement you make with a company has to include some statement of admission of wrongdoing," says Guggenheim analyst Marty Mosby. Making an admission of wrongdoing in the direct sale of loans to GSEs is relatively easy, according to Mosby, since a bank "can say that due to a lack of strict controls and overriding acceptance in the marketplace, there was mismanagement, but nothing fraudulent." But language admitting wrongdoing in the sale of private label MBS to Fannie and Freddie is "very sticky," Mosby says. "That is a critical piece of language. As soon as you admit to any wrongdoing to FHFA, it will affect lawsuits from private investors." The Wall Street Journal on Wednesday reported that a group of investors, including BlackRock ( BLK) and Neuberger Berman are seeking "at least $5.75 billion" from JPMorgan Chase, to recover losses on mortgage-backed securities sold to them by the bank.