NEW YORK ( TheStreet) -- The ouster of Alvaro de Molina from the CEO slot at GMAC Financial Services earlier this week by a board reportedly frustrated with the pace of the company's recovery begs the question: Is Citigroup's ( C) CEO Vikram Pandit next?

With rivals like JPMorgan Chase ( JPM), Goldman Sachs ( GS) and large regional banks such as U.S. Bancorp ( USB) clearly closer to business as usual than survival mode, pressure is mounting on TARP-laden financial institutions like Citi and Bank of America ( BAC), who are both still dealing with fallout from the financial crisis, to right the ship.

Citi's board is "probably grumbling," says Carter Burgess, a managing director and head of the board recruiting practice at RSR Partners. "I wouldn't doubt there is impatience because everyone feels like their reputation is at stake to get this thing right."

But there is only so much that Pandit, who has spent two years at the helm, and his team can do at once, Burgess says.

"They would love to pay back TARP tomorrow but they can't," he says. "Everybody thinks two years is enough to do anything. It's such a mess it may take five years. He is under the pressure to get it done tomorrow."

"They got to stick with this guy unless they bring one of their new directors out of retirement, which I don't think they will," he adds.

Citi, which as received $45 billion from the U.S. government, has been unwinding its non-core assets and businesses, which are now housed in its Citi Holdings portfolio. While the company has made substantial progress, including the recent decision to sell shares to the public of its insurance subsidiary Primerica, it still has a ways to go.

Citi's Vice Chairman Edward 'Ned' Kelly said at an investor presentation last week that it was difficult to give an exact time frame on when it will disentangle itself from its risky assets and businesses.

"We want to move as quickly as possible," Kelly said. "We recognize that there is a tension between eliminating the equity market overhang, which is generated by the uncertainty around the pace of decline and whatever losses might be associated with it" and "doing the sensible thing economically, given the current market conditions and volumes involved because these are large businesses."

In the meantime, Citi continues to manage and add loans in businesses like CitiFinancial, its North American consumer lending business, as well as its retail partner credit cards, in order to "optimize value," Kelly said.

Rochdale Securities analyst Richard Bove says it could take four to six years for the company to get out from under Citi Holdings at a cost of $25 billion to $30 billion a year. "It is these costs that will keep overall company's earnings under pressure," Bove writes in a Nov. 11 note.

Directors and management of weaker institutions are under "tremendous pressure" from shareholders to get their companies out of the morass created by the financial crisis, particularly as signs of economic life begin to show, observers say.

But "maybe investors are getting too optimistic too early," says Anthony Sabino, a professor of law and business at St. John's University and a private practice lawyer.

"It's a very tough call for a board to make," Sabino says. "We're at a delicate point in the economy where again there are positive signs, but we're certainly not out of the woods yet. In a lot of ways coming out of this is like building a building ... Has the cement hardened enough to put up the next floor in the building?"

Citi's Pandit is feeling the heat from a number of groups, including regulators, the board, frustrated investors, even employees and customers, who are all anxious for him to get the bank moving again. However, if Pandit "moves out of survival mode too soon, he will be crucified," Sabino says. "But if he moves slowly and is cautious then obviously the board will criticize him," all while JPMorgan is "eating their lunch."

"In troubled times like this if you're going to err; it's better to err on the side of caution. If you make a misstep now, you're really going to be in trouble," Sabino says.

Citi isn't the only big financial firm at a crossroads. Once Bank of America names a successor to outgoing CEO Ken Lewis, moving forward will be the "key issue" confronting the new CEO at BofA, he adds.

"The continuing delay in the search ... has now reached the danger level. At this point, shareholders, customers, employees are losing confidence," Sabino says. "It begets the second issue -- whoever that successor is that person is immediately going to be confronted by the board. That person will have to make a difficult choice," between remaining in survival mode or shifting gears to performance and growth.

"The bottom line it's going to be tough for them to change gears," he says.

--Written by Laurie Kulikowski in New York.