As Congress imposes greater restrictions on financial institutions that received government bailout money, a number of smaller banks are beginning to express a desire to give the money back -- and some question whether they even needed it in the first place.

Limits Congress has placed on executive compensation packages and other perks have caused widespread grumbling among banks of all sizes that have received aid through the Troubled Asset Relief Program. But some also worry about tarnishing their brand with the stigma now being associated with accepting government money, thanks to repeated bailouts for companies like Citigroup ( C), Bank of America ( BAC) and AIG ( AIG).

"My sense is a lot of banks are looking at paying back TARP money," says Jean Everett, a partner at Hiscock & Barclay's financial institutions and lending practice. "It's a tough call because you don't know what's going to happen in three to six months or three to six days ... Maybe sitting on that capital will still be a good idea even though it's not economic at these rates to lend it."

TCF Financial ( TCB), a $17 billion-asset company located outside of Minneapolis, announced Monday that it applied to the Treasury Department to redeem its preferred stock for a total price of $361.2 million, plus a final pro-rated accrued dividend. The company provides retail and commercial customers in five states across the Midwest and in Colorado and Arizona.

TCF -- known to have strong deposit gathering skills -- says it has sufficient capital to complete the redemption payment and will not have to issue any additional common equity.

Iberiabank Corp. ( IBKC), a $5.6 billion-asset Louisiana-based bank holding company that operates Iberiabank and Pulaski Bank and Trust, said last Friday it planned to return the government's $90.5 million investment in the firm.

Last fall, the Treasury Department scrambled to put together the TARP program in the wake of Lehman Brothers' bankruptcy filing and AIG's near collapse in September. The effort was aimed at shoring up capital and confidence in the banking system and freeing up credit in frozen lending channels.

The government bailout effort has since went through several iterations. Its original intention was to purchase banks' toxic assets, but soon shifted to buying preferred equity stakes in the companies to prop up capital levels. Treasury allotted $250 billion to invest in preferred stakes of large and small banks under the Capital Purchase Program -- of which roughly $197 billion has been invested.

While big financial companies like Northern Trust ( NTRS), Goldman Sachs ( GS) and even consumer banks like Wells Fargo ( WFC) have talked about repaying TARP, smaller banks that shied away from more risky product offerings make as good a case as anyone for returning the capital.

"In our opinion, the substantial majority of publicly traded banks does not, and did not, need TARP proceeds," Kevin Reynolds, an analyst at Wunderlich Securities, writes in a note on IberiaBank. "However, most publicly traded institutions were strongly 'encouraged' by the government to participate so as not to create a banking environment of haves and have-nots. That day appears to be coming to an end.

" The more Congress talks, the less that healthy banks want to participate in a program seemingly designed to prop-up the largest and most dysfunctional banks at the expense of the traditional banking sector," Reynolds writes.

Some small firms have avoided TARP altogether. Roughly 170 banks declined to take government funds, according to Keefe, Bruyette & Woods.

Just this week, Sussex Bancorp ( SBBX) of Franklin, N.J. said it had decided not to participate in TARP, even after receiving approval in December to get $10 million in capital.

One reason the company declined the funds was because it did not believe it could "profitably deploy the additional capital" in the current environment, Sussex CEO Donald Kovach said in a statement.

"Since October, our non-performing assets have stabilized, and we have seen loan demand slow. We are a very liquid, well-capitalized institution," Kovach said.

IberiaBank CEO Daryl Byrd sounds like a man who wishes he had made the same decision when his bank, which targets small- and mid-size commercial and small business customers, was offered government money. In a statement last week, he said the "recent actions, interpretations, and commentary regarding various aspects of the Capital Purchase Program places our company at an unacceptable competitive disadvantage."

The board "has determined that continued participation in this program is no longer in the best interest of our company and its shareholders," he added.

As part of President Obama's American Recovery and Reinvestment Act of 2009, signed into law last month, banks will now have an easier time redeeming government stakes, but the details are still vague.

Banks now have the option to redeem some or all of the government investment shares, without having to wait the mandatory three-year period or raise replacement private capital, as in the original rules for the Capital Purchase Program. However, companies do need to notify their primary regulator at least 30 days prior to share redemption so that regulators can sign off that the firm has sufficient capital even without the TARP funds.

TARP participants must redeem a minimum of 25% of the issue price of the preferred. They also must pay any accrued or unpaid dividends, the Treasury says.

To be sure, while several big bank CEOs have expressed interest in paying back their TARP funds as soon as possible, larger banks looking to exit the program may have more of a difficult time in doing so, as they scuffle to preserve capital.

A host of banks cut their dividend payouts over the past several weeks, including JPMorgan Chase ( JPM), PNC Financial Services ( PNC), US Bancorp ( USB) and Wells Fargo.

Banks have to be prepared to demonstrate to regulators that if they give up the TARP capital they will have enough capital to weather future bumps in the road, says Wayne Abernathy, executive vice president of financial institutions policy at ABA.

PNC CEO James Rohr, whose company received $7.6 billion of government funds, said he expected to exit its TARP agreement "as soon as appropriate."

But Rohr said he has concerns on whether he would want PNC to have lower capital levels than other large and mid-size banks just to get out of the agreement, during a conference call earlier this week.

"I don't think you want to do it until you get an economic environment that is clearly more favorable to the depositor," he added.

IberiaBank said its pro forma tier-1 capital ratios as of the end of the fourth quarter would be 9.51% compared to the reported tier-1 ratio of 11.27%. Its tangible equity ratio would have been 7.24% compared to the reported 8.91%.

IberiaBank said between early December -- when it received Treasury's investment -- and the end of February, the company's loan portfolio rose by $68 million, or 2% from new loan originations. Less than two weeks after IberiaBank received TARP funds, it completed a common stock offering of $109 million.

TCF Financial CEO and Chairman William Cooper said in a statement earlier this week that the "rules have definitely changed" for TARP participants.

Assuming the redemption of preferred stock occurred at Dec. 31, TCF's pro-forma capital ratios would have been 5.85% for tangible common equity as opposed to 6%, it said. Its tier-1 capital would have been 8.88% as opposed 11.79%, still in excess of the so-called well capitalized threshold required by regulators.

The company "still could raise our own capital if we had to," TCF Financial spokesman Jason Korstange said in response to questions regarding whether it may be safer to hold onto the capital while the economy still shakes out.

The company completed a $115 million trust-preferred security offering in August.

TCF said that in the last six weeks of the year, it originated approximately $490 million in loans and leases, 762 loan modifications and $117 million in home equity loans.

But David Rochester, an analyst at Friedman Billings Ramsey, worries about TCF's home equity exposure.

"While we continue to believe TCF Financial has an attractive deposit franchise, we expect consensus EPS to decline materially over the next six months on worsening credit and weaker fee income expectations, driving strong multiple compression," Rochester writes in a note on Tuesday. He expects TCF to cut its dividend this year.

Rochester, who rates the company at underperform, is primarily concerned with TCF's $2 billion of second-lien home equity loans, in which it does not hold the first lien position. These loans make up 15% of the company's loan portfolio, he says. "Severity is close to 100% for these loans in the event of default," he writes in the note.

Approximately 85% of TCF's home equity portfolio is from homes located in Minnesota, Illinois and Michigan -- states experiencing "meaningful increases" in unemployment and declines in home prices, factors which greatly impact a borrower's ability to make payments on mortgages, he writes.

Wunderlich's Reynolds expects more healthy institutions to pursue secondary common stock offerings in order to redeem the TARP funds, increase their tangible common equity ratios and "get the government out of their business," he writes.

"We believe investors would welcome the opportunity to move from the sidelines into the banking sector by participating in what will likely be significant common equity offerings," Reynolds wrote in the IberiaBank note.

More from Investing

Quick Read: 3 Things for Investors to Know Before Wednesday's Trading Session

Quick Read: 3 Things for Investors to Know Before Wednesday's Trading Session

Replay: Jim Cramer on the Markets, Oil, General Electric, Zillow and Micron

Replay: Jim Cramer on the Markets, Oil, General Electric, Zillow and Micron

Pegasystems Founder Explains Why He Has One of the Hottest Tech Stocks Around

Pegasystems Founder Explains Why He Has One of the Hottest Tech Stocks Around

Adobe Isn't Just Going After Shopify With Its Latest Acquisition

Adobe Isn't Just Going After Shopify With Its Latest Acquisition

Experts Break Down GDPR Risks for Investors

Experts Break Down GDPR Risks for Investors