(Updated stock prices throughout)

MCLEAN, Va. (

TheStreet.com

) --

Capital One Financial's

(COF) - Get Report

sagging credit portfolio begs a fundamental question about its future: Should it be more of a bank or a credit card company?

The 21-year-old company, known for its ubiquitous "What's in Your Wallet?" credit card advertising campaign, faces pressure as its legacy credit card and other financing businesses struggle amid the recession and looming new credit card regulations.

Meanwhile, Capital One spent the last four years expanding its business by acquiring several mid-size banking outfits, stretching from New York to New Orleans. Some observers say Capital One will have to dig deeper into banking to remain a powerhouse company, as unemployment continues to rise and stricter card legislation passed this spring by President Obama will negatively affect all credit card businesses.

Even though Chairman and CEO Richard Fairbank has said the company is working to integrate its most recent acquisition, Chevy Chase Bank of Bethesda, Md., and is focusing on building "strong, local scale positions," at least one analyst says more bank acquisitions could eventually be in the cards, given the changing credit card environment.

The regulatory changes will "make it a lot harder for the credit card business in general to really recover, just because they're no longer able to re-price for risk," says Jason Arnold, an analyst at RBC Capital Markets. Credit card firms will do "a lot less lending or charge annual fees. ... Their banking presence will take a bigger piece of the pie."

"

Having the more stable bank deposits is certainly more favorable for them," Arnold adds. "Their presence in strong markets like New York metro market and Baltimore with Chevy Chase is beneficial, so that's certainly a plus. We'll probably see more small regional and community banks have problems ahead. Just being a widely recognized name is going to be beneficial."

Capital One began building its retail banking franchise to diversify the company away from strictly consumer finance businesses, but also to garner deposits as a cheaper way to fund its lending businesses. It acquired New Orleans-based

Hibernia Bank

in 2005, which had a large presence in Louisiana and Texas. It then purchased

North Fork Bank

in 2006, which expanded the company's branch network throughout New York, New Jersey and Connecticut. The company just acquired

Chevy Chase Bank

, a smaller Bethesda, Md.-based banking outfit, in February, which added $13 billion in deposits and 240 branches primarily in the Washington and Baltimore areas.

The company, which still receives a majority of revenue from the card lending business, has been hurt by the down economy as consumers curtail their spending habits and make sure to pay their credit card bills on time -- meaning that Capital One earns less fees from late payments and charges to customers for going over their purchase limits.

As the financial crisis intensified, Capital One was forced to reposition its businesses and "de-risk" its balance sheet. The company has been running off high-cost deposits and shrinking its loan portfolio and opting instead to grow its investment securities portfolio until the economy begins to improve. Two years ago, the company shuttered its wholesale mortgage business inherited from North Fork as the credit crisis intensified. It has also significantly pulled back on its auto finance business and U.K. lending business and no longer offers installment loans.

The new changes coming from President Obama's card legislation will further significantly change the card business. Not only will credit card companies like

American Express

(AXP) - Get Report

and

Discover Financial Services

(DFS) - Get Report

be affected, but banks with a large credit card operations, such as

Bank of America

(BAC) - Get Report

,

Citigroup

(C) - Get Report

and

JPMorgan Chase

(JPM) - Get Report

also will feel the pinch.

Still, credit quality remains an issue for Capital One in the foreseeable future, as customers stop paying their bills altogether. Capital One said in a regulatory filing on Monday that monthly net charge-offs in its U.S. card business last month worsened a bit from June to 9.83% and loans that were at least 30 days past due on their payments also rose slightly to 4.83%.

While the company and other credit card competitors saw improvement in early-stage delinquencies during the second quarter, it said last month that "late-stage" delinquencies worsened in the quarter.

"Capital One has a lot of exposure to commercial real estate in their banking operations," on top of auto loans and troubled card loans, Arnold says.

Capital One shares, which recently were down 1.8% to $34.35 on Monday, have fallen about 45% from its 52-week high of $63.50 on Sept. 22, 2008. Stocks of key competitors American Express and Discover, meanwhile, have fallen 27% and 33% from their 52-week highs, respectively.

Capital One

posted a net loss to shareholders of 65 cents a share for the second quarter due to the firm's repurchase of preferred shares through the Troubled Asset Relief Program. Excluding the TARP repayment, the company would have made $224.2 million, or 53 cents a share, in the quarter, fueled by Chevy Chase Bank.

"Despite the declining loan balances, we still believe that our businesses continue to have more than sufficient scale and that our core national lending and local banking businesses are still very valuable franchises," chairman and CEO Richard Fairbank said in an earnings conference call on July 29. "

We remain well positioned to weather the storm, deliver shareholder value over the cycle and achieve our vision of combining great local banking franchises with a sustainably high return credit card business."

Capital One's business will be less exposed to the negative effects of the new legislation "because we never relied on practices like aggressive penalty re-pricing, universal default or double-cycle billing," Fairbank said. "We expect the returns in our card business to diminish modestly from pre-recession levels but to remain very attractive and well above hurdle rates. ...

In the long-term we believe the new law and the marketing competitive environment it will create could open up opportunities and be a net benefit for Capital One."

CLSA analyst Craig Maurer doesn't see the firm doing any more major bank acquisitions in the near future, unless it is at an attractive price and in the mid-Atlantic region. "They still have to integrate the systems from what they already bought," he says.

"The banks are going to find a way to replace the fees that they can't charge anymore," Maurer says. "They're already doing it," by implementing higher interest rates on customers' cards before the new regulation sets in.

Ladenburg Thalmann analysts, who initiated coverage of the company last week with a buy rating, say Capital One is "the best positioned credit card issuer to adjust to changing market conditions."

"Its nimble business model and opportunistic and proactive management approach has proven successful during past market disruptions," the analysts wrote in a note. "We expect the company to similarly take advantage of changing competitive dynamics and outmaneuver its rivals over the intermediate- to long-term."

While other card issuers are likely to consolidate with branch-based banks, Capital One has "already moved toward a broader deposit funding and retail product distribution base."

--

Written by Laurie Kulikowski in New York

.