Pessimism is building on Wall Street about the fate of the retail sector as early signs of a consumer spending slowdown continue to emerge. But when it comes to retailers like the so-called dollar stores that serve the lowest-income shopper, pessimism is nothing new.

Before

Federal Reserve

Chairman Ben Bernanke spooked the market in early May with hawkish words about inflation and interest rates, the S&P Retail Index had logged a respectable gain of 5.3% since the start of 2005. But shares of

Wal-Mart

(WMT) - Get Report

, the world's largest retailer and Wall Street's favorite bellwether for the bargain-focused consumer, was down 5.7% over the same span.

Meanwhile, Wal-Mart's smaller discount counterparts -- the dollar stores -- were hurting even worse. The largest,

Dollar General

(DG) - Get Report

, was down more than 18%, while

Family Dollar

(FDO)

and

Dollar Tree

(DLTR) - Get Report

had lost 19% and 3%, respectively.

Since the broader stock market started cracking in May, the losses have only accelerated, with the exception of Dollar Tree, which is up 6.9% over the last three months as its sales trends have shown signs of improvement. Now, with spending fatigue spreading to other echelons in the retail sector as the economy slows, will the low end make a resurgence as more consumers head for the bargain bin?

So far, such a resurgence remains elusive as the companies' bottom lines have been hurt by a sales shift toward lower-margin products, along with increased store investments.

Late Friday, Dollar General lowered its second-quarter earnings estimate to a range of 14 cents to 15 cents a share from its previous forecast of 18 to 22 cents a share. The company blamed its weaker outlook on a disappointing back-to-school shopping season and lower gross margins as its customers spent more heavily on consumable products, like food, as opposed to higher-ticket items such as clothing and home décor.

Investors responded by abandoning the stock, sending it down more than 9% on Monday. Some observers, however, saw the selling as an overreaction.

"The comps at the dollar stores have actually started to pick up lately, even as margins suffer a bit due to changes in their product mix," says Morningstar analyst Anthony Chukumba, referring to same-store sales, a key retail metric gauging sales at stores open for at least a year. "We may be starting to see a 'trade-down effect,' as people who might have been shopping at Macy's start to head to dollar stores and other discounters in order to tighten their belts a bit."

In its first quarter, Dollar General's comps growth slowed to an increase of 1.6% from the 2% gain logged for 2005. But over the last three months, it has seen a sequential increase, from 1.3% growth in May, to 2.5% in June, culminating in a 4.6% jump for July.

At Dollar Tree, there's a similar trend of gross-margin deterioration, even as its comps have increased 4.1% so far this year. Family Dollar has logged comp increases surpassing the 3% mark for each of the three quarters of its fiscal year, which ends this month.

On Wednesday, Dollar Tree posted second-quarter earnings of $29 million, or 28 cents a share, beating Wall Street's EPS estimate by a penny. Shares were lower, however, after the company forecast third-quarter earnings at or below analysts' expectations. The company's second-quarter gross margin fell to 33.2% from 34%, a decline that is expected to continue for the rest of the year.

Dollar Tree said rising fuel prices drove up transportation costs, and like Dollar General, it sold more lower-margin goods in the quarter.

All the dollar stores have been tinkering with their product lines, expanding their offerings to include more consumable items and even some higher-priced goods in an effort to win customers from discount giants like Wal-Mart,

Target

(TGT) - Get Report

and

Costco

(COST) - Get Report

. They've been spending more money on investments like adding coolers in their stores to sell perishables and alarm systems to prevent shoplifting.

The dollar-store concept saw explosive growth earlier in this decade, as investors noticed a market for a more convenient, smaller-format venue for the kind of deep-discount retailing pioneered by Wal-Mart. But that growth has since slowed as early signs of market saturation have emerged.

"There are probably enough dollar stores out there now," says HSBC analyst Mark Husson. "The returns on invested capital for these companies are going down as the number of stores in the country is going up. The barriers to entry in this industry are pitifully small, and while the historical returns have been high, that has attracted a lot of competition from all kinds of players, including private dollar-store operators, which has exacerbated the saturation issue."

Last year, Dollar Tree, which currently operates roughly 3,200 stores, recorded square footage growth of only 12.7%, down from the 21% it reported for 2004. Family Dollar, with about 6,100 stores, had to lower its new-store target for 2006 to 400 from 500.

Chukumba noted in a recent report that Dollar General, with over 8,000 stores, still has room to expand because its store base is mostly concentrated in the South. He expects the company's top-line growth to average 9% for the next five years, down from its 14% average for the last five years.

Despite the slowdown, Dollar Tree CEO Bob Sasser told analysts on a conference call Wednesday that the chain gained market share, increased its store traffic and raised the value of its average sale ticket in the second quarter. While he said gross margins will continue to decline this year, the stage is set for strong results in 2007 when consumers may be even more likely to be searching for low prices.

Husson says that even if dollar stores can find new ways to squeeze sales growth out of changes in their stores and product mix, the industry is headed for slower growth and lower returns, and that doesn't bode well for the stocks.

"They're becoming more complicated businesses, and the competition is still flooding in," Husson says. "So yes, the underlying consumer is under some pressure, but on the other hand, the dollar stores have been their own worst enemy in the race for space."