NEW YORK ( TheStreet) -- Goldman Sachs Monday downgraded its view of the discount brokerage sector to neutral from buy, citing concerns that a continued low interest rate environment will hurt margins and fee income.

As part of the call, the firm cut its recommendation on TD Ameritrade ( AMTD), to neutral from buy and lowered its price target to $21 from $24. TD Ameritrade was down 1.46% to $18.92 just after the open Monday.

Goldman also lowered target prices for E*Trade ( ETFC), OptionsXpress Holdings ( OXPS), The Charles Schwab Companies ( SCHW) and TradeStation Group ( TRAD), but left its ratings on those companies unchanged.

The firm is neutral on OptionsXpress and TradeStation, while it has a sell rating on Schwab and a buy rating on E*Trade, arguing that improving consumer credit trends and the company's potential to be acquired are reasons to be bullish on E*Trade's stock.

Schwab is the company most negatively affected by a low rate environment, while the impact is also considerable for TD Ameritrade, the Goldman report states. In order to keep customers in its money market mutual funds, for example, Schwab has had to waive the fees it ordinarily charges to offset the fact that they yield next to nothing. Those funds account for 60% of the assets Schwab manages, according to the report.

Stock prices of all five discount brokers mentioned in the report were down Monday, while the Vanguard Financials ETF ( VFH) was up 0.1% in early morning trading.

-- Written by Dan Freed in New York.