A big shareholder is taking aim at struggling publisher New York Times Co. (NYT - Get Report).

Morgan Stanley Investment Management said Tuesday it withheld votes for the Times' director nominees because it believes the company's board and management have become unaccountable to shareholders.

The firm, which says it owns more than 5% of the Times' Class A stock, called for the elimination of the dual-stock structure that leaves control of the board with minority shareholders led by the founding Sulzberger family. The Times and a number of other big media outfits have left voting control in the hands of founders, under the rationale that long-term owners look after the long-term interests of the business rather than chasing short-term profit. But Morgan Stanley said the company has failed to keep up its end of the bargain.

"MSIM believes that the dual-class voting at The New York Times Company, which is an exception to the general rule of one-share, one-vote, creates special privileges as well as responsibilities," the firm said in a Tuesday-afternoon statement out of London. "MSIM contends that the Board and management at The New York Times Company have failed to fulfill these responsibilities effectively.  While it may have at one time been designed to protect the editorial independence and the integrity of the news franchise, the dual-class voting structure now fosters a lack of accountability to all of the company's shareholders."

Morgan Stanley added that it believes that "other long-term institutional shareholders have also withheld their votes for the company's Class A director nominees." The Times couldn't immediately be reached for comment.

New York Times Co. stock has dropped 52% since its peak in June 2002, Morgan Stanley says. But "despite significant underperformance, management's total compensation is substantial and has increased considerably over this period," Morgan Stanley says. "As a long-term, committed shareholder since 1996, MSIM has privately conveyed its concerns to the company's Board and senior management on a number of occasions and has suggested substantive strategies to operate the business better and allocate capital more efficiently. However, to date, the Board and management have failed to take the actions necessary to improve operational and financial performance."

The Morgan Stanley move comes as shareholder activists have taken increasing aim at managers and directors of underperforming companies. Carl Icahn took on Time Warner ( TWX) last year in perhaps the most widely reported face-off, though that effort ended in a standstill when Time Warner chief Dick Parsons gave in to Icahn's demands for a bigger stock buyback while ignoring his pleas for a four-way company breakup.

Newspaper companies also have been afflicted by a decline in their core business, facing rising production costs and drops in circulation and advertising revenue. Earlier this spring Knight Ridder gave in to activists who demanded a sale, parceling itself off to McClatchy ( MNI - Get Report), which turned around and said it would try to get rid of 12 of the 32 acquired papers.

Benchmark Capital's Ed Atorino says any shareholder revolt could be "rather fruitless" given that the controlling family controls nine of the 13 board seats and 88% of the class B shares. Benchmark doesn't hold NYT shares.

Atorino notes that Private Capital Managment is a 14% shareholder in the New York Times Co. and that T Rowe Price also has large holdings. PCM's Bruce Sherman was the principal source of shareholder agitation at Knight Ridder, due to his unhappiness with the performance of that newspaper holding company.

On Tuesday, New York Times rose 14 cents to $25.16.