Investors may have missed one major red flag that could have tipped them off that something was amiss at Bernard Madoff's now-infamous hedge fund: He acted as his own prime broker.

Madoff was charged last week with running a Ponzi scheme he told investigators resulted in $50 billion in losses, adding up to what is believed to be the largest securities fraud ever perpetrated. Nearly all hedge funds use prime brokers, which provide critical back office functions, such as settling trades. Run by large banks, the prime brokers also frequently lend to the hedge funds and, if they start posting heavy losses, the prime broker may require them to post additional collateral.

"There could not be a bigger red flag," says Gary Shugrue head of Ascendant Capital Partners, a fund of hedge funds based in Radnor, Penn. "The prime broker really acts as the custodian would at a traditional asset manager. It's where the money is."

Even hedge funds a fraction of the size -- Madoff oversaw $17 billion, but has said to investigators he lost $50 billion -- use one or more prime brokers at large banks or securities firms such as

Goldman Sachs

(GS) - Get Report


Morgan Stanley

(GS) - Get Report


For many years, Goldman and Morgan Stanley dominated the prime brokerage business, but the struggles of those firms have created an opportunity for rivals such as

JPMorgan Chase

(JPM) - Get Report

, which beefed up its prime brokerage business considerably earlier this year with its acquisition of

Bear Stearns



(C) - Get Report

has also made prime brokerage a key part of its strategy, under Morgan Stanley alumni such as CEO Vikram Pandit and institutional business head John Havens.

European banks such as

Deutsche Bank

(DB) - Get Report

have also been making a big prime brokerage push. Ironically, some of those chasing this business were among Madoff's victims, including


(UBS) - Get Report


BNP Paribas

, which recently acquired

Bank of America's

(BAC) - Get Report

prime brokerage operations.