Investors may have missed one major red flag that could have tipped them off that something was amiss at Bernard Madoff's now-infamous hedge fund: He acted as his own prime broker.

Madoff was charged last week with running a Ponzi scheme he told investigators resulted in $50 billion in losses, adding up to what is believed to be the largest securities fraud ever perpetrated. Nearly all hedge funds use prime brokers, which provide critical back office functions, such as settling trades. Run by large banks, the prime brokers also frequently lend to the hedge funds and, if they start posting heavy losses, the prime broker may require them to post additional collateral.

"There could not be a bigger red flag," says Gary Shugrue head of Ascendant Capital Partners, a fund of hedge funds based in Radnor, Penn. "The prime broker really acts as the custodian would at a traditional asset manager. It's where the money is."

Even hedge funds a fraction of the size -- Madoff oversaw $17 billion, but has said to investigators he lost $50 billion -- use one or more prime brokers at large banks or securities firms such as Goldman Sachs ( GS - Get Report) or Morgan Stanley ( GS - Get Report).

For many years, Goldman and Morgan Stanley dominated the prime brokerage business, but the struggles of those firms have created an opportunity for rivals such as JPMorgan Chase ( JPM - Get Report), which beefed up its prime brokerage business considerably earlier this year with its acquisition of Bear Stearns. Citigroup ( C - Get Report) has also made prime brokerage a key part of its strategy, under Morgan Stanley alumni such as CEO Vikram Pandit and institutional business head John Havens.

European banks such as Deutsche Bank ( DB - Get Report) have also been making a big prime brokerage push. Ironically, some of those chasing this business were among Madoff's victims, including UBS ( UBS - Get Report) and BNP Paribas, which recently acquired Bank of America's ( BAC - Get Report) prime brokerage operations.