Updated from 2:00 p.m. EDT

Goldman Sachs ( GS) investment banker Byron Trott is leaving the company to start his own merchant banking firm with backing from Warren Buffett, according to a source familiar with situation and a published report.

Goldman CEO Lloyd Blankfein recently made a visit to Chicago for a town hall meeting of the bank's employees there, and ended up having to deal with Trott's departure during his visit, according to one banker at a rival firm. Trott works out of the firm's Chicago office.

Trott is raising as much as $2 billion for a fund that will invest in family-controlled and entrepreneurial companies, The Wall Street Journal reported Monday morning.

Buffett told the Journal that Berkshire Hathaway ( BRK-A) will invest in Trott's firm, to be called BDT Capital Partners. Buffett has praised the banker on two separate occasions in his annual letter to shareholders, writing that Trott is "a rare investment banker who puts himself in his client's shoes," adding, "I trust him completely."

Trott was the banker that advised Buffett on his $5 billion investment in Goldman last year and he was also behind a $3 billion Buffett infusion into General Electric ( GE), according to report in The New York Times last year.

Before he became an investment banker, Trott worked in the Goldman "private client" unit that advises wealthy families. That's how he got to know both the Mars and Wrigley families, heirs of the candy companies that came together in a $23 billion deal last year. Trott advised Wrigley on the deal, and Goldman Sachs and Berkshire Hathaway each contributed several million to help Mars make the purchase.

Trott did not return a call to his office and a call to a Berkshire Hathaway spokeswoman was not returned.

A Goldman spokeswoman declined to say whether the firm would invest in Trott's fund.

Many high-level executives have left large U.S. banks in recent months , including Goldman President and co-COO Jon Winkleried. A threatened 90% tax on bonus payments to institutions that have taken government funds, as Goldman has, and a general increase in government scrutiny of Wall Street firms has driven some of the moves. The bonus tax bill approved by the House earlier this month now looks unlikely to make it through Congress in its present form.

Some bank executives, such as former Citigroup ( C)M&A chief Frank Yeary and former Merrill Lynch president Greg Fleming, have gone into academia. Others have gone to work for small investment banks that have not taken government money, such as Lazard ( LAZ) or Evercore ( EVR), or foreign-owned institutions like Deutsche Bank ( DB).

Some other executives may go to hedge funds or private equity firms, though those firms are having troubles of their own. Some traders from big banks have gone to small trading firms like Broadpoint Securities or BTIG, in which Goldman has a stake. Bloomberg News recently reported that there are more than 50 such firms in the fixed income market, that look to make money on the spread between buyers and sellers, which has widened dramatically during the credit crisis.