Cowen Prices IPO Below Expectations

"It is not a very conducive environment to bring financial services companies to the market," Menlow says. "We are not in a sector rotation where broker stocks are in. The market is dealing with stocks that are lowered-tiered and that, sometimes, are just barely able to get past the expenses of Sarbanes-Oxley."

Meanwhile, the competition in the boutique banking sector is stiffening. Joseph Perella, a former Morgan Stanley(MS Quote) investment banker, recently launched his boutique bank. His former Morgan Stanley colleague Stephen Crawford also is reported to be opening a boutique investment bank. Other small banks, such as Rohatyn, also are competing for business.

Still, boutiques are hopeful their businesses will continue to thrive -- even through a downturn -- as companies opt for smaller advisers over large investment banks, due to conflicts of interest within some of the bigger firms.