NEW YORK ( TheStreet) -- After stronger-than-expected earnings reports from U.S. banking giants JPMorgan Chase ( JPM) and Citigroup ( C), it's tempting for investors to question whether the worst is over for the banking sector after recent headwinds like Federal Reserve stress tests, ratings downgrades and a Eurozone slowdown. While the prospect of a banking and housing recovery augur well for the likes of JPMorgan, Citigroup and Bank of America ( BAC), financial sector investors may still be underestimating risks to earnings - as they have in the past. A rate fixing scandal that started with Barclays ( BCS) may extend to some of the largest banks in the U.S. and Europe, and stands out as a new and potentially titanic risk. However, amid those concerns and uncertainty over sector-wide earnings, pure-play investment banks like Goldman Sachs ( GS) and traditional lenders like Wells Fargo ( WFC) -Warren Buffett's largest bank sector stock investment - may avoid the next shoe to drop in the banking industry. Since Barclays settled a rate fixing probe with U.S. and U.K. regulators on June 27, the prospect of regulatory fines and civil litigation hangs over banking giants like Bank of America, Citigroup, Deutsche Bank ( DB), UBS ( UBS) and Credit Suisse ( CS), who also are involved in the setting of key short-term interest rates, otherwise known as Libor. Diversified european consumer and investment banking conglomerates like Barclays, Deutsche Bank and Credit Suisse are most exposed to revelations of manipulation in the rate setting process, according to a Tuesday analysis by bank research firm KBW. In the U.S., similarly structured banks like JPMorgan, Citigroup and Bank of America face the same risks. Among major banking conglomerates, KBW estimates that losses could range from $3 billion to roughly $6 billion, eroding capital and future earnings. The probe is significant because of the size of the market for Libor, which is used to set over $350 trillion in financial contracts, and for the reputational damages that the regulatory inquest may wreak on the industry. Already, the process of setting the rates has been criticized as being too opaque and giving far too much power to the world's largest banks. Meanwhile, a manipulation of the rates may have unfairly impacted the borrowing costs of homeowners, municipalities and corporations, as Barclays and others allegedly tried to manipulate rate fixings during the financial crisis. Nevertheless, it is investment banks like Goldman Sachs and Morgan Stanley ( MS) and consumer-focused lenders like Wells Fargo and US Bancorp ( USB) that don't face Libor related losses since they aren't among the institutions that set the rate, signaling that an escalating Libor probe could raise winners and losers among the most watched bank stocks. For financial sector investors, who've weathered a wrenching six months of industry-wide pessimism entering 2012, excitement after first quarter earnings and a recently muddied outlook for the remainder of the year, the probe may mark yet another unanticipated earnings drain. In fact, on Tuesday, KBW analysts led by Mark Phin estimate that banks involved in the process of setting Libor face an estimated $35 billion in total regulatory and civil fines.