Comerica (CMA) - Get Report  , the Dallas-based bank facing shareholder ire as it grapples with loans to the beleaguered energy industry, now must contend with a new threat: Brexit.

The U.K.'s vote to leave the European Union has convinced many traders that the U.S. Federal Reserve is unlikely to raise interest rates this year, with some futures prices even reflecting the possibility of a cut, Goldman Sachs analysts wrote Monday in a report. That means banks won't get the boost to their lending margins that many analysts were expecting earlier this year, when Fed Chair Janet Yellen was signaling that rate hikes were in the works.

If no rate increases happen this year or next, Comerica would be the worst-hit among U.S. banks, with earnings per share falling by 4% versus 2016 estimates and 12% versus 2017, according to Goldman. The average reduction for regional banks would be 3% for 2016 and 7% for 2017.

"In the event there was a rate cut in the U.S., there would be incremental earnings downside from lower reinvestment yields," the Goldman analysts led by Richard Ramsden wrote.

The Brexit fallout adds to a growing list of concerns for Comerica CEO Ralph Babb, who already faces a chorus of complaints over the bank's performance. In April, some of its biggest shareholders, including Fiduciary Management and Invesco, traveled to an annual meeting to voice their displeasure. Mike Mayo, an analyst at brokerage CLSA, has mounted a public campaign to lobby for management changes, a strategy overhaul or a sale of the entire franchise.

Babb hired Boston Consulting Group to help conduct a companywide review, and he said earlier this month that cost cuts were likely, while a sale of the bank was probably off the table -- a decision that rankled some shareholders who were hoping for a quick exit.

More than half of Comerica's $3.1 billion of energy loans are "criticized," meaning they're considered weak or unduly risky. Meanwhile, the executive team has suffered a string of departures and retirements in recent months, including its chief financial officer, business-banking head, chief technology officer and chief diversity officer.

Comerica was especially dependent on rising interest rates to buoy profits, since more than half of its $49 billion of loans carry variable rates.

The 10-year Treasury yield, often used as a barometer of rate trends on long-term loans, tumbled to 1.57% on June 24 following the Brexit vote, from 1.74% the prior day and 1.85% at the start of June.

"While rising rates will provide a significant benefit, we are neither waiting nor relying on that to drive improved returns," Comerica spokesman Wayne Mielke said in an e-mailed statement. "We are identifying meaningful opportunities to enhance revenue, operate more efficiently and lower expenses, with the goal of building a more profitable organization that is better able to drive enhanced long-term value for shareholders."

Whereas some big home lenders like Wells Fargo might get a boost from mortgage refinancing as rates drop, Comerica is primarily a business lender and probably wouldn't benefit as much, according to Goldman. 

Comerica's shares have tumbled 17% since the Brexit vote, compared with an 11% drop for U.S. banks on average.

EXCLUSIVE LOOK INSIDE: Wells Fargo is a holding in Jim Cramer's Action Alerts PLUS charitable trust portfolio. Want to be alerted before he buys or sells the stock? Learn more now.