If Britain chooses to exit the European Union in June the shares of big banks will be affected, but not for long, analysts say.

Several large banks, including Goldman Sachs (GS) - Get Goldman Sachs Group, Inc. (GS) Report , Citigroup (C) - Get Citigroup Inc. Report , JPMorgan (JPM) - Get JPMorgan Chase & Co. (JPM) Report , and Wells Fargo (WFC) - Get Wells Fargo & Company Report use London as a hub for their EU operations.

That may change, though, if Britain votes to exit the EU early this summer.

"Essentially what we're talking about is a real estate situation," said Dick Bove, analyst at Rafferty Capital Markets.

According to analysts, big banks will take a substantial hit when it comes to relocation and staffing. The hit, though, as Bove said, will be a one-time thing.

"It's not as if they're going to lose customers," said Marty Mosby, director of bank and equity strategy at broker-dealer Vining & Sparks. "It's more of a burden they'll have to go through. Depending on how it's evolving, I think it's something that will have an unfavorable impact, you're introducing uncertainty and risk."

Mosby said that Goldman Sachs is already trading at a discount, not only because of poor first quarter earnings, but also because a potential Brexit is looming.

"If you saw a vote to say that Britain will stay in, you would see a lift in share price, and if you see a vote to leave, then you'll see more pressure," Mosby said. "We'll have to see that unfold as it gets closer, if it seems like. For the most part right now, there's some of that already priced in."

It's not just Goldman that is feeling the squeeze of a potential Brexit, though.

TheStreet Recommends

"You would still see JPMorgan, Citigroup being impacted," Mosby said. "Everybody who is doing business will be impacted."

Bove advised shareholders to hold on to their pieces of big banks, for now at least.

"People like to freak out over everything, so I'm sure they'll freak out over the fact that Britain left," Bove said. "Maybe the share price is affected, but I would think it would be a mistake to sell them based on a real estate situation."

Even if Britain exits the EU, shareholders may not have to worry at all, Mosby said.

"Even if Britain leaves the EU, doesn't mean that the EU says they're not going to allow Britain to do business with [them]," Mosby said. "In 2019, they are allowing the financial institutions to do business not in the EU, as long as their regulatory standards are as stringent."

Goldman Sachs was trading at $153.33 on Tuesday morning, down 29% from the company's year high of $218.77 it hit on June 23.

Citigroup was trading at $41.65 Tuesday morning, down 31% from its year high of $60.95, which it hit on July 22, 2015.

JPMorgan was trading at $58.67 Tuesday morning, down 17% from its year high of $70.61, which it hit on July 22, 2015 as well.

Wells Fargo was trading at $47.26 Tuesday morning, down 19% from its year high of $58.77, which it also reached on July 22, 2015.

Wells Fargo is a holding in Jim Cramer's Action Alerts PLUS Charitable Trust Portfolio. Want to be alerted before Cramer buys or sells WFC? Learn more now.