Brexit is more of a worry for the UK and Europe than the U.S., according to one expert.

"Our view is that it's a domestic issue," said David Lebovitz, global market strategist at JPMorgan Asset Management, based in New York. "This really matters for the UK and Europe. Both economies are going to slow down as a result."

He said the U.S. economy should be fine because we export a very small percentage of our GDP to the UK. "There's an element of contagion, but from an economic standpoint, we don't think there's going to be too much of an effect," Lebovitz added.

Stocks rose Wednesday for the second day in a row following steep declines on Monday and last Friday, as investors digested the UK's decision to leave the European Union. "Now, the market is realizing that we don't really know how this is going to play out," he said. "The market is taking a step back to an extent in order to wait for more details before going into complete selloff mode."

Meanwhile, the pound rose 0.8% against the dollar to almost $1.35 on Wednesday. Lebovitz expects stocks to follow the pound, similar to the way stocks moved in lockstep with oil towards the beginning of this year, although he still thinks the dollar has more sway in the markets.

"Too much dollar strength is going to keep the Fed on the sidelines and determines what happens in emerging markets, so we're much more concerned with the dollar relative to all of the other currencies, rather than thinking the pound is going to drive the bus," the analyst said.

Lebovitz thinks stocks can head mildly higher from here, with the S&P 500 wrapping up 2016 slightly above 2,100, compared to its current level of 2,036. That implies roughly 3.4% upside.