Hillary Clinton's rhetoric suggests Wall Street should be worried about its future if she is elected, but investors shouldn't necessarily take Clinton's speeches at face value.

TheStreet held a panel featuring top financial experts who discussed what the 2016 presidential election means for the retail investors. Stephen Horan, a managing director with the CFA Institute, one of the panelists, isn't convinced Clinton is so bad for Wall Street.

"If you look at the headlines, you see a lot of podium pounding and Sen. Elizabeth Warren's [D-Mass.] is the greatest thing since sliced bread and down with Wall Street," said Horan. "But three of Hillary's top five donors are major Wall Street banks: Citigroup (C) - Get Report , Morgan Stanley (MS) - Get Report and JPMorgan Chase (JPM) - Get Report . "If you talk with the heads of those institutions, what you'll probably hear is that they, despite the political rhetoric, see Clinton as a pragmatist."

In fact, Clinton was criticized by Democratic presidential rival Sen. Bernie Sanders, I-Vt., as not being tough enough on Wall Street. In an April debate on CNN, Sanders dismissed Clinton's claims that she has a tough stance on Wall Street and raised questions about the $675,000 Clinton reportedly received in speaking fees from Goldman Sachs (GS) - Get Report.

"Clinton's ties with Wall Street are actually much stronger than [Donald] Trump's," Horan said. "Trump has very few ties with Wall Street."

Robert Johnson, author of Invest With the Fed and another panelist at TheStreet's panel, found it ironic that Wall Street was backing the career politician instead of a businessman who had no career in politics.

"Obviously the biggest reason is the unpredictable nature of Trump and the uncertainty," Johnson said. "I hate to use trite investment sayings but there is one old Wall Street adage that holds true and that is the market dislikes uncertainty."

Johnson points to Trump's affinity towards unpredictability. For example, when talking about foreign policy Trump said the following to Fox News host Bill O'Reilly: "I want to be unpredictable." 

"Markets don't like unpredictable folks," Johnson said.

On the flip side, S&P analyst Scott Kessler, pointed to Trump's endorsement of a repeal of the 2010 Wall Street regulation bill Dodd-Frank. "If you talk to a lot of bankers out there, that is the number one concern they have," Kessler said. "That Dodd-Frank is going to grow and more of it is going to become implemented."