The rally in financial markets on Wednesday, led by bank and technology stocks, is a "significant moment," says TheStreet's Jim Cramer. 

The S&P 500 added 0.80%, the Dow Jones Industrial Average popped 0.94%, and the Nasdaq rose 0.83%. 

"We haven't had that kind of broad-based rally in ages," Cramer said, noting that biotech and health care stocks are gaining as well. "This is the possibility of a powerful" surge upward, he said. 

The KBW Bank Index, used to track the performance of the banking sector, jumped 1.79%. 

While Cramer said he's not sure why banks are rallying, he noted that investors may have a positive outlook on the industry, with Bank of America's (BAC) - Get Report   win in a mortgage-fraud case trumping a poor presentation at Wells Fargo's (WFC) - Get Report  annual investor day. 

"This bank move went on nothing," he added.

A jury verdict against Charlotte, N.C.-based Bank of America in a mortgage fraud case linked to Countrywide Financial, a lender it acquired in the buildup to the financial crisis, was overturned by a federal appeals court on Monday. The ruling could shield other banks from some litigation expenses related to the crisis.

At investor day, Wells Fargo's executives told investors that the bank is still struggling with regulatory issues, low interest rates, and loan losses related to the energy sector, which prompted the bank to lower its targeted return on assets to 1.4%, down from the highest range of 1.6% for the year. 

Another factor influencing the rally could be that crude oil has seen a jump after inventories showed a greater-than-expected decrease. West Texas Intermediate crude oil was up 0.76% to $48.99 a barrel. 

Ultimately, Cramer said that the rally could be pushed higher if "Janet Yellen says the right thing" as the market prepares for a potential June interest rate hike. Minutes from the Federal Reserve monetary policy committee's most recent meeting, chaired by Yellen, showed members actively considering an increase next month, a move the market hadn't expected until later in the year.

"These are how real rallies start, on nothing," Cramer said. "And then we find out why."

EXCLUSIVE LOOK INSIDE: Wells Fargo is a holding in Jim Cramer's Action Alerts PLUS charitable trust portfolio. Want to be alerted before he buys or sells WFC? Learn more now.