Companies are rushing to borrow money now, in order to lock in a lower interest rate before the Federal Reserve makes its move. Jack Kopnisky, CEO of Sterling National Bank (STL) - Get Report , said bankers should not abandon banking fundamentals simply because of a surge in the demand for credit.

"There are lots of people wanting to provide credit," said Kopnisky. "One of the challenges is making sure you stick to the fundamentals of credit, which is the biggest risk in banking. We've been very deliberate in sticking to what we know that works over the long term."

Shares of the New York-based middle market lender have risen over 14% year-to-date. The company earned 25 cents in its third quarter earnings report in late October, beating the Wall Street consensus estimate by a penny. Net income for the quarter was $24.2 million, while revenue was $112.2 million in the period.

This was the first full quarter since the company's merger with Hudson Valley Bank. The company has been aggressive in its deal-making. Earlier this year, it acquired the factoring portfolio from First Capital, in addition to payroll services provider Damian Services.

"We are very systematic in terms of our process and tactics when it comes to mergers and acquisitions," said Kopnisky. "We are actually ahead of plan in terms of getting cost reductions and revenue gains."

Kopnisky said financial regulations are indeed onerous and holding back some of the smaller banks. At around $12 billion in assets, Sterling Bank has the wherewithal to pick off some of the smaller players who cannot afford the staff to accommodate the Dodd-Frank Act's extensive set of rules.

"Our view is that you have to get to about $10 billion in assets to get the economies of scale on the earnings side to pay the costs of regulation, as well as the costs of technology upgrades," said Kopnisky.