Retail stocks have slumped this year. 

While many chalk up the declines to competition from online retail behemoth Amazon (AMZN) , famed market expert Peter Schiff, CEO of Euro Pacific Capital, doesn't agree.

"Amazon has been here for a long time - this is the worst year for retail - it's worse than 2008 and 2009," he said. "This is because consumers are broke - they have lousy jobs, they're loaded up with debt and they can't afford to buy stuff."

He pointed to Foot Locker's (FL) 25% decline last Friday. 

Schiff isn't comforted by the double-digit gains in earnings growth for the past two quarters in U.S. stocks.

"Let's see what happens if we go to a recession which is going to impeded [companies'] revenues and let's see what happens if interest rates finally rise despite what the Fed does and that's going to undermine their costs because you have a lot of companies that have levered up their balance sheets."

Schiff said companies have been resorting to debt-financed share buybacks amid low interest rates. "All that's going to come back to bite these companies when interest rates go up," Schiff said. 

Watch the full interview with Peter Schiff:

 

More from Investing

What the Charts Say About the Big Picture: Cramer's 'Off The Charts'

What the Charts Say About the Big Picture: Cramer's 'Off The Charts'

3 Must Know Market Insights From TheStreet's Jim Cramer Before Wednesday's Open

3 Must Know Market Insights From TheStreet's Jim Cramer Before Wednesday's Open

Short Disney Stock Here: Doug Kass Insider

Short Disney Stock Here: Doug Kass Insider

CHART: Even Consumer Staples Stocks Are Being Pummeled in Tuesday's Market Rout

CHART: Even Consumer Staples Stocks Are Being Pummeled in Tuesday's Market Rout

Why Google, Facebook and Other Tech Giants' Capital Spending Is Going Sky High

Why Google, Facebook and Other Tech Giants' Capital Spending Is Going Sky High