NEW YORK (TheStreet) -- Consumers are pointing a finger at retailers for a rash of security breaches that left shoppers' credit and debit cards vulnerable to identity theft, with 80% of people in a survey saying that having their credit card data compromised is worse than "getting the flu."

According to the nationwide survey of more than 2,000 shoppers by Feedzai, a San Mateo, Calif., data science company, 60% of adults say merchants are "responsible" for data breaches -- including big-name brand retailers such as Target and Neiman Marcus, which each experienced widespread data breaches recently.

"Fraud prevention is now a matter of predicting complex consumer behavior based on changing sentiments," says Pedro Bizarro, chief data scientist of Feedzai. "These findings show that consumers believe it is the merchant's responsibility, but really it is a collective problem that the industry needs to understand in order to distinguish customers from criminals and keep payment data safe."

Also see: These Are the Worst Passwords on the Internet>>

Retailers may have a case to make saying data breaches are a shared concern for shopper and retailer alike.

"It's convenient to blame merchants who are also easy targets of today's professional criminals, who are well-organized and use sophisticated technology," says Loc Nguyen, Feedzai's CMO. "However, keeping modern commerce safe is a collective responsibility for banks, merchants, government and consumers. The good guys also need to collaborate and fight fire with fire."

Consumers don't see it that way.

Besides the 60% of Americans who say security breaches are the primary responsibility of merchants, online and off, another 13% blame banks. The government is also a popular target of consumers' ire; only 5% of adults say it's up to the individual to prevent data breaches.

Also see: Your Card Data Is Safer at Banks Than Retailers>>

In terms of a major inconvenience, having their credit or debit card stolen or compromised is the "most aggravating" of "unpleasant activities" according to Feedzai. Losing a phone, getting a flu and dealing with the registry of motor vehicles fall right behind an identity theft incident.

But only one in five Americans say they will change their shopping habits (switching back to cash, for instance, or shopping more at mom and pop stores).

There's plenty of evidence to suggest that retailers, banks and consumers can all work harder to prevent data breaches and identity theft.

But for consumers, it's going to be hard to fight data thieves with one finger perpetually pointed at their banks and merchants.

No matter which finger it is.