I’m done with Starbucks, at least for now. That $4.99 triple-venti-cappuccino-with-skim-wet that I had the other day will have to be my last one. That’s because the ultimate arbiter of taste, my friend Danny Meyer, who runs the best restaurant chain in New York, told me that it’s not the right moment to flaunt expensive coffee.

If you don’t know Danny, who runs Union Square Café, Tabla, Eleven Madison Park, Gramercy Tavern and Blue Smoke, then you don’t know the current national zeitgeist. Last week Danny, who wrote the bible of the hospitality industry, Setting the Table, came to Mad Money to talk about brand loyalty and customer satisfaction.

He was marveling about how great Trader Joe's is to shop at and how much fun it is to go to a Costco (Stock Quote: COST) or a Chipotle (Stock Quote: CMG). Danny’s got a great eye for stores and for product so I asked him about Starbucks (Stock Quote: SBUX). I wanted to know if there could be a turn in the fortunes of the company.

No, he said. Not any time soon.

This is a moment in time where people want to play it cool. They don’t want to be seen as big spenders. The $4.99 Starbucks latte has become our everyday corporate jet! Danny says people “feel bad” about spending that kind of money and they feel that their coworkers are eying them as seekers of the extravagant, especially given that Dunkin Donuts and McDonald’s (Stock Quote: MCD) serves a good cup of java for half the price.

Beyond that, the service at my Starbucks now leaves a lot to be desired. I got served the wrong cup of coffee the other day, without an apology. I waited for what seemed like forever to get a cup, at least much longer than I used to. I believe this may be because the other Starbucks two blocks away recently shuttered, making mine considerably more crowded. If business stays brisk, they may continue to have quality problems. I also noticed that right outside the store’s window was a man in a street cart serving coffee for half the price. He was able to handle twice as many customers in about a quarter of the time.

Now, maybe Starbucks will get its act together. Maybe it will improve hospitality and service some time soon. But with times this tough, maybe Danny’s right. You don’t want to show people you are made of money when they might be hurting, and a $4.99 drink at Starbucks now gives off that signal. Starbucks, I’m told, is conscious of the problem and is trying to cut the price. Until they do, however, they’re too expensive for someone who wants to be really low key about wealth during this incredibly difficult period.

So I’ll be drinking an extra-large Dunkin’ coffee. It’s at least a buck cheaper than Starbucks.

—For the best rates on CDs, mortgages, savings, credit cards and more, enter your ZIP code at BankingMyWay.com.