NEW YORK (MainStreet) — How much should you pay in investment fees, given an average $10,000 investment?

The answer varies — and somewhat surprisingly, based on the state you call home.

According to investment fee services firm FeeX, if you live in West Virginia or Vermont you're paying way more in fees than, say, if you live in New Mexico or Wyoming. Here is FeeX's list of "top fee" and "bottom fee" states when it comes to investment fees.

Top five states paying the most in investment account fees annually:

  • West Virginia: $106
  • Vermont: $93
  • South Carolina: $80
  • New Hampshire: $78
  • Florida: $77

Top five states paying the least in investment account fees annually:

  • New Mexico: $19
  • Wyoming: $23
  • Hawaii: $31
  • Oklahoma: $36
  • Indiana: $41

(See the guide — and see where your state ranks on average investment fees here.)

WATCH: More personal finance videos on MainStreet | More videos from Scott Gamm

View Today’s Auto Loan Refi Rates

Why the disparity in statewide investment fees? It's a combination of state regulations and what the market will bear.

Paying more for investment fees isn't just a nagging problem. If you do it long enough, it can really erode your portfolio performance.

"We were shocked to see that investors in some states are paying so much more than investors in others," FeeX CEO Yoav Zurel says. "We already know that 75% of investors in all states pay more in fees than they need to, and, because these fees compound year after year, they can erase a third of the money you'd otherwise have for retirement."

Zurel notes that spending even a little time vetting your portfolios investment fees is worth the effort.

"Every investor should spend five minutes finding out how much of their hard-earned money is lost to financial fees, because saving now could help with future financial milestones like purchasing a house or funding a child's education." He also advocates reviewing your investment goals to make sure those fees are being used for the right purposes — you don't want to be paying investment charges on portfolio management services that aren't cutting it, Zurel says.

There's not much you can do, however, if you live in a state such as West Virginia, where fees are high. If you're in an investment plan you like, and you live in a high-fee state, you're going to pay the piper.

— By Brian O'Connell for MainStreet