NEW YORK (MainStreet) — Investing with your company 401(k) plan can be a mistake and lead to lower returns because the fees are too excessive or the investment options are few and far in between and generate low yields.

Other company retirement plans simply have too many fees, which erode the principal amount, especially if the employer does not match contributions.

In the majority of plans, the employer absorbs the administration and other fees so the employee doesn't have to pay them, said Cal Brown, market manager for Savant Wealth Management in McLean, Virg.

What is common is the employee pays the expense ratio for the mutual fund. Any expense ratios over 1% fee against your returns are high, he said.

"Generally, if your mutual fund is an American large company stock fund, the expense ratio should be low," he said. "If it's a mutual fund investing in emerging markets, the expense ratio is high."

Employees wind up paying higher fees in new plans or ones that are smaller, said Bill Keen, CEO of Keen Wealth Advisors in Overland Park, Kan.

"New start-up 401(k) plans may not qualify for the lower expense structure that larger plans can enjoy," he said. "In some cases, these plans have investments with sales loads and high internal expenses."

Employees should examine their mutual funds closely, because technically inferior fund options which are expensive or poor performers will affect your returns by 1% to 5% annually. Over a long period of time, compounding that poor return can cost you a lot of money, Brown said.

Some 401(k) plans match 100% of the employee contributions up to 3% to 5% of their salary. For an employee who makes $100,000 and receives a 3% match, if the employee contributes $3,000, then the employer will also match with a contribution of $3,000, making it a $6,000 contributed total. Other employers may only match 25% or 50% up to 3% to 5% of their salary.

However, if your company is generous and matches at 25% or 50%, then take advantage of what is essentially free money. In other words, if your fund is underperforming by 5% per year, but you are gaining 25% from the matching contribution, "the obvious answer is to take the 25%," he said.

Even if your employer matches only 50% of the first 6%, you should stick with the company 401(k) plan even if it is less than stellar, said Regi Armstrong president and senior client wealth manager of the Armstrong Wealth Management Group, an advisory firm with $195 million in assets under management in Florence, S.C.

If your company doesn't match 401(k) contributions or has mediocre investment options, you may be better off with an IRA, said Kalen Holliday, spokesperson for Covestor, an online investment management company with offices in Boston and London. Covestor has over 100 portfolios available to IRA accounts that have low or no management fees.

"One of the things people should keep in mind is that they contribute to their company 401(k) and an IRA," she said.

For people who change jobs frequently, IRAs can be a good option as they provide a place to consolidate your retirement savings and avoid having to roll over your 401ks frequently, Holliday said. A recent Fidelity Investments study showed that average IRA contributions hit an all-time high in 2013 of $4,150, a 4.7% increase since 2012.

TST Recommends

"What we've witnessed recently at Covestor is people are rolling over their 401(k)s and consolidating IRAs into our no-management fee core portfolios," she said. "These low-cost passive investment options appeal to people saving for retirement."

If you have an expensive plan or one with few investment options, it may be better to allocate your funds to a non-retirement account. One option is to contribute the maximum amount to you and your spouse's Roth IRA(s). For those under 50, this means a couple can invest a combined $11,000 in 2014 in Roth IRAs.

"The younger you are, the more valuable this is likely to be," said Armstrong. "If it is properly invested, the funds can grow tax-free potentially, which could be very valuable in retirement."

The benefit of contributing to a Roth is that while there are no deductions when putting money in, the money is completely tax-free when it is withdrawn, said Elle Kaplan, CEO of Lexion Capital Management, a 100% woman-owned asset management firm in New York.

"This is especially ideal for a young person when their level of earning will likely make a deduction fairly insignificant, but is a sound and profitable choice for anyone who is eligible," she said.

Bypassing your company 401(k) plan can be one of the best decisions for employees to make, said Pamela Yellen, a financial security expert and author in Sante Fe.

In spite of new fee-disclosure rules that went into effect in 2012, they don't provide a simple figure for the annual cost you're paying. Only 7% of employees have made changes to their investments as a result of receiving information about fees, she said.

"Even a 1% fee will eat away 28% of your savings over time, according to the Department of Labor and most 401(k) plans charge fees above 1%," Yellen said.

You might be sitting on a tax time bomb. While people love the idea of deferring taxes, they need to ask themselves what direction they think tax rates will go over the long term she said. If you think they're going up, and you're successful in growing your nest-egg, you'll only end up paying higher taxes on a bigger number.

"Putting your money in a 401(k) is like putting it in prison," Yellen said. "There are strict rules governing how much you can access, when you can do it and when you must do it in order to avoid penalties. Those rules can and do change at any time."

Another concern is that you will not know the value of your 401(k) on the day you plan to tap into it, she said.

"You'll always worry the market might crash right before you planned to start taking distributions," Yellen said.

If you are just starting out in your career and are in a low or no income tax situation, investing in your company retirement plan is not likely to be beneficial, said Keen.

"This can make tying up money in a qualified account not as attractive as if the client were in a higher tax bracket and able to utilize the tax deductibility of the contribution," he said.

--Written by Ellen Chang for MainStreet