Saddled with debt or facing a large bill such as one due to the IRS, consumers often turn to their retirement portfolios and take out a 401(k) loan or the principal amount from an IRA.

Financial advisors recommend that Americans refrain from taking out these types of loans against their own retirement savings because making up the difference can be a costly mistake and one that they regret later on.

While borrowing from a 401(k) account appears to be a quick solution when consumers are cash-strapped, financial advisors tend to steer their clients away from this option. Although 401(k) loans do not require a credit check unlike obtaining a credit card or personal loan, employers will typically demand that you repay the loan within a short period if you leave your job or get fired.

The majority of 401(k) plans allows employees to borrow up to 50% of the vested account balance or up to $50,000 at rate based on the prime rate, plus 1% to 2%, which amounts to approximately 6%, said Jon Ulin, a managing principal of Ulin & Co. Wealth Management in Boca Raton, Fla. Qualified retirement plans typically require that the loans need to be repaid within a five-year period.

"The interest is paid back to your 401(k) account and not to the plan sponsor," he said. "As you are really paying the loan interest cost back to your own account, it really is not an expense such as taking a cash advance from a credit card or home equity line of credit."

Borrowing from your retirement portfolio is frowned upon because there is a hidden opportunity cost of employees missing out on growth from their monthly investments. It  can "have a direct effect on your future retirement nest egg," Ulin said.

A study conducted by the Employee Benefits Research Institute (EBRI) found that since 2000, nearly 20% of all 401(k) participants still had outstanding loans.

"We have also seen many employees who take out 401(k) loans reduce or completely stop their retirement savings contributions while the loan is outstanding," he said.

Raiding funds from your 401(k) can result in high "opportunity costs" since a moderate to moderate aggressive allocated portfolio could return an average of 6% to 8% per year, Ulin said.

The opportunity cost of not investing $50,000 for one year is about $3,000 to $4,000 in tax-deferred gains in your 401(k), he said.

"Based on the rule of 72, your money could double every ten years with just a 7% return, so you could be costing your retirement nest egg double what you are taking out if you decide to not pay it off and take a taxable distribution, plus an applicable penalties if you are younger than 59.5," Ulin said.

Another cheaper option is to borrow from your IRA, but the money must be returned within 60 days or the proceeds will be taxed as ordinary income, plus another 10% penalty if you are under 59.5 years old, he said. IRA loans are not revolving ones and you can only borrow from your IRA once every 365 days.

A Roth IRA "may be the best place to tap quick cash in case of an emergency since the principal amount can be taken out at any time" without paying taxes or penalties, Ulin said.

The goal is to save up enough to cover expenses for six months or longer.

"The number one lesson for most Americans is to focus on building up an emergency reserve bucket," he said.

Borrowing from any retirement account creates a "permanent setback to your plan," said Greg McBride, chief financial analyst for Bankrate, the NewYork-based financial content company.

Since only one in four adults has adequate emergency savings, working a second job might be a better solution since the labor market is really tight.

"Work some additional hours and bring in extra cash rather than digging a hole deeper by borrowing more money as interest rates are about to go up," he said. "Everybody wants to lean on the easiest option of borrowing which is getting more expensive. At some point you have to start saving money and you won't get there by borrowing more money."

Credit cards are the most common form of debt most consumers use for emergencies, but they tend to be also the most expensive option, said Bruce McClary, spokesman for the National Foundation for Credit Counseling, a Washington, D.C.-based non-profit organization. The current average rate is 15% and interest payments will easily increase the total amount of debt.

The average amount of credit card debt consumers have is $7,800 at a 15% APR. An individual who makes minimum payments of 3% of the total amount of debt means it would take him 198 months or 16.5 years to repay the total, including paying $5,365.20 in interest.

"Many people only consider the amount of the minimum payment, but they should focus on the amount of interest they will pay instead," he said.

Other consumers will opt for short-term or payday loans to fill the gap, McClary said.

"These are typically small dollar loans, but they come with very high interest rates and other fees," he said. "There are differences in the way payday loan interest rates are regulated in each state, so they can range from 36% to as much as 600% or more."

Investors who have a well-diversified stock portfolio of quality stocks, bonds or ETFs could choose to borrow from a broker, said C.J. Brott, founder of Capital Ideas, a registered investment advisor in Dallas.

"You can borrow fairly cheaply from a firm like Interactive Brokers," he said. "Some smaller regional firms like mine will lend at 0.25 basis points over the broker loan rate, which is the interest rate set on bank loans to brokers for margin loans and is also very cheap. The best part about a margin loan is it has no maturity, which means it is very flexible."

More from Roth IRAs

Retirement Daily: Weekly Roundup April 13 2018

Retirement Daily: Weekly Roundup April 13 2018

Contributing to Your IRA? Don't Forget the Deadline

Contributing to Your IRA? Don't Forget the Deadline

Five Types of IRAs You Need to Know About for Retirement

Five Types of IRAs You Need to Know About for Retirement

Buy This Dividend-Paying Titan Amid Market Decline

Buy This Dividend-Paying Titan Amid Market Decline

Why Apple Is Still the Best Stock to Own for Your Retirement

Why Apple Is Still the Best Stock to Own for Your Retirement