If a stranger or someone close to you is exploiting your age or health for access to your money, you'd better believe that's a personal finance issue.

ReKeithen Miller as certified financial planner with Palisades Hudson Financial Group in Atlanta notes that financial elder abuse isn't always outright fraud. In most instances, it is a close friend or family member taking advantage of victims' diminished capabilities to pressure or influence them to act against their best interests.

For example, some people who are not capable of handling major transactions may still be very capable of handling routine financial matters like paying bills. Other who are highly functional when medicated but can't make sound decisions without their prescriptions. Those with vision or hearing loss may be able to make transactions just fine in person, but struggle to do so over the phone.

"Those who are most vulnerable to financial exploitation usually retain some degree of financial independence but rely on someone else for their other needs," Miller says. "This opens the door for a third party to exploit the relationship."

The Justice Department says the red flags for this kind of abuse are everywhere. If there are sudden changes in bank accounts or practices -- including unexplained withdrawals of large sums of money -- when the customer is accompanied by an unfamiliar person, that's a sign. If new names pop up on that older person's bank signature card or unauthorized withdrawals are made using that person's ATM card. If a will or other estate-planning documents change abruptly, if funds or valuables go missing, if the standards of care or housing slip, if forged signatures start cropping up in financial transactions or if relatives suddenly appear and claim (or outright take) property, possessions or money, those are all problematic. Also, if an older friend or relative says they're being exploited, don't shrug it off.

"Abuse isn't always the result of a carefully plotted scheme," Miller says. "Family members' emotions can run high, and fights can result in collateral financial damage to an individual unequipped to defend his or her own wishes."

The best way to defend against that exploitation is to prepare, even if that means surrendering some financial independence. Studies have discovered that financial decision-making peaks around age 53 and gradually declines, even among healthy individuals. It's part of the aging process, but Fidelity Investments discovered that 60% of older adults worry about burdening their families with the task of managing the finances.

"The inevitability of losing financial independence is something for which we all need to plan," says Suzanne Schmitt, vice president of family engagement for Fidelity Investments. "That's why it's important for families to be in sync about what needs to happen in the event it's necessary to take the financial keys away from a loved one. By engaging in conversations now and having a strong support system in place, families can help loved ones gracefully transition into that next phase of their lives."

While only 9% of older adults (aged 50-80) surveyed by Fidelity felt they'd never lose the ability to manage their day-to-day finances, 60% admit to having witnessed it happen to a friend or family member—and 40% actually helped manage their own parents' finances. Financial firm Edward Jones found that 43% someone close to them who had or is currently dealing with Alzheimer's disease. That includes 51% of Baby Boomers who've known someone who's struggled with Alzheimer's.

"Health care expenses can be difficult to project, especially when you are decades away from retirement," said Scott Thoma, principal and investment strategist for Edward Jones. "Unexpected conditions and medical expenses that manifest later in life pose a great threat not only to physical and mental health, but also to the financial well-being of both the care receiver and the caregiver. That's why it is critical to start preparing early -- proactive planning can ultimately help individuals protect their assets over the long term, even if health complications emerge during retirement."

That's a discussion you want to have while you can still opt to do so. While Fidelity found that nearly 75% of older Americans surveyed say it's very important to maintain the ability to manage day-to-day finances, there are a number of instances where adult children often need to step in and get more involved whether the parent wants them to or not. When a parent or loved one makes a direct request for financial assistance, when age starts to become a significant factor (like when a parent turns 75 or older) and when there is awareness of a change in circumstances, Fidelity notes that it's best for adult children to step in if there's no plan in place. That puts the burden on parents to formulate a clear strategy for their later years.

"When you get right down to it, we're independent right up to the point we're not," says Fidelity's Schmitt. "Well before a tipping point has been reached, families need to be prepared and make sure they have a strong transition plan in place — and the good news is, there are several benefits to building a strong family financial safety net. Doing so allows parents the ability to maintain their current lifestyle for as long as possible, helps them preserve their assets and may increase the likelihood they won't fall victim to fraud."

Palisades Financial's Miller notes that, when you or your loved one is still capable, you should set up a durable financial power of attorney. The POA, along with other documents such as a health care directive, empowers a trusted individual or individuals to make decisions on another's behalf. That said, be alert for cases of "undue influence" where a person uses a position of power to influence someone else to do something he or she would not have done otherwise. For example, if an adult child who does not get along with his siblings tries persuade his mother that his siblings mistreat her and that she should trust him with her finances exclusively, that may not be the best person to grant power of attorney.

"Appointing someone to this position gives him or her a great deal of power, so it is essential to be careful about who you choose," he says.

If your loved one is no longer capable of executing such documents, you may want to consider guardianship or conservatorship. A legal guardian is authorized to make personal decisions, such as choices about housing and medical care, on behalf of an individual who no longer has the capacity. A conservator is appointed to make financial decisions. Both are court-appointed positions, and Miller warns that the process can be complicated and will take time.

Just be on guard for third parties who want to take on that role without the court appointing them to it. Miller notes that people in their 70s and 80s are often targeted for free-lunch seminar pitches. While treating someone to a free lunch does not automatically signal bad motives, anyone invited to such events should be especially wary of hard-sell approaches. Also, never assume that someone with professional certifications is inherently trustworthy.

"Not all certifications have a rigorous evaluation process or require the professional to maintain certain standards," Miller says. "Make sure that anyone providing advice to your loved one observes a fiduciary standard—meaning he or she puts the interests of the client first."

Finally, stay on the lookout for "affinity fraud." If someone suddenly tries to use a shared affiliation, such as religious beliefs, to build unwarranted trust with your loved one, keep it at arm's length.

"A financial professional who attends your loved one's church or lives in her neighborhood deserves just as much scrutiny as a complete stranger," Miller says.

This article is commentary by an independent contributor. At the time of publication, the author held no positions in the stocks mentioned.

More from Retirement

Video: When Planning for Retirement, Don't Underestimate Your Life Span

Video: When Planning for Retirement, Don't Underestimate Your Life Span

7 Ways Your Financial Adviser Should Help You Survive Rising Interest Rates

7 Ways Your Financial Adviser Should Help You Survive Rising Interest Rates

4 Critical Questions to Ask Your Financial Advisor

4 Critical Questions to Ask Your Financial Advisor

Nearly 50% of Americans Concerned They Won't Have Enough Money in Retirement

Nearly 50% of Americans Concerned They Won't Have Enough Money in Retirement

From Football to Finance: NFL Hall of Famer Mike Haynes Has Some Advice For You

From Football to Finance: NFL Hall of Famer Mike Haynes Has Some Advice For You