While most attempt to care for the elderly with compassion and kindness, more and more are looking at that group and their financial savings as prey.

Both the number of incidents and the financial impact of elder financial abuse may be higher than most people realize — with about 20% of family and friends saying they knew an elder who experienced financial abuse, according to a recent study by insurance company Allianz Life. Possibly even more disturbing is more than one-third of active caregivers — or those that could be soon — said the elderly they care for have experienced financial abuse or exploitation with a loss, and 40% of the elderly they care for have experienced abuse more than once.

While many are worried if they are squirreling away enough into an IRA or their 401(k), Jamie Hopkins, professor of retirement income planning at The American College, said a research report published this year in the Journal of Retirement that the number one ethical concern that retirement planners had for their clients was elder abuse.

"However, when consumers are surveyed about their own retirement concerns, they rarely list elder abuse as a major concern, as most people do not feel that it is likely to happen to them," Hopkins said.

How do you — or someone in your family — avoid elder financial abuse? The key is to avoid becoming a victim in the first place, said Anne Tinyo, head of life management services at Wells Fargo Private Bank. Tinyo said the some tips to avoid being a victim can include direct depositing all checks, and having a trusted third person review bank statements.

She also recommends choosing a caregiver with caution.

"Do not assume that by hiring a caregiver through a bonded agency you are guaranteed to get someone who has been checked," Tinyo said. "There is no current law requiring mandatory background checks for in-home caregivers in all states. Make sure there is supervision of care in the home, whether through a trusted family member or friend or by hiring a certified care manager."

Lynette Whiteman, executive director of the Caregiver Volunteers of Central Jersey, said biggest risk her group sees is when the elderly are completely isolated from any sort of support system. She said many of the elderly become so reliant on whoever is taking care of them that they are fearful of losing this person and being alone. Whiteman recommends there should never be one person solely responsible for an elderly person.

"If the family does not live far away, split the responsibilities for care, in some shape or form between siblings so there are checks and balances," Whiteman said. "If you have an aide, make sure a neighbor stops in or be a squeaky wheel and call the agency often."

Unfortunately for many, the abuse often can come from those least expected, said Pat Brault, a principal and regional director at Wipfli Hewins Investment Advisors in Minnesota.

"A common example of elder financial abuse is when a trusted family member takes advantage of senior by manipulating them to provide financial assistance to them," Brault said. "Situations like this can get to the point of threatening the financial situation of the senior."

Brault said in cases where a senior has caregiver relationships with family members or others, some warning signs to watch out can include a caregiver who is preventing the senior from communicating freely or unusual bank or brokerage account activities.

"It's critical for seniors, family members and advisors to engage in open, honest communication at all times," Brault said. "One of the biggest challenges for family members or advisors who observe abuse is simply approaching the topic."