NEW YORK (MainStreet) — Housing prices are rising — by 3% over the past year — but that's just the leading edge of the problem for Americans struggling to afford a home.

Compare Today’s Low Mortgage Rates

Finding money for a down payment and monthly payments is proving difficult for a growing number of would-be homeowners, many of whom can't afford a new home for one specific reason: They owe too much in student loan debt.

According to NeighborWorks America, a Washington, D.C., nonprofit, roughly 25% of Americans know a family member, friend or co-worker who can't buy a home due to student loan debt.

Average student loan debt in the U.S. stands at $29,400, and that figure should continue to escalate given the steady and continuing rise in college costs in recent years.

Collegeboard.com says costs rose by 2.9% for tuition and fees for in-state students at public four-year colleges and universities in 2013-14, following increases of 4.5% in 2012-13 and 8.5% in 2011-12 (before adjusting for inflation).

TheStreet Recommends

NeighborWorks reports that most Americans (60%) want to own a home, but big issues such as student loan debt and a tight lending environment work against them, keeping otherwise credit-worthy buyers out of the market.

Nearly half of Americans (49%) who have student loan debt call their college loans the biggest obstacle to buying a home, even over "lack of a down payment" and "lack of job security." Women are particularly affected, as they represent 58% of all student loan debt holders.

"Earning a postsecondary degree is increasingly critical in the United States, but student debt is preventing some Americans from purchasing a home and fully fulfilling this 'American Dream' aspiration," says Chuck Wehrwein, chief executive at NeighborWorks America.

Burdensome student loan debt has a "spider web" effect that hurts the entire U.S. economy, he says.

"If we don't mitigate the effect student loan burden is having and will have for years to come on homeownership, the country will lose a significant amount of economic activity, and hundreds of thousands of people will be unable to benefit from the stability and financial value that homeownership has been proven to offer," Wehrwein says.

By Brian O'Connell for MainStreet