NEW YORK ( TheStreet) -- Rising home prices have helped pull more than 3 million borrowers from underwater over the past year, but in the hardest-hit areas, the crisis is far from over. According to Zillow, there are still millions of borrowers who have debt so far in excess of the value of their homes, that it would likely take years for them to regain equity in their homes. The national negative equity rate fell in the second quarter to 12.2 million borrowers. That's 23.8% of all homeowners with a mortgage who owe much more than their homes are worth. In the previous quarter, there were 13 million underwater borrowers while in the year ago quarter there as many as 15.3 million. While the negative equity rate is improving, about 57% of homeowners in negative equity are underwater by 20% or more. Roughly one in seven owe more than twice what their homes are worth. Zillow expects home prices to rise by 4.8% over the next year. If home prices rise at that pace annually, it would take a homeowner underwater by 20% roughly four years to reach positive equity, according to Zillow. Between rising interest rates, flat incomes and a still lackluster job market, it is unclear if home prices will indeed expand at that pace for four years. "The frustratingly slow pace of negative equity declines in the face of such robust home value appreciation is a direct result of the fact that many people in the hardest-hit markets are underwater by an enormous amount," said Stan Humphries, chief economist at Zillow. "Because of this, negative equity will be a factor in these markets for years to come, constraining the supply of homes for sale and keeping people out of the market who might otherwise get involved." Zillow also calculates the "effective" negative equity rate, which includes borrowers who have less than 20% equity in their homes. Such borrowers usually will find it difficult to sell and trade-up to a bigger home because listing a home for sale and buying a new one involves not only a new downpayment but considerable closing costs.