NEW YORK (MainStreet) — Six months after graduation, former students begin to payback their college loans. More often than not, the money goes to a loan servicer, who is charged with collecting monthly payments and, if the borrower struggles, manages repayment plans and loan consolidations. These loan servicers, which student loan borrowers typically know nothing about until they start writing their checks, will hold a hammer over their financial lives for years to come.

Also see:Why A Dying Poor Man May Not Have Health Coverage >>

But the loan servicers are under the gun also. Direct Loans from the federal government are assigned to servicers based on Department of Education surveys that rank the servicers according to user satisfaction.

"The servicer rankings matter most for the four main for-profit servicers -- Sallie Mae, Great Lakes Educational Services, PHEAA Federal Loan Servicing and NelNet -- because their allocations are based on performance," said Mark Kantrowitz, vice president and publisher of Edvisors Network. "Borrowers are randomized across these servicers to ensure that the ratings form the basis of an apples-to-apples comparison."

The non-profit servicers—there are currently 11--have a similar rating system, but input from college financial aid staff is omitted. Default metrics look at delinquency rates, not the percentage of borrowers and the percentage of dollars that go into default. The Health Care and Education Reconciliation Act of 2010 created a carve-out for the non-profit servicers, so they will continue to get loan volume even if their performance isn't as good as the four main servicers.

"These servicers are guaranteed to continue to receive their initial loan servicing volume, no matter what," said Kantrowitz.

So for the four main servicers, the rankings are critical for each coming year. "The full-year rankings suggest that Nelnet will get 30% of new volume, PHEAA Federal Loan Servicing 26%, Great Lakes 26% and Sallie Mae 18%," said Kantrowitz. "If you look at past performance, Nelnet had done much worse. It was a wake-up call for them and they took steps to improve performance, with better quality control at their call center, increasing call center hours and the like." The Department of Education publishes a formula on how it converts these scores to new volume.

Also see: Is Your Appearance Ruining Your Career? >>

Drawing conclusions from the responses to the servicer ranking survey can be tricky. Great Lakes Education Services performed the best overall when it came to borrower and school satisfaction, yet it has the most loan defaults. Nelnet scores the best on loan performance, but is an also-ran in the satisfaction surveys.

One of the worst things about this system is that for the most part, it's difficult to change servicers if you think you're being played by the one you've been assigned to. It you are reassigned, it can seem like a random act.

Also see: Who is the Richest Comedian?>>

"While you generally can't call and request that your loan be switched to a specific servicer, there are some triggers that could lead to a servicer change," said Rohit Chopra, the Consumer Financial Protection Bureau's student loan ombudsman. "For example, with federal student loans, if you consolidate multiple types of federal student loans or submit a certification form for loan forgiveness, you might find that you have a new servicer." That's cold comfort for borrowers who are desperate to dump their under-performing servicer.

The CFPB has released its ranking of the four largest Direct Loan servicers based on publicly available Servicer Performance Reports from the Department of Education, released each quarter.

--Written by John Sandman for MainStreet