The fourth quarter provision for loan losses increased $148,000, or 19%, to $909,000 for the three months ended December 31, 2012 compared to $761,000 for the comparable 2011 period due to strong growth in the loan portfolio during this time period. Net charge-offs for the fourth quarter were $511,000, or 0.13% annualized as a percentage of average loans outstanding, an increase from $263,000, or 0.07% annualized as a percentage of average loans outstanding, in the prior year period. Provision expense continues to be assessed in correlation to the Company's loan growth and risk profile.
On a linked quarter basis, non-interest expense increased $518,000 to $15.0 million in the fourth quarter 2012, while core operating expense basically remained level, increasing by only $15,000 to $14.5 million. The aforementioned $503,000 in compensation and other expenses related to the Jack Henry & Associates service disruption were not included in fourth quarter core operating expense. Core salaries and employee benefits expense increased $506,000, or 6%, over the linked quarter due to both the increase in FTE ($328,000) and the increase in commissions and incentive expense related to record loan originations ($272,000). One significant driver to controlling expense over the linked quarter was the decrease of $324,000 in professional fees. The fourth quarter 2012 non-interest expense as a percentage of average assets and the efficiency ratio on a core basis were 2.96% and 70.10%, respectively.
For the year 2012, core operating expense totaled $54.0 million and increased by $8.9 million, or 20%, from total core operating expense of $45.1 million in 2011. The primary driver in the increase in core operating expense in 2012 was the $7.3 million increase in core salaries and employee benefits expense attributable to the infrastructure investment as the Company transitions to a commercial banking model from a mutual thrift model. FTE increased to 331 at December 31, 2012 from 281 at December 31, 2011 supporting expansion in the revenue driving commercial and residential mortgage banking divisions. This infrastructure growth will provide the Company with the ability to continue strong organic growth to mitigate potential margin compression in 2013. Additionally, due to the hard freeze of the Company's pension plan as of December 31, 2012, the fourth quarter of 2012 will be the last quarter of higher pension expense, having estimated an incremental $1.1 million in annual savings following the freeze.