"We think some people may have skipped over it on the form – even with the software, in some instances – just completing the return as they did last year. We've been surprised by that," Everson said.

For this year only, most taxpayers can claim a refund of $30 to $60. The amount depends on the size of your family, for taxes the IRS mistakenly collected on long-distance phone services. So far, 30 percent of tax filers are failing to claim the refund, and half of the taxpayers paid preparers to complete their return, Forbes reported.

On the other hand, the IRS received huge telephone refund claims, allegedly fraudulent, for around $10,000 early in the filing season. "That's a lot of phone usage. Even my teenage kids can't generate that much phone usage," Everson joked. A subsequent crackdown on fraud appears to be effective, he said.

Other challenges for the IRS included extensions of tax breaks that did not become final until the end of December. The agency had to hussle to implement the changes in time for filing deadlines, and incorporate the changes into software programs.

A new direct-deposit service was also difficult: Filers can split their refund and have it sent electronically to different financial institutions. Everson said about 55,000 people took advantage of the service, that’s of 74 million returns processed so far, but he expects it to become more popular.