New York's top court will decide whether a convicted sex offender's return home constitutes child neglect by him and his wife even though there was no evidence he did anything inappropriate with his children.

The Court of Appeals is set to hear arguments Tuesday in the case of a Poughkeepsie couple identified as Grace and James C. He pleaded guilty in Manhattan to patronizing a 17-year-old prostitute and raping a 14-year-old girl, spent a year in jail and went home as a registered sex offender to his wife, four sons and daughter in 2007.

Dutchess County social workers alleged neglect on his part for returning and his failure to participate in treatment and by the mother for letting him return.

A Family Court judge agreed. A midlevel court reversed, finding no showing of "actual danger" to their children.