Argyle may have been winless since Boxing Day, but they were clearly determined to get one over their former manager, whose defection to the Walkers Stadium had provoked anger throughout the West Country club.
Matt Oakley had the ball in the net after three minutes but his effort was ruled offside, before the former Derby midfielder dragged a shot wide of the far post.
Luke McCormick then clawed away from just beneath the bar Steve Howard's looping shot as City had the better of the opening exchanges.
Howard was fortunate to escape a caution after his raised arm connected with McCormick's face, leaving the keeper dazed and in need of treatment.
Plymouth's response was strikes from Halmosi and Steven MacLean which flew narrowly wide.
Argyle's enterprise was rewarded on 34 minutes when the home defence failed to clear Krizstian Timor's knock down from a Paul Connolly free-kick and, when the loose ball arrived at the feet of Halmosi, he took full advantage of the confusion to drive home from eight yards.
McCormick kept Plymouth ahead before the break when he instinctively palmed away a Steve Clemence drive.
After the restart, Howard headed a Clemence corner on to the roof of the net, before Ben Alnwick had to get down quickly to prevent a Jim Paterson drive being diverted into his own net by Patrick Kisnorbo.
Substitute Zsolt Laczko nodded over from a good position as Leicester continued to labour against Plymouth's more vibrant performers.
Jermaine Easter seemed certain to wrap up the game when he stretched the Foxes' rearguard only to be denied by Alnwick's acrobatic save.
Leicester rallied late on and were almost awarded a penalty for a handball by Lee Hodges.
Then in the dying moments, substitute Matty Fryatt was foiled by a finger-tip save from McCormick, who sealed a tremendous personal performance when he hung on to Gareth McAuley's shot deep in time added on at the end.