Nigel Pearson's men had gone almost three months without a win ahead of West Ham's visit to the King Power Stadium, but finally they earned a vital victory.
Former Champions League winner Esteban Cambiasso proved his worth with the opening goal after just 12 minutes, but David Nugent missed a penalty shortly after which would have put Leicester 2-0 up.
They were made to pay as Cheikhou Kouyate grabbed an equaliser just past the half hour mark, with the in-form Senegal international pulling West Ham level.
However, Leicester did finally snatch all three points as midfielder Andy King swept the ball home in the 86th minute, giving the Foxes a dramatic late win.
With Sunderland only in action tomorrow, Leicester are now just four points adrift of safety - and have given themselves a real chance of the great escape.
Yet for West Ham, it's another defeat - and with Sam Allardyce out of contract at the end of the season, the pressure is mounting on him, with fans keen to see a change in the dugout after another disappointing afternoon.