Baggies goalkeeper was the hero, but he was not happy with his own performance.
The Baggies were pushed hard by the League One side, drawing 1-1 after 120 minutes, but Foster saved penalties from Martin Samuelsen and Lee Angol to send his side into the FA Cup fifth round where they will face Reading.
Foster may have relished his chance to become a hero after extra time, but he admits he was not satisfied with his own performance prior to that, and felt he should have saved Jon Taylor's opening goal.
Speaking to The BBC after the game, Foster said he wanted to apologise to West Brom supporters for not saving the effort.
He said: "It wasn't easy, it was a very tough game. Fair play to Peterborough, they pushed us all the way. We knew they would try to get on top of us and they came at us from the start. It was a decent cup tie.
"I have to apologise to our fans, I should have saved their goal. It got a deflection and some of the pace was taken out of it, it was scrappy.
"But we reacted well and got the goal back quickly. After that I thought we were the only one team that looked like scoring. I love shootouts! I think most goalies do. We've got nothing to lose."
Foster has a good track record in penalty shootouts, notably helping Manchester United win the Carling Cup against Tottenham Hotspur at Wembley in 2009.
Fortunately last night he came through once again, and did not make his night worse by ending up on the losing side.