The pair struggled throughout the match, which Arsenal won 2-1, and Jamie Redknapp was just one of many pundits to air his frustration at Pochettino for tinkering with his team - during Sky Sports' coverage of the League Cup clash.
However, the Argentine manager insists that he made the right calls, saying that Fazio and Vorm both deserved to start in the fixture.
"No, I don't regret playing Fazio," he said. "You can make the mistakes and in football it always happens. They are not machines.
"Today he played because he deserved to play and afterwards you can play well or not. This is football and it's not boring because you never know.
"Always you need to make a decision and I think it was the right decision. But maybe when you lose it's not the right decision, and you understand this is the game.
"I don't think it was a gamble to change the goalkeeper. Michel Vorm deserved to play and I think it is a competition, and it was a good opportunity for him. We need to show respect to the players."
Vorm was at fault for the first goal Spurs conceded, as he parried a long-range Alex Oxlade-Chamberlain shot straight into Mathieu Flamini's path. Fazio then made a mishap for the second, with his poor clearance falling perfectly for Flamini to smash into the net.
The two mistakes ultimately cost Tottenham the match, in what was a very frustrating evening at White Hart Lane.
Fazio and Vorm are very much back-up players at Spurs, and there have been suggestions that neither should have played in the North London derby, as the game carries too much importance for supporters.
After their displays, it is unlikely that either will be back in the first-team any time soon, with Spurs' next match coming against Manchester City in the Premier League on Saturday.