Mauricio Pochettino's side had taken the lead through an Emmanuel Adebayor header in the first half - but the Magpies hit straight back through a Sammy Ameobi equaliser seconds after the break.
And Perez finished the job for Alan Pardew's men, heading home later on in the half with what proved to be the winner for the visitors.
The striker, who was a surprise stand-in for the injured Papiss Cisse, was making his first start for Newcastle and duly repaid Pardew's faith in him.
The celebrations that followed are likely to be remembered by the Spaniard for a long time to come, with the picture above showing just how much it meant to Magpies fans. Tottenham supporters will want to forget them rather more quickly, however...
Perez, 21, is barely visible in the picture as he is being mobbed by fellow team-mates. Is this the start of something beautiful for Newcastle?