The 28-year-old spent two years in Glasgow, before leaving Rangers for Ipswich Town last summer.
During his time at Ibrox, Waghorn faced Celtic on six occasions across all competitions.
And although the English forward misses the "outrageous" rivalry that comes hand-in-hand with such a fixture, he admits that Celtic supporters would occasionally 'cross the line' when encountering him away from football.
"Those games against Celtic are on a different level, not only on the field but off the field too with everything else that comes with it," he told The Scottish Sun. "The week leading up to the game there’s so much hype in the press and they just look for anything to wind somebody up.
“The downside is that you can’t really leave your house for a few days afterwards. As a family man I probably found that side a little bit tough.
“Going to the park with my little boy, you’re just trying to have a nice day, and you’d have Celtic fans, adults, coming up to you and approaching you. That’s not nice. There were times when it probably crossed the line a little bit."
Waghorn endured a disappointing last season at Rangers, only netting seven goals in 32 Scottish Premiership outings.
The former Wigan star played a pivotal role in helping the Gers - then under Mark Warburton - win promotion from the Championship during the 2015-16 campaign, but struggled with the step up in class.
Nevertheless, he has clearly landed on his feet since leaving Rangers and Celtic behind, and scored 16 Championship goals for Town last term.