Walker had given the away side the lead at Parkhead.
The 22-year-old had given the away side the lead during the opening moments of the game, but an equaliser from Gary Mackay-Steven as well as the first two goals of Patrick Roberts' loan spell at Parkhead ensured that Ronny Deila's men remained well out in front at the top of the Scottish Premiership.
And after reflecting on the game, Walker conceded that the champions deserved the victory but insisted that he was still proud of his side's performance.
"Did 3-1 flatter them? Probably not, they are a good team, they created chances," he told The Scottish Sun.
"But so did we and we are disappointed to lose the game given the way we played."
The fact that Hearts are hugely disappointed following a relatively narrow defeat away to the league leaders highlights how far they have come since Neilson took charge at the start of last season. In their first season back in the top flight, the club appear to have third place - and a place in the Europa League first qualifying round for next season - secured, which is a tremendous achievement given the off-field struggles of previous years.
However, the fact that an arguably under-par Celtic team came out on top proves that Deila's men are still able to do enough against most of the sides in the division even when they are not at their best.