The pair came together just before half-time in the 2-2 draw at the Emirates Stadium on Wednesday evening.
The Uruguayan striker had attempted to delay an Arsenal penalty by holding onto the ball, something Ramsey attempted to stop.
Cavani did not take kindly to that, first striking the Welshman in the chest and then slightly catching him in the face, something Ramsey attempted to point out to the referee.
It was unsavoury, to say the least, and both Ferdinand and Gerrard believe he should be embarrassed by what he did.
"I think he should be a little embarrassed by that. I thought the Welsh are meant to be tough,” Gerrard told BT Sport.
“I think there was a bit of over-reacting from Ramsey. I think if he was a foreign player we'd be going mad,” added Ferdinand.
On the other hand, former Arsenal striker Ian Wright moved to defend the midfielder.
Ramsey did not go to ground when struck by Cavani nor did he overreact but showed the kind of restraint a number of players would not have.
And Wright believes Cavani could have been sent off for not only the incident with Ramsey but also for time wasting in general.
“Cavani has taken the ball to bide time, maybe put a bit more pressure on Giroud and I think he's very lucky," he added.
"To throw his hand like that... I don't think it's that bad, but his arm is very aggressive, he could easily have gone for that."