The Dutchman was disappointed in his centre-back after he was sent off against Manchester City.
The United centre-back received two yellow cards - both for very avoidable fouls - against Manchester City, while Sergio Aguero scored a second-half winner for Manuel Pellegrini's side against 10 men.
And the Man United boss says that, when you are a on a yellow card as a player, you 'cannot do' what Smalling did.
Van Gaal told Sky Sports: “You know as a player when you are on a yellow card. So you cannot do what you did [referring to Smalling's sending off].
“I say a lot of things to the players. It’s not always that they follow what I have said. It’s also emotion. Sometimes you cannot control your emotion. It was not very smart."
However, the Dutchman did defend his side's efforts after Smalling's sending off: "I don’t think our 10 men could have given more. The willpower was unbelievable.
“We can be very proud but at the end we have zero points and it’s the goals that count.
“We were so close in spite of the fact our team was playing with 10 men. We still created more chances in the second half. We were close but not close enough.”