The Rams won the East Midlands clash 1-0 thanks to a second half goal from Markus Olsson.
Speaking to BBC Radio Nottinghamshire, Mills said: "There's no time to dwell on it. We have to move on quickly.
"It's obviously been a bit of a baptism of fire for the new staff. If we had got a point today then we could say it's a great start, but looking at it it was a good point at Hull, but disappointing today."
Forest had sacked manager Dougie Freedman just a week ahead of the fixture, who had led them to a win over Derby earlier in the season, a game also decided by just a single goal.
Paul Williams has been placed as interim manager, with Andy Reid assisting as a player coach.
The win keeps Derby's momentum going in their quest for promotion, currently sitting in fifth position in the Championship table.
Forest meanwhile are struggling for direction, having lost five of their six games before Freedman's sacking, and now a further two games without a win.
They have dropped to 15th position, and need a couple of wins just to eradicate any slight fears of relegation.