The forward has not played for the Eagles since mid-December.
The 22-year-old has not featured for Alan Pardew's side since mid-December after sustaining a calf strain during the victory over Stoke (via Express), but Millen has explained that he will be available for the game against Remi Garde's men.
"Connor Wickham has been back in training although he's not fully match-fit. He will be in the squad," Millen said, as reported in the Express.
"He's going to train today and we'll assess him. Medically he's fine, he's over his injury which is great news."
The game against the Potters was something of a bittersweet experience for the former Sunderland man. While his participation in the clash was halted by the injury, he bagged his first goal for the club since his £8m move in the summer (via Express), which helped Palace to a narrow 2-1 victory.
And while he has hardly been prolific during his time at Selhurst Park, his absence was highlighted in the club's next three games, where they failed to score a single goal.
However, they ended their winless run with a victory in the FA Cup third round tie with Southampton, which should give them plenty of confidence going into the game with the league's bottom side. Wickham's potential return should provide a further boost as they look to dent Villa's survival hopes even further, while also strengthening their own chances of remaining well inside the top-half.