Southampton face Everton and Aston Villa in their next two matches.
The Saints head into their two consecutive away games coming off the back of a 3-1 win over Newcastle United at the weekend, a result that consolidates their position in the top seven.
While a top four spot is now a difficult task, Koeman wants his side to challenge nonetheless as they currently sit seven points behind Manchester City in fourth with only five games remaining.
The Saints need to keep the pressure on, and Koeman believes at least four points from the next two matches will help their cause in what is a very unpredictable Premier League.
“I’m not worried when we play at home because we are a team at home that create always opportunities to score,” the Dutch boss explained to the Daily Echo.
“I’m more worried about the away games and we get two away now – Everton first and then Aston Villa. To keep the pressure to the team above us on the table we need at least four points out of the two.
“We have to improve and we have to keep our home level and that we need to show in the two away games.”
Southampton have started to pick up form over the past two months, winning eight and drawing two of their last 13 Premier League matches which has seen them move up the table.
While a top-four spot does seem beyond them at this stage of the campaign, a fifth-place finish would see them enter the Europa League and only three points separates the Saints from Manchester United who currently hold fifth spot.
The south coast club still have Tottenham Hotspur and Manchester City to play which will be challenging, yet Koeman’s men have already proved that on their day they are a match for any team in the league.
Both Everton and Villa are hugely out of form, and they are two games Southampton should be looking to win.