Current chief executive James Leigh-Pemberton will become UKFI's chairman and will "continue to play a key role" in the running of UKFI.
Holbourn said: It is a privilege to be appointed to lead UKFI. We have an outstanding team here and I look forward to working with them and our investee companies in my new position to continue to deliver good results for the taxpayer.
UKFI was created in 2008 due to the number of financial assets the government took onto its books in the wake of the financial crisis.
It looks after the government's shareholdings in the Royal Bank of Scotland and Lloyds Banking Group as well as UK Asset Resolution and its subsidiaries, the remnants of Bradford and Bingley and Northern Rock.
The government has been working to sell off its holding in the two high street banks RBS and Lloyds. Government shares in Lloyds bank are expected to resume in coming months after being put on hold due to a sell off of banking stocks at the beginning of the year.
The government still holds on to a nine per cent stake in Lloyds, worth around £4bn.
RBS is much further from returning to private ownership, with shares in the bank way under what the government bought them for and the bank struggling to return to profitability.
Leigh-Pemberton said: We are fortunate at UKFI to have a very strong team, which will flourish under Olly’s leadership. His judgement and experience are great assets to the organisation, and I look forward to continuing to work closely with Olly and the team as UKFI becomes part of the larger UKGI group.
Former UBS banker Peter Norton will take on the new role of head of banking and capital markets at UKFI. At UBS he worked on advisory and capital raising assignments for UK and European financial institutions.