Sources say that Sir Robert McAlpine, Laing O'Rourke and Miller Construction are now left in the hunt for the American Express project.
The news means the end of the road for a fourth challenger - Skanska.
It is understood that the client is in talks with the three remaining firms and that a decision is expected within the next few weeks.
The office complex, to be known as Amex House, will be constructed on the car park site of the present Amercian Express building in Mighell Street.
Some demolition work is also involved in the job.
The project centres on a nine-storey call centre and a four-storey data centre.
Work on site is expected to kick off early in 2010.
Lawmaker escorted out of UT gameThree chase £100m office job