Health bosses have decided to press ahead with plans for the site next to the Royal Marsden and Institute for Cancer Research.
Members of Keep Our St Helier Hospital say the meeting on the 3rd where the decision was made was held behind closed doors.
The group, which has been campaigning on this proposal for 5 years, has launched a petition against it.