Council leader Ian Harvey, who was one of the six to leave, said there had been serious concerns of "bullying, misogyny, hounding and intimidation".
Leaving the authority with no overall control, the remaining Tories have called for the six to stand for election.
The Conservatives do not recognise the allegations, saying they were "shameful, sordid, and not worthy of a response".