The council is proposing redeveloping the historic building into a boutique hotel.
But an online petition launched by campaign group Save Royal Kingston to keep it as it is was signed by over 1,600 people.
An Asset of Community Value is land or property of importance to a community which is subject to additional protection from development.