Councillors have given the green light for the redevelopment.
The new building will also contain 5-a-side rooftop football pitches, a climbing wall, flexible community spaces and a café.
Demolishment of the current building, which closed in December 2019, starts later in the summer.