The Serbia international was given the ban after he pushed the referee during their FA Cup defeat to Manchester United last month.
A Football Association appeal to have the punishment extended was rejected.
Silva admits he can't wait to have his top scorer back.