Remove pandas to make build working again

[FlorianSW]

Upgrading to the latest version of pandas would've worked, too. However, the server stats API is not finnisshed, not run on regular users workloads (except they intentionally enabled it) and was most likely not finished at all. No need in keeping it for much longer.