Remove some glider_buffer destroy listeners

[emersion]

Now that locking a buffer guarantees it won't be destroyed until it's unlocked, we can probably remove a bunch of buffer destroy listeners.