I had a fully-charged drone base waiting to launch, and it was destroyed by creeper. The sprite of the drone stays on the map. Rebuilding a drone on the same site will mask it but not delete it.
Sure enough..... looks like I broke this during the beta. You are right that it is essentially just a graphical leftover. The drone object actually gets destroyed, its "sprite" just gets left over. This only happens when the drone is on the base. If the drone is in the air it gets cleaned up.
Thanks for the catch...