PropertyPtr's public inheritance of shared_ptr reveals too many internals

In my opinion, a PropertyPtr should not expose the shared_ptr methods to the user, we should refactor this.