Well, it was easy, but there was downtime. When you remove the interface from the bgroup the network settings are saved to the bgroup and not the interface. This means that from the moment you remove the interface, it becomes unavailable. At that moment you need to set the zone for the bgroup to NULL and the interface IP to 0.0.0.0 to free up the IP. When this is done you need to set the configuration back to the interface port that you removed from the bgroup. The policies and other objects do stay without any changes. Still for someone that wants to do this, be aware the the interface will be unavailable during the configuration phase (2-3 minutes).
The bgroup indeed did not provided any added value for what I needed. Thanks.
For your info. Case closed 🙂