No having it does cause surprises when route-maps are not applied when the peer comes up. Changing it will cause surprise to existing users.
The default should be changed but a migration script should add address-family ipv4-unicast to old configs so that existing config continue to work as before, being explicit about what's going on.
If there was no no-default-ipv4-unicast, insert "address-family ipv4-unicast" into existing neighbours that don't have it so that they work as before and clearly show that they allow both AFs.