It's a Dust seal, it keeps out Dust, Dirt, And water.
Even with sealed bearings, the last thing you want is dirt or water in there.
Removing them is a bad idea, and will Not improve "rolling resistance"
If you want less rolling resistance, install ceramic wheel bearings.
Once you push a bike with those, you'll be amazed at the difference, and is the only way to achieve what you're looking for(as well as adding lighter, chain, sprockets, wheels, and rotors).