Great idea.
Better fuel mileage, better ride, and quieter.
All you lose is traction.
If traction is an issue, for the price of steel wheels, have traction tires on steel wheels for winter, and steer tires on polished wheels for spring, summer, and fall.