We've had a similar experience this weekend. All of our faucets, toilets and shower heads were clogged with what looked like rusty synthetic fish eggs. Because it affected our cold water pipes as well as the hot water ones, we ruled out the hot water heater. The water was clear through the value just before the water softener so the debris was most likely not from the city. We finally called our water softener company who told us that the screen that prevents the beads in our water softener from escaping must have broken.

We finally called our water softener company who told us that the screen that prevents the beads in our water softener from escaping must have broken.
If the softener's a rental unit then the softener company should be responsible for flushing out your water pipes and faucets. If it's a unit you purchaseed then I would contact the manufacture and see if they are responsible for any part of the expense. Good luck, Tom