Hello,
Yesterday I went to give my kids a bath and the shower water was rust colored (Not too unusual, we usually run it clear and it's fine), and then we lost all water. I checked every water source in the house, same thing. We have well water, and a brand new oil furnace with an on demand tankless water heater. My husband took off the screens to one of the faucets and out gushed this brown muddy substance and then a lot of this black sediment. If I were to guess it feels like fine charcoal, and when you rub it between your fingers it stains. He proceeded to do this to ever water source, and all had the same result. Today each time I turn on a faucet we are getting more black sediment, water pressure isn't great, I am guessing as a result of this issue. I saw other posts where people have suggested anodes, and braided pipe. This substance isn't oily though, and would we still have an anode on a tankless system? Any information would be helpful, I am not so concerned about the stuff making anyone sick as I am that this is the start to a much bigger issue? Any information would be great! Thank you for your time. :)

You have a well problem. Run an outside spigot into a bucket to see what is coming from the well. The black sediment could be sand. Mixed with manganese. You need a well man ASAP.
You also need a water treatment system. Manufacturers of tankless water heaters recommend water treatment when applying a TWH on well water to remove hardness, iron, manganese, etc from the water. Tankless water heaters are highly efficient and consequently it is imperative to feed them with quality water or they loose their efficiency and have a very short life span.
Sediment coming from the well in this amount generally indicates a failed screen on well point or a hole in the casing of the well.
RJ

Thank you so much for your input! We are getting it looked at. The diagram was also very helpful.