I have old light boxes that are wired for on/off switched fixtures. How would I re-wire them, along with a switch box, to use regular light fixtures?

Switches on fixture? How many lights? Say you have 3 along wall that have power all the time. Turn breaker off that powers those lights, take note of other power loss, maybe an outlet, remove, separate wires at outlet, and see if lights go out, Could install combination switch/receptical, where receptacle was.

If confident in cutting hole in wall, fishing wires from receptacle to new switch location,
What do you have? Stud wall, and drywall? Block, fir strip, and drywall? Wood?