Modern computers generally have switching power regulators within the motherboard to prevent overvoltage and undervoltage to individual components. The battery may have a hi/low cycle as you described, but most computers are fault tolerant.
A switching regulator will generally draw a little extra amperage to compensate for a voltage drop. Even the regulated power supplies provided with most laptops show some ripple on the oscilloscope. Throughout the life of the power supply voltages will vary as much as 20% on some machines, and the computer will still be fully functional.
A computer that does not fair well with battery supplied power may want to try a
DC to DC Switching Regulator, which will more appropriately level off the voltage rises and drops associated with a battery. It's a bit more heavy duty than those within the laptop itself.
My biggest concern in this process would be the type of power supply originally used with the computer. Some power supplies have a pulse code or variable core voltage to prevent non-OEM devices being used with the computer. This would be much more difficult to overcome from a project standpoint. You would have to experiment to find out.