I am employed by a company in Texas and started working from home upon the birth of our daughter.

I spend part of my time working out of our home here in Texas. However, sometimes I also work out of my parents' home in Delaware (for childcare help).

Texas does not have a state income tax. Could I potentially owe any income tax to Delaware under this arrangement? Advice would be much appreciated!

You will file tax return in both the states. One state is your tax home and other state is where you are part year resident.
In the state where you are part year resident, report income you earned while in that state.
In the state that is your tax home, report your worldwide income for full year. Also in this state claim credit for taxes paid to the other state.
Read this: Your U.S. Tax Return: Working in two or more states