I'm trying to remember a film from my youth. Basically the plot is about a southern family whose father has passed on and the mother isn't in the picture. (I believe in the movie she had died in child birth) The rest is basically the kids trying to survive on their own and hide it from the neighbours in their small but nosey community.

You might be thinking of "Where The Lilies Bloom" from 1974...
Set in the Appalachian Mtns, 3 sisters and 1 brother

I think that is right. I seem to remember "Lilly" in the title. Thank you.