My mother died leaving a minor child and minor grandchild in which she had custody, behind. How is custody determined when no will was written

Usually it reverts back to the parents if she was a guardian and never adopted. Otherwise the children would go to the state.

Of course, a relative or close friend often takes them.