I remember a family living out in a desert and their house is visited by aliens in a small UFO and little aliens, they play around with the children of the house.

Are you referring to ET?

Try also Mac and Me (1988) (http://www.imdb.com/title/tt0095560/)

The Day Time Ended (1980) (http://www.monstershack.net/sp/index.php/the-day-time-ended-1980/)?