You need to be sure he has time to be a kid and not pushed to far and to fast, Many gifted children are ruined by not being allowed to also be kids. He will have plenty of time latter to expand on his gifts if he is truly gifted and not just mature for his age.

If you must, you start by not spending all of your time with him, allow him a chance to also be away from you. To be with kids just his age. Not to groups where all parents are just sitting there watching but time he can develop without you there.