I have heard all sorts of horror stories about what will happen to my retirement accounts if I am a permanent resident in the US but with a UK passport.. please clarify

You can start with telling us what horror stories you've heard. I for one haven't heard any of them to say if they are true or not.

I agree; I know of no horror stories for such situations.

My next door neighbor, not a citizen, has been retired and is drawing SS and IRA income. An individual can get Social Security retirement benefits so long as he or she is “lawfully present” in the United States. The person need not become a U.S. citizen. IRA's are the same.