First, you need to read the instruction on how to post. If you are asking about HIPAA we need more info about who you are, what is your relationship to the client, why you took the picture etc. In other words you need to give some detail then ask your questions.

We take photos often of our patients, to use for reference to medical needs latter. Did you take the photo as part of the job,

But the real issue is what you did with it.

Context is key as stated above. A photo taken by a friend is just that a photo. A photo taken by an agent of a medical entity could be held to violate that persons HIPPA rights (though that still depends on the context).

If by client you mean MH or MR, then likely it could be viewed as a violation if it was placed in the context of "here is a client of ours" {picture}. That would id them as a client. Of course that will depend on the specific regulations of your particular regulating body.

Immediately deleting a picture, that has been seen by no one else--I can not imagine a circumstance where that could be said to be a violation, but check with entity counsel to be sure.