Attachment 47777The hole in the wood perhaps is a bit too large. The gap must be filled in with something.
Right now it is wobbly side to side. Wood dough was used but was unsatisfactory.maybe some type of sleeve
Or other device into that gap. A pipe clamp below and a pipe clamp on above prevents the wood from sliding down the pole. Problem is the gap between the pipe and the wood perhaps 1/4

How to stop the pipe or wood from moving depends on what you've got going on and how much weight or pressure will be on the wood or the pipe. What is your project? Can you show a picture of the whole thing?

What is that pipe for? Looks too thin for water... being that close to the wood it better not be something that ever gets hot.. though it looks far too small for a chimney.

I have used, a spray foam that hardens and can be trimmed to look neat. The pipe of course if properly secured should not have a lot of movement anyway.

storm collar? cheaper version of Noritz N Vent 4" Storm Collar Pipe SCP4 | eBay