At school in the 1950s a teacher read a book to the class about some children who discovered a priests hole in the house they were staying in. This priests hole lead down to a beach that smugglers used.

Possibly one of The Famous Five series (http://en.wikipedia.org/wiki/The_Famous_Five_%28series%29). Try Five Go to Smuggler's Top.