prove the volume of hexagonal = 0.866a^2c?

You need to find the area of the face of the hexagonal prism, and multiply that by its height to get its volume. So first to develop a formula for the area of a hexagon: break the hexagon into 6 triangles, and find the area of each traingle. I assume that the length 'a' is from each corner to the opposite corner. You should be able to see that the length of each side of the hexagon is equal to 1/2 x a, then use basic trigonometry to find the height and base of each of the 6 triangles. Remember that the area of each triangle is one half base times height. Add the 6 areas up, and you have the total area of the face of the hexagonal prism. Can you take it from here?