how can I prove cot^2X-1/1+cot^2X=2cos^2X-1

I assume what you mean is:

[cot^2(x) -1]/[1+cot^2(x)] = 2 cos^2(x) -1

Right?

First step is convert the cotangent function into cosine over sine. Then multiply through by sin^2(x). Then all you need do is apply the fact that sin^2(x) = 1- cos^2(x)

yeah have reference to equations like this.
Trigonometry Equations - EquationSheet