The other day my wife was cleaning the house and we lost all power. Now none of the breakers tripped and there was power coming into the house. We got power for the fridge and TV coming from our neighbors right now. When we reconnected the lines to the breakers and checked the outlets we have 120 coming through but when we plug in even a small desk lamp it drops the voltage down to about 30. Is it in the breaker or is there a serious problem with the wiring?

Yes there is a serious problem, and most likely is a loose or corroded shared circuit neutral, or can even be in the utility lines, overhead connections,or meter, etc.

Unfortunately, this really is not a DIY project, really best to call in a good service electrician to troubleshoot the system.