Although I personally like ladybugs and it doesn't bother me when they find their way inside my home, there actually has been a rise in allergic reactions and asthma attacks associated with the Japanese variety.  
This website should help you.
Lady bugs /Asian Beetles/ Ladybugs
Right now, the best thing you can do is just vacuum them up as you see them.  
Next fall, you might want to do as the website suggests. 
I get a terrible cluster fly infestation in the wintertime because I live in the middle of the woods.  So, every fall (early) I have a local pest control company come out and spray the entire exterior of the house.  Since I started doing that, the problem has been resolved.  You may want to consider having a company come in next fall and spray for you so that you don't have to deal with all the chemicals yourself.