If I live in CT and work in NY, should medicare be taken out of my paycheck?

Of course medicare tax should be taken out - any reason why you think it shouldn't be? Also social security tax, NY state income tax, and federal income tax should be withheld.

Agreed; FICA (Social Security and Medicare) taxes are due regardless of where you work if you are a U.S. citizen and the source of the income is from the United States.