they might return once you stop the pill. For women who experience migraines the week of taking placebos on birth control pills, migraine may worsen when you go off the pill, says Carlson.