I was in Japan in October, the weather was cooler than expected. I wore shorts and a light jacket. I was thankful that I travel with a folding umbrella, as it poured a few afternoons.
I cruised Alaska a few years ago in May was expecting cold weather and I wore shorts everyday.
I guess what I am trying to say is, pack for cold and warm.