Maybe you just need to take a break from wearing. If you feel like shaving, I say shave it. Hair or no hair. It's all subjective and what you are comfortable with.

I could never shave mine because my head looks fine with hair but without would like like Brain's from Pinky and the Brain.

If you can get away with it, go for it.