It's simply not an issue. Your side hair does grow and gets a bit untidy and then it gets a trim, and no one pays enough attention to someone else's hair to notice that the top always stays the same length.

It is hard to overstate how little other people are interested in your appearance. If you want to test this, grow a moustache and then shave it off. I can tell you from personal experience that 85% - 95% of the people you know won't notice. Three months later when the subject of moustaches comes up someone might say "didn't you use to have a moustache?". That's how much they register the details of your appearance.