What people will notice is if something is wrong with the balance between the amount of hair on top, compared with the amount of hair on the sides.
It will be a subliminal thing that something is not quite right.
I'm sorry to say that its around 30 that men typically start to recede.
If you temple hair is creeping backwards, then your hairline usually goes with it.

Here's a guide that might help you: