If you speak to him about it, he will think 'wow, I really must have affected him for him to be talking about it five years later.' And you don't want him to think that he is that important in your life.
If you nut him, you will lose your job.
Fuck with his mind. Don't even acknowledge the fact that you remember him. Or if that's not possible, don't acknowledge that you remember he called you a cunt. Show him that that is how insignificant in your life he is. Treat him with the very minimum of civility you can get away with, but take the opportunities that may arise to make him look like a twat within the confines of your job. You're an intelligent guy, use it to make him squirm!