Well, I wouldn't freak out but I wouldn't relax about it either. It could be a number of ailments. Why not just get a referral to the ENT now and get it checked out instead of agonizing about it for another month?

Tell your Dr. that you used to smoke and that you are in a higher risk category.
A biopsy is the only way to know for sure.