I would see another ENT. He may be good by but he clearly isn't up to date. These days base of tongue cancer - HPV related - is on the rise. There is no such thing as a "history" that would indicate the potential for oral cancer of this type. If you've had even one sexual partner you could have had HPV. This cancer is also found in the tonsil as well - truth be told. Usually HPV related oral cancer affects people between 30 and 60. There is not often pain associated with it, it tends to be diagnosed by a difficulty swallowing or swollen lymphnodes. This is why it is found usually in the later stages.

The cancer he may have been referring to was the smokers/drinkers non HPV oral cancer. (From your intro I assume you do both - though probably not to excess) often this cancer hits people who are older mostly male who drink and smoke a lot. (Though people like myself who do none of those things get it as well.) the difference between the two is location. Non HPV cancers are usually located in the oral cavity - tongue, gums, cheek, palate. HPV- in your throat, or tonsil.

I would get a second opinion. Take care