It may seem surprising that such an odd-looking obscure word for the study of flags should have gained acceptance, but it’s relatively common and is recorded in most recent dictionaries (they have to be fairly new, since the word was coined only in the 1950s).
In 1971, he [Crampton] and Barraclough launched the Flag Institute, which maintains and distributes up-to-date information about national flags and emblems throughout the world. In the same year, at the fourth international Congress of Vexillology, in Turin, the institute was accepted as a member of the International Federation.
The Times (London), 7 Jun. 1997.
The word comes from the Latin vexillum for a flag, which derives from the verb vehere, to carry (from which we get vehicle as well). A related Latin term was vexillum, for a body of men grouped under one flag. This suggests that the original Latin referred to a flag that was carried rather than flown from a mast.
Someone who studies flags is a vexillologist, and the adjective is the mildly tongue-twisting vexillological. These two terms may be modern, but the Latin root turns up in a number of obscure terms, such as vexillator for a banner-bearer in a mystery or miracle play. Vexillum is also used in modern botany for the large external petal of a legume flower.