It is easy to become lost in the fog surrounding the source of this word. There are — or at least, there have been — several meanings to it, which don’t all come from the same source.
At one time josser was a mildly contemptuous word for a man of the cloth, a clergyman or padre, a term that was better known in Australia than elsewhere. This came from the seaports of the Malay peninsula, having been taken from the pidgin English joss for a Chinese idol or religious image (hence joss stick and good joss, meaning good luck). Joss isn’t a Chinese word, however, but from deos, an archaic form of the Portuguese word for God, which makes the name for a clergyman entirely apt, if deviously derived. Then there’s the sense of joss for the boss, which is from an obscure English dialect word that was taken to Australia in the middle of the nineteenth century.
Our word is used in Polari (or Palari, or other spellings), which was once the private language of showmen and travelling folk. The word is originally from Romany, the language of the Gypsies, and literally means an outsider; it was used for a person who wasn’t born to the trade but who joined a circus or travelling fair as an adult. A member of a circus community might once have said (for Polari is now virtually extinct), “Nante palari before the josser cul”, “Don’t speak Palari before that outsider”.
There’s yet another meaning for josser, for a man, often an old man or one regarded with mild contempt (so it’s quite close in meaning to geezer). This is the sense that most often turns up in British and Irish literature during a period of about half a century from the 1880s on. An example is in Five Tales, by John Galsworthy (for whom it was a favourite word): “He lowered himself to the ground, and moved in the bluish darkness towards the gate of his daughter’s house. Bob Pillin walked beside him, thinking: ‘Poor old josser, he is gettin’ a back number!’ ” Was this a variation on the clergyman sense, or from the English dialect word for the boss, or from the circus sense? Nobody seems to know.