This comes from a great period of Scottish letters, that of the rivalry between the Edinburgh Review and Blackwood’s Magazine from 1817 onwards. The latter was a Tory journal, full of what one contemporary called sentimental Jacobitism, which was created specifically to oppose the Whig policies of its rival.
Among its writers were Thomas de Quincy, James Hogg (the Ettrick Shepherd), and John Wilson. The last of these was soon to be appointed by political shenanigans on the part of the Tory council of Edinburgh to be Professor of Moral Philosophy at the local university, a post for which he was entirely unqualified (he relied on his friends to write his lectures for him, though a biographer commented that “he probably managed to become something of an expert on the subject by the time he retired in 1851”).
This did nothing to stop his prodigious literary output for the magazine under the pen name of Christopher North, which was by turns fearless, violent, measured and scurrilous; no better word could be imagined to describe its tone than Wilson’s own invention in its pages of squabash for the process of utterly demolishing some rival in print. He seems to have made it up, a fanciful compound of squash and bash, though there’s a Scots word stramash for an uproar or commotion that might have contributed to its invention.
The noun is not recorded outside his own works, but the verb survived as a rarity for a while after his death. It seems to have died out near the end of the nineteenth century.