He now began effectually to follow out his bent for limning and etching, and, after a few trials, abandoned his trade as a barber. In 1784, he published his first caricature, which represented a half-crazed Jacobite gentleman, named laird Robertson, who was wont to amuse the citizens of Edinburgh by cutting caricatured resemblances of public characters, which he fixed on the head of his stick, and whose figure was perfectly known to all the inhabitants. The portrait, accordingly, excited some attention, and the author was induced to attempt others. The style assumed by Mr Kay was the stippled or dotted style, and nothing could equal the felicity of the likeness. From that time forward, till he was about eighty years of age, this untutored son of genius pursued his vocation, taking off, one after another, the whole of the public and eccentric persons who appeared in the Scottish capital, and occasionally caricaturing any jocular incident that happened to attract attention. To speak of his portraits as caricatures is doing them signal injustice. They were the most exact and faithful likenesses that could have been represented by any mode of art. He drew the man as he walked the street every day: his gait, his costume, every peculiarity of his appearance, done to a point, and no defect perceptible except the stiffness of the figures. Indeed, he may be said to have rather resembled one of the prosopographuses or apographs of modern times, than a living artist trusting to his eye and hand. Hence, nothing can be more valuable in the way of engraved portraits, than his representations of the distinguished men who adorned Edinburgh in the latter part of the eighteenth century—the Blairs, the Smiths, and the Robertsons. It was only in certain instances that his productions could be considered as caricatures, namely, in those combinations by which he meant to burlesque any ridiculous public transaction: and even here, his likenesses displayed all his usual correctness. During a considerable part of his career, Mr Kay was a professed miniature painter, and executed some specimens which, for delicacy and finish, would surprise such individuals as have only been accustomed to inspect his published etchings. It is said, that his only fault in this capacity, was a rigid and unbending adherence to likeness—a total want of the courtly system practised, in so eminent a degree, by Lawrence and other fashionable painters. Once, it is related, he was "trysted" with an exceedingly ill-looking man, much pimpled, who, to add to the distresses of the artist, came accompanied by a fair nymph to whom he was about to be married. Honest Kay did all he could in favour of this gentleman, so far as, omitting the ravages of bacchanalianism would go; but still he could not satisfy his customer, who earnestly appealed to his inamorata as to the injustice which he conceived to be done to him, and the necessity of improving the likeness, for so he termed the flattery which he conceived to be necessary. Quite tired at length with this literally ugly customer, and greatly incensed, the miniaturist exclaimed, with an execration, that he would "paint every plook in the puppy’s’face: would that please him!" It is needless to remark, that in this, as in other instances, Mr Kay lost by his unbending accuracy of delineation.

During almost the whole of his career as an artist, Mr Kay had a small print-shop in the Parliament Square, the window of which was usually stuck full of his productions. He etched in all nearly nine hundred plates, forming a complete record of the public characters, of every grade and kind, including many distinguished strangers, who made a figure in Edinburgh for nearly half a century. It may be safely affirmed, that no city in the empire can boast of so curious a chronicle. From the first to the last, there is a remarkable similarity in his style. After forty years’ experience, he was just as deficient in grouping, and other acquired gifts in the art, as when he first began to use the graver. It would almost appear as if nature had designed him for that peculiar style alone, in which he so much excelled all other men, and had denied him every common effect of his art, which other men generally attain with ease.

In a profile of himself, executed about the year 1785, Mr Kay appears with a handsome aquiline countenance, of much delicacy and ingenuity of expression. In his latter days, when the writer of this notice first saw him, he was a slender but straight old man, of middle size, and usually dressed in a garb of antique cut; of simple habits, and quiet, unassuming manners. His head was of a singular structure, presenting a very remarkable protuberance in the forehead, where phrenologists, we believe, place the organs of observation: in Kay, the profile of this feature formed the arc of a perfect circle, beginning under the hair, and terminating at the root of the nose. According to the information of his widow, (a second spouse, whom he married in 1787,) he cared for, and could settle at no employment, except that of etching likenesses. He would suddenly quit his lucrative employment in miniature-drawing, in order to commit some freak of his fancy to copper, from which, perhaps, no profit was to be hoped for. It was the conviction of this lady, that, if he had devoted himself to the more productive art, he would soon have acquired a competency.