Oval miniature (badly damaged) in watercolour on ivory in a moulded gilt wooden frame. The sitter is shown when a captain, head and shoulders, turned to his left with only his right shoulder and epaulette visible but looking out at the viewer, against a neutral brown background. Though apparently balding he wears his own hair frizzed over his ears and possibly powdered, has blue eyes, and is in the captain's full dress of 1795-1812 with a white shirt and neckcloth.

Kelly was commissioned lieutenant on 16 May 1776. He served in the 'Hope', sloop, the following year and was promoted to commander on 1 April 1782 and captain on 8 August 1783 in the 'Adamant' , 50, in which he was flag-captain to Admiral Sir Richard Hughes in the Leeward Islands until 1786. In 1793, at the start of the French Revolutionary War, he was appointed to the 'Solebay', 32-gun frigate, and served under Sir John Jervis at the capture of Martinique, including ashore leading men against Fort Bourbon. In 1795 he moved to the 'Veteran' and in 1797 to the 80-gun 'Gibraltar', which he was still commanding in 1801 under Sir John Borlase Warren off Cadiz when Warren's squadron re-entered the Mediterranean to join Keith's fleet in search of the French squadron of Ganteaume.

Some time early in 1803 he appears to have been dismissed from his ship by court martial at Gibraltar, apparently for not enforcing the rigours of the Articles of War against mutinous members of his crew, including the striking of an officer, because he sympathized with their viewpoint, which was reported against him. A letter of 20 March from Sir Richard Strachan to Benjamin Tucker, asking him to use his influence to rescind this with the Earl of St Vincent, appeared in lot 105 of Charles Miller's maritime sale of 21 October 2009. He may have been briefly in the 'Temeraire' and then the 'Caesar' early in 1805, however, but had handed the former over to Eliab Harvey and the latter to Strachan before Trafalgar. He became Rear-Admiral of the Blue in the post-Trafalgar promotions of 9 November 1805; of the White on 28 April 1808; of the Red on 25 October 1809 and Vice Admiral of the Blue on 31 July 1810. It is unlikely he had any active service as a flag-officer, though this remains to be confirmed. He died at Kelly House, Launceston, Cornwall, the home of his brother Arthur, on 2 May 1811 (Royal Cornwall Gazette, 11 May). From the little information about him in standard sources, Kelly appears to be one of those officers who was fortunate in becoming a post-captain relatively early, perhaps for reasons of family influence rather than notable merit. He enjoyed long service in that role with apparently little greater distinction, and possibly ending in some disapproval, before benefiting (albeit in retirement) from the system of automatic subsequent promotions through flag rank and the increases in half-pay salary that came with them.