By this time in Maryland, as in the other colonies, the tide of self government was running strong. Meanwhile the feudal characteristics of the Calvert regime had been losing ground. With the coming of the French and Indian War, relations between the Proprietary and the lower house of the Assembly became strained, owing to the discontent of the delegates with the power and prerequisites exercised and claimed by the "court" or governor's party. This discontent culminated time and again in complete stoppage of legislative business.

To this crisis Frederick was all but indifferent. He spent much time in travel on the Continent, and indulged a flair for literary pursuits. He produced a privately published book of travel and volumes of verse in which his contemporaries took little interest. Drawing for generations large revenues from the Province –the last Lord Baltimore received £13,000 a year (more than $250,000 in today's American dollars) –it is perhaps not remarkable that the later Calverts regarded Maryland primarily as a source of revenue. To his credit, however, Frederick appointed as successive governors of Maryland two men of ability, Horatio Sharpe and Robert Eden, the latter his brother-in-law who served till 1776.

Frederick's marriage to the Lady Diana Egerton, daughter of the Duke of Bridgewater, was cut short by her death in 1758. When he died in 1771, without ever having visited Maryland, he left no legitimate children, but devised the province to a son, Henry Harford, from whom Harford County takes its name. Though there was a contest in England over the matter, Harford was recognized as proprietary by the Maryland Assembly, which after the Revolution of 1776, voted $50,000 to extinguish his claims. Thus severed the last tie between Maryland and the Calvert blood.

These portraits passed to Frederick's sister, Caroline, ancestress of Sir Timothy Eden and his brother, Sir Anthony Eden, British prime minister, 1955-1957.

The likeness of Frederick is the work of Johann Ludwig Tietz of Germany, who painted persons prominent in the Hanoverian court, including Queen Charlotte. The paper, ink, and books indicated his Lordship's literary inclinations.