D. Michael Quinn discusses the origin of "Mormon"; argues it may come from the Scottish name "Moorman" or from "Mormo" (a spectre).
D. Michael Quinn, Early Mormonism and the Magic World View, rev. ed. (Salt Lake City: Signature Books, 1998), 197
The name “Mormon” has several non-magic parallels. It is one of the forms of the Scottish name for “the official in charge of the cattle on the marsh or waste ground.” It is phonetically close to “Moorman” or “Moormen,” a designation for the Moors. The common view of Smith’s time was that the Moors were special devotees of the occult sciences.
More aligned with the spirit references in the Book of Mormon was LeLoyer’s seventeenth-century A Treatise of Specters. It identified “Mormo” as a spirit in the form of a woman that “terrified little children.” Less than fifty years later, a published sermon referred to the “frightful apparitions of ghosts and Mormos.” Folk language employed “Mormo” to mean “a spectre,” but dictionaries available in Palmyra in the 1820s defined “Mormo” as “bug bear; false terror.” A year after the publication of the Book of Mormon, a caustic newspaper article claimed such a derivation for Mormon, which was repeated in the era’s most widely circulated anti-Mormon book. Joseph Smith later denied this by using a combination of English and what he understood to be the Egyptian word “mon” to assert that Mormon meant “more good.” Without specifying its meaning of frightful apparition,” a 1948 article in the church’s magazine identified Mormon as one of the books’ names that had been “common to the English language as used in America.”