Leslie Rumble argues that Joseph directly took inspiration from View of the Hebrews.
Leslie Rumble, "The Book of Mormon," The Homiletic and Pastoral Review, January 1960, 338–339
The book was not solely a product of his imagination. There were more than enough ideas ready at hand for him to draw upon as he went along.
The main theme of his narrative is to be found in Ethan Smith’s View of the Hebrews; or the Ten Tribes of Israel in America, published in 1823, four years before Joseph claimed to have unearthed the golden plates. Ethan Smith, a Vermont clergyman, maintained that the ancient Americans were Jewish in origin, that they had built up a great civilization, that they had their own inspired prophets, and that they were destined by God to gather the last remnants of Israel in the latter days, all being converted to Christianity. He gave long citations from Isaiah, and placed great reliance upon the prophecy that the "stick of Judah” and the “stick of Joseph” would eventually be united.
In developing this story there was room for many incidents invented by Joseph himself, but these were chiefly of various adventures and wars which are not beyond the capacity of any highly-imaginative youth of his years.