The heavy metals, including cadmium (Cd), chromium (Cr), copper (Cu), mercury (Hg), nickel (Ni), lead (Pb), zinc (Zn), and the metalloid arsenic (As) were detected in surface and core soil samples collected from a tobacco growing region in Shandong Peninsula on the east coast of China to evaluate their pollution levels, ecological and health risks, and to analyze their spatial and vertical distributions. The heavy metal sources were identified quantitatively using the positive matrix factorization (PMF) receptor model. In accordance, most of the soils did not have accumulations and were not contaminated by As, Cr, Cu, Ni, Pb, and Zn. High accumulations of Cd and Hg occurred in the soils, posing an ecological risk to the local agricultural environment, while Cr and Ni levels presented a carcinogenic health risk to humans. Four main sources of heavy metals in the soils were identified. Correspondingly Ni and Cr were mainly originated from natural sources, Hg from coal combustion, Cd from agricultural practices, Cu, Pb, and Zn from agricultural practices and industrial activities, and As from industrial activities.