The modern evolutionary synthesis suggests that both environmental variation and genetic diversity are critical determinants of pathogen success. However, the relative contribution of these two sources of variation is not routinely measured. To estimate the relative contribution of plasticity and genetic diversity for virulence-associated phenotypes in a generalist plant pathogen, we grew a population of 15 isolates of Botrytis cinerea from throughout the world, under a variety of in vitro and in planta conditions. Under in planta conditions, phenotypic differences between the isolates were determined by the combination of genotypic variation within the pathogen and environmental variation. In contrast, phenotypic differences between the isolates under in vitro conditions were predominantly determined by genetic variation in the pathogen. Using a correlation network approach, we link the phenotypic variation under in vitro experimental conditions to phenotypic variation during plant infection. This study indicates that there is a high level of phenotypic variation within B. cinerea that is controlled by a mixture of genetic variation, environment, and genotype × environment. This argues that future experiments into the pathogenicity of B. cinerea must account for the genetic and environmental variation within the pathogen to better sample the potential phenotypic space of the pathogen.