Non-native pests, climate change, and their interactions are likely to alter relationships between trees and tree-associated organisms with consequences for forest health. To understand and predict such changes, factors structuring tree-associated communities need to be determined. Here, we analysed the data consisting of records of insects and fungi collected from dormant twigs from 155 tree species at 51 botanical gardens or arboreta in 32 countries. Generalized dissimilarity models revealed similar relative importance of studied climatic, host-related and geographic factors on differences in tree-associated communities. Mean annual temperature, phylogenetic distance between hosts and geographic distance between locations were the major drivers of dissimilarities. The increasing importance of high temperatures on differences in studied communities indicate that climate change could affect tree-associated organisms directly and indirectly through host range shifts. Insect and fungal communities were more similar between closely related vs. distant hosts suggesting that host range shifts may facilitate the emergence of new pests. Moreover, dissimilarities among tree-associated communities increased with geographic distance indicating that human-mediated transport may serve as a pathway of the introductions of new pests. The results of this study highlight the need to limit the establishment of tree pests and increase the resilience of forest ecosystems to changes in climate.