Background: The divergent glacial histories of southern and northern Europe affect present-day species diversity at coarse-grained scales in these two regions, but do these effects also penetrate to the more fine-grained scales of local communities? Methodology/Principal Findings: We carried out a cross-scale analysis to address this question for vascular plants in two mountain regions, the Alps in southern Europe and the Scandes in northern Europe, using environmentally paired vegetation plots in the two regions (n = 403 in each region) to quantify four diversity components: (i) total number of species occurring in a region (total gamma-diversity), (ii) number of species that could occur in a target plot after environmental filtering (habitat-specific gamma-diversity), (iii) pair-wise species compositional turnover between plots (plot-to-plot beta-diversity) and (iv) number of species present per plot (plot alpha-diversity). We found strong region effects on total gamma-diversity, habitat-specific gamma-diversity and plot-to-plot beta-diversity, with a greater diversity in the Alps even towards distances smaller than 50 m between plots. In contrast, there was a slightly greater plot alpha-diversity in the Scandes, but with a tendency towards contrasting region effects on high and low soil-acidity plots. Conclusions/Significance: We conclude that there are strong regional differences between coarse-grained (landscape-to regional-scale) diversity components of the flora in the Alps and the Scandes mountain ranges, but that these differences do not necessarily penetrate to the finest-grained (plot-scale) diversity component, at least not on acidic soils. Our findings are consistent with the contrasting regional Quaternary histories, but we also consider alternative explanatory models. Notably, ecological sorting and habitat connectivity may play a role in the unexpected limited or reversed region effect on plot alpha-diversity, and may also affect the larger-scale diversity components. For instance, plot connectivity and/or selection for high dispersal ability may increase plot alpha-diversity and compensate for low total gamma-diversity.