Fencing is an important and common grassland management strategy for restoring areas degraded by grazing. However, experimental evidence regarding the differences between the effects of short-term and long-term fencing on grassland structure remains insufficient, and the mechanisms by which fencing influences plant diversity and community structure are not yet fully understood. A long-term fencing experiment was conducted in a Tibetan meadow. We investigated Hill diversity, aboveground net primary productivity (ANPP), plant community height, species richness, and the biomass and height of four plant functional groups (including grasses, forbs, legumes, and sedges) after 1, 2, 3, and 13 years of fencing. This allowed us to compare the differing impacts of fencing on diversity and community structure over time. Additionally, we collected data on environmental factors such as soil compaction, soil moisture, soil nutrients, and plant litter biomass to elucidate the effects of fencing on plant communities. Our results indicated that short-term fencing increased Hill diversity within plant communities. Conversely, long-term fencing significantly decreased Hill diversity in alpine meadows. Furthermore, fencing reduced aboveground net primary productivity across all four assessment periods. Plant diversity exhibited a positive correlation with soil compaction and soil moisture, but a negative correlation with community height, litter biomass, and litter height. Fencing primarily influenced the diversity of litter biomass, community height, and soil nitrate nitrogen content across all biotic and abiotic factors. Our findings demonstrate that the effects of short-term and long-term fencing on plant communities differ markedly. Long-term fencing can adversely affect plant communities through litter accumulation and alterations in soil physical properties; therefore, the duration of fencing must be considered in grassland restoration efforts.