For people it is mostly the car accidents they cause and the millions of dollars in damages and even lives lost in those accidents. But environmentally, when populations are as high as they are now, they strip forests of wildflowers and most vegetative growth below about 5 feet. Forests have no upcoming generations of trees as they never get very large before the deer strip them of leaves and they die.
But even in the rural but built up areas, where there is enough grass and low vegetation to feed a thundering herd of deer, they manage to find a way to be a bugger. Here is a deer stretching hard to reach that last apple that is within the deer browse zone, right in the back yard of a house, with another house nearby visible in the photo.
I like deer, but here in the eastern US we just have way too darn many of the things.