If large numbers of plants are built, they will eventually pose some problems, even in the desert. They could take up immense amounts of land and damage the environment. Already, building a plant in California requires hiring a licensed tortoise wrangler to capture and relocate endangered desert tortoises.
Often in desert construction the entire lot is bulldozed for convenience and it doesn't have to be. This applies to one acre lots (from personal experience) to larger plots as mentioned. The compressed surface often only allows the invasive non-native Russian thistle (aka tumbleweed) to grow. I've never seen the dominant native vegetation in the Mojave, the creosote bush, grow back in disturbed plots.