I am not a physicist, so the following may be my misunderstanding. Symmetry seems theoretically impossible, except at one instant. If there was a perfectly symmetric piece of matter (after rotating or reflecting it around some axis, the set of locations of its atoms would be the same as before, just a possibly different atom in each location), then in the next instant of time, its atoms would move to unpredictable locations by the Heisenberg uncertainty principle (the location and momentum of a particle cannot be simultaneously determined). This is because the locations of the atoms would be known by symmetry in the first instant, thus their momenta unknown.

Symmetry may not provide complete information about the locations of the atoms, but constrains their possible locations. Such an upper bound on the uncertainty about locations puts a lower bound on the uncertainty about momenta. Momentum uncertainty creates location uncertainty in the next instant.

Symmetry is probably an approximation: rotating or reflecting a piece of matter, its atoms are in locations close to the previous locations of its atoms. Again, an upper bound on the location uncertainty about the atoms should put a lower bound on the momentum uncertainty. If the atoms move in uncertain directions, then the approximate location symmetry would be lost at some point in time, both in the future and the past.