Vinegar's acid chemically reacts with the natural stone, causing dullness, burning, or etching, and the acid eats away at the sealant, exposing stones underneath to wear and tear.
It further compromises the surface's integrity to cause chips, pockmarks, and cracks. So, for pavers made from anything like cement, brick, limestone, or clay, vinegar is a no-go.