Looking at your approach in the damage system, the idea arises to make additional divisions in the geometry. If it is damaged, just delete parts of the polygon. You can use math to recalculate vertexes for smooth shapes, and when combined, you can get almost any shape without blurring the texture.
For example, you can make a grid of hexes and remove hexes when damaged, it won’t look bad.