This is a common problem with the A3/S3 armrest button, and after breaking on me for over 4 years now I've decided to finally fix it (with the help of some buddies on the net). Follow the simple instructions below and you should be happy.

 1) Remove the 4 small screws "hidden" in the glove box fuzz.

2) Locate where the clip has snapped, the two small  plastic bits above are the ones that have broken off and need fixing.

3) drill out clean holes where the plastic has broken. use a 4mm dia drill head.

4) Place screws through the two new holes and secure in place using adhesive (I used superglue) but you could use some JB Weld epoxy or something like that.

5) make sure when you refit the steel release pieces on either side of the armrest are sandwiched between the new metal bits and the plastic bits that haven’t broken as in the pic:

That's it you're done, now enjoy the armrest with the assurance it'll never break again (hopefully)