I actually missed my A4 after trading in for the WRX. I don't miss the performance on the A4 but the brake system and the quieter ride. Also, the paint on the A4 is much better quality when stuff on the road got on your car (small rocks...) the WRX easily get scratched by those. Also, driving fast on a wet road, I feel more comfortable w/ the Audi for stability/handling. don't mean to praise the Audi but these are the facts since I own both (well, used to).
On the other hand, everyone knows price/performance is the main reason most Audi owners trading in theirs for the WRX.