Vaughan Trammell-designed contemporary sells for $5.7 million