compute a scalar triple product