hessian matrix 2x^3-3x^2-6xy(x-y-1)