5xy^2-[3xy^2-(4xy^2-2x^y)]+x^2y-xy^2
=5xy^2-3xy^2+(4xy^2-2x^y)+x^2y-xy^2
=5xy^2-3xy^2+4xy^2-2x^y+x^2y-xy^2
=5xy^2-x^2y