-3a^3b^2+9a^2b^2+30ab^2
=-3ab^2(a^2-3a-10)
=-3ab^2(a^2-2a+1-a-11)
=-3ab^2[(a-1)^2-(a+11)]