y=cos^2x-sin^2-√3cos(3π/2+2x)+1
=cos2x-√3sin2x+1
=2cos(2x+π/3)+1
当2kπ+π/2≤2x+π/3≤2kπ+3π/2时,函数单调递减
2kπ+π/2≤2x+π/3≤2kπ+3π/2
2kπ+π/6≤2x≤2kπ+7π/6
kπ+π/12≤x≤kπ+7π/12
所以单调递减区间：[kπ+π/12,kπ+7π/12],k∈Z
最大值：ymax=3
最小值：ymin=-1