trigonometry - Expressing $costheta - sqrt{3}sintheta = rsin(theta - alpha)$

My book explains that $a\cos\theta + b\sin\theta$ is a sine (or cosine) graph with a particular amplitude/shift (i.e. $r\sin(\theta + \alpha)$) and shows me some steps to solve for $r$ and $\alpha$:

$$r\sin(\theta + \alpha) \equiv a\cos\theta + b\sin\theta$$

$$\Rightarrow r\sin\theta\cos\alpha + r\cos\theta\sin\alpha \equiv a\cos\theta + b\sin\theta$$

I see the basic trig identity $\sin(a + b) \equiv \sin a\cos b + \sin b\cos a$ is being used, and the coefficients $a$ and $b$ are easily identified as:

$$r\sin\alpha = a$$
$$r\cos\alpha = b$$

Then the book squares and adds the equations:

$$r^2(\sin^2\alpha + \cos^2\alpha) = a^2 + b^2 \Rightarrow r = \sqrt{a^2 + b^2}$$

I see that the basic identity $\sin^2\alpha + \cos^2\alpha \equiv 1$ is being used here.

All the examples in the book involve $r$ being a positive quantity.

An exercise asks me to express $\cos\theta - \sqrt{3}\sin\theta$ in the form $r\sin(\theta - \alpha)$. I tried this and ran into problems as the squaring and square rooting process only ever produces a positive $r$, but the answer is $-2\sin(\theta - \frac{1}{6}\pi)$.

My workings:

$\cos\theta - \sqrt{3}\sin\theta = r\sin(\theta - \alpha) = r\sin\theta\cos\alpha - r\sin\alpha\cos\theta$

$\Rightarrow r\cos\alpha = -\sqrt{3}$, $r\sin\alpha = -1$

$\Rightarrow r^2\cos^2\alpha = 3$, $r^2\sin^2\alpha = 1 => r^2(\cos^2\alpha + \sin^2\alpha) = 4$

$\Rightarrow r^2 = 4 \Rightarrow r = \pm2$.

My question is how should the process in the book be refined so one knows whether $r$ should be positive or negative?

(sorry if this is a bit long winded for such a basic question but I thought showing what I do and don't know might get me an answer targeted at my simple level!).

Answer

In your specific case you get either
$$\sin\alpha = \frac{1}{2}\text{ and } \cos\alpha=-\frac{\sqrt{3}}{2}\text{ for } r=2$$
or
$$\sin\alpha = -\frac{1}{2}\text{ and } \cos\alpha=\frac{\sqrt{3}}{2}\text{ for } r=-2$$
In the first case you get $\alpha=\frac{5\pi}{6}$, and in the second case you get $\alpha=-\frac{\pi}{6}$. (I use the function $r\sin(\theta + \alpha)$ with a positive sign of $\alpha$). You have to be careful when computing $\alpha$ because you usually cannot simply take the principle branch of the functions $\arcsin(x)$ and $\arccos(x)$. You have to choose a value that satisfies both equations (the one for $\sin(\alpha)$ and the one for $\cos(\alpha)$).