Michael's Wiki

For univariate Gaussian binary classification: Optimal decision regions are defined by decision boundaries in $x$ where $P(c_1)P(x|\theta_1) = P(c_2)P(x|\theta_2)$.

<latex> \begin{align*}

• P(c_1)P(x|\theta_1) &= P(c_2)P(x|\theta_2)

P(c_1)\frac{1}{\sqrt{2\pi}\sigma_1} e^{-\frac{1}{2\sigma_1^2}(\mu_1-x)^2} &=

• P(c_2)\frac{1}{\sqrt{2\pi}\sigma_2} e^{-\frac{1}{2\sigma_2^2}(\mu_2-x)^2}

\end{align*} </latex>

Cancel common terms and take the log

<latex> \begin{align*}
P(c_1)\sigma_2^2 e^{-\frac{1}{2\sigma_1^2}(\mu_1-x)^2} &=

• P(c_2)\sigma_1^2 e^{-\frac{1}{2\sigma_2^2}(\mu_2-x)^2}

\log P(c_1) + \log \sigma_2 - \frac{1}{2\sigma_1^2}(\mu_1-x)^2 &=

• \log P(c_2) + \log \sigma_1 - \frac{1}{2\sigma_2^2}(\mu_2-x)^2

\frac{1}{2\sigma_2^2}(\mu_2-x)^2 - \frac{1}{2\sigma_1^2}(\mu_1-x)^2&=

• \log P(c_2) - \log P(c_1) + \log \sigma_1 - \log \sigma_2

\frac{1}{\sigma_2^2}(\mu_2-x)^2 - \frac{1}{\sigma_1^2}(\mu_1-x)^2 &=

• 2 \log \frac{P(c_2) \sigma_1 }{ P(c_1) \sigma_2 }

\sigma_1^2(\mu_2-x)^2 - \sigma_2^2(\mu_1-x)^2 &=

• 2 \sigma_1^2 \sigma_2^2 \log \frac{P(c_2) \sigma_1 }{ P(c_1) \sigma_2 }

\sigma_1^2(\mu_2^2 - 2\mu_2x + x^2) - \sigma_2^2(\mu_1^2 - 2x\mu_1 + x^2) &=

• 2 \sigma_1^2 \sigma_2^2 \log \frac{P(c_2) \sigma_1 }{ P(c_1) \sigma_2 }

\sigma_1^2\mu_2^2 - 2\sigma_1^2\mu_2x + \sigma_1^2x^2 - \sigma_2^2\mu_1^2 + 2\sigma_2^2\mu_1x - \sigma_2^2x^2&=

• 2 \sigma_1^2 \sigma_2^2 \log \frac{P(c_2) \sigma_1 }{ P(c_1) \sigma_2 }

\sigma_1^2x^2 - \sigma_2^2x^2 + 2\sigma_2^2\mu_1x - 2\sigma_1^2\mu_2x - \sigma_2^2\mu_1^2 + \sigma_1^2\mu_2^2 &=

• 2 \sigma_1^2 \sigma_2^2 \log \frac{P(c_2) \sigma_1 }{ P(c_1) \sigma_2 }

\end{align*} </latex>

Use the quadratic formula:

<latex> \begin{align*}

• 0 &= (\sigma_1^2 - \sigma_2^2)x^2 + \left(2(\sigma_2^2\mu_1 - \sigma_1^2\mu_2) \right) x
• + \sigma_1^2\mu_2^2 - \sigma_2^2\mu_1^2 - 2 \sigma_1^2 \sigma_2^2 \log \frac{P(c_2) \sigma_1 }{ P(c_1) \sigma_2 }

x &= \frac{-b \pm \sqrt(b^2-4ac)}{2a} \end{align*} \begin{align*}

• a &= (\sigma_1^2 - \sigma_2^2)

b &= 2(\sigma_2^2\mu_1 - \sigma_1^2\mu_2)
c &= \sigma_1^2\mu_2^2 - \sigma_2^2\mu_1^2 - 2 \sigma_1^2 \sigma_2^2 \log \frac{P(c_2) \sigma_1 }{ P(c_1) \sigma_2 } \end{align*} </latex>