However, as we increase the dimension of a Gaussian random variable, its geometric properties change dramatically. As the dimension $d$ of a standard normall high-dimensional random vector $\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$ increases, fewer randomly generated points (vectors) from this distribution are likely to land close to the mean. Instead, the random variables strongly concentrate around a sphere of radius $\sqrt{d}.$ (Recall also that standard normal variables are spherically symmetric $-$ all random vectors of the same magnitude are equally likely.) In that sense, high-dimensional standard normal random variables behave similarly to uniformly random variables from a high-dimensional sphere or a ball (both of radius $\sqrt{d}$), which is the reason why all three of these distributions feature prominently in the analysis of randomized algorithms, with the analysis often being exchangeable between the three specific distributions. But we are digressing... Let us now formally argue what we claimed above: that high-dimensional (standard normal) Gaussians concentrate around the sphere of radius $\sqrt{d}.$
Recall the definition of the Euclidean (or $\ell_2$) norm of a vector: $\|\mathbf{x}\|_2 = \sqrt{\sum_{i=1}^d x_i^2},$ where $x_1, x_2, \dots, x_d$ are the entries of $\mathbf{x}$, which, as we discussed before has the interpretation of what we intuitively understand as the length of a vector. We prove the following:
Theorem 1-Concentration of high-dimensional Gaussians $-$ Gaussian Annulus Theorem Let $\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I}),$ where the dimension is $d \geq 2.$ Then for any $t \in (0, \sqrt{d}],$ we have $\|\mathbf{z}\|_2 \in [\sqrt{d} - t, \sqrt{d} + t]$ with probability at least $1 - 2 e^{-t^2/8}.$
Proof
To prove the theorem, we use the Chernoff method. You will recall that Chernoff method boils down to finding a moment generating function of a random variable and using Markov's inequality. The question here is: which random variable should we work with?
The somewhat obvious choice is to try to directly bound $\|\mathbf{z}\|_2$, but it is not entirely clear how to do that. However, if we square it, then we get a sum of squares of independent univariate Gaussians: $\|\mathbf{z}\|_2^2 = z_1^2 + z_2^2 + \dots + z_d^2,$ where each $z_i \sim \mathcal{N}(0, 1).$ Observe that $\mathbb{E}[z_i^2] = 1,$ because $z_i \sim \mathcal{N}(0, 1),$ for all $i = 1, 2, \dots, d,$ and it helps to center (make expectation equal to zero) each $z_i^2$ by considering $z_i^2 - 1$. Now we can use independence to conclude that
$$
\mathbb{E}_{\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})}[e^{\lambda (\sum_{i=1}^d(z_i^2 - 1))}] = \mathbb{E}_{z_1 \sim \mathcal{N}(0, 1), \dots, z_d \sim \mathcal{N}(0, 1) }[e^{\lambda (z_1^2-1)}e^{\lambda (z_2^2 - 1)}\cdots e^{\lambda (z_d^2-1)}] = \Big(\mathbb{E}_{z_1 \sim \mathcal{N}(0, 1)}[e^{\lambda (z_1^2-1)}] \Big)^d.
$$
Now we can focus on finding (or bounding) $\mathbb{E}_{z_1 \sim \mathcal{N}(0, 1)}[e^{\lambda (z_1^2-1)}]$. Writing this expectation explicitly by recalling the formula for the pdf of a standard normal random variable, we have
\begin{align}
\mathbb{E}_{z_1 \sim \mathcal{N}(0, 1)}[e^{\lambda (z_1^2 - 1)}] &= \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{+\infty} e^{\lambda (z_1^2 - 1)} e^{-\frac{z_1^2}{2}}dz_1 \\
&= e^{-\lambda}\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{+\infty} e^{-\frac{z_1^2(1 - 2\lambda)}{2}}dz_1.
\end{align}
If $1 - 2\lambda > 0$, then $\frac{\sqrt{1-2\lambda}}{\sqrt{2\pi}} e^{-\frac{z_1^2(1 - 2\lambda)}{2}}$ is the pdf of a Gaussian random variable with mean zero and variance $1/(1-2\lambda).$ If $1 - 2\lambda \leq 0,$ the above integral is infinite. Thus, we conclude that for $\lambda < \frac{1}{2},$ we have $\mathbb{E}_{z_1 \sim \mathcal{N}(0, 1)}[e^{\lambda (z_1^2-1)}] = \frac{e^{-\lambda}}{\sqrt{1-2\lambda}}.$ We could now try applying Markov inequality to $e^{\lambda (\|\mathbf{z}\|_2^2-d)}$ for $\lambda \in (0, 1/2),$ using
$$
\mathbb{E}_{\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})}[e^{\lambda (\|\mathbf{z}\|_2^2-d)}] =
\frac{e^{-\lambda d}}{(1-2\lambda)^{d/2}}.
$$
However, this would only allow us to argue that $\|\mathbf{z}\|_2 \leq c \sqrt{d}$ with high probability for some constant $c > 1,$ which is weaker than what we are trying to prove. (You are encouraged to try this on your own.)
A key insight to prove the stated result is the following inequality that we state here without proof:
$$
\frac{e^{-\lambda}}{\sqrt{1 - 2\lambda}} \leq e^{2\lambda^2}, \quad \forall \lambda: |\lambda|\leq 1/4.
$$
Thus, we can apply Markov's inequality to the random variable $e^{\lambda (\|\mathbf{z}\|_2^2 - d)}$ (in the same way we have used the Chernoff method) but now we need to ensure that $\lambda \leq 1/4.$ Note that by what we have shown above, we have that for $|\lambda| \leq 1/4,$
$$
\mathbb{E}_{\mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I})}[e^{\lambda (\|\mathbf{z}\|_2^2-d)}] \leq e^{2d\lambda^2}.
$$
Before proceeding, recall (from the statement of the theorem) that we are interested in bounding the probability that $|\|\mathbf{z}\|_2 - \sqrt{d}| \geq t$ for $t \in (0, \sqrt{d}]$. Multiplying both sides by $\|\mathbf{z}\|_2 + \sqrt{d}$, this is the same as bounding the probability that $|\|\mathbf{z}\|_2^2 - {d}| \geq t(\|\mathbf{z}\|_2 + \sqrt{d}).$ Because this inequality is guaranteed to hold whenever $|\|\mathbf{z}\|_2^2 - {d}| \geq t\sqrt{d},$ it suffices to bound above the probability that $|\|\mathbf{z}\|_2^2 - {d}| \geq t\sqrt{d}.$
It remains now to apply Markov inequality to both $e^{\lambda(\|\mathbf{z}\|_2^2 - d)} \geq e^{\lambda t\sqrt{d}}$ and $e^{-\lambda(\|\mathbf{z}\|_2^2 - d)} \geq e^{\lambda t\sqrt{d}}$ for $\lambda \in (0, 1/4]$ to bound the probabilities that $\|\mathbf{z}\|_2^2 - d \geq t\sqrt{d}$ and $-(\|\mathbf{z}\|_2^2 - d) \geq t\sqrt{d}$, respectively, and obtain a two-sided tail bound. In particular, we have that
\begin{align}
\mathbb{P}[|\|\mathbf{z}\|_2^2 - {d}| \geq t\sqrt{d}] &= \mathbb{P}[e^{\lambda(\|\mathbf{z}\|_2^2 - d)} \geq e^{\lambda t\sqrt{d}}] + \mathbb{P}[e^{-\lambda(\|\mathbf{z}\|_2^2 - d)} \geq e^{\lambda t\sqrt{d}}] \\
&\leq 2 e^{2d\lambda^2 - \lambda t\sqrt{d}}.
\end{align}
As we saw before, the above probability is minimized when $2d\lambda^2 - \lambda t\sqrt{d}$ is minimized, which happens for $\lambda = \frac{t}{4\sqrt{d}}.$ Since $t \in (0, \sqrt{d}],$ this choice also guarantees that $\lambda \in (0, 1/4],$ so the above bound applies. Plugging in this choice, we get
$$
\mathbb{P}[|\|\mathbf{z}\|_2^2 - {d}| \geq t\sqrt{d}] \leq 2 e^{-t^2/8},
$$
which is exactly what was claimed in the theorem statement. $\blacksquare$
