Signal Processing

$ % macros % MathJax \newcommand{\bigtimes}{\mathop{\vcenter{\hbox{$\Huge\times\normalsize$}}}} % prefix with \! and postfix with \!\! % sized grouping symbols \renewcommand {\aa} [1] {\left\langle {#1} \right\rangle} % <> angle brackets \newcommand {\bb} [1] {\left[ {#1} \right]} % [] brackets \newcommand {\cc} [1] {\left\{ {#1} \right\}} % {} curly braces \newcommand {\mm} [1] {\lVert {#1} \rVert} % || || double norm bars \newcommand {\nn} [1] {\lvert {#1} \rvert} % || norm bars \newcommand {\pp} [1] {\left( {#1} \right)} % () parentheses % unit \newcommand {\unit} [1] {\bb{\mathrm{#1}}} % measurement unit % math \newcommand {\fn} [1] {\mathrm{#1}} % function name % sets \newcommand {\setZ} {\mathbb{Z}} \newcommand {\setQ} {\mathbb{Q}} \newcommand {\setR} {\mathbb{R}} \newcommand {\setC} {\mathbb{C}} % arithmetic \newcommand {\q} [2] {\frac{#1}{#2}} % quotient, because fuck \frac % trig \newcommand {\acos} {\mathrm{acos}} % \mathrm{???}^{-1} is misleading \newcommand {\asin} {\mathrm{asin}} \newcommand {\atan} {\mathrm{atan}} \newcommand {\atantwo} {\mathrm{atan2}} % at angle = atan2(y, x) \newcommand {\asec} {\mathrm{asec}} \newcommand {\acsc} {\mathrm{acsc}} \newcommand {\acot} {\mathrm{acot}} % complex numbers \newcommand {\z} [1] {\tilde{#1}} \newcommand {\conj} [1] {{#1}^\ast} \renewcommand {\Re} {\mathfrak{Re}} % real part \renewcommand {\Im} {\mathrm{I}\mathfrak{m}} % imaginary part % quaternions \newcommand {\quat} [1] {\tilde{\mathbf{#1}}} % quaternion symbol \newcommand {\uquat} [1] {\check{\mathbf{#1}}} % versor symbol \newcommand {\gquat} [1] {\tilde{\boldsymbol{#1}}} % greek quaternion symbol \newcommand {\guquat}[1] {\check{\boldsymbol{#1}}} % greek versor symbol % vectors \renewcommand {\vec} [1] {\mathbf{#1}} % vector symbol \newcommand {\uvec} [1] {\hat{\mathbf{#1}}} % unit vector symbol \newcommand {\gvec} [1] {\boldsymbol{#1}} % greek vector symbol \newcommand {\guvec} [1] {\hat{\boldsymbol{#1}}} % greek unit vector symbol % special math vectors \renewcommand {\r} {\vec{r}} % r vector [m] \newcommand {\R} {\vec{R}} % R = r - r' difference vector [m] \newcommand {\ur} {\uvec{r}} % r unit vector [#] \newcommand {\uR} {\uvec{R}} % R unit vector [#] \newcommand {\ux} {\uvec{x}} % x unit vector [#] \newcommand {\uy} {\uvec{y}} % y unit vector [#] \newcommand {\uz} {\uvec{z}} % z unit vector [#] \newcommand {\urho} {\guvec{\rho}} % rho unit vector [#] \newcommand {\utheta} {\guvec{\theta}} % theta unit vector [#] \newcommand {\uphi} {\guvec{\phi}} % phi unit vector [#] \newcommand {\un} {\uvec{n}} % unit normal vector [#] % vector operations \newcommand {\inner} [2] {\left\langle {#1} , {#2} \right\rangle} % <,> \newcommand {\outer} [2] {{#1} \otimes {#2}} \newcommand {\norm} [1] {\mm{#1}} \renewcommand {\dot} {\cdot} % dot product \newcommand {\cross} {\times} % cross product % matrices \newcommand {\mat} [1] {\mathbf{#1}} % matrix symbol \newcommand {\gmat} [1] {\boldsymbol{#1}} % greek matrix symbol % ordinary derivatives \newcommand {\od} [2] {\q{d #1}{d #2}} % ordinary derivative \newcommand {\odn} [3] {\q{d^{#3}{#1}}{d{#2}^{#3}}} % nth order od \newcommand {\odt} [1] {\q{d{#1}}{dt}} % time od % partial derivatives \newcommand {\de} {\partial} % partial symbol \newcommand {\pd} [2] {\q{\de{#1}}{\de{#2}}} % partial derivative \newcommand {\pdn} [3] {\q{\de^{#3}{#1}}{\de{#2}^{#3}}} % nth order pd \newcommand {\pdd} [3] {\q{\de^2{#1}}{\de{#2}\de{#3}}} % 2nd order mixed pd \newcommand {\pdt} [1] {\q{\de{#1}}{\de{t}}} % time pd % vector derivatives \newcommand {\del} {\nabla} % del \newcommand {\grad} {\del} % gradient \renewcommand {\div} {\del\dot} % divergence \newcommand {\curl} {\del\cross} % curl % differential vectors \newcommand {\dL} {d\vec{L}} % differential vector length [m] \newcommand {\dS} {d\vec{S}} % differential vector surface area [m^2] % special functions \newcommand {\Hn} [2] {H^{(#1)}_{#2}} % nth order Hankel function \newcommand {\hn} [2] {H^{(#1)}_{#2}} % nth order spherical Hankel function % transforms \newcommand {\FT} {\mathcal{F}} % fourier transform \newcommand {\IFT} {\FT^{-1}} % inverse fourier transform % signal processing \newcommand {\conv} [2] {{#1}\ast{#2}} % convolution \newcommand {\corr} [2] {{#1}\star{#2}} % correlation % abstract algebra \newcommand {\lie} [1] {\mathfrak{#1}} % lie algebra % other \renewcommand {\d} {\delta} % optimization %\DeclareMathOperator* {\argmin} {arg\,min} %\DeclareMathOperator* {\argmax} {arg\,max} \newcommand {\argmin} {\fn{arg\,min}} \newcommand {\argmax} {\fn{arg\,max}} % waves \renewcommand {\l} {\lambda} % wavelength [m] \renewcommand {\k} {\vec{k}} % wavevector [rad/m] \newcommand {\uk} {\uvec{k}} % unit wavevector [#] \newcommand {\w} {\omega} % angular frequency [rad/s] \renewcommand {\TH} {e^{j \w t}} % engineering time-harmonic function [#] % classical mechanics \newcommand {\F} {\vec{F}} % force [N] \newcommand {\p} {\vec{p}} % momentum [kg m/s] % \r % position [m], aliased \renewcommand {\v} {\vec{v}} % velocity vector [m/s] \renewcommand {\a} {\vec{a}} % acceleration [m/s^2] \newcommand {\vGamma} {\gvec{\Gamma}} % torque [N m] \renewcommand {\L} {\vec{L}} % angular momentum [kg m^2 / s] \newcommand {\mI} {\mat{I}} % moment of inertia tensor [kg m^2/rad] \newcommand {\vw} {\gvec{\omega}} % angular velocity vector [rad/s] \newcommand {\valpha} {\gvec{\alpha}} % angular acceleration vector [rad/s^2] % electromagnetics % fields \newcommand {\E} {\vec{E}} % electric field intensity [V/m] \renewcommand {\H} {\vec{H}} % magnetic field intensity [A/m] \newcommand {\D} {\vec{D}} % electric flux density [C/m^2] \newcommand {\B} {\vec{B}} % magnetic flux density [Wb/m^2] % potentials \newcommand {\A} {\vec{A}} % vector potential [Wb/m], [C/m] % \F % electric vector potential [C/m], aliased % sources \newcommand {\I} {\vec{I}} % line current density [A] , [V] \newcommand {\J} {\vec{J}} % surface current density [A/m] , [V/m] \newcommand {\K} {\vec{K}} % volume current density [A/m^2], [V/m^2] % \M % magnetic current [V/m^2], aliased, obsolete % materials \newcommand {\ep} {\epsilon} % permittivity [F/m] % \mu % permeability [H/m], aliased \renewcommand {\P} {\vec{P}} % polarization [C/m^2], [Wb/m^2] % \p % electric dipole moment [C m], aliased \newcommand {\M} {\vec{M}} % magnetization [A/m], [V/m] \newcommand {\m} {\vec{m}} % magnetic dipole moment [A m^2] % power \renewcommand {\S} {\vec{S}} % poynting vector [W/m^2] \newcommand {\Sa} {\aa{\vec{S}}_t} % time averaged poynting vector [W/m^2] % quantum mechanics \newcommand {\bra} [1] {\left\langle {#1} \right|} % <| \newcommand {\ket} [1] {\left| {#1} \right\rangle} % |> \newcommand {\braket} [2] {\left\langle {#1} \middle| {#2} \right\rangle} $

This is a review of signal processing.

Let $N \in \setZ^+$ and $m,n \in [0, N-1]$ $$ f(m) : \setZ \rightarrow \setC $$ periodicity $$ f(m + N) = f(m) $$ This is a vectorspace. Impulse Function $$ \delta(m) \equiv \begin{cases} 0 \quad\text{if}\quad m \neq 0 \\ 1 \quad\text{if}\quad m = 0 \end{cases} $$ $$ f(m) = \sum_{m'=0}^{N-1} \delta(m - m') f(m') $$ Unit Step Function $$ u(m) \equiv \begin{cases} 0 \quad\text{if}\quad m < 0 \\ 1 \quad\text{if}\quad m \geq 0 \end{cases} $$ $$ \delta(m) = u(m) - u(m - 1) $$ Standard Inner Product $$ \inner{f}{g} \equiv \sum_{m=0}^{N-1} \conj{f(m)} g(m) $$ DFT Basis Function $N \in \setZ^+$ and $m,n \in [0, N-1]$ (Unitary convention) $$ b_n(m) = \frac{1}{\sqrt{N}} e^{i \frac{2 \pi n}{N} m} $$ $$ \inner{b_n}{b_{n'}} = \delta_{nn'} $$ where $$ \delta_{nn'} = \begin{cases} 0 \quad\text{if}\quad n \neq n' \\ 1 \quad\text{if}\quad n = n'\end{cases} $$

$$ \inner{b_n}{b_{n'}} = \sum_{m=0}^{N-1} \conj{\bb{\frac{1}{\sqrt{N}} e^{i \frac{2 \pi n}{N} m}}} \bb{\frac{1}{\sqrt{N}} e^{i \frac{2 \pi n'}{N} m}} $$ or $$ \inner{b_n}{b_{n'}} = \frac{1}{N} \sum_{m=0}^{N-1} e^{i \frac{2 \pi [n' - n]}{N} m} $$ if $n = n'$, then $[n' - n] = 0$ and thus all exponentials become $e^0 = 1$ so $$ \inner{b_n}{b_{n}} = \frac{1}{N} \sum_{m=0}^{N-1} 1 = \frac{1}{N} N = 1 $$ otherwise, let $n \neq n'$. to evaluate this sum, a lemma is needed. Lemma. Let $f:\setZ \rightarrow \setC$ and $A, b, c \in \setC$ $$ f(m) = A b^{c m} $$ $$ S = \sum_{m=0}^{N-1} A b^{c m} $$ $$ S = A \bb{1 + b^c + b^{c 2} + \dots + b^{c [N - 2]} + b^{c [N - 1]}} $$ $$ b^c S = A \bb{b^c + b^{c 2} + b^{c 3} + \dots + b^{c [N - 1]} + b^{c N}} $$ $$ S - b^c S = A \bb{1 - b^{c N}} $$ $$ S = A \frac{1 - b^{c N}}{1 - b^c} $$ Apply this by letting $A = 1/N, b = e, c = i 2 \pi [n' - n] / N$. Then $$ \inner{b_n}{b_{n'}} = \frac{1}{N} \frac{1 - e^{i 2 \pi [n' - n]}}{1 - e^{i \frac{2 \pi [n' - n]}{N}}} = \frac{1}{N} \frac{1 - 1}{1 - e^{i \frac{2 \pi [n' - n]}{N}}} = 0 $$ which follows because the denominator is non-zero and $e^{i 2 \pi [n' - n]} = 1$ due to the argument being an integer multiple of $2 \pi$. Combining these cases, you get $$ \inner{b_n}{b_{n'}} = \delta_{nn'} $$

$$ f(m) = \sum_{n=0}^{N-1} b_n(m) c_n $$ $$ c_n = \inner{b_n}{f} $$

$$ \inner{b_n}{f} = \inner{b_n}{\sum_{n'=0}^{N-1} b_{n'} c_{n'}} = \sum_{n'=0}^{N-1} \inner{b_n}{b_{n'}} c_{n'} = \sum_{n'=0}^{N-1} \delta_{nn'} c_{n'} = c_n $$

Linearity $$ T(f(m) a + g(m) b) = T(f(m)) a + T(g(m)) b $$ Shift Invariance (often called time/space invariant) $$ g(m) = T(f(m')) $$ $$ g(m - m_0) = T(f(m' - m_0)) $$ Let $T$ be a linear shift invariant function. Then $$ T(f(m)) = T\pp{\sum_{m'=0}^{N-1} \delta(m - m') f(m')} = \sum_{m'=0}^{N-1} T(\delta(m - m')) f(m') = \sum_{m'=0}^{N-1} h(m - m') f(m') $$ where $h = T(\delta)$.

Convolution $$ [\conv{f}{g}](m) \equiv \sum_{m'=0}^{N-1} f(m - m') g(m') $$

Properties

Linearity $$ \conv{f}{[g + h]} = \conv{f}{g} + \conv{f}{h} $$ $$ \conv{f}{[a g + b h]} = \sum_{m'=0}^{N-1} f(m - m') [a g(m') + b h(m')] = a \sum_{m'} f(m - m') g(m') + b \sum_{m'} f(m - m') h(m') = a [\conv{f}{g}] + b [\conv{f}{h}] $$ Shift Invariance $$ \conv{f}{g} = [\conv{f}{g}](m + m_0) $$ Associativity $$ \conv{[\conv{f}{g}]}{h} = \conv{f}{[\conv{g}{h}]} $$ $$ [\conv{[\conv{f}{g}]}{h}](m) = \sum_{m'=0}^{N-1} \bb{\sum_{m''=0}^{N-1} f(m - m' - m'') g(m'')} h(m') = \ = \conv{f}{[\conv{g}{h}]} $$ $$ d_{m_0} (m) = \delta(m - m_0) $$ $$ [\conv{d_{m_0}}{f}](m) = f(m - m_0) $$ $$ [\conv{f}{g}] = \sum_{m=0}^{N-1} f([m - m_0] - m') g(m') = $$ Commutivity $$ \conv{f}{g} = \conv{g}{f} $$

$$ [\conv{f}{g}](m) = \sum_{m'=0}^{N-1} f(m - m') g(m') $$ Let $m - m' = m''$. Then $$ [\conv{f}{g}](m) = \sum_{m''=0}^{N-1} f(m'') g(m - m'') $$

Eigenfunctions of LTI systems

$$ q_\omega(m) = A e^{i \omega m} $$ $$ [\conv{q_\omega}{h}](m) = \sum_{m'=0}^{N-1} A e^{i \omega [m - m']} h(m') = A e^{i \omega m} \sum_{m'=0}^{N-1} e^{-i \omega m'} h(m') = q_\omega \lambda_\omega $$ where $$ \lambda_\omega = \sum_{m'=0}^{N-1} e^{-i \omega m'} h(m') $$ is the associated eigenvalue of eigenvector $q_\omega$ of the linear function of convolution by $h$. $$ [\conv{h}{}] q_\omega = \lambda_\omega q_\omega $$