Author: marmara

Differentiation and integration of functions. Theorem:  Derivative of unit step function is dirac delta $$\begin{eqnarray} \frac{d}{dt}U(t) = \delta(t) \end{eqnarray}$$ Proof: Applying the definition of derivative…

Periodic Dirac Delta Functions Consider the functions \begin{eqnarray} P_1(t) &=& \frac{\sin[(N+\frac{1}{2}) t]}{\sin(\frac{t}{2})} \\ P_2(t) &=& \frac{\cos[(N+\frac{1}{2}) t]}{\sin(\frac{t}{2})} \\ \end{eqnarray} $P_1(t)$ is periodic with a period…

Below, we will derive the backprop rules for a convolutional layer. As this derivation is complex compared to the fully connected one, we will do…

Better optimization algorithms These perform progressively better than simple gradient descent. In the following, superscript $i$ on $v^i$ denotes $i$th component of the vector $v$.…

Consider the forced LTI differential equation \begin{eqnarray} \frac{d^n}{dt^n} y(t) + a_{n-1} \frac{d^{n-1}}{dt^{n-1}}y(t)  + \ldots +a_2\frac{d^2}{dt^2}y(t)+ a_1 \frac{d}{dt}y(t) + a_0 y(t) = f(t) \end{eqnarray} Our aim…

An nxn matrix always has n eigenvalues, counting multiplicities. While these eigenvalues may repeat, their total count, including repetitions, always sums to n. However, an…

Examples of Differential Equations Spring-mass systems Population increase in bacteria Lotke-Volterra Logistics equation Planetary Orbits Rocket Equation Electric motor Car power equation Military encounter Lorenz…

In this section, we will only be interestd in LTI difference equations and ignore the more general cases. Definition:  Define the operator $\Delta$ as \begin{eqnarray}…