🔬 Tutorial problems gamma \(\gamma\)#
Note
This problems are designed to help you practice the concepts covered in the lectures. Not all problems may be covered in the tutorial, those left out are for additional practice on your own.
\(\gamma\).1#
Consider an \((n \times n)\) Vandermonde matrix [this one can be named :)] of the form
Show that the determinant of \(V\) is given by
for the cases \(n=2\), \(n=3\) and \(n=4\)
Properties of the determinants help in finding an elegant solution.
Using the fact that the determinant is invariant to Gaussian elementary row operations we perform the following derivation:
(expand along the first row)
(take out common factor from each row)
It is not hard to show (by long division of polynomials and mathematical induction) that
Continuing the derivation:
We can then repeat the same procedure:
subtract the first row from all other
notice regularities in the polynomials
expand the determinant along the first row
take out common factors from each row
It is then obvious how to prove the main formula by mathematical induction.
\(n=2\)
\(n=3\)
\(n=4\)
Follow the same scheme, using
\(x^2-y^2=(x-y)(x+y)\)
\(x^3-y^3=(x-y)(x^2+xy+y^2)\)
\(\gamma\).2#
Consider a function \(f : \mathbb{R}^N \ni x \mapsto x^{T}Bx \in \mathbb{R}\), where \(N \times N\) matrix \(B\) is square but not symmetric.
Show that the same function can be represented as \(x^{T}Ax\) where \(A\) is symmetric.
Given a square matrix \(M\), you can use the identity \(M = \tfrac{1}{2}(M+M') + \tfrac{1}{2}(M-M')\) where the first component is symmetric and the second is not symmetric.
If \(A\) and \(B\) are conformable for matrix multiplication, then \((AB)^{T} = B^T A^T\).
In general a matrix can be decomposed into a symmetric and an anti-symmetric part as follows:
Note that the symmetric part is invariant to the transpose, \(M^T_s=M_s\) while the antisymmetric part changes its sign under transposition: \(M^T_a=-M_a\)
Let us examine the purely antisymmetric matrix
Which implies that \(q=0\) has to hold true for all \(x\)!
Then, it is clear that the \(A = \frac{1}{2}(B+B^T)\) solves the problem.
\(\gamma\).3#
In which direction should one move from a given point in order to increase the value of the function most rapidly:
\(\quad\) \(f(x,y) = 4x^2y\) from the point \((2,3)\)
\(\quad\) \(f(x,y) = y^2 e^{3x}\) from the point \((0,3)\)
Present your answer as a vector of length 1.
[Simon and Blume, 1994]: Exercises 14.18, 14.19
Review the definition and facts about the gradient of a multivariate functions.
The direction of most rapid ascent of the multivariate function is given by its gradient.
1.
We need to make sure that the direction vector had length 1, so we divide by its norm
The direction vector is then \((48, 16)/16\sqrt{10} = (3/\sqrt{10}, 1/\sqrt{10})\)
2.
We need to make sure that the direction vector had length 1, so we divide by its norm
The direction vector is then \((27, 6)/3\sqrt{85} = (9/\sqrt{85}, 2/\sqrt{85})\)
\(\gamma\).4#
A critical point of a multivariate function is the point at which all partial derivatives are zero.
Compute the critical points of the following functions:
\(\quad\) \(x^4+x^2-6xy + 3y^2\)
\(\quad\) \(x^2-6xy+2y^2+10x+2y-5\)
\(\quad\) \(xy^2+x^3y-xy\)
\(\quad\) \(3x^4+3x^2y-y^3\)
\(\quad\) \(x^2+6xy+y^2-3yz+4z^2-10x-5y-21z\)
\(\quad\) \((x^2+2y^2+3z^2) e^{-(x^2+y^2+z^2)}\)
[Simon and Blume, 1994]: Exercises 17.1, 17.2
Solution algorithm:
compute partial derivatives
solve the system of equations formed by the partial derivatives equalized to zeros
Correct answers:
\((0,0), (1,1), (-1,-1)\)
\((13/7,16/7)\)
\((0,0), (0,1), (1,0), (-1,0), (1/\sqrt{5},2/5), (-1/\sqrt{5},2/5)\)
\((0,0), (1/2,-1/2), (-1/2,-1/2)\)
\((2,1,3)\)
\((0,0,0), (1,0,0), (-1,0,0), (0,1,0), (0,-1,0), (0,0,1), (0,0,-1)\)
\(\gamma\).5#
Consider a function \(f : \mathbb{R}^N \ni x \mapsto x'Ax \in \mathbb{R}\), where \(N \times N\) matrix \(A\) is symmetric.
Using the product rule of multivariate calculus, derive the gradient and Hessian of \(f\). Make sure that all multiplied vectors and matrices are conformable.
You can assume that \(x\) is a column vector, and that any vector function of \(x\) is also a column vector.
A possible answer:
Represent the quadratic form as a dot product of two functions \(f(x) = x' A x = h(x) \cdot g(x)\), where \(h(x) = x\) and \(g(x) = A x\). Then \(Dh(x) = I\) (identity matrix) and \(Dg(x) = A\).
The last Jacobian can be easity derived by representing matrix multiplication as a linear combination of columns. Differentiating with respect to each element of \(x\) then yields a Jacobian composed of columns of matrix \(A\), therefore equal to it.
Applying the dot product rule of differentiation we have
The last transformation is transpose of a product + utilizing symmetry of \(A\).
The final answer is the \(1 \times N\) matrix (row vector)
\(\gamma\).6#
Compute directional derivative of \(f(x,y) = xy^2 + x^3y\) at the point \((4,-2)\) in the direction of the vector \((1/\sqrt{10},3/\sqrt{10})\).
Proceed in two different ways:
first, using the definition of the directional derivative, write down a function \(g \colon \mathbb{R} \to \mathbb{R}\) of \(h\) given as a slice of the original function \(f(x,y)\) through the given point in the given direction; then differentiate this function and compute the derivative at \(h=0\)
second, use the gradient formula; and verify that the same answer is obtained
[Simon and Blume, 1994]: Exercise 14.20
Follow the example in the lecture notes.
Denote \((x_0,y_0) = (4,-2)\), and \(v = (1/\sqrt{10},3/\sqrt{10})\).
Note that \(\|v\| = \sqrt{(1/\sqrt{10})^2+(3/\sqrt{10})^2} = (1+3^2)/10 = 1\).
First approach
Let \(g(h)\) denote the univariate function of \(h\) obtained by slicing the function \(f(x,y)\) through the point \((x_0,y_0)\) in the direction of the vector \(v\):
We have
Evaluating the last expression at the given \((x_0,y_0)\) and \(v\) we get
Second approach
Compute the partial derivatives and the gradient