Make homework 1 V0.1

HW1/Solutions_HW1.ipynb

@@ -0,0 +1,153 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "provenance": []
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "source": [
+        "### **Problem 1(5)**:\n",
+        "\n",
+        "Consider the matrix $H(v) = \\hat{1} - 2vv^T$, where $v$ is a unit column vector. What is the rank of the matrix $H(v)$? Prove that it is orthogonal.\n"
+      ],
+      "metadata": {
+        "id": "8rZACDYdX9AU"
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "### **Solution:**\n",
+        "\n",
+        "Let we have a vector $v = (v_{1} v_{2} ... v_{n})^T$ that, by condition, is unit, i.e. $v^Tv=1$.Then the matrix $\\underset{n\\times{}n}H$ has the following form:\n",
+        "\n",
+        "$H(v) = \\hat{1} - 2vv^T = (h_{ij}), 1\\leq{i,j}\\leq{n}$, where $h_{ij} = \\begin{equation}\n",
+        "    \\begin{cases}\n",
+        "      1-2v_{i}^2, i = j\\\\\n",
+        "      -2v_{i}v_{j}, i \\ne j\n",
+        "    \\end{cases}.\n",
+        "\\end{equation}$\n",
+        "\n",
+        "It can be seen that the matrix $H$ is symmetrical. Let us prove that it is orthogonal, which also means that the equality $HH^T = \\hat{1}$. On the other hand, due to symmetry:\n",
+        "\n",
+        "$HH^T = H^2 = (\\hat{1} - 2vv^T)^2 = \\hat{1} - 4vv^T + 4(vv^T)(vv^T) \\overset{v^Tv=1} = \\hat{1} - 4vv^T + 4vv^T = \\hat{1}$.\n",
+        "\n",
+        "It follows that the matrix is orthoganal. It also follows that the $rk(H) = n$.\n",
+        "\n"
+      ],
+      "metadata": {
+        "id": "QLstl0WUan2w"
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "### **Problem 2(10)**  \n",
+        "Prove the following inequalities and provide examples of x and A when they turn into equalities:\n",
+        "\n",
+        "*   $∥x∥_{2} ≤ \\sqrt{m}∥x∥_{∞}$\n",
+        "*   $∥A∥_{∞} ≤ \\sqrt{n}∥A∥_{2}$\n",
+        "\n",
+        "where $x$ is a vector of $m$ components and $A$ is $m × n$ matrix.\n",
+        "\n"
+      ],
+      "metadata": {
+        "id": "imeyAc5D93on"
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "**1)** By definition we have:  \n",
+        "$\\parallel x∥_{2} = \\sqrt{\\sum_{n=1}^{m} x_i^2}$ and $\\parallel x∥_{∞} = \\underset{1 ≤ i ≤ m}\\max{|x_i|}$  \n",
+        "Then the original inequality can be rewritten in the form:  \n",
+        "$\\sqrt{\\sum_{n=1}^{m} x_i^2} ≤ \\sqrt{m}\\underset{1 ≤ i ≤ m}\\max{|x_i|}$  \n",
+        "Let's square both sides:  \n",
+        "$\\sum_{n=1}^{m} x_i^2 ≤ m (\\underset{1 ≤ i ≤ m}\\max{|x_i|})^2 ⇒ \\sum_{n=1}^{m} x_i^2 ≤ m \\underset{1 ≤ i ≤ m}\\max{x_i^2} ⇔ \\sum_{n=1}^{m} x_i^2 ≤ \\sum_{n=1}^{m} \\underset{1 ≤ i ≤ m}\\max{x_i^2}$  \n",
+        "Hence its validity is obvious and it is clear that equality is achieved for arbitrary $m ∈ ℕ$ if $x_{1} = x_{2} = ... = x_{m}$, otherwise if $m = 1$."
+      ],
+      "metadata": {
+        "id": "qP9zP8Da-uZf"
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "### **Problem 3(5)**:\n",
+        "\n",
+        "Assuming $u$ and $v$ are $m$-vectors, consider the matrix $A = 1 + uv^T$ which is a rank-one perturbation of identity. Can it be singular? Assuming it is not, compute its inverse. You may look for it in a form of $A^{-1} = 1 + \\alpha{}uv^T$ for some scalar $\\alpha$ and evaluate $\\alpha$.\n"
+      ],
+      "metadata": {
+        "id": "H9RDz5NmqXcx"
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "### **Solution:**  \n",
+        "**1)** Matrix $A$ is singular if there is a $m$-vector $x \\ne 0$ such that:  \n",
+        "$(1 + uv^T)x = 0 ⇔ x + uv^Tx = 0 ⇔ uv^Tx = -x.$\n",
+        "Note that $v^Tx$ is a scalar, so the matrix $A$ is singular if $x = \\beta{}u$, $\\beta{} \\ne 0$.  \n",
+        "From here we can see:  \n",
+        "$uv^Tx = -x ⇔ \\beta{}uv^Tu = -\\beta{}u ⇔ v^Tu = -1$.  \n",
+        "Thus, matrix $A$ is singular if $v^Tu = -1$  \n",
+        "\n",
+        "**2)** Assume that matrix $A$ has an inverse of the form $A^{-1} = 1 + αuv^T$.  \n",
+        "$AA^{-1} = 1 ⇔ (1 + uv^T)(1 + αuv^T) = 1 ⇔ αuv^T + uv^T + αuv^Tuv^T = O$.  \n",
+        "Since $v^Tu$ is a scalar, we denote it as $γ \\ne -1$ (otherwise the matrix will be singular). Then the resulting equality will be written in the form:  \n",
+        "$αuv^T + uv^T + αuv^Tuv^T = O ⇔ αuv^T + uv^T + αγuv^T = O ⇔ (α + 1 + αγ)uv^T = O ⇔ α + 1 + αγ = 0 ⇔ α = -\\frac{1}{1 + γ} = -\\frac{1}{1 + v^Tu}$.  \n",
+        "Thus, the matrix $A$ has the inverse of the form $A^{-1} = 1 + αuv^T$ and $α = -\\frac{1}{1 + v^Tu}, v^Tu \\ne -1$"
+      ],
+      "metadata": {
+        "id": "R_uvoqagr0td"
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "### **Problem 4(5)**:\n",
+        "\n",
+        "Prove that for any unitary matrix $U$ one has $\\parallel UA \\parallel_{F} = \\parallel AU\\parallel_{F} = \\parallel A \\parallel_{F}$."
+      ],
+      "metadata": {
+        "id": "WwlMFp33i0PH"
+      }
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "### **Solution:**  \n",
+        "Let's write down the expression by definition:\n",
+        "\n",
+        "$\\parallel UA \\parallel_{F} = \\sqrt{tr[(UA)^*(UA)]} = \\sqrt{tr[A^*U^*UA]}$.\n",
+        "\n",
+        "Let us take advantage of the fact that $U$ is unitary, i.e. $U^*U = I$. That:\n",
+        "\n",
+        "$\\parallel UA \\parallel_{F} = \\sqrt{tr[A^*U^*UA]} = \\sqrt{tr[A^*A]} = \\parallel A \\parallel_{F}$\n",
+        "\n",
+        "Similarly for $AU$ we get:\n",
+        "\n",
+        "$\\parallel AU \\parallel_{F} = \\sqrt{tr[(AU)^*(AU)]} = \\sqrt{tr[U^*A^*AU]}$.\n",
+        "\n",
+        "Let's use the trace property $tr(AB) = tr(BA)$. So:\n",
+        "$\\parallel AU \\parallel_{F} = \\sqrt{tr[U^*A^*AU]} = \\sqrt{tr[A^*AUU^*]} = \\sqrt{tr[A^*A]} = \\parallel A \\parallel_{F}$.\n",
+        "\n",
+        "Thus $\\parallel UA \\parallel_{F} = \\parallel AU\\parallel_{F} = \\parallel A \\parallel_{F}$."
+      ],
+      "metadata": {
+        "id": "Tvy4TARzl_Rc"
+      }
+    }
+  ]
+}