initial commit

LICENSE

+BSD 3-Clause License
+
+Copyright (c) 2020, Regents of the University of Nebraska
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+* Neither the name of the University of Nebraska nor the names of its
+  contributors may be used to endorse or promote products derived from
+  this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

+# Homework 2
+
+This repository cotains:
+* utils/display_images.py: subroutine for displaying images
+* cifar10.py: subroutines for loading and downloading cifar10 data
+* omp.py: STARTER CODE for OMP whitening subroutine
+* dictlearn.py: STARTER CODE for dictlearn subroutine
+
+
+## Instructions
+### 1. Complete zca.py:
+
+`zca.py` contains the following subroutine
+
+```python
+def zca_white(x):
+  """ perform zca whitening 
+  
+      inputs:
+        x: numpy array of images
+
+      outputs:
+        y: numpy array of whitened images        
+  """
+
+  # *** put code for zca whitening here ***
+
+  return y
+```
+
+which need to completed using only the `numpy` package.  
+**No other packages should be used**   
+
+### 2. Complete ica.py
+
+`ica.py` includes the following subroutines that need to be completed. The first
+subroutine `sample` should sample patches from images.  
+
+```python
+def sample(x, patch_size=16, num_patches=10):
+    ''' randomly sample patches from x 
+    
+        inputs:
+            x: numpy array of images
+            patch_size: patch dims for patch_size x patch_size images 
+                        (default 16)
+            num_patches: number of patches to sample (default 10)
+
+        outputs:
+            y: numpy array of patches
+    '''
+
+    return y
+```
+
+The second subroutine `ica` should perform gradient descent with the backtracking 
+line search to adapted the learning ratefor the ICA objective function. 
+
+```python
+def ica(x, **args):
+    ''' perform independent component analysis (ICA)
+    
+        inputs:
+            x: numpy array of images
+            args:
+                lr: learning rate (default 1e-3)
+                nsteps: maximum iterations (default 1000)
+                k: number of latent variables (defualt 20)
+        
+        returns:
+            L: numpy array of loss function value for all iterations
+            W: numpy array of ICA basis vectors
+    '''
+
+    # default parameters
+    if not len(args):
+        args['lr'] = 1
+        args['nsteps'] = 200
+        args['k'] = 64
+
+    lr=args['lr']
+    nsteps = args['nsteps']
+    k = args['k']
+    
+    # ***initialize variables here***
+
+    '''training loop using graident descent'''
+    for step in range(nsteps):
+        # ***insert gradient descent code here***
+        ''' use backtracking line search '''
+        
+        # print loss
+        print('step: {} / {}, L: {}'.format(step, nsteps, L[step]))
+        
+    return L, W
+```
+
+`ica` and `sample` need to completed only the following packages:
+* `numpy`
+* `scipy.linalg`
+**No other packages should be used**
+
+### Parameters
+`ica.py` also provides a sample main that loads cifar10 using `cifar10.py`,  
+whitens the images using `zca.py`, performs ICA using the `ica(x,**args)`, and displays
+and displays the learned basis images `W`.
+
+**Note that values of parameters such as the learning rate `lr`, number of basis
+ images `k`, and number of optimization steps `nsteps` may need to be changed**
+
+### Example
+
+The following image show an example result of applying ICA to whitened cifar10 data
+
+![Test Image 1](cifar_ica_basis_64.png)
+
+