Skip to content
Snippets Groups Projects
Commit a1483fa9 authored by Zeynep Hakguder's avatar Zeynep Hakguder
Browse files

model for pa1

parent bbbdda05
Branches
Tags
No related merge requests found
Show whitespace changes
Inline Side-by-side
model.ipynb → ProgrammingAssignment_1/model.ipynb
+ 0
− 14
View file @ a1483fa9
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# JUPYTER NOTEBOOK TIPS # JUPYTER NOTEBOOK TIPS
Each rectangular box is called a cell. Each rectangular box is called a cell.
* ctrl+ENTER evaluates the current cell; if it contains Python code, it runs the code, if it contains Markdown, it returns rendered text. * ctrl+ENTER evaluates the current cell; if it contains Python code, it runs the code, if it contains Markdown, it returns rendered text.
* alt+ENTER evaluates the current cell and adds a new cell below it. * alt+ENTER evaluates the current cell and adds a new cell below it.
* If you click to the left of a cell, you'll notice the frame changes color to blue. You can erase a cell by hitting 'dd' (that's two "d"s in a row) when the frame is blue. * If you click to the left of a cell, you'll notice the frame changes color to blue. You can erase a cell by hitting 'dd' (that's two "d"s in a row) when the frame is blue.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Supervised Learning Model Skeleton # Supervised Learning Model Skeleton
We'll use this skeleton for implementing different supervised learning algorithms. Please complete "preprocess" and "partition" methods below. We'll use this skeleton for implementing different supervised learning algorithms. Please complete "preprocess" and "partition" methods below.
%% Cell type:markdown id: tags:
This step is for reading the dataset and for extracting features and labels. The "preprocess" function should return an *n x d* "features" array, and an *n x 1* "labels" array, where *n* is the number of examples and *d* is the number of features in the dataset. In cases where there is a big difference between the scales of features, we want to normalize the features to have values in the same range [0,1]. Since this is not the case with this dataset, we will not do normalization.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
#TODO: GIVE! #TODO: GIVE!
def preprocess(feature_file, label_file): def preprocess(feature_file, label_file):
''' '''
Args: Args:
feature_file: str feature_file: str
file containing features file containing features
label_file: str label_file: str
file containing labels file containing labels
Returns: Returns:
features: ndarray features: ndarray
nxd features nxd features
labels: ndarray labels: ndarray
nx1 labels nx1 labels
''' '''
# You might find np.genfromtxt useful for reading in the file. Be careful with the file delimiter, # You might find np.genfromtxt useful for reading in the file. Be careful with the file delimiter,
# e.g. for comma-separated files use delimiter=',' argument. # e.g. for comma-separated files use delimiter=',' argument.
# TODO # TODO
raise NotImplementedError raise NotImplementedError
return features, labels return features, labels
``` ```
%% Cell type:markdown id: tags:
Next, you'll need to split your dataset into training, validation and test sets. The "partition" function should take as input the size of the whole dataset and randomly sample a proportion *t* of the dataset as test partition and a proportion of *v* as validation partition. The remaining will be used as training data. For example, to keep 30% of the examples as test and %10 as validation, set *t* = 0.3 and *v* = 0.1. You should choose these values according to the size of the data available to you. The "split" function should return indices of the training, validation and test sets. These will be used to index into the whole training set.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
#TODO: GIVE! #TODO: GIVE!
def partition(size, t, v = 0): def partition(size, t, v = 0):
''' '''
Args: Args:
size: int size: int
number of examples in the whole dataset number of examples in the whole dataset
t: float t: float
proportion kept for test proportion kept for test
v: float v: float
proportion kept for validation proportion kept for validation
Returns: Returns:
test_indices: ndarray test_indices: ndarray
1D array containing test set indices 1D array containing test set indices
val_indices: ndarray val_indices: ndarray
1D array containing validation set indices 1D array containing validation set indices
''' '''
# np.random.permutation might come in handy. Do not sample with replacement! # np.random.permutation might come in handy. Do not sample with replacement!
# Be sure not to use the same indices in test and validation sets! # Be sure not to use the same indices in test and validation sets!
# use the first np.ceil(size*t) for test, # use the first np.ceil(size*t) for test,
# the following np.ceil(size*v) for validation set. # the following np.ceil(size*v) for validation set.
# TODO # TODO
raise NotImplementedError raise NotImplementedError
return test_indices, val_indices return test_indices, val_indices
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In cases, where data is not abundantly available, we resort to getting an error estimate from average of error on different splits of error. In this case, every fold of data is used for testing and for training in turns, i.e. assuming we split our data into 3 folds, we'd In cases, where data is not abundantly available, we resort to getting an error estimate from average of error on different splits of error. In this case, every fold of data is used for testing and for training in turns, i.e. assuming we split our data into 3 folds, we'd
* train our model on fold-1+fold-2 and test on fold-3 * train our model on fold-1+fold-2 and test on fold-3
* train our model on fold-1+fold-3 and test on fold-2 * train our model on fold-1+fold-3 and test on fold-2
* train our model on fold-2+fold-3 and test on fold-1. * train our model on fold-2+fold-3 and test on fold-1.
We'd use the average of the error we obtained in three runs as our error estimate. Implement function "kfold" below. We'd use the average of the error we obtained in three runs as our error estimate. Implement function "kfold" below.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# TODO: Programming Assignment 2 # TODO: Programming Assignment 2
def kfold(indices, k): def kfold(indices, k):
''' '''
Args: Args:
indices: ndarray indices: ndarray
1D array with integer entries containing indices 1D array with integer entries containing indices
k: int k: int
Number of desired splits in data.(Assume test set is already separated.) Number of desired splits in data.(Assume test set is already separated.)
Returns: Returns:
fold_dict: dict fold_dict: dict
A dictionary with integer keys corresponding to folds. Values are (training_indices, val_indices). A dictionary with integer keys corresponding to folds. Values are (training_indices, val_indices).
val_indices: ndarray val_indices: ndarray
1/k of training indices randomly chosen and separates them as validation partition. 1/k of training indices randomly chosen and separates them as validation partition.
train_indices: ndarray train_indices: ndarray
Remaining 1-(1/k) of the indices. Remaining 1-(1/k) of the indices.
e.g. fold_dict = {0: (train_0_indices, val_0_indices), e.g. fold_dict = {0: (train_0_indices, val_0_indices),
1: (train_0_indices, val_0_indices), 2: (train_0_indices, val_0_indices)} for k = 3 1: (train_0_indices, val_0_indices), 2: (train_0_indices, val_0_indices)} for k = 3
''' '''
return fold_dict return fold_dict
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Choice of distance metric plays an important role in the performance of *k*-NN. Let's start with implementing a distance method in the "distance" function below. It should take two data points and the name of the metric and return a scalar value. Choice of distance metric plays an important role in the performance of *k*-NN. Let's start with implementing a distance method in the "distance" function below. It should take two data points and the name of the metric and return a scalar value.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
#TODO: Programming Assignment 1 #TODO: Programming Assignment 1
def distance(x, y, metric): def distance(x, y, metric):
''' '''
Args: Args:
x: ndarray x: ndarray
1D array containing coordinates for a point 1D array containing coordinates for a point
y: ndarray y: ndarray
1D array containing coordinates for a point 1D array containing coordinates for a point
metric: str metric: str
Euclidean, Hamming Euclidean, Hamming
Returns: Returns:
''' '''
if metric == 'Euclidean': if metric == 'Euclidean':
raise NotImplementedError raise NotImplementedError
elif metric == 'Hammming': elif metric == 'Hammming':
raise NotImplementedError raise NotImplementedError
else: else:
raise ValueError('{} is not a valid metric.'.format(metric)) raise ValueError('{} is not a valid metric.'.format(metric))
return dist # scalar distance btw x and y return dist # scalar distance btw x and y
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We will extend "Model" class while implementing supervised learning algorithms. We will extend "Model" class while implementing supervised learning algorithms.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
class Model: class Model:
# preprocess_f and partition_f expect functions # preprocess_f and partition_f expect functions
# use kwargs to pass arguments to preprocessor_f and partition_f # use kwargs to pass arguments to preprocessor_f and partition_f
# kwargs is a dictionary and should contain t, v, feature_file, label_file # kwargs is a dictionary and should contain t, v, feature_file, label_file
# e.g. {'t': 0.3, 'v': 0.1, 'feature_file': 'some_file_name', 'label_file': 'some_file_name'} # e.g. {'t': 0.3, 'v': 0.1, 'feature_file': 'some_file_name', 'label_file': 'some_file_name'}
def __init__(self, preprocessor_f, partition_f, **kwargs): def __init__(self, preprocessor_f, partition_f, **kwargs):
self.features, self.labels = preprocessor_f(kwargs['feature_file'], kwargs['label_file']) self.features, self.labels = preprocessor_f(kwargs['feature_file'], kwargs['label_file'])
self.size = len(self.labels) # number of examples in dataset self.size = len(self.labels) # number of examples in dataset
self.feat_dim = self.features.shape[1] # number of features self.feat_dim = self.features.shape[1] # number of features
self.val_indices, self.test_indices = partition_f(self.size, kwargs['t'], kwargs['v']) self.val_indices, self.test_indices = partition_f(self.size, kwargs['t'], kwargs['v'])
self.val_size = len(self.val_indices) self.val_size = len(self.val_indices)
self.test_size = len(self.test_indices) self.test_size = len(self.test_indices)
self.train_indices = np.delete(np.arange(self.size), np.append(self.test_indices, self.val_indices), 0) self.train_indices = np.delete(np.arange(self.size), np.append(self.test_indices, self.val_indices), 0)
self.train_size = len(self.train_indices) self.train_size = len(self.train_indices)
def fit(self): def fit(self):
raise NotImplementedError raise NotImplementedError
def predict(self, test_points): def predict(self, test_points):
raise NotImplementedError raise NotImplementedError
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## General supervised learning performance related functions ## General supervised learning performance related functions
### (To be implemented later when it is indicated in other notebooks) ### (To be implemented later when it is indicated in other notebooks)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Implement the "conf_matrix" function that takes as input an array of true labels (*true*) and an array of predicted labels (*pred*). It should output a numpy.ndarray. Implement the "conf_matrix" function that takes as input an array of true labels (*true*) and an array of predicted labels (*pred*). It should output a numpy.ndarray.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# TODO: Programming Assignment 1 # TODO: Programming Assignment 1
def conf_matrix(true, pred): def conf_matrix(true, pred):
''' '''
Args: Args:
true: ndarray true: ndarray
nx1 array of true labels for test set nx1 array of true labels for test set
pred: ndarray pred: ndarray
nx1 array of predicted labels for test set nx1 array of predicted labels for test set
Returns: Returns:
ndarray ndarray
''' '''
raise NotImplementedError raise NotImplementedError
tp = tn = fp = fn = 0 tp = tn = fp = fn = 0
# calculate true positives (tp), true negatives(tn) # calculate true positives (tp), true negatives(tn)
# false positives (fp) and false negatives (fn) # false positives (fp) and false negatives (fn)
# returns the confusion matrix as numpy.ndarray # returns the confusion matrix as numpy.ndarray
return np.array([tp,tn, fp, fn]) return np.array([tp,tn, fp, fn])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
ROC curves are a good way to visualize sensitivity vs. 1-specificity for varying cut off points. Now, implement a "ROC" function that predicts the labels of the test set examples using different *threshold* values in "predict" and plot the ROC curve. "ROC" takes a list containing different *threshold* parameter values to try and returns two arrays; one where each entry is the sensitivity at a given threshold and the other where entries are 1-specificities. ROC curves are a good way to visualize sensitivity vs. 1-specificity for varying cut off points. Now, implement a "ROC" function that predicts the labels of the test set examples using different *threshold* values in "predict" and plot the ROC curve. "ROC" takes a list containing different *threshold* parameter values to try and returns two arrays; one where each entry is the sensitivity at a given threshold and the other where entries are 1-specificities.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# TODO: Programming Assignment 1 # TODO: Programming Assignment 1
def ROC(model, indices, value_list): def ROC(model, indices, value_list):
''' '''
Args: Args:
model: a fitted supervised learning model model: a fitted supervised learning model
indices: ndarray indices: ndarray
1D array containing indices 1D array containing indices
value_list: ndarray value_list: ndarray
1D array containing different threshold values 1D array containing different threshold values
Returns: Returns:
sens: ndarray sens: ndarray
1D array containing sensitivities 1D array containing sensitivities
spec_: ndarray spec_: ndarray
1D array containing 1-specifities 1D array containing 1-specifities
''' '''
# use predict method to obtain predicted labels at different threshold values # use predict method to obtain predicted labels at different threshold values
# use conf_matrix to calculate tp, tn, fp, fn # use conf_matrix to calculate tp, tn, fp, fn
# calculate sensitivity, 1-specificity # calculate sensitivity, 1-specificity
# return two arrays # return two arrays
raise NotImplementedError raise NotImplementedError
return sens, spec_ return sens, spec_
``` ```
......
model_solution.ipynb → ProgrammingAssignment_1/model_solution.ipynb
+ 0
− 14
View file @ a1483fa9
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# JUPYTER NOTEBOOK TIPS # JUPYTER NOTEBOOK TIPS
Each rectangular box is called a cell. Each rectangular box is called a cell.
* ctrl+ENTER evaluates the current cell; if it contains Python code, it runs the code, if it contains Markdown, it returns rendered text. * ctrl+ENTER evaluates the current cell; if it contains Python code, it runs the code, if it contains Markdown, it returns rendered text.
* alt+ENTER evaluates the current cell and adds a new cell below it. * alt+ENTER evaluates the current cell and adds a new cell below it.
* If you click to the left of a cell, you'll notice the frame changes color to blue. You can erase a cell by hitting 'dd' (that's two "d"s in a row) when the frame is blue. * If you click to the left of a cell, you'll notice the frame changes color to blue. You can erase a cell by hitting 'dd' (that's two "d"s in a row) when the frame is blue.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Supervised Learning Model Skeleton # Supervised Learning Model Skeleton
We'll use this skeleton for implementing different supervised learning algorithms. Please complete "preprocess" and "partition" methods below. We'll use this skeleton for implementing different supervised learning algorithms. Please complete "preprocess" and "partition" methods below.
%% Cell type:markdown id: tags:
This step is for reading the dataset and for extracting features and labels. The "preprocess" function should return an *n x d* "features" array, and an *n x 1* "labels" array, where *n* is the number of examples and *d* is the number of features in the dataset. In cases where there is a big difference between the scales of features, we want to normalize the features to have values in the same range [0,1]. Since this is not the case with this dataset, we will not do normalization.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
def preprocess(feature_file, label_file): def preprocess(feature_file, label_file):
''' '''
Args: Args:
feature_file: str feature_file: str
file containing features file containing features
label_file: str label_file: str
file containing labels file containing labels
Returns: Returns:
features: ndarray features: ndarray
nxd features nxd features
labels: ndarray labels: ndarray
nx1 labels nx1 labels
''' '''
# You might find np.genfromtxt useful for reading in the file. Be careful with the file delimiter, # You might find np.genfromtxt useful for reading in the file. Be careful with the file delimiter,
# e.g. for comma-separated files use delimiter=',' argument. # e.g. for comma-separated files use delimiter=',' argument.
# TODO # TODO
raise NotImplementedError raise NotImplementedError
return features, labels return features, labels
``` ```
%% Cell type:markdown id: tags:
Next, you'll need to split your dataset into training, validation and test sets. The "partition" function should take as input the size of the whole dataset and randomly sample a proportion *t* of the dataset as test partition and a proportion of *v* as validation partition. The remaining will be used as training data. For example, to keep 30% of the examples as test and %10 as validation, set *t* = 0.3 and *v* = 0.1. You should choose these values according to the size of the data available to you. The "split" function should return indices of the training, validation and test sets. These will be used to index into the whole training set.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
#TODO: GIVE! #TODO: GIVE!
def partition(size, t, v = 0): def partition(size, t, v = 0):
''' '''
Args: Args:
size: int size: int
number of examples in the whole dataset number of examples in the whole dataset
t: float t: float
proportion kept for test proportion kept for test
v: float v: float
proportion kept for validation proportion kept for validation
Returns: Returns:
test_indices: ndarray test_indices: ndarray
1D array containing test set indices 1D array containing test set indices
val_indices: ndarray val_indices: ndarray
1D array containing validation set indices 1D array containing validation set indices
''' '''
# np.random.permutation might come in handy. Do not sample with replacement! # np.random.permutation might come in handy. Do not sample with replacement!
# Be sure not to use the same indices in test and validation sets! # Be sure not to use the same indices in test and validation sets!
# use the first np.ceil(size*t) for test, # use the first np.ceil(size*t) for test,
# the following np.ceil(size*v) for validation set. # the following np.ceil(size*v) for validation set.
# TODO # TODO
raise NotImplementedError raise NotImplementedError
return test_indices, val_indices return test_indices, val_indices
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
In cases, where data is not abundantly available, we resort to getting an error estimate from average of error on different splits of error. In this case, every fold of data is used for testing and for training in turns, i.e. assuming we split our data into 3 folds, we'd In cases, where data is not abundantly available, we resort to getting an error estimate from average of error on different splits of error. In this case, every fold of data is used for testing and for training in turns, i.e. assuming we split our data into 3 folds, we'd
* train our model on fold-1+fold-2 and test on fold-3 * train our model on fold-1+fold-2 and test on fold-3
* train our model on fold-1+fold-3 and test on fold-2 * train our model on fold-1+fold-3 and test on fold-2
* train our model on fold-2+fold-3 and test on fold-1. * train our model on fold-2+fold-3 and test on fold-1.
We'd use the average of the error we obtained in three runs as our error estimate. Implement function "kfold" below. We'd use the average of the error we obtained in three runs as our error estimate. Implement function "kfold" below.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# TODO: Programming Assignment 2 # TODO: Programming Assignment 2
def kfold(indices, k): def kfold(indices, k):
''' '''
Args: Args:
indices: ndarray indices: ndarray
1D array with integer entries containing indices 1D array with integer entries containing indices
k: int k: int
Number of desired splits in data.(Assume test set is already separated.) Number of desired splits in data.(Assume test set is already separated.)
Returns: Returns:
fold_dict: dict fold_dict: dict
A dictionary with integer keys corresponding to folds. Values are (training_indices, val_indices). A dictionary with integer keys corresponding to folds. Values are (training_indices, val_indices).
val_indices: ndarray val_indices: ndarray
1/k of training indices randomly chosen and separates them as validation partition. 1/k of training indices randomly chosen and separates them as validation partition.
train_indices: ndarray train_indices: ndarray
Remaining 1-(1/k) of the indices. Remaining 1-(1/k) of the indices.
e.g. fold_dict = {0: (train_0_indices, val_0_indices), e.g. fold_dict = {0: (train_0_indices, val_0_indices),
1: (train_0_indices, val_0_indices), 2: (train_0_indices, val_0_indices)} for k = 3 1: (train_0_indices, val_0_indices), 2: (train_0_indices, val_0_indices)} for k = 3
''' '''
return fold_dict return fold_dict
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Choice of distance metric plays an important role in the performance of *k*-NN. Let's start with implementing a distance method in the "distance" function below. It should take two data points and the name of the metric and return a scalar value. Choice of distance metric plays an important role in the performance of *k*-NN. Let's start with implementing a distance method in the "distance" function below. It should take two data points and the name of the metric and return a scalar value.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
#TODO: Programming Assignment 1 #TODO: Programming Assignment 1
def distance(x, y, metric): def distance(x, y, metric):
''' '''
Args: Args:
x: ndarray x: ndarray
1D array containing coordinates for a point 1D array containing coordinates for a point
y: ndarray y: ndarray
1D array containing coordinates for a point 1D array containing coordinates for a point
metric: str metric: str
Euclidean, Hamming Euclidean, Hamming
Returns: Returns:
''' '''
if metric == 'Euclidean': if metric == 'Euclidean':
raise NotImplementedError raise NotImplementedError
elif metric == 'Hammming': elif metric == 'Hammming':
raise NotImplementedError raise NotImplementedError
else: else:
raise ValueError('{} is not a valid metric.'.format(metric)) raise ValueError('{} is not a valid metric.'.format(metric))
return dist # scalar distance btw x and y return dist # scalar distance btw x and y
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
We will extend "Model" class while implementing supervised learning algorithms. We will extend "Model" class while implementing supervised learning algorithms.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
class Model: class Model:
# preprocess_f and partition_f expect functions # preprocess_f and partition_f expect functions
# use kwargs to pass arguments to preprocessor_f and partition_f # use kwargs to pass arguments to preprocessor_f and partition_f
# kwargs is a dictionary and should contain t, v, feature_file, label_file # kwargs is a dictionary and should contain t, v, feature_file, label_file
# e.g. {'t': 0.3, 'v': 0.1, 'feature_file': 'some_file_name', 'label_file': 'some_file_name'} # e.g. {'t': 0.3, 'v': 0.1, 'feature_file': 'some_file_name', 'label_file': 'some_file_name'}
def __init__(self, preprocessor_f, partition_f, **kwargs): def __init__(self, preprocessor_f, partition_f, **kwargs):
self.features, self.labels = preprocessor_f(kwargs['feature_file'], kwargs['label_file']) self.features, self.labels = preprocessor_f(kwargs['feature_file'], kwargs['label_file'])
self.size = len(self.labels) # number of examples in dataset self.size = len(self.labels) # number of examples in dataset
self.feat_dim = self.features.shape[1] # number of features self.feat_dim = self.features.shape[1] # number of features
self.val_indices, self.test_indices = partition_f(self.size, kwargs['t'], kwargs['v']) self.val_indices, self.test_indices = partition_f(self.size, kwargs['t'], kwargs['v'])
self.val_size = len(self.val_indices) self.val_size = len(self.val_indices)
self.test_size = len(self.test_indices) self.test_size = len(self.test_indices)
self.train_indices = np.delete(np.arange(self.size), np.append(self.test_indices, self.val_indices), 0) self.train_indices = np.delete(np.arange(self.size), np.append(self.test_indices, self.val_indices), 0)
self.train_size = len(self.train_indices) self.train_size = len(self.train_indices)
def fit(self): def fit(self):
raise NotImplementedError raise NotImplementedError
def predict(self, test_points): def predict(self, test_points):
raise NotImplementedError raise NotImplementedError
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## General supervised learning performance related functions ## General supervised learning performance related functions
### (To be implemented later when it is indicated in other notebooks) ### (To be implemented later when it is indicated in other notebooks)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Implement the "conf_matrix" function that takes as input an array of true labels (*true*) and an array of predicted labels (*pred*). It should output a numpy.ndarray. Implement the "conf_matrix" function that takes as input an array of true labels (*true*) and an array of predicted labels (*pred*). It should output a numpy.ndarray.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# TODO: Programming Assignment 1 # TODO: Programming Assignment 1
def conf_matrix(true, pred): def conf_matrix(true, pred):
''' '''
Args: Args:
true: ndarray true: ndarray
nx1 array of true labels for test set nx1 array of true labels for test set
pred: ndarray pred: ndarray
nx1 array of predicted labels for test set nx1 array of predicted labels for test set
Returns: Returns:
ndarray ndarray
''' '''
raise NotImplementedError raise NotImplementedError
tp = tn = fp = fn = 0 tp = tn = fp = fn = 0
# calculate true positives (tp), true negatives(tn) # calculate true positives (tp), true negatives(tn)
# false positives (fp) and false negatives (fn) # false positives (fp) and false negatives (fn)
# returns the confusion matrix as numpy.ndarray # returns the confusion matrix as numpy.ndarray
return np.array([tp,tn, fp, fn]) return np.array([tp,tn, fp, fn])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
ROC curves are a good way to visualize sensitivity vs. 1-specificity for varying cut off points. Now, implement a "ROC" function that predicts the labels of the test set examples using different *threshold* values in "predict" and plot the ROC curve. "ROC" takes a list containing different *threshold* parameter values to try and returns two arrays; one where each entry is the sensitivity at a given threshold and the other where entries are 1-specificities. ROC curves are a good way to visualize sensitivity vs. 1-specificity for varying cut off points. Now, implement a "ROC" function that predicts the labels of the test set examples using different *threshold* values in "predict" and plot the ROC curve. "ROC" takes a list containing different *threshold* parameter values to try and returns two arrays; one where each entry is the sensitivity at a given threshold and the other where entries are 1-specificities.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# TODO: Programming Assignment 1 # TODO: Programming Assignment 1
def ROC(model, indices, value_list): def ROC(model, indices, value_list):
''' '''
Args: Args:
model: a fitted supervised learning model model: a fitted supervised learning model
indices: ndarray indices: ndarray
1D array containing indices 1D array containing indices
value_list: ndarray value_list: ndarray
1D array containing different threshold values 1D array containing different threshold values
Returns: Returns:
sens: ndarray sens: ndarray
1D array containing sensitivities 1D array containing sensitivities
spec_: ndarray spec_: ndarray
1D array containing 1-specifities 1D array containing 1-specifities
''' '''
# use predict method to obtain predicted labels at different threshold values # use predict method to obtain predicted labels at different threshold values
# use conf_matrix to calculate tp, tn, fp, fn # use conf_matrix to calculate tp, tn, fp, fn
# calculate sensitivity, 1-specificity # calculate sensitivity, 1-specificity
# return two arrays # return two arrays
raise NotImplementedError raise NotImplementedError
return sens, spec_ return sens, spec_
``` ```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment