Skip to content

Commit 2789855

Browse files
ptyadanaptyadana
committed
Support Vector Machine SVM module completed
1 parent 93e697e commit 2789855

File tree

6 files changed

+353
-0
lines changed

6 files changed

+353
-0
lines changed

‎ML - Applied Machine Learning - Algorithms/03.Support Vector Machine/01.Support Vector Machines - Hyperparameters.ipynb

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"## Support Vector Machines: Hyperparameters\n",
8+
"\n",
9+
"Import [Support Vector Machines](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html) from `sklearn` and explore the hyperparameters."
10+
]
11+
},
12+
{
13+
"cell_type": "markdown",
14+
"metadata": {},
15+
"source": [
16+
"### Import Support Vector Machines Algorithm"
17+
]
18+
},
19+
{
20+
"cell_type": "code",
21+
"execution_count": 4,
22+
"metadata": {},
23+
"outputs": [
24+
{
25+
"data": {
26+
"text/plain": [
27+
"SVC()"
28+
]
29+
},
30+
"execution_count": 4,
31+
"metadata": {},
32+
"output_type": "execute_result"
33+
}
34+
],
35+
"source": [
36+
"from sklearn.svm import SVC\n",
37+
"\n",
38+
"SVC()"
39+
]
40+
},
41+
{
42+
"cell_type": "code",
43+
"execution_count": 2,
44+
"metadata": {},
45+
"outputs": [
46+
{
47+
"data": {
48+
"text/plain": [
49+
"['C',\n",
50+
" '__abstractmethods__',\n",
51+
" '__class__',\n",
52+
" '__delattr__',\n",
53+
" '__dict__',\n",
54+
" '__dir__',\n",
55+
" '__doc__',\n",
56+
" '__eq__',\n",
57+
" '__format__',\n",
58+
" '__ge__',\n",
59+
" '__getattribute__',\n",
60+
" '__getstate__',\n",
61+
" '__gt__',\n",
62+
" '__hash__',\n",
63+
" '__init__',\n",
64+
" '__init_subclass__',\n",
65+
" '__le__',\n",
66+
" '__lt__',\n",
67+
" '__module__',\n",
68+
" '__ne__',\n",
69+
" '__new__',\n",
70+
" '__reduce__',\n",
71+
" '__reduce_ex__',\n",
72+
" '__repr__',\n",
73+
" '__setattr__',\n",
74+
" '__setstate__',\n",
75+
" '__sizeof__',\n",
76+
" '__str__',\n",
77+
" '__subclasshook__',\n",
78+
" '__weakref__',\n",
79+
" '_abc_impl',\n",
80+
" '_check_n_features',\n",
81+
" '_check_proba',\n",
82+
" '_compute_kernel',\n",
83+
" '_decision_function',\n",
84+
" '_dense_decision_function',\n",
85+
" '_dense_fit',\n",
86+
" '_dense_predict',\n",
87+
" '_dense_predict_proba',\n",
88+
" '_estimator_type',\n",
89+
" '_get_coef',\n",
90+
" '_get_param_names',\n",
91+
" '_get_tags',\n",
92+
" '_impl',\n",
93+
" '_more_tags',\n",
94+
" '_pairwise',\n",
95+
" '_predict_log_proba',\n",
96+
" '_predict_proba',\n",
97+
" '_repr_html_',\n",
98+
" '_repr_html_inner',\n",
99+
" '_repr_mimebundle_',\n",
100+
" '_sparse_decision_function',\n",
101+
" '_sparse_fit',\n",
102+
" '_sparse_kernels',\n",
103+
" '_sparse_predict',\n",
104+
" '_sparse_predict_proba',\n",
105+
" '_validate_data',\n",
106+
" '_validate_for_predict',\n",
107+
" '_validate_targets',\n",
108+
" '_warn_from_fit_status',\n",
109+
" 'break_ties',\n",
110+
" 'cache_size',\n",
111+
" 'class_weight',\n",
112+
" 'coef0',\n",
113+
" 'coef_',\n",
114+
" 'decision_function',\n",
115+
" 'decision_function_shape',\n",
116+
" 'degree',\n",
117+
" 'epsilon',\n",
118+
" 'fit',\n",
119+
" 'gamma',\n",
120+
" 'get_params',\n",
121+
" 'kernel',\n",
122+
" 'max_iter',\n",
123+
" 'n_support_',\n",
124+
" 'nu',\n",
125+
" 'predict',\n",
126+
" 'predict_log_proba',\n",
127+
" 'predict_proba',\n",
128+
" 'probA_',\n",
129+
" 'probB_',\n",
130+
" 'probability',\n",
131+
" 'random_state',\n",
132+
" 'score',\n",
133+
" 'set_params',\n",
134+
" 'shrinking',\n",
135+
" 'tol',\n",
136+
" 'verbose']"
137+
]
138+
},
139+
"execution_count": 2,
140+
"metadata": {},
141+
"output_type": "execute_result"
142+
}
143+
],
144+
"source": [
145+
"dir(SVC())"
146+
]
147+
},
148+
{
149+
"cell_type": "code",
150+
"execution_count": null,
151+
"metadata": {},
152+
"outputs": [],
153+
"source": []
154+
}
155+
],
156+
"metadata": {
157+
"kernelspec": {
158+
"display_name": "Python 3",
159+
"language": "python",
160+
"name": "python3"
161+
},
162+
"language_info": {
163+
"codemirror_mode": {
164+
"name": "ipython",
165+
"version": 3
166+
},
167+
"file_extension": ".py",
168+
"mimetype": "text/x-python",
169+
"name": "python",
170+
"nbconvert_exporter": "python",
171+
"pygments_lexer": "ipython3",
172+
"version": "3.8.3"
173+
}
174+
},
175+
"nbformat": 4,
176+
"nbformat_minor": 2
177+
}

‎ML - Applied Machine Learning - Algorithms/03.Support Vector Machine/02.Support Vector Machines - Fit and evaluate a model.ipynb

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"## Support Vector Machines: Fit and evaluate a model\n",
8+
"\n",
9+
"Using the Titanic dataset from [this](https://www.kaggle.com/c/titanic/overview) Kaggle competition.\n",
10+
"\n",
11+
"In this section, we will fit and evaluate a simple Support Vector Machines model."
12+
]
13+
},
14+
{
15+
"cell_type": "markdown",
16+
"metadata": {},
17+
"source": [
18+
"### Read in Data\n",
19+
"\n",
20+
"![CV](img/CV.png)\n",
21+
"![Cross-Val](img/Cross-Val.png)"
22+
]
23+
},
24+
{
25+
"cell_type": "code",
26+
"execution_count": 3,
27+
"metadata": {},
28+
"outputs": [],
29+
"source": [
30+
"import joblib\n",
31+
"import pandas as pd\n",
32+
"from sklearn.svm import SVC\n",
33+
"from sklearn.model_selection import GridSearchCV\n",
34+
"\n",
35+
"import warnings\n",
36+
"warnings.filterwarnings('ignore', category=FutureWarning)\n",
37+
"warnings.filterwarnings('ignore', category=DeprecationWarning)\n",
38+
"\n",
39+
"train_features = pd.read_csv('../Data/train_features.csv')\n",
40+
"train_labels = pd.read_csv('../Data/train_labels.csv', header=None)"
41+
]
42+
},
43+
{
44+
"cell_type": "markdown",
45+
"metadata": {},
46+
"source": [
47+
"### Hyperparameter tuning\n",
48+
"\n",
49+
"![c](img/c.png)"
50+
]
51+
},
52+
{
53+
"cell_type": "code",
54+
"execution_count": 7,
55+
"metadata": {},
56+
"outputs": [],
57+
"source": [
58+
"def print_results(results):\n",
59+
" print('BEST PARAMS: {}\\n'.format(results.best_params_))\n",
60+
" \n",
61+
" means = results.cv_results_['mean_test_score']\n",
62+
" stds = results.cv_results_['std_test_score']\n",
63+
" for mean, std, params in zip(means, stds, results.cv_results_['params']):\n",
64+
" print('{} (+-{}) for {}'.format(round(mean,3), round(std * 2,3), params))"
65+
]
66+
},
67+
{
68+
"cell_type": "code",
69+
"execution_count": 8,
70+
"metadata": {},
71+
"outputs": [
72+
{
73+
"name": "stdout",
74+
"output_type": "stream",
75+
"text": [
76+
"BEST PARAMS: {'C': 0.1, 'kernel': 'linear'}\n",
77+
"\n",
78+
"0.796 (+-0.115) for {'C': 0.1, 'kernel': 'linear'}\n",
79+
"0.654 (+-0.06) for {'C': 0.1, 'kernel': 'rbf'}\n",
80+
"0.796 (+-0.115) for {'C': 1, 'kernel': 'linear'}\n",
81+
"0.661 (+-0.048) for {'C': 1, 'kernel': 'rbf'}\n",
82+
"0.796 (+-0.115) for {'C': 10, 'kernel': 'linear'}\n",
83+
"0.684 (+-0.07) for {'C': 10, 'kernel': 'rbf'}\n"
84+
]
85+
}
86+
],
87+
"source": [
88+
"svc = SVC()\n",
89+
"parameters = {\n",
90+
" 'kernel': ['linear', 'rbf'],\n",
91+
" 'C': [0.1, 1, 10]\n",
92+
"}\n",
93+
"\n",
94+
"cv = GridSearchCV(svc, parameters, cv=5)\n",
95+
"cv.fit(train_features, train_labels.values.ravel())\n",
96+
"\n",
97+
"print_results(cv)"
98+
]
99+
},
100+
{
101+
"cell_type": "code",
102+
"execution_count": 9,
103+
"metadata": {},
104+
"outputs": [
105+
{
106+
"data": {
107+
"text/plain": [
108+
"SVC(C=0.1, kernel='linear')"
109+
]
110+
},
111+
"execution_count": 9,
112+
"metadata": {},
113+
"output_type": "execute_result"
114+
}
115+
],
116+
"source": [
117+
"cv.best_estimator_"
118+
]
119+
},
120+
{
121+
"cell_type": "markdown",
122+
"metadata": {},
123+
"source": [
124+
"### Write out pickled model"
125+
]
126+
},
127+
{
128+
"cell_type": "code",
129+
"execution_count": 11,
130+
"metadata": {},
131+
"outputs": [
132+
{
133+
"data": {
134+
"text/plain": [
135+
"['../Pickled_Models/SVM_model.pkl']"
136+
]
137+
},
138+
"execution_count": 11,
139+
"metadata": {},
140+
"output_type": "execute_result"
141+
}
142+
],
143+
"source": [
144+
"joblib.dump(cv.best_estimator_, '../Pickled_Models/SVM_model.pkl')"
145+
]
146+
},
147+
{
148+
"cell_type": "code",
149+
"execution_count": null,
150+
"metadata": {},
151+
"outputs": [],
152+
"source": []
153+
}
154+
],
155+
"metadata": {
156+
"kernelspec": {
157+
"display_name": "Python 3",
158+
"language": "python",
159+
"name": "python3"
160+
},
161+
"language_info": {
162+
"codemirror_mode": {
163+
"name": "ipython",
164+
"version": 3
165+
},
166+
"file_extension": ".py",
167+
"mimetype": "text/x-python",
168+
"name": "python",
169+
"nbconvert_exporter": "python",
170+
"pygments_lexer": "ipython3",
171+
"version": "3.8.3"
172+
}
173+
},
174+
"nbformat": 4,
175+
"nbformat_minor": 2
176+
}

‎ML - Applied Machine Learning - Algorithms/03.Support Vector Machine/img/CV.png

289 KB
Loading

‎ML - Applied Machine Learning - Algorithms/03.Support Vector Machine/img/Cross-Val.png

96.2 KB
Loading

‎ML - Applied Machine Learning - Algorithms/03.Support Vector Machine/img/c.png

30.1 KB
Loading

‎ML - Applied Machine Learning - Algorithms/Pickled_Models/SVM_model.pkl

19.9 KB
Binary file not shown.

0 commit comments

Comments
 (0)