가중치의 제곱에 비례하는 노이즈를 Cost Function에 추가(가중치 감쇠:Weight Decay)
1. Data Preprocessing
# MNIST Data_Set Load
from tensorflow.keras.datasets import mnist
(X_train, y_train), (X_test, y_test) = mnist.load_data()
1-1. Reshape and Normalization
X_train = X_train.reshape((60000, 28 * 28))
X_test = X_test.reshape((10000, 28 * 28))
#Normalization
X_train = X_train.astype(float) / 255
X_test = X_test.astype(float) / 255
1-2. One Hot Encoding
from tensorflow.keras.utils import to_categorical
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)
2. MNIST Keras Modeling
from tensorflow.keras import models
from tensorflow.keras import layers
from tensorflow.keras import regularizers
mnist = models.Sequential()
mnist.add(layers.Dense(512, activation = 'relu',
kernel_regularizer = regularizers.l2(0.00001),
input_shape=(28 * 28,)))
mnist.add(layers.Dense(256, activation = 'relu',
kernel_regularizer = regularizers.l2(0.00001)))
mnist.add(layers.Dense(10, activation = 'softmax'))
mnist.summary()
2-2. Model Compile
mnist.compile(loss = 'categorical_crossentropy',
optimizer = 'rmsprop',
metrics = ['accuracy'])
2-3. Model Fit
%%time
Hist_mnist = mnist.fit(X_train, y_train,
epochs = 100,
batch_size = 128,
validation_split = 0.2)
2-4. 학습 결과 시각화
import matplotlib.pyplot as plt
epochs = range(1, len(Hist_mnist.history['loss']) + 1)
plt.figure(figsize = (9, 6))
plt.plot(epochs, Hist_mnist.history['loss'])
plt.plot(epochs, Hist_mnist.history['val_loss'])
plt.ylim(0, 0.25)
plt.title('Training & Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend(['Training Loss', 'Validation Loss'])
plt.grid()
plt.show()
2-5. Model Evaluate
loss, accuracy = mnist.evaluate(X_test, y_test)
print('Loss = {:.5f}'.format(loss))
print('Accuracy = {:.5f}'.format(accuracy))
훈련과정에서 네트워크의 일부 출력 특성의 연결을 무작위로 제외 시킴
1. Data Preprocessing
from tensorflow.keras.datasets import mnist
(X_train, y_train), (X_test, y_test) = mnist.load_data()
1-1. Reshape and Normalization
X_train = X_train.reshape((60000, 28 * 28))
X_test = X_test.reshape((10000, 28 * 28))
#Normalization
X_train = X_train.astype(float) / 255
X_test = X_test.astype(float) / 255
1-2. One Hot Encoding
from tensorflow.keras.utils import to_categorical
y_train = to_categorical(y_train)
y_test = to_categorical(y_test
2. Keras Modeling
2-1. Model Define
from tensorflow.keras import models
from tensorflow.keras import layers
mnist = models.Sequential()
mnist.add(layers.Dense(512, activation = 'relu', input_shape=(28 * 28,)))
mnist.add(layers.Dropout(0.4))
mnist.add(layers.Dense(256, activation = 'relu'))
mnist.add(layers.Dropout(0.2))
mnist.add(layers.Dense(10, activation = 'softmax'))
mnist.summary()
2-2. Model Compile
mnist.compile(loss = 'categorical_crossentropy',
optimizer = 'rmsprop',
metrics = ['accuracy'])
2-3. Model Fit
%%time
Hist_mnist = mnist.fit(X_train, y_train,
epochs = 100,
batch_size = 128,
validation_split = 0.2)
2-4. 학습 결과 시각화
import matplotlib.pyplot as plt
epochs = range(1, len(Hist_mnist.history['loss']) + 1)
plt.figure(figsize = (9, 6))
plt.plot(epochs, Hist_mnist.history['loss'])
plt.plot(epochs, Hist_mnist.history['val_loss'])
plt.ylim(0, 0.25)
plt.title('Training & Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend(['Training Loss', 'Validation Loss'])
plt.grid()
plt.show()
2-5. Model Evaluate
loss, accuracy = mnist.evaluate(X_test, y_test)
print('Loss = {:.5f}'.format(loss))
print('Accuracy = {:.5f}'.format(accuracy))
활성화 함수의 입력값을 정규화 과정을 수행하여 전달
Gradient Vanishing 문제 해결 및 더 큰 Learning Rate를 사용 가능
1. Data Preprocessing
from tensorflow.keras.datasets import mnist
(X_train, y_train), (X_test, y_test) = mnist.load_data()
1-1. Reshape and Normalization
X_train = X_train.reshape((60000, 28 * 28))
X_test = X_test.reshape((10000, 28 * 28))
X_train = X_train.astype(float) / 255
X_test = X_test.astype(float) / 255
1-2. One Hot Encoding
from tensorflow.keras.utils import to_categorical
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)
2. MNIST Keras Modeling
2-1. Model Define
from tensorflow.keras import models
from tensorflow.keras import layers
mnist = models.Sequential()
mnist.add(layers.Dense(512, input_shape=(28 * 28,)))
mnist.add(layers.BatchNormalization())
mnist.add(layers.Activation('relu'))
mnist.add(layers.Dense(256))
mnist.add(layers.BatchNormalization())
mnist.add(layers.Activation('relu'))
mnist.add(layers.Dense(10, activation = 'softmax'))
mnist.summary()
2-2. Model Compile
mnist.compile(loss = 'categorical_crossentropy',
optimizer = 'rmsprop',
metrics = ['accuracy'])
2-3.Model Fit
%%time
Hist_mnist = mnist.fit(X_train, y_train,
epochs = 100,
batch_size = 128,
validation_split = 0.2)
2-4. 학습 결과 시각화
import matplotlib.pyplot as plt
epochs = range(1, len(Hist_mnist.history['loss']) + 1)
plt.figure(figsize = (9, 6))
plt.plot(epochs, Hist_mnist.history['loss'])
plt.plot(epochs, Hist_mnist.history['val_loss'])
plt.ylim(0, 0.25)
plt.title('Training & Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend(['Training Loss', 'Validation Loss'])
plt.grid()
plt.show()
2-5.Model Evaluate
loss, accuracy = mnist.evaluate(X_test, y_test)
print('Loss = {:.5f}'.format(loss))
print('Accuracy = {:.5f}'.format(accuracy))
CNN(Convolutional Neural Network)-이미지 데이터 셋을 이용한 CNN Modeling (0) | 2022.06.24 |
---|---|
CNN(Convolutional Neural Network)-MNIST _Categorical Classification (0) | 2022.06.24 |
DNN(Deep Neural Network)-MNIST _Categorical Classification (0) | 2022.06.17 |
DNN(Deep Neural Network)-IMDB _Binary Classification (0) | 2022.06.17 |
TensorFlow (0) | 2022.06.13 |