상세 컨텐츠

본문 제목

DNN(Deep Neural Network)-MNIST _Categorical Classification

인공지능/딥러닝

by 2^7 2022. 6. 20. 15:49

본문

L2 Regularization

가중치의 제곱에 비례하는 노이즈를 Cost Function에 추가(가중치 감쇠:Weight Decay)

1. Data Preprocessing

# MNIST Data_Set Load

from tensorflow.keras.datasets import mnist

(X_train, y_train), (X_test, y_test) = mnist.load_data()

1-1. Reshape and Normalization

X_train = X_train.reshape((60000, 28 * 28))
X_test = X_test.reshape((10000, 28 * 28))
#Normalization

X_train = X_train.astype(float) / 255
X_test = X_test.astype(float) / 255

1-2. One Hot Encoding

from tensorflow.keras.utils import to_categorical

y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

2. MNIST Keras Modeling

  • 모델 신경망 구조 정의
    • 2개의 Hidden Layers & 768개의 Nodes
    • Model Capacity는 기존과 동일
    • L2 Regularization 적용
from tensorflow.keras import models
from tensorflow.keras import layers
from tensorflow.keras import regularizers

mnist = models.Sequential()
mnist.add(layers.Dense(512, activation = 'relu',
                       kernel_regularizer = regularizers.l2(0.00001),
                       input_shape=(28 * 28,)))
mnist.add(layers.Dense(256, activation = 'relu',
                       kernel_regularizer = regularizers.l2(0.00001)))
mnist.add(layers.Dense(10, activation = 'softmax'))
mnist.summary()


2-2. Model Compile

mnist.compile(loss = 'categorical_crossentropy',
              optimizer = 'rmsprop',
              metrics = ['accuracy'])

2-3. Model Fit

%%time

Hist_mnist = mnist.fit(X_train, y_train,
                       epochs = 100,
                       batch_size = 128,
                       validation_split = 0.2)


2-4. 학습 결과 시각화

import matplotlib.pyplot as plt

epochs = range(1, len(Hist_mnist.history['loss']) + 1)

plt.figure(figsize = (9, 6))
plt.plot(epochs, Hist_mnist.history['loss'])
plt.plot(epochs, Hist_mnist.history['val_loss'])
plt.ylim(0, 0.25)
plt.title('Training & Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend(['Training Loss', 'Validation Loss'])
plt.grid()
plt.show()


2-5. Model Evaluate

loss, accuracy = mnist.evaluate(X_test, y_test)

print('Loss = {:.5f}'.format(loss))
print('Accuracy = {:.5f}'.format(accuracy))


Dropout

훈련과정에서 네트워크의 일부 출력 특성의 연결을 무작위로 제외 시킴

1. Data Preprocessing

from tensorflow.keras.datasets import mnist

(X_train, y_train), (X_test, y_test) = mnist.load_data()

1-1. Reshape and Normalization

X_train = X_train.reshape((60000, 28 * 28))
X_test = X_test.reshape((10000, 28 * 28))
#Normalization

X_train = X_train.astype(float) / 255
X_test = X_test.astype(float) / 255

1-2. One Hot Encoding

from tensorflow.keras.utils import to_categorical

y_train = to_categorical(y_train)
y_test = to_categorical(y_test

2. Keras Modeling

2-1. Model Define

  • 모델 신경망 구조 정의
    • 2개의 Hidden Layers & 768개의 Nodes
    • Model Capacity는 기존과 동일
    • Dropout Layers 적용
from tensorflow.keras import models
from tensorflow.keras import layers

mnist = models.Sequential()
mnist.add(layers.Dense(512, activation = 'relu', input_shape=(28 * 28,)))
mnist.add(layers.Dropout(0.4))
mnist.add(layers.Dense(256, activation = 'relu'))
mnist.add(layers.Dropout(0.2))
mnist.add(layers.Dense(10, activation = 'softmax'))
mnist.summary()


2-2. Model Compile

mnist.compile(loss = 'categorical_crossentropy',
              optimizer = 'rmsprop',
              metrics = ['accuracy'])

2-3. Model Fit

%%time

Hist_mnist = mnist.fit(X_train, y_train,
                       epochs = 100,
                       batch_size = 128,
                       validation_split = 0.2)


2-4. 학습 결과 시각화

import matplotlib.pyplot as plt

epochs = range(1, len(Hist_mnist.history['loss']) + 1)

plt.figure(figsize = (9, 6))
plt.plot(epochs, Hist_mnist.history['loss'])
plt.plot(epochs, Hist_mnist.history['val_loss'])
plt.ylim(0, 0.25)
plt.title('Training & Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend(['Training Loss', 'Validation Loss'])
plt.grid()
plt.show()


2-5. Model Evaluate

loss, accuracy = mnist.evaluate(X_test, y_test)

print('Loss = {:.5f}'.format(loss))
print('Accuracy = {:.5f}'.format(accuracy))


Batch Normalization

활성화 함수의 입력값을 정규화 과정을 수행하여 전달

Gradient Vanishing 문제 해결 및 더 큰 Learning Rate를 사용 가능

1. Data Preprocessing

from tensorflow.keras.datasets import mnist

(X_train, y_train), (X_test, y_test) = mnist.load_data()

1-1. Reshape and Normalization

X_train = X_train.reshape((60000, 28 * 28))
X_test = X_test.reshape((10000, 28 * 28))
X_train = X_train.astype(float) / 255
X_test = X_test.astype(float) / 255

1-2. One Hot Encoding

from tensorflow.keras.utils import to_categorical

y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

2. MNIST Keras Modeling

2-1. Model Define

  • 모델 신경망 구조 정의
    • 활성화 함수의 입력값을 정규화 과정을 수행하여 전달
    • Gradient Vanishing 문제 해결 및 더 큰 Learning Rate를 사용 가능
from tensorflow.keras import models
from tensorflow.keras import layers

mnist = models.Sequential()
mnist.add(layers.Dense(512, input_shape=(28 * 28,)))
mnist.add(layers.BatchNormalization())
mnist.add(layers.Activation('relu'))
mnist.add(layers.Dense(256))
mnist.add(layers.BatchNormalization())
mnist.add(layers.Activation('relu'))
mnist.add(layers.Dense(10, activation = 'softmax'))
mnist.summary()


2-2. Model Compile

mnist.compile(loss = 'categorical_crossentropy',
              optimizer = 'rmsprop',
              metrics = ['accuracy'])

2-3.Model Fit

%%time

Hist_mnist = mnist.fit(X_train, y_train,
                       epochs = 100,
                       batch_size = 128,
                       validation_split = 0.2)


2-4. 학습 결과 시각화

import matplotlib.pyplot as plt

epochs = range(1, len(Hist_mnist.history['loss']) + 1)

plt.figure(figsize = (9, 6))
plt.plot(epochs, Hist_mnist.history['loss'])
plt.plot(epochs, Hist_mnist.history['val_loss'])
plt.ylim(0, 0.25)
plt.title('Training & Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend(['Training Loss', 'Validation Loss'])
plt.grid()
plt.show()


2-5.Model Evaluate

loss, accuracy = mnist.evaluate(X_test, y_test)

print('Loss = {:.5f}'.format(loss))
print('Accuracy = {:.5f}'.format(accuracy))

728x90

관련글 더보기