아래 내용은 Udemy에서 Pytorch: Deep Learning and Artificial Intelligence를 보고 정리한 내용이다.
GANs Theory
CNNs and RNNs are just neural networks with shared weights
이전의 모델과 비교가 된다.
Gans is a system
has two neural networks: generator and discriminator
위조범죄자 : 상점에 가서 가짜 돈을 사용하여 물건을 구매하려고 한다.
shop owner: 진짜인지 가짜인지 판단한다.
GAN the loss function
2개가 필요하다. discriminator generator
discriminator 은 classify real and fake images - binary classification
the correct loss is the binary cross entropy
Generator loss
Feeze the discriminator layers so only the generator is trained
real = 1 , fake = 0
Latent space
Generator is like the opposite of a feature transformer/embedding
image -> vecotr vs vecotr -> image
GAN pseudocode
#1. load in the data x, y = get_mnist()
#2. create networks
D = nn.Sequential(....)
G = nn.Sequential(....)
#3. loss and optimizer
crit = nn.BCEWithLogitsLoss()
optim_D = Adam(D.parameters())
optim_G = Adam(G.parameters())
#4. Get outputs and loss
D_out = [D(real), D(fake)] //then find loss, etc.
G_out = D(G(noise)) //combined_model(noise)
#4. gradient descent loop
for epoch in epochs:
#train discriminator
real_images = sample from x
fake_images = g.predict(noise)
d.train_on_batch(real_images, ones)
d.train_on_batch(fake_images, zeros)
#train generator
combined_model.train_on_batch(noise, ones)
GAN code preparation
Generator is hard but discriminator is easy
Discriminator pytorch
D= nn.Sequential(
nn.Linear(784, 512),
nn.LeakyReLU(0.2),
nn.Linear(512,256),
nn.LeakyReLU(0.2),
nn.Linear(256, 1)
)
Generator pytorch
latent_dim = 100
G = nn.Sequential(
nn.Linear(latent_dim, 256),
nn.LeakyReLU(0.2),
nn.BatchNorm1d(256, momentum=0.7),
nn.Linear(256,512),
nn.LeakyReLU(0.2),
nn.BatchNorm1d(512, momentum=0.7),
nn.Linear(512,1024),
nn.LeakyReLU(0.2),
nn.BatchNorm1d(1024, momentum=0.7),
nn.Linear(1024, 784),
nn.Tanh()
)
GANs optimizer and loss pytorch
criterion = nn.BCEWithLogitsLoss()
d_optimizer = torch.optim.Adam(D.parameters(), lr = 0.0002, betas=(0.5, 0.999))
g_optimizer = torch.optim.Adam(G.parameters(), lr = 0.0002, betas=(0.5, 0.999))
# 0, 1로 return
def scale_image(img):
out = (img+1) / 2
return out
ones_ = torch.ones(batch_size, 1).to(device)
zeros_ = torch.zeros(batch_size, 1).to(device)
d_losses = []
g_losses = []
for epoch in range(200):
for inputs, _ in data_loader:
n = inputs.size(0)
inputs = inputs.reshape(n, 784).to(device)
ones = ones_[:n]
zeros = zeros_[:n]
###train discriminator
real_outputs = D(inputs)
d_loss_real = criterion(real_outputs, ones)
#fake images
noise = torch.randn(n, latent_dim).to(device)
fake_images = G(noise)
fake_outputs = D(fake_images)
d_loss_fake = criterion(fake_outputs, zeros)
#gradient descent
d_loss = 0.5 * (d_loss_real + d_loss_fake)
d_optimizer.zero_grad()
g_optimizer.zero_grad()
d_loss.backward()
d_optimizer.step()
#train generator
for _ in range(2):
noise = torch.randn(n, latent_dim).to(device)
fake_images = G(noise)
fake_outputs = D(fake_images)
g_loss = criterion(fake_outputs, ones)
d_optimizer.zero_grad()
g_optimizer.zero_grad()
g_loss.backward()
g_optimizer.step()
d_losses.append(d_loss.item())
g_losses.append(g_loss.item())
print(f"Epoch: {epoch} , d_loss : {d_loss.item()} ,g_loss:{g_loss.item()}")
fake_images = fake_images.reshape(-1, 1, 28,28)
save_image(scale_image(fake_images) , f"gan_images/{epoch+1}.png")
'교육동영상 > 02. pytorch: Deep Learning' 카테고리의 다른 글
12. Stock Trading Project with Deep Reinforcement Learning (0) | 2021.01.06 |
---|---|
10. Deep Reinforcement Learning (0) | 2021.01.04 |
09. Transfer Learning (0) | 2020.12.23 |
08. Recommender System (0) | 2020.12.21 |
07. NLP (0) | 2020.12.16 |