您好,登錄后才能下訂單哦!
如何進(jìn)行RNN總結(jié)及sin與cos擬合應(yīng)用,很多新手對(duì)此不是很清楚,為了幫助大家解決這個(gè)難題,下面小編將為大家詳細(xì)講解,有這方面需求的人可以來學(xué)習(xí)下,希望你能有所收獲。
一、RNN總結(jié)
一個(gè)簡(jiǎn)單的RNN模型由輸入層,一個(gè)隱藏層,一個(gè)輸出層組成。
我們給出這個(gè)抽象圖對(duì)應(yīng)的具體圖,能夠很清楚的看到,上一時(shí)刻的隱藏層是如何影響當(dāng)前時(shí)刻的隱藏層的。
基于RNN還可以繼續(xù)擴(kuò)展到雙向循環(huán)神經(jīng)網(wǎng)絡(luò),深度循環(huán)神經(jīng)網(wǎng)絡(luò)。RNN公式如下:
定義RNN類,代碼如下:
from torch import nn class RNN(nn.Module): def __init__(self): super(RNN, self).__init__() self.rnn = nn.RNN( input_size=INPUT_SIZE, # The number of expected features in the input `x` hidden_size=32, # The number of features in the hidden state `h` num_layers=1, # Number of recurrent layers batch_first=True # batch維度是否在前,If ``True``, tensors as `(batch, seq, feature)` ) self.out = nn.Linear(32, 1) # 線性變換 def forward(self, x, h_state): out, h_state = self.rnn(x, h_state) return out, h_state
Tips: 1. RNN的訓(xùn)練算法是BPTT,它的基本原理核BP算法一致,包含同樣的三個(gè)步驟:1) 前向計(jì)算每個(gè)神經(jīng)元的輸出值;2)反向計(jì)算每個(gè)神經(jīng)元的誤差項(xiàng)?_j值,它是誤差函數(shù)E對(duì)神經(jīng)元j的加權(quán)輸出net_j的偏導(dǎo)數(shù);3)計(jì)算每個(gè)權(quán)重的梯度。
2. RNN的梯度消失核爆炸,根據(jù)公式的指數(shù)形式,β大于或小于1都將造成梯度消失核爆炸問題。
如何避免:1) 梯度爆炸:設(shè)置一個(gè)梯度閾值,當(dāng)梯度超過這個(gè)閾值的時(shí)候可以直接截取 (Gradient Clipping(pytorch nn.utils.clip_grad_norm )) ;好的參數(shù)初始化方式,如He初始化; 非飽和的激活函數(shù)(如 ReLU) ; 批量規(guī)范化(Batch Normalization); LSTM 。2)梯度消失:改進(jìn)網(wǎng)絡(luò)LSTM,加入了forget gate。
二、sin與cos擬合應(yīng)用
函數(shù)sin擬合為cos,模型黑盒子類似sin(π/2+α)= cosα
import torch from torch import nn import numpy as np import matplotlib.pyplot as plt # 定義超參數(shù) TIME_STEP = 10 INPUT_SIZE = 1 learning_rate = 0.001 class RNN(nn.Module): def __init__(self): super(RNN, self).__init__() self.rnn = nn.RNN( input_size=INPUT_SIZE, hidden_size=32, num_layers=1, batch_first=True ) self.out = nn.Linear(32, 1) def forward(self, x, h_state): # r_out.shape:seq_len,batch,hidden_size*num_direction(1,10,32) r_out, h_state = self.rnn(x, h_state) out = self.out(r_out).squeeze() return out, h_state rnn = RNN() criterion = nn.MSELoss() optimizer = torch.optim.Adam(rnn.parameters(), lr=learning_rate) h_state = None plt.figure(1, figsize=(12, 5)) plt.ion() # 開啟動(dòng)態(tài)交互 for step in range(100): start, end = step * np.pi, (step + 1) * np.pi steps = np.linspace(start, end, TIME_STEP, dtype=np.float32, endpoint=False) x_np = np.sin(steps) # x_np.shape: 10 y_np = np.cos(steps) # y_np.shape: 10 x = torch.from_numpy(x_np[np.newaxis, :, np.newaxis]) # x.shape: 1,10,1 y = torch.from_numpy(y_np) # y.shape: 10 prediction, h_state = rnn(x, h_state) h_state = h_state.data loss = criterion(prediction, y) optimizer.zero_grad() loss.backward() optimizer.step() plt.plot(steps, y_np.flatten(), 'r-') plt.plot(steps, prediction.data.numpy().flatten(), 'b-') plt.draw() plt.pause(.05) plt.ioff() plt.show()
看完上述內(nèi)容是否對(duì)您有幫助呢?如果還想對(duì)相關(guān)知識(shí)有進(jìn)一步的了解或閱讀更多相關(guān)文章,請(qǐng)關(guān)注億速云行業(yè)資訊頻道,感謝您對(duì)億速云的支持。
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如果涉及侵權(quán)請(qǐng)聯(lián)系站長(zhǎng)郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。