您好,登錄后才能下訂單哦!
查看源碼
Linear 的初始化部分:
class Linear(Module): ... __constants__ = ['bias'] def __init__(self, in_features, out_features, bias=True): super(Linear, self).__init__() self.in_features = in_features self.out_features = out_features self.weight = Parameter(torch.Tensor(out_features, in_features)) if bias: self.bias = Parameter(torch.Tensor(out_features)) else: self.register_parameter('bias', None) self.reset_parameters() ...
需要實現(xiàn)的內(nèi)容:
計算步驟:
@weak_script_method def forward(self, input): return F.linear(input, self.weight, self.bias)
返回的是:input * weight + bias
對于 weight
weight: the learnable weights of the module of shape :math:`(\text{out\_features}, \text{in\_features})`. The values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where :math:`k = \frac{1}{\text{in\_features}}`
對于 bias
bias: the learnable bias of the module of shape :math:`(\text{out\_features})`. If :attr:`bias` is ``True``, the values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{in\_features}}`
實例展示
舉個例子:
>>> import torch >>> nn1 = torch.nn.Linear(100, 50) >>> input1 = torch.randn(140, 100) >>> output1 = nn1(input1) >>> output1.size() torch.Size([140, 50])
張量的大小由 140 x 100 變成了 140 x 50
執(zhí)行的操作是:
[140,100]×[100,50]=[140,50]
以上就是本文的全部內(nèi)容,希望對大家的學習有所幫助,也希望大家多多支持億速云。
免責聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉載和分享為主,文章觀點不代表本網(wǎng)站立場,如果涉及侵權請聯(lián)系站長郵箱:is@yisu.com進行舉報,并提供相關證據(jù),一經(jīng)查實,將立刻刪除涉嫌侵權內(nèi)容。