溫馨提示×

如何在PyTorch中進(jìn)行超參數(shù)搜索

小樊
150
2024-03-14 11:08:32

在PyTorch中進(jìn)行超參數(shù)搜索通常有兩種常用的方法：

  1. 使用Grid Search：通過定義一個超參數(shù)的候選值列表，對所有可能的組合進(jìn)行窮舉搜索，選擇表現(xiàn)最好的超參數(shù)組合。可以使用GridSearchCV類來實現(xiàn)這一過程。
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import accuracy_score
from torch import nn, optim
from torch.utils.data import DataLoader

# Define your model
class MyModel(nn.Module):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super(MyModel, self).__init__()
        self.hidden = nn.Linear(input_dim, hidden_dim)
        self.relu = nn.ReLU()
        self.output = nn.Linear(hidden_dim, output_dim)
    
    def forward(self, x):
        x = self.hidden(x)
        x = self.relu(x)
        x = self.output(x)
        return x

# Define your dataset and dataloader
# dataset = ...
# dataloader = DataLoader(dataset, batch_size=64, shuffle=True)

# Define parameter grid
param_grid = {
    'hidden_dim': [64, 128, 256],
    'learning_rate': [0.001, 0.01, 0.1]
}

# Create a GridSearchCV object
grid_search = GridSearchCV(MyModel, param_grid, scoring='accuracy', cv=3)

# Fit the model
grid_search.fit(dataloader)

# Print best parameters
print(grid_search.best_params_)
  1. 使用Random Search：與Grid Search不同，Random Search是隨機(jī)地在指定的參數(shù)空間中采樣，從而更有效地搜索超參數(shù)空間?？梢允褂肦andomizedSearchCV類來實現(xiàn)這一過程。
from sklearn.model_selection import RandomizedSearchCV
from sklearn.metrics import accuracy_score
from torch import nn, optim
from torch.utils.data import DataLoader

# Define your model
# Define your model
class MyModel(nn.Module):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super(MyModel, self).__init__()
        self.hidden = nn.Linear(input_dim, hidden_dim)
        self.relu = nn.ReLU()
        self.output = nn.Linear(hidden_dim, output_dim)
    
    def forward(self, x):
        x = self.hidden(x)
        x = self.relu(x)
        x = self.output(x)
        return x

# Define your dataset and dataloader
# dataset = ...
# dataloader = DataLoader(dataset, batch_size=64, shuffle=True)

# Define parameter grid
param_dist = {
    'hidden_dim': [64, 128, 256],
    'learning_rate': [0.001, 0.01, 0.1]
}

# Create a RandomizedSearchCV object
random_search = RandomizedSearchCV(MyModel, param_dist, n_iter=10, scoring='accuracy', cv=3)

# Fit the model
random_search.fit(dataloader)

# Print best parameters
print(random_search.best_params_)

無論選擇哪種方法，超參數(shù)搜索是一個耗時的過程，需要謹(jǐn)慎選擇超參數(shù)的范圍和步長，以及合適的評估指標(biāo)來評估模型性能。

0 踩