2021-06-09 记事

Save/Load state_dict 权重加载保存

Save:
torch.save(model.state\_dict(), PATH)
Load:
model \= TheModelClass(*args, **kwargs) model.load_state_dict(torch.load(PATH)) model.eval()
pytorch-lightning 中hp_metric使用

https://pytorch-lightning.readthedocs.io/en/stable/extensions/logging.html
hp_metric作为调参的指标值，需要自己指定不然无意义。

If you want to track a metric in the tensorboard hparams tab, log scalars to the key hp_metric. If tracking multiple metrics, initialize TensorBoardLogger with default_hp_metric=False and call log_hyperparams only once with your metric keys and initial values. Subsequent updates can simply be logged to the metric keys. Refer to the following for examples on how to setup proper hyperparams metrics tracking within LightningModule.

Using default_hp_metric

def validation_step(self, batch, batch_idx):
self.log(“hp_metric”, some_scalar)

Using custom or multiple metrics (default_hp_metric=False)

def on_train_start(self):
self.logger.log_hyperparams(self.hparams, {“hp/metric_1”: 0, “hp/metric_2”: 0})

def validation_step(self, batch, batch_idx):
self.log(“hp/metric_1”, some_scalar_1)
self.log(“hp/metric_2”, some_scalar_2)

In the example, using hp/ as a prefix allows for the metrics to be grouped under “hp” in the tensorboard scalar tab where you can collapse them.

Sigmoid函数

Sigmoid 函数是一个在生物学中常见的S型函数，也称为S型生长曲线。 [1] 在信息科学中，由于其单增以及反函数单增等性质，Sigmoid函数常被用作神经网络的激活函数，将变量映射到0,1之间。

LR的sigmoid函数得到的值并不是天生就能被认为是概率。只是因为我们拿正确的概率分布和它的cross-entropy作为loss function去训练它，才使它逼近了我们想去逼近的那个概率分布模型。

sigmoid函数的好处是，可以让每个特征的贡献度之和分布到整个实数区间上。

二分类任务

Sigmoid配合
torch.nn.BCELoss
![[img/d3407574780bf65d59002f5555327bab.png]]

m = nn.Sigmoid()
loss = nn.BCELoss()
input = torch.randn(3, requires_grad=True)
target = torch.empty(3).random_(2)#input和target尺寸相同，即公式中的和
output = loss(m(input), target)#把input放入sigmoid的在放入loss中
output,m(input),target
<h1>(tensor(0.6829), tensor([0.5136, 0.5192, 0.4780]), tensor([1., 0., 0.]))</h1>

{"alias": ["陈华"], "subject_id": "10344084", "type": ["Human"], "data": [{"predicate": "义项描述", "object": "琼剧演员"}, {"predicate": "标签", "object": "娱乐人物"}, {"predicate": "标签", "object": "人物"}, {"predicate": "标签", "object": "演员"}], "subject": "陈华"}