pytorch_lightning 配合Wandb示例

pytorch_lightning 配合　WandbLogger　示例

官方文档
https://docs.wandb.ai/v/zh-hans/

竟然还有中文文档。

基础依赖

!pip install pytorch-lightning wandb -q
os.environ["WANDB_SILENT"] = "true"
os.environ["WANDB_API_KEY"] = "ea464ba3c978ec8b8837c5989d4fcf"

wandb sweep

from pytorch_lightning.loggers import WandbLogger
import wandb
sweep_config = {
  "method": "bayes",   # Random search https://docs.wandb.ai/guides/sweeps/configuration#search-strategy
  "metric": {           # We want to maximize val_acc
<h1>"name": "val_acc",</h1>
<h1>"goal": "maximize"</h1>
<pre><code>  "name": "val_loss",
  "goal": "minimize",
</code></pre>
<h1>"target": 0.99</h1>
},
  "parameters": {
        "batch_size":{
            "values": [16,32]
        },
        "optimizer_name":{
            "values": ["AdamW"]
        },
        "learning_rate": {
            # log uniform distribution between exp(min) and exp(max)
            "distribution": "log_uniform",
            "min": -9.21,   # exp(-9.21) = 1e-4
            "max": -4.61    # exp(-4.61) = 1e-2
        }
  }        

}
sweep_id = wandb.sweep(sweep_config, project="bart_14363f")

def sweep_iteration():  <br />
    # set up W&B logger
    wandb.init()    # required to have access to <code>wandb.config</code>
    wandb_logger = WandbLogger()
    train_loader=DataLoader(traindataset,batch_size=wandb.config.batch_size, shuffle=True, )
    val_loader=DataLoader(devdataset,batch_size=wandb.config.batch_size, shuffle=False, )
    model=LitAutoSeq2seq(learning_rate=wandb.config.learning_rate,optimizer_name=wandb.config.optimizer_name)   
<pre><code>early_stop_callback = EarlyStopping(
   monitor='val_loss',
   min_delta=0.000,
   patience=20,
   verbose=True,
   mode='min'
)
lr_monitor = LearningRateMonitor(logging_interval='step')


# setup Trainer
trainer = pl.Trainer(
    gpus=1,
#     min_epochs=1,
    precision=16,amp_level='O2',
</code></pre>
<h1>checkpoint_callback=checkpoint_callback,</h1>
<h1>resume_from_checkpoint="../input/nernotebook61fed2341fmodel/LitAutoMark-out.ckpt",</h1>
<pre><code>    auto_select_gpus=True,
    callbacks=[lr_monitor,early_stop_callback],
    deterministic=True,
</code></pre>
<h1>auto_scale_batch_size='binsearch',</h1>
<h1>auto_lr_find=True,</h1>
<h1>max_epochs=wandb.config.epochs,</h1>
<pre><code>    max_epochs=2,
    logger=wandb_logger,
</code></pre>
<h1>accumulate_grad_batches=wandb.config.accumulate_grad_batches</h1>
<pre><code>    accumulate_grad_batches=2
)
# train
trainer.fit(model,train_dataloader=train_loader,val_dataloaders=val_loader)
</code></pre>
<h1>运行</h1>
count = 30 # number of runs to execute
wandb.agent(sweep_id, function=sweep_iteration,count =count)
<h1>while not sweep.done():</h1>
<h1>sweep.print_status()</h1>
<h1>sweep.step()</h1>
<h1>time.sleep(5)</h1>

值得一说的是这里的示例还是蛮多的。
https://docs.wandb.ai/v/zh-hans/examples

PyTorch Lightning

W&B报告​

Colab笔记本

用PyTorch Lightning做语义分割

W&B仪表盘​

Github 仓库​

https://docs.wandb.ai/v/zh-hans/integrations/pytorch-lightning