Default / 默认 · 8月 31, 2021

pytorch_lightning 配合Wandb示例

内容纲要

pytorch_lightning 配合 WandbLogger 示例

官方文档
https://docs.wandb.ai/v/zh-hans/

竟然还有中文文档。

基础依赖

!pip install pytorch-lightning wandb -q
os.environ["WANDB_SILENT"] = "true"
os.environ["WANDB_API_KEY"] = "ea464ba3c978ec8b8837c5989d4fcf"

wandb sweep


from pytorch_lightning.loggers import WandbLogger import wandb sweep_config = { "method": "bayes", # Random search https://docs.wandb.ai/guides/sweeps/configuration#search-strategy "metric": { # We want to maximize val_acc <h1>"name": "val_acc",</h1> <h1>"goal": "maximize"</h1> <pre><code> "name": "val_loss", "goal": "minimize", </code></pre> <h1>"target": 0.99</h1> }, "parameters": { "batch_size":{ "values": [16,32] }, "optimizer_name":{ "values": ["AdamW"] }, "learning_rate": { # log uniform distribution between exp(min) and exp(max) "distribution": "log_uniform", "min": -9.21, # exp(-9.21) = 1e-4 "max": -4.61 # exp(-4.61) = 1e-2 } } } sweep_id = wandb.sweep(sweep_config, project="bart_14363f") def sweep_iteration(): <br /> # set up W&B logger wandb.init() # required to have access to <code>wandb.config</code> wandb_logger = WandbLogger() train_loader=DataLoader(traindataset,batch_size=wandb.config.batch_size, shuffle=True, ) val_loader=DataLoader(devdataset,batch_size=wandb.config.batch_size, shuffle=False, ) model=LitAutoSeq2seq(learning_rate=wandb.config.learning_rate,optimizer_name=wandb.config.optimizer_name) <pre><code>early_stop_callback = EarlyStopping( monitor='val_loss', min_delta=0.000, patience=20, verbose=True, mode='min' ) lr_monitor = LearningRateMonitor(logging_interval='step') # setup Trainer trainer = pl.Trainer( gpus=1, # min_epochs=1, precision=16,amp_level='O2', </code></pre> <h1>checkpoint_callback=checkpoint_callback,</h1> <h1>resume_from_checkpoint="../input/nernotebook61fed2341fmodel/LitAutoMark-out.ckpt",</h1> <pre><code> auto_select_gpus=True, callbacks=[lr_monitor,early_stop_callback], deterministic=True, </code></pre> <h1>auto_scale_batch_size='binsearch',</h1> <h1>auto_lr_find=True,</h1> <h1>max_epochs=wandb.config.epochs,</h1> <pre><code> max_epochs=2, logger=wandb_logger, </code></pre> <h1>accumulate_grad_batches=wandb.config.accumulate_grad_batches</h1> <pre><code> accumulate_grad_batches=2 ) # train trainer.fit(model,train_dataloader=train_loader,val_dataloaders=val_loader) </code></pre> <h1>运行</h1> count = 30 # number of runs to execute wandb.agent(sweep_id, function=sweep_iteration,count =count) <h1>while not sweep.done():</h1> <h1>sweep.print_status()</h1> <h1>sweep.step()</h1> <h1>time.sleep(5)</h1>

值得一说的是这里的示例还是蛮多的。
https://docs.wandb.ai/v/zh-hans/examples

PyTorch Lightning

W&B报告

Colab笔记本

用PyTorch Lightning做语义分割

W&B仪表盘​

Github 仓库​

https://docs.wandb.ai/v/zh-hans/integrations/pytorch-lightning

%d 博主赞过: