Tensorboard Cheet Sheet (SummaryWriter)

Posted: Oct 21, 2023 ·  Updated: Oct 27, 2023 · 3 min.
Cheet sheet

List some commonly used commands for tensorboard.

Tensorboard

Part of the Beginner's Guide series:

  1. Git Command Guide
  2. Coding Python in Linux Server
  3. Create A Site by Hugo
  4. Using tmux to manage your terminal
  5. Tensorboard Cheet Sheet (SummaryWriter)This post!
  6. Debugging Python with pdb
  7. Customize your PyTorch Dataset
  8. Cheetsheet for Python os module

Some commonly used commands for Tensorboard.

Initialize the writer Link to this heading

Start the Tensorboard Link to this heading

To start Tensorboard, run the following command in the terminal:

bash
1tensorboard --logdir=/path/to/log --port=6006

I always make mistake with the path, so I recommend you to use the absolute path.

The output will be like:

bash
1TensorBoard 2.5.0 at http://localhost:6006/ (Press CTRL+C to quit)

Or, you can start Tensorboard from Vscode.

Initialize the writer Link to this heading

python
1from torch.utils.tensorboard import SummaryWriter
2writer = SummaryWriter('runs/exp-1') # the result will be saved in runs/exp-1
3writer = SummaryWriter() # the result will be saved in runs/Jul21-10-53-36 like format
4writer = SummaryWriter(comment='exp-1') # the result will be saved in runs/Jul21-10-53-36-exp-1 like format

Close the writer Link to this heading

After recording ALL the data, you should close the writer.

python
1writer.close()

Add Log to Tensorboard Link to this heading

Add scalar Link to this heading

The most common used of Tensorboard is to add scalar. The output will be a curve of your index, such as loss, accuracy vs epoch.

def add_scalar(tag, scalar_value, global_step=None, walltime=None)

python
1writer.add_scalar('train/looss', loss, epoch) # Here is an example of adding loss
2writer.add_scalar('directory/like/tag', value, index) # Value is the y-axis, index is the x-axis

Add image Link to this heading

add_image allows you to add image to Tensorboard. It can be used to visualize the result of your model.

def add_image(tag, img_tensor, global_step=None, walltime=None, dataformats='CHW')

python
 1writer.add_image('image', image, index) 
 2# image format: torch.Tensor, numpy.ndarray
 3# default channel order: (Channel, Height, Width)
 4# if you want to change the channel order, use the following code
 5## with argument dataformats:
 6## "HWC" for (Height, Width, Channel)
 7## "HW" for (Height, Width)
 8## "CHW" for (Channel, Height, Width)
 9# By default, torch.Tensor -> (C,H,W), numpy.ndarray -> (H,W,C)
10writer.add_image('image', image, index, dataformats='HWC')

Add histogram Link to this heading

add_histogram allows you to add histogram to Tensorboard. It can be used to visualize the distribution of your model.

def add_histogram(tag, values, global_step=None, bins='tensorflow', walltime=None, max_bins=None)

  • values: torch.Tensor of Any shape. the histogram is computed over the flattened array. Type==(castable to float64)
  • bins
python
1writer.add_histogram('histogram', values, index)

Add Graph Link to this heading

to be continued

Add Hyperparameter Tuning Result Link to this heading

to be continued

Add Embedding Link to this heading

add_embedding allows you to add embedding to Tensorboard. It can be used to visualize the result of your model. It provides several ways to visualize the embedding, for example, PCA, t-SNE, etc.

def add_embedding(tensor, metadata=None, label_img=None, global_step=None, tag='default')

  • tensor: torch.Tensor of shape (n_samples, n_features)
  • metadata: list of labels, col1: index, col2: label
  • label_img: torch.Tensor of shape (n_samples, C, H, W), this is optional
python
1features = model(images)
2labels = targets
3writer.add_embedding(features, metadata=labels, label_img=images)

Reference Link to this heading