[WIP: Feb2023] TorchSharp Tensorboard

[GeorgeS2019]

We now have Tensorboard!

PyTorch code
TorchSharp code

• add_scalar

Add scalar data to summary.

add_scalar(tag, scalar_value, global_step=None, walltime=None, new_style=False, double_precision=False)

• add_scalars

Adds many scalar data to summary.

add_scalars(main_tag, tag_scalar_dict, global_step=None, walltime=None)

• add_histogram

Add histogram to summary. #939

add_histogram(tag, values, global_step=None, bins='tensorflow', walltime=None, max_bins=None)

• add_image

Add image data to summary. #936

add_image(tag, img_tensor, global_step=None, walltime=None, dataformats='CHW')

• add_images

Add batched image data to summary.

add_images(tag, img_tensor, global_step=None, walltime=None, dataformats='NCHW')

more ...

[GeorgeS2019]

Related discussion:

PyTorch for Deep Learning: Creating and Deploying Deep Learning Applications

• Chapter7: Tensorboard Example

import torch
import torch.nn as nn
import torch.utils.data
import torchvision
from functools import partial
from torch import optim
from torch.utils.tensorboard import SummaryWriter
from torchvision import datasets, transforms

[NiklasGustafsson]

@GeorgeS2019 -- please file this as an issue instead of a Discussion topic.

[GeorgeS2019]

• add_histogram

Add histogram to summary.

add_histogram(tag, values, global_step=None, bins='tensorflow', walltime=None, max_bins=None)

Output from ChatGPT

TensorBoard is a tool for visualizing TensorFlow runs. In TorchSharp, TensorBoard can be used to visualize the histograms of tensors by using the SummaryWriter class and the AddHistogram method.

Here's an example of how to use TensorBoard to visualize the histograms of the weights and biases of a linear layer in TorchSharp:

using Torch;
using Torch.Schedulers;
using Torch.Utils;
using Torch.Optim;
using System;
using System.IO;

namespace TensorboardExample
{
    class Program
    {
        static void Main(string[] args)
        {
            // Define the neural network
            var model = new Sequential();
            model.Add(new Linear(2, 128));
            model.Add(new ReLU());
            model.Add(new Linear(128, 1));

            // Define the optimizer
            var optimizer = new SGD(model.Parameters(), 0.01);

            // Define the loss function
            var loss = new MSELoss();

            // Define the data
            var inputs = new Tensor(new[] { 1.0f, 2.0f, 3.0f, 4.0f, 5.0f }, new[] { 5, 2 });
            var targets = new Tensor(new[] { 2.0f, 4.0f, 6.0f, 8.0f, 10.0f }, new[] { 5, 1 });

            // Train the model
            int epochCount = 100;
            int logInterval = 10;
            int batchSize = inputs.Shape[0];
            for (int epoch = 0; epoch < epochCount; epoch++)
            {
                optimizer.ZeroGrad();
                var output = model.Forward(inputs);
                var l = loss.Forward(output, targets);
                l.Backward();
                optimizer.Step();

                // Log the histograms of the weights and biases
                if (epoch % logInterval == 0)
                {
                    var summaryWriter = new SummaryWriter();
                    summaryWriter.UseEventFile(Path.Combine(Directory.GetCurrentDirectory(), "runs", "example"));
                    var linear = model[0] as Linear;
                    summaryWriter.AddHistogram("weights", linear.Weight, epoch);
                    summaryWriter.AddHistogram("biases", linear.Bias, epoch);
                    summaryWriter.Flush();
                }
            }

            // Print the final loss
            Console.WriteLine("Loss: " + loss.Forward(model.Forward(inputs), targets).ToScalar());
        }
    }
}

This code creates a summary writer and uses it to log the histograms of the weights and biases of the first linear layer in the network. The histograms are logged every 10 epochs.

[NiklasGustafsson]

Another example of ChatGPT getting things wrong. Tensorboard histograms are not supported by TorchSharp yet, and when it does, it will follow PyTorch naming conventions rather than standard .NET naming.