Skip to content
This repository has been archived by the owner on Mar 5, 2022. It is now read-only.

.NET Core global tools to retrieve Azure Diagnostics logs from a WADLogsTable or blobs and write them to a CSV file.

License

Notifications You must be signed in to change notification settings

gabrielweyer/dotnet-wad-to-csv

Folders and files

NameName
Last commit message
Last commit date

Latest commit

author
Gabriel Weyer
Dec 29, 2019
fbd1060 · Dec 29, 2019

History

44 Commits
Dec 29, 2019
Dec 29, 2019
Dec 29, 2019
Jun 20, 2018
Jan 3, 2019
Jan 13, 2019
Jun 20, 2018
Jun 20, 2018
Dec 29, 2019
Dec 29, 2019
Dec 29, 2019
Dec 29, 2019
Dec 29, 2019
Jun 26, 2018

Repository files navigation

Azure Diagnostics to CSV

.NET Core global tools to retrieve Azure Diagnostics logs from a WADLogsTable or blobs and write them to a CSV file.

Package Release
dotnet-wad-to-csv NuGet
dotnet-blob-to-csv NuGet
CI Status Platform(s) Framework(s)
AppVeyor Build Status Windows netcoreapp3.1

Installation

> dotnet tool install -g dotnet-wad-to-csv
> dotnet tool install -g dotnet-blob-to-csv

WAD to CSV Usage

> dotnet wad-to-csv -l <last> -o <output-file-path>
> dotnet wad-to-csv -f <from> -o <output-file-path>
> dotnet wad-to-csv -f <from> -t <to> -o <output-file-path>
  • <last>: based on the time designator of the ISO 8601 durations. This duration is then substracted from the current UTC time. For example:
    • 5M: get all the logs for the last 5 minutes
    • 1H: get all the logs for the last 1 hour
    • You can combine them too if you feel like it, i.e. 2H3M5S would get all the logs for the last 7385 seconds
  • <from>: ISO 8601 date time expressed in UTC. Cannot be combined with <last>, can be combined with <to>.
    • Valid date time: 2018-06-24T23:12:15
    • The time component can be omitted: 2018-06-24
  • <to>: ISO 8601 date time expressed in UTC. Must be be combined with <from>.
  • <output-file-path>: where you wish to write the output file, does not need to exist but should be valid. If a file exists with the same name it will be replaced.

The tool will prompt you for a Shared Access Signature so that it doesn't get saved to your CLI history. I recommend you restrict the SAS to:

  • Read and List permissions
  • Tables service
  • Container and Object resource types
  • A short expiration time

WAD to CSV will attempt to obfuscate settings that have been logged due to this issue in the Microsoft.WindowsAzure.ConfigurationManager.

Blob to CSV Usage

> dotnet blob-to-csv -l <last> -o <output-file-path> -c <container> -p <prefix>
> dotnet blob-to-csv -f <from> -o <output-file-path> -c <container> -p <prefix>
> dotnet blob-to-csv -f <from> -t <to> -o <output-file-path> -c <container> -p <prefix>
  • <last>: based on the time designator of the ISO 8601 durations. This duration is then substracted from the current UTC time. For example:
    • 5M: get all the logs for the last 5 minutes
    • 1H: get all the logs for the last 1 hour
    • You can combine them too if you feel like it, i.e. 2H3M5S would get all the logs for the last 7385 seconds
  • <from>: ISO 8601 date time expressed in UTC. Cannot be combined with <last>, can be combined with <to>.
    • Valid date time: 2018-06-24T23:12:15
    • The time component can be omitted: 2018-06-24
  • <to>: ISO 8601 date time expressed in UTC. Must be be combined with <from>.
  • <output-file-path>: where you wish to write the output file, does not need to exist but should be valid. If a file exists with the same name it will be replaced.
  • <container>: The name of the container. For https://account.blob.core.windows.net/container-name/prefix/2018/06/22/00/e872fe-54660.applicationLog.csv, the container name is container-name.
  • <prefix>: The prefix (if any). For https://account.blob.core.windows.net/container-name/prefix/2018/06/22/00/e872fe-54660.applicationLog.csv, the prefix is prefix.

The tool will prompt you for a Shared Access Signature so that it doesn't get saved to your CLI history. I recommend you restrict the SAS to:

  • Read and List permissions
  • Blobs service
  • Container and Object resource types
  • A short expiration time

Output file format

Generated,Level,Message
2018-06-18T09:50:28.155T,Information,"Some logging event I wrote"
  • Generated is expressed in UTC
  • The Level is written as-is for blobs and converted from the Level column for the WADLogsTable:
    • 1: Fatal
    • 2: Error
    • 3: Warning
    • 4: Information
    • 5: Verbose
    • Everything else: Undefined

Note: this format is compatible with the Splunk csv source type.

About

.NET Core global tools to retrieve Azure Diagnostics logs from a WADLogsTable or blobs and write them to a CSV file.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published