Skip to content

Commit

Permalink
docs: Add mdox remote/local link checking and move web preprocess (t…
Browse files Browse the repository at this point in the history
…hanos-io#4357)

* Add mdox remote and local link checking

Signed-off-by: Saswata Mukherjee <[email protected]>

* Add mdox transform

Signed-off-by: Saswata Mukherjee <[email protected]>

* Update mdox and config

Signed-off-by: Saswata Mukherjee <[email protected]>

* Make YAML DRY & fix prev-release

Signed-off-by: Saswata Mukherjee <[email protected]>

* Localize tip links; Switch to Origin.Path

Signed-off-by: Saswata Mukherjee <[email protected]>
  • Loading branch information
saswatamcode authored Aug 17, 2021
1 parent 7a8d189 commit 9d6b004
Show file tree
Hide file tree
Showing 62 changed files with 375 additions and 414 deletions.
6 changes: 3 additions & 3 deletions .bingo/Variables.mk
Original file line number Diff line number Diff line change
Expand Up @@ -89,11 +89,11 @@ $(JSONNETFMT): $(BINGO_DIR)/jsonnetfmt.mod
@echo "(re)installing $(GOBIN)/jsonnetfmt-v0.17.0"
@cd $(BINGO_DIR) && $(GO) build -mod=mod -modfile=jsonnetfmt.mod -o=$(GOBIN)/jsonnetfmt-v0.17.0 "github.com/google/go-jsonnet/cmd/jsonnetfmt"

MDOX := $(GOBIN)/mdox-v0.2.2-0.20210617084122-22b44c491197
MDOX := $(GOBIN)/mdox-v0.2.2-0.20210810104227-dd2d81061a27
$(MDOX): $(BINGO_DIR)/mdox.mod
@# Install binary/ries using Go 1.14+ build command. This is using bwplotka/bingo-controlled, separate go module with pinned dependencies.
@echo "(re)installing $(GOBIN)/mdox-v0.2.2-0.20210617084122-22b44c491197"
@cd $(BINGO_DIR) && $(GO) build -mod=mod -modfile=mdox.mod -o=$(GOBIN)/mdox-v0.2.2-0.20210617084122-22b44c491197 "github.com/bwplotka/mdox"
@echo "(re)installing $(GOBIN)/mdox-v0.2.2-0.20210810104227-dd2d81061a27"
@cd $(BINGO_DIR) && $(GO) build -mod=mod -modfile=mdox.mod -o=$(GOBIN)/mdox-v0.2.2-0.20210810104227-dd2d81061a27 "github.com/bwplotka/mdox"

MINIO := $(GOBIN)/minio-v0.0.0-20200527010300-cccf2de129da
$(MINIO): $(BINGO_DIR)/minio.mod
Expand Down
2 changes: 1 addition & 1 deletion .bingo/mdox.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ go 1.16

replace github.com/Kunde21/markdownfmt/v2 => github.com/bwplotka/markdownfmt/v2 v2.0.0-20210616121647-559e77044d46

require github.com/bwplotka/mdox v0.2.2-0.20210617084122-22b44c491197
require github.com/bwplotka/mdox v0.2.2-0.20210810104227-dd2d81061a27
2 changes: 1 addition & 1 deletion .bingo/variables.env
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ JSONNET="${GOBIN}/jsonnet-v0.17.0"

JSONNETFMT="${GOBIN}/jsonnetfmt-v0.17.0"

MDOX="${GOBIN}/mdox-v0.2.2-0.20210617084122-22b44c491197"
MDOX="${GOBIN}/mdox-v0.2.2-0.20210810104227-dd2d81061a27"

MINIO="${GOBIN}/minio-v0.0.0-20200527010300-cccf2de129da"

Expand Down
65 changes: 65 additions & 0 deletions .mdox.prev-release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
version: 1

inputDir: "$(OUTPUT_CONTENT_DIR)/$(tags)-git-docs"
outputDir: "$(OUTPUT_CONTENT_DIR)/$(tags)"
extraInputGlobs:
- "CHANGELOG.md"
- "SECURITY.md"
- "MAINTAINERS.md"
- "CONTRIBUTING.md"
- "CODE_OF_CONDUCT.md"

gitIgnored: true
localLinksStyle:
hugo:
indexFileName: "_index.md"

transformations:

- glob: "../../../CHANGELOG.md"
path: /thanos/CHANGELOG.md
frontMatter:
template: |
title: Changelog
type: docs
menu: thanos
lastmod: "{{ .Origin.LastMod }}"
- glob: "../../../MAINTAINERS.md"
path: /thanos/MAINTAINERS.md
frontMatter:
template: |
title: Maintainers
type: docs
menu: thanos
lastmod: "{{ .Origin.LastMod }}"
- glob: "../../../SECURITY.md"
path: /thanos/SECURITY.md
frontMatter:
template: |
title: Security
type: docs
menu: thanos
lastmod: "{{ .Origin.LastMod }}"
- glob: "../../../CODE_OF_CONDUCT.md"
path: /contributing/CODE_OF_CONDUCT.md
frontMatter:
template: |
title: Code of Conduct
type: docs
menu: contributing
lastmod: "{{ .Origin.LastMod }}"
- glob: "../../../CONTRIBUTING.md"
path: /contributing/CONTRIBUTING.md
frontMatter:
template: |
title: Contributing
type: docs
menu: contributing
lastmod: "{{ .Origin.LastMod }}"
- glob: "*.md"
path: /thanos/*
19 changes: 19 additions & 0 deletions .mdox.validate.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
version: 1

validators:
# Validators to skip checking PR/issue links of Thanos, Prometheus and Cortex.
- regex: '(^http[s]?:\/\/)(www\.)?(github\.com\/)thanos-io\/thanos(\/pull\/|\/issues\/)'
type: 'githubPullsIssues'
- regex: '(^http[s]?:\/\/)(www\.)?(github\.com\/)prometheus\/prometheus(\/pull\/|\/issues\/)'
type: 'githubPullsIssues'
- regex: '(^http[s]?:\/\/)(www\.)?(github\.com\/)cortexproject\/cortex(\/pull\/|\/issues\/)'
type: 'githubPullsIssues'
# Ignore Thanos release links.
- regex: '(^http[s]?:\/\/)(www\.)?(github\.com\/)thanos-io\/thanos(\/releases\/)'
type: 'ignore'
# Causes http stream errors with statuscode 0 sometimes. But is safe to skip.
- regex: 'slack\.cncf\.io'
type: 'ignore'
# 301 errors even when curl-ed.
- regex: 'envoyproxy\.io'
type: 'ignore'
146 changes: 146 additions & 0 deletions .mdox.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
version: 1

inputDir: "docs"
outputDir: "website/docs-pre-processed/tip"
extraInputGlobs:
- "CHANGELOG.md"
- "SECURITY.md"
- "MAINTAINERS.md"
- "CONTRIBUTING.md"
- "CODE_OF_CONDUCT.md"

gitIgnored: true
localLinksStyle:
hugo:
indexFileName: "_index.md"

transformations:

- glob: "../CHANGELOG.md"
path: /thanos/CHANGELOG.md
frontMatter:
template: |
title: Changelog
type: docs
menu: thanos
lastmod: "{{ .Origin.LastMod }}"
backMatter: &docBackMatter
template: |
Found a typo, inconsistency or missing information in our docs?
Help us to improve [Thanos](https://thanos.io) documentation by proposing a fix [on GitHub here](https://github.com/thanos-io/thanos/edit/main/{{ .Origin.Path }}) :heart:
- glob: "../MAINTAINERS.md"
path: /thanos/MAINTAINERS.md
frontMatter:
template: |
title: Maintainers
type: docs
menu: thanos
lastmod: "{{ .Origin.LastMod }}"
backMatter: *docBackMatter

- glob: "../SECURITY.md"
path: /thanos/SECURITY.md
frontMatter:
template: |
title: Security
type: docs
menu: thanos
lastmod: "{{ .Origin.LastMod }}"
backMatter: *docBackMatter

- glob: "../CODE_OF_CONDUCT.md"
path: /contributing/CODE_OF_CONDUCT.md
frontMatter:
template: |
title: Code of Conduct
type: docs
menu: contributing
lastmod: "{{ .Origin.LastMod }}"
backMatter: *docBackMatter

- glob: "../CONTRIBUTING.md"
path: /contributing/CONTRIBUTING.md
frontMatter:
template: |
title: Contributing
type: docs
menu: contributing
lastmod: "{{ .Origin.LastMod }}"
backMatter: *docBackMatter

- glob: "**/README.md"
path: _index.md
frontMatter: &justTitleFrontMatter
template: |
title: "{{ .Origin.FirstHeader }}"
backMatter: *docBackMatter

- glob: "getting-started.md"
path: /thanos/getting-started.md
popHeader: false
frontMatter: &weightOneFrontMatter
template: |
type: docs
title: "{{ .Origin.FirstHeader }}"
menu: thanos
weight: 1
lastmod: "{{ .Origin.LastMod }}"
backMatter: *docBackMatter

- glob: "quick-tutorial.md"
path: /thanos/quick-tutorial.md
popHeader: false
frontMatter: *weightOneFrontMatter
backMatter: *docBackMatter

- glob: "proposals-*/README.md"
path: /proposals-*/_index.md
frontMatter: *justTitleFrontMatter
backMatter: *docBackMatter

- glob: "README.md"
path: /thanos/_index.md
frontMatter: *justTitleFrontMatter
backMatter: *docBackMatter

- glob: "*.md"
path: /thanos/*
popHeader: false
frontMatter:
template: |
type: docs
title: "{{ .Origin.FirstHeader }}"
menu: thanos
lastmod: "{{ .Origin.LastMod }}"
backMatter: *docBackMatter

- glob: "operating/*.md"
path: /operating/*
frontMatter:
template: |
type: docs
title: "{{ .Origin.FirstHeader }}"
menu: operating
lastmod: "{{ .Origin.LastMod }}"
backMatter: *docBackMatter

- glob: "contributing/*.md"
path: /contributing/*
frontMatter:
template: |
type: docs
title: "{{ .Origin.FirstHeader }}"
menu: contributing
lastmod: "{{ .Origin.LastMod }}"
backMatter: *docBackMatter

- glob: "components/*.md"
path: /components/*
frontMatter:
template: |
type: docs
title: "{{ .Origin.FirstHeader }}"
menu: components
lastmod: "{{ .Origin.LastMod }}"
backMatter: *docBackMatter
18 changes: 9 additions & 9 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re

Highlights:

- New Thanos component, [Query Frontend](https://thanos.io/tip/components/query-frontend.md/) has more options and supports shared cache (currently: Memcached).
- New Thanos component, [Query Frontend](docs/components/query-frontend.md) has more options and supports shared cache (currently: Memcached).
- Added debug mode in Thanos UI that allows to filter Stores to query from by their IPs from Store page (!). This helps enormously in e.g debugging the slowest store etc. All raw Thanos API allows passing `storeMatch[]` arguments with `__address__` matchers.
- Improved debuggability on all Thanos components by exposing [off-CPU profiles thanks to fgprof endpoint](https://github.com/felixge/fgprof).
- Significantly improved sidecar latency and CPU usage for metrics fetches.
Expand Down Expand Up @@ -302,7 +302,7 @@ Highlights:

Highlights:

- Added new Thanos component: [Query Frontend](https://thanos.io/v0.15/components/query-frontend/) responsible for response caching, query scheduling and parallelization (based on Cortex Query Frontend).
- Added new Thanos component: [Query Frontend](https://thanos.io/v0.15/components/query-frontend.md/) responsible for response caching, query scheduling and parallelization (based on Cortex Query Frontend).
- Added various new, improved UIs to Thanos based on React: Querier BuildInfo & Flags, Ruler UI, BlockViewer.
- Optimized Sidecar, Store, Receive, Ruler data retrieval with new TSDB ChunkIterator (capping chunks to 120 samples), which fixed various leaks.
- Fixed sample limit on Store Gateway.
Expand Down Expand Up @@ -344,7 +344,7 @@ Highlights:
- [#2973](https://github.com/thanos-io/thanos/pull/2973) Add Thanos Query Frontend component.
- [#2980](https://github.com/thanos-io/thanos/pull/2980) Bucket Viewer: Migrate block viewer to React.
- [#2725](https://github.com/thanos-io/thanos/pull/2725) Add bucket index operation durations: `thanos_bucket_store_cached_series_fetch_duration_seconds` and `thanos_bucket_store_cached_postings_fetch_duration_seconds`.
- [#2931](https://github.com/thanos-io/thanos/pull/2931) Query: Allow passing a `storeMatch[]` to select matching stores when debugging the querier. See [documentation](https://thanos.io/tip/components/query.md/#store-filtering)
- [#2931](https://github.com/thanos-io/thanos/pull/2931) Query: Allow passing a `storeMatch[]` to select matching stores when debugging the querier. See [documentation](docs/components/query.md#store-filtering)

### Changed

Expand Down Expand Up @@ -422,7 +422,7 @@ sse_config:
### Changed

- [#2194](https://github.com/thanos-io/thanos/pull/2194) Updated to golang v1.14.2.
- [#2505](https://github.com/thanos.io/thanos/pull/2505) Store: Removed obsolete `thanos_store_node_info` metric.
- [#2505](https://github.com/thanos-io/thanos/pull/2505) Store: Removed obsolete `thanos_store_node_info` metric.
- [#2513](https://github.com/thanos-io/thanos/pull/2513) Tools: Moved `thanos bucket` commands to `thanos tools bucket`, also moved `thanos check rules` to `thanos tools rules-check`. `thanos tools rules-check` also takes rules by `--rules` repeated flag not argument anymore.
- [#2548](https://github.com/thanos-io/thanos/pull/2548/commits/53e69bd89b2b08c18df298eed7d90cb7179cc0ec) Store, Querier: remove duplicated chunks on StoreAPI.
- [#2596](https://github.com/thanos-io/thanos/pull/2596) Updated Prometheus dependency to [@cd73b3d33e064bbd846fc7a26dc8c313d46af382](https://github.com/prometheus/prometheus/commit/cd73b3d33e064bbd846fc7a26dc8c313d46af382) which falls in between v2.17.0 and v2.18.0.
Expand Down Expand Up @@ -468,7 +468,7 @@ sse_config:

### Added

- [#2252](https://github.com/thanos-io/thanos/pull/2252) Query: add new `--store-strict` flag. More information available [here](/docs/proposals/202001_thanos_query_health_handling.md).
- [#2252](https://github.com/thanos-io/thanos/pull/2252) Query: add new `--store-strict` flag. More information available [here](docs/proposals-done/202001-thanos-query-health-handling.md).
- [#2265](https://github.com/thanos-io/thanos/pull/2265) Compact: add `--wait-interval` to specify compaction wait interval between consecutive compact runs when `--wait` is enabled.
- [#2250](https://github.com/thanos-io/thanos/pull/2250) Compact: enable vertical compaction for offline deduplication (experimental). Uses `--deduplication.replica-label` flag to specify the replica label on which to deduplicate (hidden). Please note that this uses a NAIVE algorithm for merging (no smart replica deduplication, just chaining samples together). This works well for deduplication of blocks with **precisely the same samples** like those produced by Receiver replication. We plan to add a smarter algorithm in the following weeks.
- [#1714](https://github.com/thanos-io/thanos/pull/1714) Compact: the compact component now exposes the bucket web UI when it is run as a long-lived process.
Expand Down Expand Up @@ -507,7 +507,7 @@ sse_config:
### Added

- [#2003](https://github.com/thanos-io/thanos/pull/2003) Query: Support downsampling for /series.
- [#1952](https://github.com/thanos-io/thanos/pull/1952) Store Gateway: Implemented [binary index header](https://thanos.io/tip/proposals/201912_thanos_binary_index_header.md/). This significantly reduces resource consumption (memory, CPU, net bandwidth) for startup and data loading processes as well as baseline memory. This means that adding more blocks into object storage, without querying them will use almost no resources. This, however, **still means that querying large amounts of data** will result in high spikes of memory and CPU use as before, due to simply fetching large amounts of metrics data. Since we fixed baseline, we are now focusing on query performance optimizations in separate initiatives. To enable experimental `index-header` mode run store with hidden `experimental.enable-index-header` flag.
- [#1952](https://github.com/thanos-io/thanos/pull/1952) Store Gateway: Implemented [binary index header](docs/proposals-done/201912-thanos-binary-index-header.md). This significantly reduces resource consumption (memory, CPU, net bandwidth) for startup and data loading processes as well as baseline memory. This means that adding more blocks into object storage, without querying them will use almost no resources. This, however, **still means that querying large amounts of data** will result in high spikes of memory and CPU use as before, due to simply fetching large amounts of metrics data. Since we fixed baseline, we are now focusing on query performance optimizations in separate initiatives. To enable experimental `index-header` mode run store with hidden `experimental.enable-index-header` flag.
- [#2009](https://github.com/thanos-io/thanos/pull/2009) Store Gateway: Minimum age of all blocks before they are being read. Set it to a safe value (e.g 30m) if your object storage is eventually consistent. GCS and S3 are (roughly) strongly consistent.
- [#1963](https://github.com/thanos-io/thanos/pull/1963) Mixin: Add Thanos Ruler alerts.
- [#1984](https://github.com/thanos-io/thanos/pull/1984) Query: Add cache-control header to not cache on error.
Expand Down Expand Up @@ -856,7 +856,7 @@ This version moved tarballs to Golang 1.12.5 from 1.11 as well, so same warning

:warning: **IMPORTANT** :warning: This is the last release that supports gossip. From Thanos v0.5.0, gossip will be completely removed.

This release also disables gossip mode by default for all components. See [this](docs/proposals/201809_gossip-removal.md) for more details.
This release also disables gossip mode by default for all components. See [this](docs/proposals-done/201809-gossip-removal.md) for more details.

:warning: This release moves Thanos docker images (NOT artifacts by accident) to Golang 1.12. This release includes change in GC's memory release which gives following effect:

Expand Down Expand Up @@ -1009,7 +1009,7 @@ New Store tracing span: \* `store_query_gate_ismyturn` shows how long it took fo

- Support for gzip compressed configuration files before envvar substitution for reloader package.
- `bucket inspect` command for better insights on blocks in object storage.
- Support for [Tencent COS](docs/storage.md#tencent-cos-configuration) object storage.
- Support for [Tencent COS](docs/storage.md#tencent-cos) object storage.
- Partial Response disable option for StoreAPI and QueryAPI.
- Partial Response disable button on Thanos UI
- We have initial docs for goDoc documentation!
Expand Down Expand Up @@ -1130,7 +1130,7 @@ Note lots of necessary breaking changes in flags that relates to bucket configur

## [v0.1.0](https://github.com/thanos-io/thanos/releases/tag/v0.1.0) - 2018.09.14

Initial version to have a stable reference before [gossip protocol removal](/docs/proposals/201809_gossip-removal.md).
Initial version to have a stable reference before [gossip protocol removal](docs/proposals-done/201809-gossip-removal.md).

### Added

Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ If you encounter a security vulnerability, please refer to [Reporting a Vulnerab

When contributing a complex change to Thanos repository, please discuss the change you wish to make within a Github issue, in Slack, or by another method with the owners of this repository before making the change.

Adding a large new feature or/and component to Thanos should be done by first creating a [proposal](docs/contributing/proposal-process.md) document outlining the design decisions of the change, motivations for the change, and any alternatives that might have been considered.
Adding a large new feature or/and component to Thanos should be done by first creating a [proposal](docs/proposals-done) document outlining the design decisions of the change, motivations for the change, and any alternatives that might have been considered.

## General Naming

Expand Down
2 changes: 1 addition & 1 deletion MAINTAINERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ Self explanatory ones:

## Storage plugins maintainers

Maintainers of bucket storage clients are available [here](/docs/storage.md#implementations)
Maintainers of bucket storage clients are available [here](docs/storage.md#supported-clients)

## How to be maintainer?

Expand Down
Loading

0 comments on commit 9d6b004

Please sign in to comment.