DM: support PCRE regexp syntax for routing

[takaidohigasi]

Is your feature request related to a problem?

I'd like to consolidate many schema by a simple route config.

Describe the feature you'd like

if there are databases for source like

• mercari_a
• mercari_b
• mercari_c
• mercari_d
• mercari_e

following config do not work

  route-a:
    schema-pattern: "mercari_a"
    target-schema: "mercari"
  route-others:
    schema-pattern: "mercari_(?!a).*"
    target-schema: "mercari-other"

Describe alternatives you've considered

No response

Teachability, Documentation, Adoption, Migration Strategy

No response

[takaidohigasi]

I might mistake the repository to create issue
https://github.com/pingcap/tidb/blob/master/pkg/util/table-router/router.go

[takaidohigasi]

I understood Golang regxp support RE2 syntax, not PCRE

[takaidohigasi]

I understand there isn't lightweight safe proper library for Golang...

[lance6716]

You can use mercari_[^a] in RE2 syntax. I think it's confusing to have two RE flavors and they can be converted to the other in most cases.

[kennytm]

the actual wildcard syntax is mercari_[!a]. you need to prefix with ~ to use the regexp syntax. but the doc still seems to only recommend the wildcards * and ? at the end

schema-pattern: 'mercari_[!a]'
# or
schema-pattern: '~^mercari_[^a]'

[takaidohigasi]

sorry My example was a bit wrong and I think I can't cope with existing RE2 syntax

• mercari_sample
• mercari_sample2
• mercari_sample_hoge
• mercari_asample
• mercari_bsample
• ...

following config do not work

  route-a:
    schema-pattern: "mercari_sample"
    target-schema: "mercari"
  route-others:
    schema-pattern: "^mercari_(?!sample$).*$"

[kennytm]

It won't work because assertions (?!x) is not "regular" and thus not part of RE2 syntax. There is no plan to support regex flavors other than Golang's built-in one.

Route-rules do not allow overlapping matches¹, but if the "other" schemas are a finite set you could always do

routes:
  route_a:
    schema-pattern: mercari_sample
    target-schema: mercari
  route_others:
    schema-pattern: '~^mercari_(sample2|sample_hoge|asample|bsample|...etc...)$'
    target-schema: mercari_others

Footnotes

1. I think this can be relaxed, given that route-rules being an array with order, we can easily break the tie by favor the first matching rule. ↩