terraform parser error \"NUMBER\" is not a valid template control keyword. (grok pattern in template file breaks trivy)

[armas-mk]

Description

Trivy crashes during config scanning a terraform plan if a .tftpl template file with grok pattern is rendered via the templatefile() built-in function in Terraform.

Desired Behavior

trivy shouldn't try to process keywords escaped with %%. The templatefile syntax is correct and terraform validate reports no problem and can deploy with terraform with template rendered into desired output.

Actual Behavior

When templatefile has grok_pattern: '%%{TIMESTAMP_ISO8601:time} [%%{NUMBER:pid}] %%{GREEDYDATA:message}'
trivy throws
ERROR [terraform parser] Error parsing file module="root" file_path="main.tf" err="main.tf:604,63-69: Invalid template control keyword; \"NUMBER\" is not a valid template control keyword."

Reproduction Steps

1. Place a valid grok pattern in a terraform template file with proper %% escaping
2. run trivy config on the tfplan that has the rendered template

Target

None

Scanner

Misconfiguration

Output Format

Table

Mode

Standalone

Debug Output

2024-09-19T12:13:19+01:00	DEBUG	No plugins loaded
2024-09-19T12:13:19+01:00	DEBUG	Default config file "file_path=trivy.yaml" not found, using built in values
2024-09-19T12:13:19+01:00	DEBUG	Cache dir	dir="/home/user/.cache/trivy"
2024-09-19T12:13:19+01:00	DEBUG	Cache dir	dir="/home/user/.cache/trivy"
2024-09-19T12:13:19+01:00	DEBUG	Parsed severities	severities=[MEDIUM HIGH CRITICAL]
2024-09-19T12:13:19+01:00	INFO	[misconfig] Misconfiguration scanning is enabled
2024-09-19T12:13:19+01:00	DEBUG	[misconfig] Checks successfully loaded from disk
2024-09-19T12:13:19+01:00	DEBUG	Enabling misconfiguration scanners	scanners=[azure-arm cloudformation dockerfile helm kubernetes terraform terraformplan-json terraformplan-snapshot]
2024-09-19T12:13:19+01:00	DEBUG	Initializing scan cache...	type="memory"
2024-09-19T12:13:20+01:00	DEBUG	Scanning files for misconfigurations...	scanner="Terraform Plan JSON"
2024-09-19T12:13:20+01:00	DEBUG	[tfjson scanner] Scanning file	file_path="test.tfplan.json"
2024-09-19T12:13:20+01:00	DEBUG	[terraform scanner] Scanning directory	file_path="."
2024-09-19T12:13:20+01:00	DEBUG	[rego] Overriding filesystem for checks
2024-09-19T12:13:20+01:00	DEBUG	[rego] Embedded libraries are loaded	count=11
2024-09-19T12:13:20+01:00	DEBUG	[rego] Embedded checks are loaded	count=508
2024-09-19T12:13:20+01:00	DEBUG	[rego] Checks from disk are loaded	count=195
2024-09-19T12:13:20+01:00	DEBUG	[rego] Overriding filesystem for data
2024-09-19T12:13:20+01:00	DEBUG	[terraform parser] Setting project/module root	module="root" file_path="."
2024-09-19T12:13:20+01:00	DEBUG	[terraform parser] Parsing FS	module="root" file_path="."
2024-09-19T12:13:20+01:00	DEBUG	[terraform parser] Parsing	module="root" file_path="main.tf"
2024-09-19T12:13:21+01:00	ERROR	[terraform parser] Error parsing file	module="root" file_path="main.tf" err="main.tf:604,63-69: Invalid template control keyword; \"NUMBER\" is not a valid template control keyword."
2024-09-19T12:13:21+01:00	INFO	[terraform scanner] Scanning root module	file_path="."
2024-09-19T12:13:21+01:00	DEBUG	[terraform parser] Setting project/module root	module="root" file_path="."
2024-09-19T12:13:21+01:00	DEBUG	[terraform parser] Parsing FS	module="root" file_path="."
2024-09-19T12:13:21+01:00	DEBUG	[terraform parser] Parsing	module="root" file_path="main.tf"
2024-09-19T12:13:21+01:00	ERROR	[terraform parser] Error parsing file	module="root" file_path="main.tf" err="main.tf:604,63-69: Invalid template control keyword; \"NUMBER\" is not a valid template control keyword."
2024-09-19T12:13:21+01:00	DEBUG	[terraform parser] Loading module	module="root" module="root"
2024-09-19T12:13:21+01:00	INFO	[terraform parser] No files found, nothing to do.	module="root"
2024-09-19T12:13:21+01:00	DEBUG	[terraform executor] Adapting modules...
2024-09-19T12:13:21+01:00	DEBUG	[terraform executor] Adapted module(s) into state data.	count=0
2024-09-19T12:13:21+01:00	DEBUG	[terraform executor] Using max routines	count=19
2024-09-19T12:13:21+01:00	DEBUG	[terraform executor] Initialized Go check(s).	count=775
2024-09-19T12:13:21+01:00	DEBUG	[rego] Scannning inputs	count=1
2024-09-19T12:13:21+01:00	DEBUG	[terraform executor] Finished applying rules.
2024-09-19T12:13:21+01:00	DEBUG	[terraform executor] Applying ignores...
2024-09-19T12:13:21+01:00	DEBUG	OS is not detected.
2024-09-19T12:13:21+01:00	INFO	Detected config files	num=1
2024-09-19T12:13:21+01:00	DEBUG	Scanned config file	file_path=""
2024-09-19T12:13:21+01:00	DEBUG	[vex] VEX filtering is disabled

Operating System

Ubuntu 22.04.4 LTS

Version

Version: 0.55.2
Vulnerability DB:
  Version: 2
  UpdatedAt: 2024-05-24 06:12:01.989181315 +0000 UTC
  NextUpdate: 2024-05-24 12:12:01.989180995 +0000 UTC
  DownloadedAt: 2024-05-24 11:57:40.984020511 +0000 UTC
Check Bundle:
  Digest: sha256:5c4c5f5b566ddbc83888108ba8d6c6f23098cf51a848a8d4f40e2865f18861aa
  DownloadedAt: 2024-09-19 10:53:13.005812206 +0000 UTC

Checklist

• Run trivy clean --all
• Read the troubleshooting

[nikpivkin]

Hi @armas-mk !

Could you give a minimally reproducible example?

[armas-mk]

Hey @nikpivkin , here is an easily reproducible example

1. Create main.tf file

provider "local" {}

resource "local_file" "rendered_template" {
  content  = templatefile("${path.module}/template.tpl", {
  })
  filename = "${path.module}/rendered_template.txt"
}

2. Create template.tpl file

# Grok Pattern Template
grok_pattern = "%%{TIMESTAMP_ISO8601:time} \\[%%{NUMBER:pid}\\] %%{GREEDYDATA:message}"

3. Run terraform init
4. create tfplan
   terraform plan --out test.tfplan
5. convert tfplan to json
   terraform show --json test.tfplan > test.tfplan.json
6. run trivy

$ trivy config test.tfplan.json -d
2024-09-19T13:10:44+01:00	DEBUG	No plugins loaded
2024-09-19T13:10:44+01:00	DEBUG	Default config file "file_path=trivy.yaml" not found, using built in values
2024-09-19T13:10:44+01:00	DEBUG	Cache dir	dir="/home/user/.cache/trivy"
2024-09-19T13:10:44+01:00	DEBUG	Cache dir	dir="/home/user/.cache/trivy"
2024-09-19T13:10:44+01:00	DEBUG	Parsed severities	severities=[UNKNOWN LOW MEDIUM HIGH CRITICAL]
2024-09-19T13:10:44+01:00	INFO	[misconfig] Misconfiguration scanning is enabled
2024-09-19T13:10:44+01:00	DEBUG	[misconfig] Checks successfully loaded from disk
2024-09-19T13:10:44+01:00	DEBUG	Enabling misconfiguration scanners	scanners=[azure-arm cloudformation dockerfile helm kubernetes terraform terraformplan-json terraformplan-snapshot]
2024-09-19T13:10:44+01:00	DEBUG	Initializing scan cache...	type="memory"
2024-09-19T13:10:44+01:00	DEBUG	Scanning files for misconfigurations...	scanner="Terraform Plan JSON"
2024-09-19T13:10:44+01:00	DEBUG	[tfjson scanner] Scanning file	file_path="test.tfplan.json"
2024-09-19T13:10:44+01:00	DEBUG	[terraform scanner] Scanning directory	file_path="."
2024-09-19T13:10:44+01:00	DEBUG	[rego] Overriding filesystem for checks
2024-09-19T13:10:44+01:00	DEBUG	[rego] Embedded libraries are loaded	count=11
2024-09-19T13:10:44+01:00	DEBUG	[rego] Embedded checks are loaded	count=508
2024-09-19T13:10:44+01:00	DEBUG	[rego] Checks from disk are loaded	count=195
2024-09-19T13:10:44+01:00	DEBUG	[rego] Overriding filesystem for data
2024-09-19T13:10:45+01:00	DEBUG	[terraform parser] Setting project/module root	module="root" file_path="."
2024-09-19T13:10:45+01:00	DEBUG	[terraform parser] Parsing FS	module="root" file_path="."
2024-09-19T13:10:45+01:00	DEBUG	[terraform parser] Parsing	module="root" file_path="main.tf"
2024-09-19T13:10:45+01:00	ERROR	[terraform parser] Error parsing file	module="root" file_path="main.tf" err="main.tf:4,49-55: Invalid template control keyword; \"NUMBER\" is not a valid template control keyword."
2024-09-19T13:10:45+01:00	INFO	[terraform scanner] Scanning root module	file_path="."
2024-09-19T13:10:45+01:00	DEBUG	[terraform parser] Setting project/module root	module="root" file_path="."
2024-09-19T13:10:45+01:00	DEBUG	[terraform parser] Parsing FS	module="root" file_path="."
2024-09-19T13:10:45+01:00	DEBUG	[terraform parser] Parsing	module="root" file_path="main.tf"
2024-09-19T13:10:45+01:00	ERROR	[terraform parser] Error parsing file	module="root" file_path="main.tf" err="main.tf:4,49-55: Invalid template control keyword; \"NUMBER\" is not a valid template control keyword."
2024-09-19T13:10:45+01:00	DEBUG	[terraform parser] Loading module	module="root" module="root"
2024-09-19T13:10:45+01:00	INFO	[terraform parser] No files found, nothing to do.	module="root"
2024-09-19T13:10:45+01:00	DEBUG	[terraform executor] Adapting modules...
2024-09-19T13:10:45+01:00	DEBUG	[terraform executor] Adapted module(s) into state data.	count=0
2024-09-19T13:10:45+01:00	DEBUG	[terraform executor] Using max routines	count=19
2024-09-19T13:10:45+01:00	DEBUG	[terraform executor] Initialized Go check(s).	count=775
2024-09-19T13:10:45+01:00	DEBUG	[rego] Scannning inputs	count=1
2024-09-19T13:10:45+01:00	DEBUG	[terraform executor] Finished applying rules.
2024-09-19T13:10:45+01:00	DEBUG	[terraform executor] Applying ignores...
2024-09-19T13:10:45+01:00	DEBUG	OS is not detected.
2024-09-19T13:10:45+01:00	INFO	Detected config files	num=1
2024-09-19T13:10:45+01:00	DEBUG	Scanned config file	file_path=""
2024-09-19T13:10:45+01:00	DEBUG	[vex] VEX filtering is disabled

[nikpivkin]

Thanks

[nikpivkin]

Track #7557