Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic happens when opening a boltdb #705

Open
aa624545345 opened this issue Mar 6, 2024 · 2 comments
Open

Panic happens when opening a boltdb #705

aa624545345 opened this issue Mar 6, 2024 · 2 comments

Comments

@aa624545345
Copy link

I'm not sure whether this is an issue, but it's confusing.
In my scenario, when containerd service was running, the host powered off. Next time host started, then containerd started, panic happened when containerd was trying to open the meta.db.

panic: assertion failed: Page expected to be: 3393, but self identifies as 6868074383292906018

goroutine 31 [running]:
go.etcd.io/bbolt._assert(...)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/vendor/go.etcd.io/bbolt/db.go:1359
go.etcd.io/bbolt.(*page).fastCheck(0x7f8a70d41000, 0xd41)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/vendor/go.etcd.io/bbolt/page.go:57 +0x1df
go.etcd.io/bbolt.(*Tx).page(0xc000372fa0?, 0x5625f8aea50b?)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/vendor/go.etcd.io/bbolt/tx.go:534 +0x8a
go.etcd.io/bbolt.(*Tx).forEachPageInternal(0x0?, {0xc000120460?, 0x1, 0xa}, 0xc000373078)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/vendor/go.etcd.io/bbolt/tx.go:546 +0x65
go.etcd.io/bbolt.(*Tx).forEachPage(...)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/vendor/go.etcd.io/bbolt/tx.go:542
go.etcd.io/bbolt.(*Tx).checkBucket(0xc00027a0e0, 0xc00027a0f8, 0xc000373240, 0xc000373210, {0x5625fa6f2ba0?, 0x5625fb4ab008}, 0xc000064b40)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/vendor/go.etcd.io/bbolt/tx_check.go:83 +0x126
go.etcd.io/bbolt.(*DB).freepages(0x5625f9de40bb?)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/vendor/go.etcd.io/bbolt/db.go:1181 +0x229
go.etcd.io/bbolt.(*DB).loadFreelist.func1()
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/vendor/go.etcd.io/bbolt/db.go:412 +0xc5
sync.(*Once).doSlow(0xc000164888?, 0x10?)
	/usr/local/go/src/sync/once.go:74 +0xc2
sync.(*Once).Do(...)
	/usr/local/go/src/sync/once.go:65
go.etcd.io/bbolt.(*DB).loadFreelist(0xc0001646c0?)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/vendor/go.etcd.io/bbolt/db.go:408 +0x47
go.etcd.io/bbolt.Open({0xc000026f80, 0x31}, 0xf9de3628?, 0xc0004a1628)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/vendor/go.etcd.io/bbolt/db.go:290 +0x40c
github.com/containerd/containerd/metadata/plugin.init.1.func1(0xc000128380)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/metadata/plugin/plugin.go:159 +0x78c
github.com/containerd/containerd/plugin.(*Registration).Init(0xc0000d2a80, 0xc000128380)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/plugin/plugin.go:127 +0x2e
github.com/containerd/containerd/services/server.New({0x5625fa6fc9e0, 0xc000666230}, 0xc000614700)
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/services/server/server.go:236 +0x13f8
github.com/containerd/containerd/cmd/containerd/command.App.func1.1()
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/cmd/containerd/command/main.go:194 +0x87
created by github.com/containerd/containerd/cmd/containerd/command.App.func1
	/root/rpmbuild/BUILD/containerd.io-1.7.6/_build/src/github.com/containerd/containerd/cmd/containerd/command/main.go:191 +0x8ac

I can provide the meta.db if possible.

@fuweid
Copy link
Member

fuweid commented Mar 6, 2024

@ahrtr
Copy link
Member

ahrtr commented Jun 20, 2024

REF containerd/containerd#9929 (comment)

The root page id (3393) is pointing to part of leaf page.

===============================================
Page ID: 3391
Page Type: leaf
Total Size: 12288 bytes
Overflow pages: 2
Item Count: 3

It looks like there is a potential bug in freelist management, e.g allocate allocates used pages, and bbolt crashes or terminates for whatever reason right after writing the such data pages, but before updating the meta pages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants