[tapsend]: Enforce unique script keys #1181

guggero · 2024-11-08T14:34:17Z

Fixes #1168 (or at least prevents it from happening in the future).

This PR enforces script keys to be unique per top-level MS-SMT tree (assetID/groupKey) within a single transfer.

To be able to test this properly, we also make sure selected UTXOs are released by the freighter if the shipment of a parcel results in an error.
That change then impacts some assumptions in the integration tests, which requires additional commits to fix.

coveralls · 2024-11-08T14:43:43Z

Pull Request Test Coverage Report for Build 11838616455

Details

57 of 119 (47.9%) changed or added relevant lines in 6 files are covered.
26 unchanged lines in 6 files lost coverage.
Overall coverage increased (+0.02%) to 41.053%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
itest/assertions.go	0	4	0.0%
tapsend/send.go	39	46	84.78%
tapfreighter/wallet.go	0	18	0.0%
tapfreighter/chain_porter.go	0	33	0.0%

Files with Coverage Reduction	New Missed Lines	%
itest/assertions.go	1	0.0%
asset/asset.go	2	81.13%
asset/mock.go	3	92.2%
tapgarden/caretaker.go	4	68.5%
commitment/tap.go	4	83.91%
universe/interface.go	12	50.22%

Totals
Change from base Build 11829805032:	0.02%
Covered Lines:	25222
Relevant Lines:	61438

💛 - Coveralls

guggero · 2024-11-08T16:23:53Z

I'll look into the failed itest on Monday, looks like a test is using duplicate keys.

tapsend/send.go

Roasbeef · 2024-11-08T21:04:17Z

tapsend/send.go

+		for idx := range vPkt.Outputs {
+			vOut := vPkt.Outputs[idx]
+
+			// Check if the script key has already been used in this


Isn't it ok for us to have a script key duplicate across Bitcoin transaction outputs?

I think a duplicate script key for the same asset would be an issue, even without those assets being children of an asset split. If that occurred, the exclusion proof would be a proof of inclusion, but for a different leaf vs. the existing model where the exclusion proof terminates in a nil leaf.

exclusion proof terminates in a nil leaf.

Yes, exactly. The way our exclusion proofs currently work is that they expect no leaf to be present at the exclusion location (which is defined by the script key).

Hmm yeh, I do wonder if there's a small change we can make the the exclusion proofs here to patch this portion.

That would be optimal, yes. But not sure what you had in mind here? Prove that the exact asset leave isn't in another output? But how would you define equality here so that some trickery with a slight modification (e.g. just change the script version) isn't possible?

Also, while brainstorming on this with @jharveyb, the question came up if this restriction could ever be an issue for HTLCs? Since it's possible that multiple MPP shards have the same script key (since they use the same payment hash).

Roasbeef · 2024-11-08T21:08:51Z

tapsend/send_test.go

+		vPkt.SetInputAsset(0, &a)
+
+		for i, outputKey := range outputKeys {
+			vPkt.Outputs[i] = &tappsbt.VOutput{


Shouldn't the output index be considered here when determining uniqueness?

No, that's kind of the point of this fix. See comment(s) above.

jharveyb

Looks solid, thanks for catching this!

We may want to add similar checks earlier in the freighter eventually but this should address the root cause IMO.

tapsend/send.go

jharveyb · 2024-11-11T14:02:50Z

tapsend/send_test.go

+			},
+		},
+		{
+			name: "collision, same group packets, same keys",


would be good to test collision based on asset ID + script key in addition to group key + script key.

I would like to voice something similar. I think this test case should be added. It's a case that increased my understanding of the PR, and if IIUC it's also a edge case that should be tested.

{ name: "no collision, multi asset packets, same keys", vPackets: []*tappsbt.VPacket{ makeVPacket( assetID1, nil, []asset.ScriptKey{ scriptKey1, }, ), makeVPacket( assetID2, nil, []asset.ScriptKey{ scriptKey1, }, ), }, },

Thanks, added two more test cases.

jharveyb · 2024-11-11T14:04:28Z

tapfreighter/coin_select.go

+			return c.AnchorPoint.String()
+		},
+	)
+	log.Infof("Identified %v eligible asset inputs for send of %d to %v: "+


nit: this is counting anchor inputs and not asset inputs?

IIUC the anchor inputs may contain multiple eligible asset inputs, which we will select from later.

I think if we have multiple eligible assets in the same anchor, we'd still get multiple *AnchoredCommitment structs returned. So IMO this is correct.

tapfreighter/chain_porter.go

itest/send_test.go

gijswijs

Very nice!

I feel that the tapfreighter commits should be their own PR? They don't have anything to do with enforcing unique script keys. Same thing goes fee estimation itest.

Or maybe just mention it in the PR that a number of other fixes were implemented as well, just for future reference.

gijswijs · 2024-11-11T16:46:49Z

tapsend/send.go

+			// Check if the script key has already been used in this
+			// transaction.
+			scriptKey := asset.ToSerialized(vOut.ScriptKey.PubKey)
+			if _, ok := perAssetMap[scriptKey]; ok {


Let me see if I understand this:
In theory it would be ok to reuse scriptKeys over different tapCommitmentKeys?

Yes, since those would have different paths in the anchored tapCommitment.

I think this statement is correct:

"For a given transfer, across all anchor outputs, assets can share a TapCommitmentKey(), or an AssetCommitmentKey(), but not both."

If both are shared, for any two assets, then we can't generate valid exclusion proofs.

This test is very useful for debugging user-supplied proofs that failed for some reason. We expand the test to also verify the inclusion and exclusion proofs.

guggero · 2024-11-14T12:39:47Z

They don't have anything to do with enforcing unique script keys. Same thing goes fee estimation itest.

Actually, they kind of do. To be able to test the check introduced with the PR, we want to be able to try again. That's why we added the fix for releasing the leased inputs. Which then changes a bunch of assumptions in the itests.
Updated the PR body and commit messages to make that more clear.

This helper method returns the asset specifier for the virtual packet. Because a single virtual packet is only supposed to carry inputs and outputs of a single asset ID, the TAP commitment key is valid for the whole vPacket.

To avoid asset leaves colliding on the same asset ID and script key.

Previously we only released/unlocked the BTC level outputs we leased from the lnd wallet when an error occurred. But if we're still in a state where nothing has been written to disk and the user can try again, we can safely release/unlock the asset level UTXOs as well to avoid them being locked for 10 minutes.

This test demonstrates that we get the correct error when we attempt to send to the same address twice in the same transaction. This also shows that we can re-try normally if a send fails before it was written to disk.

The fee estimation test assumed that UTXOs would be locked after an error and couldn't be used anymore. This is no longer the case and we need to adjust the test. We clean up the test a bit while we're at it.

Because we now check all active and passive packets for uniqueness in their commitment keys, this issue has surfaced. If we're spending multiple different assets from the same on-chain input commitment, we used to create duplicate passive assets. That wasn't a problem until now because within the trees everything would just be overwritten and collapsed back to a single asset. But because have the uniqueness check on the virtual packet slice level, this duplication became apparent.

guggero · 2024-11-14T13:54:51Z

Thanks to the failing integration test I noticed two things that I needed to fix (with the latest push):

For grouped assets, the asset ID goes into the asset-level commitment key, not just the script key. So the commitment keys actually look like this (and uniqueness needs to be guaranteed across those within a single transfer):

	TAP level	asset level
grouped asset	`SHA256(group_key)`	`SHA256(asset_id \|\| script_key)`
non-grouped asset	`asset_id`	`SHA256(script_key)`

When spending multiple input assets from the same on-chain input where other passive assets were present, we would duplicate those passive assets when creating passive virtual packets for them. This wasn't really an issue before since within the commitment trees they'd just be collapsed/overwritten into a single asset. But because the new uniqueness test operates on the slice of virtual packets, this was discovered. So a simple de-duplication fix was added in the last commit.

jharveyb · 2024-11-14T16:17:26Z

* For grouped assets, the asset ID goes into the asset-level commitment key, not just the script key. So the commitment keys actually look like this (and uniqueness needs to be guaranteed across those within a single transfer):
TAP level asset level
grouped asset SHA256(group_key) SHA256(asset_id || script_key)
non-grouped asset asset_id SHA256(script_key)

May be a motivation to use the Specifier type in more places.

guggero requested review from gijswijs and jharveyb November 8, 2024 14:34

Roasbeef reviewed Nov 8, 2024

View reviewed changes

guggero mentioned this pull request Nov 11, 2024

[bug]: Not all transfer proofs generated #1168

Closed

jharveyb approved these changes Nov 11, 2024

View reviewed changes

gijswijs approved these changes Nov 11, 2024

View reviewed changes

dstadulis added the assets label Nov 11, 2024

dstadulis assigned guggero Nov 11, 2024

dstadulis added this to the v0.5 (v0.4.2 rename) milestone Nov 11, 2024

gijswijs reviewed Nov 11, 2024

View reviewed changes

jharveyb self-requested a review November 11, 2024 17:24

guggero added 2 commits November 14, 2024 13:25

proof: verify inclusion and exclusion proofs of proof suffix

a57d8ed

This test is very useful for debugging user-supplied proofs that failed for some reason. We expand the test to also verify the inclusion and exclusion proofs.

tapfreighter+tapsend: fix error msg, don't shadown pkg name

c59f68b

guggero added 8 commits November 14, 2024 13:59

tappsbt: add AssetSpecifier to vPacket

cb45a9b

This helper method returns the asset specifier for the virtual packet. Because a single virtual packet is only supposed to carry inputs and outputs of a single asset ID, the TAP commitment key is valid for the whole vPacket.

tapsend: disallow colliding script keys in packets

a23f084

To avoid asset leaves colliding on the same asset ID and script key.

tapfreighter: log selected inputs correctly

0628d0b

tapfreighter: add ReleaseCoins to asset wallet interface

59841f0

itest: add test for duplicate address send

0198a10

This test demonstrates that we get the correct error when we attempt to send to the same address twice in the same transaction. This also shows that we can re-try normally if a send fails before it was written to disk.

itest: fix fee estimation test assumptions

f20a875

The fee estimation test assumed that UTXOs would be locked after an error and couldn't be used anymore. This is no longer the case and we need to adjust the test. We clean up the test a bit while we're at it.

guggero force-pushed the enforce-unique-script-key branch from 6a69824 to ae69e0a Compare November 14, 2024 13:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[tapsend]: Enforce unique script keys #1181

[tapsend]: Enforce unique script keys #1181

guggero commented Nov 8, 2024 •

edited

Loading

coveralls commented Nov 8, 2024 •

edited

Loading

guggero commented Nov 8, 2024

Roasbeef Nov 8, 2024

jharveyb Nov 11, 2024

guggero Nov 11, 2024

Roasbeef Nov 11, 2024

guggero Nov 12, 2024

Roasbeef Nov 8, 2024

guggero Nov 11, 2024

jharveyb left a comment

jharveyb Nov 11, 2024

gijswijs Nov 11, 2024

guggero Nov 14, 2024

jharveyb Nov 11, 2024

guggero Nov 14, 2024

gijswijs left a comment

gijswijs Nov 11, 2024

jharveyb Nov 11, 2024

guggero commented Nov 14, 2024

guggero commented Nov 14, 2024

jharveyb commented Nov 14, 2024

[tapsend]: Enforce unique script keys #1181

Are you sure you want to change the base?

[tapsend]: Enforce unique script keys #1181

Conversation

guggero commented Nov 8, 2024 • edited Loading

coveralls commented Nov 8, 2024 • edited Loading

Pull Request Test Coverage Report for Build 11838616455

Details

💛 - Coveralls

guggero commented Nov 8, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jharveyb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gijswijs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

guggero commented Nov 14, 2024

guggero commented Nov 14, 2024

jharveyb commented Nov 14, 2024

guggero commented Nov 8, 2024 •

edited

Loading

coveralls commented Nov 8, 2024 •

edited

Loading