Minimize needed S3 permissions for backups

[JohnXLivingston]

Hello,

Currently, when trying to add a S3 account for backups, Virtualmin requires the s3:ListAllMyBuckets permission. If the IAM user has not this permission, adding the s3 account fails.

The problem is that s3:ListAllMyBuckets is a global permission, and allows listing all account's buckets! The "My" is misleading.

When we have a lot of servers, potentially attributed to different customers, this make impossible to properly isolate all buckets.
Of course, we can define some policies on S3 to only allow access to one bucket... But this makes still possible to get the bucket list. And that could lead to data leaks. Indeed, sometime just knowing bucket names can help to guess some information... like the number of customers we have.
And sometimes, we have buckets that are not even related to Virtualmin backups!

Backuping and restoring a server should not need any global S3 permissions. It should just need the permissions to write/read/delete the bucket we have configured.

Here is the procedure we setup in the company I work for, to handle virtualmin backups:

• create a dedicated IAM user for each server to backup
• add this user in a special group, that gives following permissions:
  • allow s3:CreateBucket, ListBucket, s3:GetObject, s3:PutObject, s3:DeleteObject on ressources arn:aws:s3:::${aws:username}-xxxxxx-com and arn:aws:s3:::${aws:username}-xxxxxxx-com/* (this allows us to dynamically compute the associated bucket name)
  • for now, we must allow s3:ListAllMyBuckets, because Virtualmin needs it (but this is the part we want to remove)
• configure virtualmin to backup in the bucket "arn:aws:s3:::${aws:username}-xxxxxx-com"

For the record, here is the exact S3 group policy we use for now:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Resource": "arn:aws:s3:::*",
			"Action": [
				"s3:ListAllMyBuckets"
			]
		},
		{
			"Effect": "Allow",
			"Action": [
				"s3:CreateBucket",
				"s3:ListBucket",
				"s3:GetObject",
				"s3:PutObject",
				"s3:DeleteObject"
			],
			"Resource": [
				"arn:aws:s3:::${aws:username}-xxxxxx-com",
				"arn:aws:s3:::${aws:username}-xxxxxx-com/*"
			]
		}
	]
}

And that is the group policy we'd like to use:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Action": [
				"s3:CreateBucket",
				"s3:ListBucket",
				"s3:GetObject",
				"s3:PutObject",
				"s3:DeleteObject"
			],
			"Resource": [
				"arn:aws:s3:::${aws:username}-xxxxxx-com",
				"arn:aws:s3:::${aws:username}-xxxxxx-com/*"
			]
		}
	]
}

So, as said, for now we have to allow s3:ListAllMyBuckets. But, IMHO, there is no good reason to ask for this permission.
Here are the changes I propose to fix:

• on the "add new S3 account" page, don't try to list buckets to test the S3 access key. Find another way to test the connection, that requires no special rights.
• on the "S3 buckets" page, just show a screen saying something like "You don't have s3:ListAllMyBuckets right, you can't list buckets". This page can be completely optional, it is not needed to configure, schedule, or restore backups.
• ensure that when configuring the scheduled backup, there is no need for any global AWS permission.
• ensure that when restoring a S3 backup, there is no need for any global AWS permission.

[jcameron]

Yes, someone else reported this issue - we plan to fix it in the next release by detecting if an S3 account is valid but not allowed to list buckets. That should allow IAM accounts with limited permissions to be used.

Ideally I would prefer to use an S3 API command to test the connection that used permissions not associated with any bucket, but I can't find one.

[JohnXLivingston]

Yes, someone else reported this issue - we plan to fix it in the next release by detecting if an S3 account is valid but not allowed to list buckets. That should allow IAM accounts with limited permissions to be used.

That was probably also me. Ilia asked me yesterday to create an issue here, to discuss the feature.

Ideally I would prefer to use an S3 API command to test the connection that used permissions not associated with any bucket, but I can't find one.

Maybe you can try the same command, but handle the error code? Maybe the error code is not the same when you are not logged in and when you don't have the right to list buckets? (or maybe the error message)

[jcameron]

Maybe you can try the same command, but handle the error code? Maybe the error code is not the same when you are not logged in and when you don't have the right to list buckets? (or maybe the error message)

Yes, that's exactly how it will work in the next release.