Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception thrown in synchronous call "bleeds over" to asynchronous call #2869

Open
Paril opened this issue Jan 23, 2024 · 3 comments
Open

Exception thrown in synchronous call "bleeds over" to asynchronous call #2869

Paril opened this issue Jan 23, 2024 · 3 comments
Assignees
Labels
bug This issue is a bug. p2 This is a standard priority issue queued This issues is on the AWS team's backlog

Comments

@Paril
Copy link

Paril commented Jan 23, 2024

Describe the bug

We're running a small test suite of things after upgrading our aging S3 client library to this one. One of the things we've done is added asynchronous/parallel uploading with putObjectAsync, but our usages of getObject only ever fetch one object so we have that synchronous.

If we force a failure condition from getObject (such as fetching an object that doesn't exist), the next calls to any Promise-based functions throw a 'ghost' exception of the synchronous call, despite it not relating to any of the promises we're waiting on.

Expected Behavior

No exception thrown

Current Behavior

Exceptions thrown. Here's the exception log we're dealing with;

exception 'Aws\S3\Exception\S3Exception' with message 'Error executing "PutObject" on "https://s3.amazonaws.com/xxxxx.com/files/sigs/test-0.png"; AWS HTTP error: Unable to open "/var/www/xxxxx.com/public_html/files/sigs/test-get-successv2.png" using mode "w+": fopen(/var/www/xxxxx.com/public_html/files/sigs/test-get-successv2.png): Failed to open stream: Permission denied'

RuntimeException: Unable to open "/var/www/xxxxx.com/public_html/files/sigs/test-get-successv2.png" using mode "w+": fopen(/var/www/xxxxx.com/public_html/files/sigs/test-get-successv2.png): Failed to open stream: Permission denied in /var/www/xxxxx.com/public_html/vendor/guzzlehttp/psr7/src/Utils.php:361
Stack trace:
#0 [internal function]: GuzzleHttp\Psr7\Utils::GuzzleHttp\Psr7\{closure}()
#1 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/psr7/src/Utils.php(373): fopen()
#2 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/psr7/src/LazyOpenStream.php(47): GuzzleHttp\Psr7\Utils::tryFopen()
#3 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/psr7/src/StreamDecoratorTrait.php(33): GuzzleHttp\Psr7\LazyOpenStream->createStream()
#4 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/psr7/src/StreamDecoratorTrait.php(144): GuzzleHttp\Psr7\LazyOpenStream->__get()
#5 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php(414): GuzzleHttp\Psr7\LazyOpenStream->write()
#6 [internal function]: GuzzleHttp\Handler\CurlFactory::GuzzleHttp\Handler\{closure}()
#7 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(171): curl_multi_exec()
#8 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(189): GuzzleHttp\Handler\CurlMultiHandler->tick()
#9 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(251): GuzzleHttp\Handler\CurlMultiHandler->execute()
#10 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(227): GuzzleHttp\Promise\Promise->invokeWaitFn()
#11 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(272): GuzzleHttp\Promise\Promise->waitIfPending()
#12 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(229): GuzzleHttp\Promise\Promise->invokeWaitList()
#13 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(272): GuzzleHttp\Promise\Promise->waitIfPending()
#14 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(229): GuzzleHttp\Promise\Promise->invokeWaitList()
#15 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(69): GuzzleHttp\Promise\Promise->waitIfPending()
#16 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Utils.php(121): GuzzleHttp\Promise\Promise->wait()
#17 /var/www/xxxxx.com/public_html/xxxxx/S3.php(228): GuzzleHttp\Promise\Utils::unwrap()
#18 /var/www/xxxxx.com/public_html/xxxxx/s3test.php(124): s3media->put_many()
#19 /var/www/xxxxx.com/public_html/xxxxx/s3test.php(62): task_s3test->testPutMany()
#20 /var/www/xxxxx.com/public_html/xxxxx/lab.php(461): task_s3test->runTask()
#21 /var/www/xxxxx.com/public_html/xxxxx/controller.php(25): module_lab->run()
#22 /var/www/xxxxx.com/public_html/xxxxx/controller.php(34): module_controller->check_special_modules()
#23 /var/www/xxxxx.com/public_html/index.php(13): module_controller->get_page()
#24 {main}

Next Aws\S3\Exception\S3Exception: Error executing "PutObject" on "https://s3.amazonaws.com/xxxxx.com/files/sigs/test-0.png"; AWS HTTP error: Unable to open "/var/www/xxxxx.com/public_html/files/sigs/test-get-successv2.png" using mode "w+": fopen(/var/www/xxxxx.com/public_html/files/sigs/test-get-successv2.png): Failed to open stream: Permission denied in /var/www/xxxxx.com/public_html/vendor/aws/aws-sdk-php/src/WrappedHttpHandler.php:196
Stack trace:
#0 /var/www/xxxxx.com/public_html/vendor/aws/aws-sdk-php/src/WrappedHttpHandler.php(98): Aws\WrappedHttpHandler->parseError()
#1 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(209): Aws\WrappedHttpHandler->Aws\{closure}()
#2 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(174): GuzzleHttp\Promise\Promise::callHandler()
#3 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/RejectedPromise.php(49): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}()
#4 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/TaskQueue.php(52): GuzzleHttp\Promise\RejectedPromise::GuzzleHttp\Promise\{closure}()
#5 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(238): GuzzleHttp\Promise\TaskQueue->run()
#6 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(272): GuzzleHttp\Promise\Promise->waitIfPending()
#7 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(229): GuzzleHttp\Promise\Promise->invokeWaitList()
#8 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(272): GuzzleHttp\Promise\Promise->waitIfPending()
#9 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(229): GuzzleHttp\Promise\Promise->invokeWaitList()
#10 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Promise.php(69): GuzzleHttp\Promise\Promise->waitIfPending()
#11 /var/www/xxxxx.com/public_html/vendor/guzzlehttp/promises/src/Utils.php(121): GuzzleHttp\Promise\Promise->wait()
#12 /var/www/xxxxx.com/public_html/xxxxx/S3.php(228): GuzzleHttp\Promise\Utils::unwrap()
#13 /var/www/xxxxx.com/public_html/xxxxx/s3test.php(124): s3media->put_many()
#14 /var/www/xxxxx.com/public_html/xxxxx/s3test.php(62): task_s3test->testPutMany()
#15 /var/www/xxxxx.com/public_html/xxxxx/lab.php(461): task_s3test->runTask()
#16 /var/www/xxxxx.com/public_html/xxxxx/controller.php(25): module_lab->run()
#17 /var/www/xxxxx.com/public_html/xxxxx/controller.php(34): module_controller->check_special_modules()
#18 /var/www/xxxxx.com/public_html/index.php(13): module_controller->get_page()
#19 {main}

Reproduction Steps

In our runTask, we're running two separate tasks: one is a get, which tries to get a file into a path it does not have permissions to write to, test-get-successv2.png. This is supposed to fail, and is failing as expected.

However, the next task is then supposed to do an unrelated put_many to paths that, when executed without the get having run prior, succeed with no errors. It is only when running with the get that the put_many then throws exceptions.

Our "get" function is very simple, only running this (with error checking, etc):

return $this->s3->getObject($params);

Our put_many is a bit more complex, but all it really does is do several $this->s3->putObjectAsync's, puts them in an array, and then \GuzzleHttp\Promise\Utils::unwrap's them. That unwrap is what's throwing the ghost exception from the prior getObject.

Note that the paths in the exception were not sent at all to put_many.

Possible Solution

No response

Additional Information/Context

No response

SDK version used

3.275.1

Environment details (Version of PHP (php -v)? OS name and version, etc.)

PHP 8.0, rest should be unrelated

@Paril Paril added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jan 23, 2024
@Paril
Copy link
Author

Paril commented Jan 24, 2024

Here's a SSCEE of the problem. The only requirements is that:
a) all files and keynames referenced exist (except in the case of put dest files, obviously)
b) the PHP script does not have write access to the destination file for the get (ie, getObject must throw an exception)

	private function testBroken()
	{
		$bucketName	= '....'; // put in your bucket here
		$s3			= new Aws\S3\S3Client([
			'region' => 'us-east-1',
			'version' => 'latest'
		]);

		$get_key_name = 'files/sigs/test-put.png'; // must exist on S3
		$get_save_to = 'files/sigs/test-saveas.png'; // must either not exist, or not have permission to access

		try
		{
			// test get; this should throw
			$params = [
				'Bucket' => $bucketName,
				'Key' => $get_key_name
			];

			$params['SaveAs'] = $get_save_to;

			$file = $s3->getObject($params);
		}
		catch (Exception)
		{
			printf('got error, as expected\n');
		}

		try
		{
			// test put_many, this should not throw
			$put_files = [
				[ 'path' => 'files/sigs/test-0.png', 'key' => 'files/sigs/test-0.png' ], // source files must exist!
				[ 'path' => 'files/sigs/test-1.png', 'key' => 'files/sigs/test-1.png' ] ];
			$put_promises = [];

			foreach ($put_files as $file)
			{
				$put_promises[] = $s3->putObjectAsync([
					'Bucket' => $bucketName,
					'Key' => $file['key'],
					'ACL' => 'public-read',
					'SourceFile' => $file['path']
				]);
			}

			\GuzzleHttp\Promise\Utils::unwrap($put_promises);
			printf('put success\n');
		}
		catch (Exception)
		{
			printf('got error, unexpected!!\n');
		}
	}

Note that the following will happen:

  • got error, as expected will be logged. this is expected, obviously, as the file should not have write permissions. This is not the problem, and is expected!
  • got error, unexpected!! will be logged next. This is the bug. The exception, if you introspect it, will be the exception from the first get failure and not related to the put at all.

If you remove the getObject call entirely, the puts will succeed.

@yenfryherrerafeliz yenfryherrerafeliz self-assigned this Jan 24, 2024
@yenfryherrerafeliz yenfryherrerafeliz added investigating This issue is being investigated and/or work is in progress to resolve the issue. and removed needs-triage This issue or PR still needs to be triaged. labels Jan 24, 2024
@yenfryherrerafeliz
Copy link
Contributor

Hi @Paril, thanks for reporting this. I was able to reproduce the issue by following your example. I am working on root causing it. I will provide updates as soon as possible.

Thanks!

@yenfryherrerafeliz
Copy link
Contributor

Hi @Paril, we are still working on getting this fixed; however, in the meantime, a workaround would be to provide a custom http handler in the client as following:

$s3 = new S3Client([
        'region' => getenv('TEST_REGION'),
        'version' => 'latest',
        'http_handler' => function (RequestInterface $request, $options) {
            return \Aws\default_http_handler()($request, $options);
        }
    ]);

I will provide more updates here, once we get this fixed.

Thanks!

@yenfryherrerafeliz yenfryherrerafeliz added p2 This is a standard priority issue queued This issues is on the AWS team's backlog labels Jan 26, 2024
@RanVaknin RanVaknin removed the investigating This issue is being investigated and/or work is in progress to resolve the issue. label Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. p2 This is a standard priority issue queued This issues is on the AWS team's backlog
Projects
None yet
Development

No branches or pull requests

3 participants