Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maximum function nesting level of '256' reached, aborting! in /vendor/yiisoft/yii2-elasticsearch/ActiveDataProvider.php:82 #201

Closed
ghost opened this issue Jan 3, 2019 · 31 comments
Labels
status:need more info status:to be verified Needs to be reproduced and validated. type:bug Bug
Milestone

Comments

@ghost
Copy link

ghost commented Jan 3, 2019

On "minimum-stability": stable work fine !

Infinite loop after Yii2 update:
yiisoft/yii2@a5182e5

What steps will reproduce the problem?

  1. php composer.phar create-project --prefer-dist yiisoft/yii2-app-basic basic
  2. Set elastic componet auth data ...
  3. Changes in composer.json

"minimum-stability": stable --> dev,
"php": >=5.4.0 --> >=7.0
"require": add "yiisoft/yii2-elasticsearch": "~2.1"

Finaly composer.json

{
    "name": "yiisoft/yii2-app-basic",
    "description": "Yii 2 Basic Project Template",
    "keywords": ["yii2", "framework", "basic", "project template"],
    "homepage": "http://www.yiiframework.com/",
    "type": "project",
    "license": "BSD-3-Clause",
    "support": {
        "issues": "https://github.com/yiisoft/yii2/issues?state=open",
        "forum": "http://www.yiiframework.com/forum/",
        "wiki": "http://www.yiiframework.com/wiki/",
        "irc": "irc://irc.freenode.net/yii",
        "source": "https://github.com/yiisoft/yii2"
    },
    "minimum-stability": "dev",
    "require": {
        "php": ">=7.0",
        "yiisoft/yii2": "~2.0.14",
        "yiisoft/yii2-bootstrap": "~2.0.0",
        "yiisoft/yii2-swiftmailer": "~2.0.0",
        "yiisoft/yii2-elasticsearch": "~2.1"
    },
    "require-dev": {
        "yiisoft/yii2-debug": "~2.0.0",
        "yiisoft/yii2-gii": "~2.0.0",
        "yiisoft/yii2-faker": "~2.0.0",
        "codeception/base": "^2.2.3",
        "codeception/verify": "~0.3.1",
        "codeception/specify": "~0.4.3"
    },
    "config": {
        "process-timeout": 1800,
        "fxp-asset": {
            "enabled": false
        }
    },
    "scripts": {
        "post-install-cmd": [
            "yii\\composer\\Installer::postInstall"
        ],
        "post-create-project-cmd": [
            "yii\\composer\\Installer::postCreateProject",
            "yii\\composer\\Installer::postInstall"
        ]
    },
    "extra": {
        "yii\\composer\\Installer::postCreateProject": {
            "setPermission": [
                {
                    "runtime": "0777",
                    "web/assets": "0777",
                    "yii": "0755"
                }
            ]
        },
        "yii\\composer\\Installer::postInstall": {
            "generateCookieValidationKey": [
                "config/web.php"
            ]
        }
    },
    "repositories": [
        {
            "type": "composer",
            "url": "https://asset-packagist.org"
        }
    ]
}
  1. \app\controllers\SiteController::actionIndex
    /**
     * Displays homepage.
     * @return string
     */
    public function actionIndex()
    {
        $query = new \yii\elasticsearch\Query();
        $query->limit(10);
        $query->from('report', 'report');

        $provider = new \yii\elasticsearch\ActiveDataProvider([
            'query'      => $query,
            'pagination' => [
                'pageSize' => 2
            ],
        ]);

        $models =  $provider->getModels();

        var_dump($models);
        die;

        return $this->render('index');
    }
  1. php yii serve
  2. curl -X GET 'http://localhost:8080'

What's expected?

\yii\elasticsearch\ActiveDataProvide::getModels() return array

What do you get instead?

[Wed Jan 16 20:17:25 2019] PHP Error:  Maximum function nesting level of '256' reached, aborting! in basic/vendor/yiisoft/yii2/base/Component.php on line 307
[Wed Jan 16 20:17:25 2019] PHP Stack trace:
[Wed Jan 16 20:17:25 2019] PHP   1. {main}() basic/web/index.php:0
[Wed Jan 16 20:17:25 2019] PHP   2. yii\web\Application->run() basic/web/index.php:12
[Wed Jan 16 20:17:25 2019] PHP   3. yii\web\Application->handleRequest() basic/vendor/yiisoft/yii2/base/Application.php:386
[Wed Jan 16 20:17:25 2019] PHP   4. yii\web\Application->runAction() basic/vendor/yiisoft/yii2/web/Application.php:103
[Wed Jan 16 20:17:25 2019] PHP   5. app\controllers\SiteController->runAction() basic/vendor/yiisoft/yii2/base/Module.php:528
[Wed Jan 16 20:17:25 2019] PHP   6. yii\base\InlineAction->runWithParams() basic/vendor/yiisoft/yii2/base/Controller.php:157
[Wed Jan 16 20:17:25 2019] PHP   7. call_user_func_array:{basic/vendor/yiisoft/yii2/base/InlineAction.php:57}() basic/vendor/yiisoft/yii2/base/InlineAction.php:57
[Wed Jan 16 20:17:25 2019] PHP   8. app\controllers\SiteController->actionIndex() basic/vendor/yiisoft/yii2/base/InlineAction.php:57
[Wed Jan 16 20:17:25 2019] PHP   9. yii\elasticsearch\ActiveDataProvider->getModels() basic/controllers/SiteController.php:74
[Wed Jan 16 20:17:25 2019] PHP  10. yii\elasticsearch\ActiveDataProvider->prepare() basic/vendor/yiisoft/yii2/data/BaseDataProvider.php:117
[Wed Jan 16 20:17:25 2019] PHP  11. yii\elasticsearch\ActiveDataProvider->prepareModels() basic/vendor/yiisoft/yii2/data/BaseDataProvider.php:104
[Wed Jan 16 20:17:25 2019] PHP  12. yii\elasticsearch\ActiveDataProvider->getPagination() basic/vendor/yiisoft/yii2-elasticsearch/ActiveDataProvider.php:89
[Wed Jan 16 20:17:25 2019] PHP  13. yii\elasticsearch\ActiveDataProvider->prepareTotalCount() basic/vendor/yiisoft/yii2/data/BaseDataProvider.php:199
[Wed Jan 16 20:17:25 2019] PHP  14. yii\elasticsearch\ActiveDataProvider->getQueryResults() basic/vendor/yiisoft/yii2-elasticsearch/ActiveDataProvider.php:117
[Wed Jan 16 20:17:25 2019] PHP  15. yii\elasticsearch\ActiveDataProvider->prepare() basic/vendor/yiisoft/yii2-elasticsearch/ActiveDataProvider.php:50
[Wed Jan 16 20:17:25 2019] PHP  16. yii\elasticsearch\ActiveDataProvider->prepareModels() basic/vendor/yiisoft/yii2/data/BaseDataProvider.php:104
[Wed Jan 16 20:17:25 2019] PHP  17. yii\elasticsearch\ActiveDataProvider->getPagination() basic/vendor/yiisoft/yii2-elasticsearch/ActiveDataProvider.php:89
[Wed Jan 16 20:17:25 2019] PHP  18. yii\elasticsearch\ActiveDataProvider->prepareTotalCount() basic/vendor/yiisoft/yii2/data/BaseDataProvider.php:199
[Wed Jan 16 20:17:25 2019] PHP  19. yii\elasticsearch\ActiveDataProvider->getQueryResults() basic/vendor/yiisoft/yii2-elasticsearch/ActiveDataProvider.php:117
[Wed Jan 16 20:17:25 2019] PHP  20. yii\elasticsearch\ActiveDataProvider->prepare() basic/vendor/yiisoft/yii2-elasticsearch/ActiveDataProvider.php:50
[Wed Jan 16 20:17:25 2019] PHP  21. yii\elasticsearch\ActiveDataProvider->prepareModels() basic/vendor/yiisoft/yii2/data/BaseDataProvider.php:104
...

Additional info

Q A
Yii version 2.0.16-dev
PHP version 7.1.25
Operating system Mojave
@ghost
Copy link
Author

ghost commented Jan 16, 2019

...
"minimum-stability": "dev",
"require": {
"php": ">=7.0",
...
"yiisoft/yii2": "~2.0.14",
"yiisoft/yii2-elasticsearch": "~2.1",
},

@samdark samdark added type:bug Bug status:to be verified Needs to be reproduced and validated. labels Jan 16, 2019
@samdark
Copy link
Member

samdark commented Jan 16, 2019

What code do you call to have this error?

@ghost
Copy link
Author

ghost commented Jan 16, 2019

In SearchModel on call getModels method:

$query = new yii\elasticsearch\Query();
$provider = new ActiveDataProvider([
'query' => $query,
]);
$provider->getModels();

@samdark
Copy link
Member

samdark commented Jan 16, 2019

What's in the query?

@ghost
Copy link
Author

ghost commented Jan 16, 2019

What's in the query?

{"size":50,"query":{"constant_score":{"filter":{"bool":{"must":[{"range":{"reg_date":{"gte":"2019-01-16T00:00:00+03:00"}}},{"range":{"reg_date":{"lte":"2019-01-16T23:59:59+03:00"}}}]}}}},"sort":[{"reg_date":"desc"}]}

@np25071984
Copy link
Contributor

I can't reproduce the problem.

ghopper@TM1701 ~/Projects/yii2.test ((HEAD detached at a5182e5d5)) $ uname -a
Linux farm 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
ghopper@TM1701 ~/Projects/yii2.test ((HEAD detached at a5182e5d5)) $ curl -X GET 'http://localhost:9200'
{
  "name" : "node-1",
  "cluster_name" : "elastic_cluster",
  "version" : {
    "number" : "2.3.1",
    "build_hash" : "bd980929010aef404e7cb0843e61d0665269fc39",
    "build_timestamp" : "2016-04-04T12:25:05Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}

SiteController.php

    public function actionAbout()
    {

//        for ($i = 11; $i < 60; $i++) {
//            $city = new City();
//            $city->primaryKey = $i;
//            $r = rand(5, 50);
//            $city->attributes = [
//                'message' => 'message#' . $i,
//                'reg_date' => date(\DateTime::ISO8601, strtotime(" +{$r} day"))
//            ];
//            $city->save();
//        }

        $query = City::find()
            ->where(['>', 'reg_date', date(\DateTime::ISO8601, strtotime(" +5 day"))])
            ->andWhere(['<', 'reg_date', date(\DateTime::ISO8601, strtotime(" +15 day"))])
            ->orderBy(['reg_date' => SORT_DESC])
            ->limit(10);
        $provider = new ActiveDataProvider([
            'query' => $query,
            'pagination' => [ 'pageSize' => 10 ],
        ]);
        $tmp = $provider->getModels();

        return $this->render('about', [
                'provider' => $provider,
            ]
        );
    }

And in about.php view:

<?= GridView::widget([
    'dataProvider' => $provider,
    'id' => 'grid-id',
    'columns' => [
        'message',
        [
            'class' => ActionColumn::className(),
            // you may configure additional properties here
        ],
    ],
]) ?>

Everything works fine! Pagination, total row amount, sorting and so on. Do I misunderstood something?

@samdark
Copy link
Member

samdark commented Jan 16, 2019

Aha, I think "yiisoft/yii2-elasticsearch ~2.1" matters.

@ghost
Copy link
Author

ghost commented Jan 16, 2019

What version is right for
"number": "5.6.2",
"lucene_version": "6.6.1"

@samdark
Copy link
Member

samdark commented Jan 16, 2019

2.1.x

@ghost
Copy link
Author

ghost commented Jan 16, 2019

didn't help

@samdark
Copy link
Member

samdark commented Jan 16, 2019

@GHopperMSK have you checked with 2.1 branch?

@ghost
Copy link
Author

ghost commented Jan 16, 2019

Depend on minimum stability
On stable use yiisoft/yii2-elasticsearch (2.0.5) and it work fine
Errors appear on dev stability and "yiisoft/yii2-elasticsearch": "~2.1"

@ghost
Copy link
Author

ghost commented Jan 16, 2019

Change description and steps for reproduce

@np25071984
Copy link
Contributor

np25071984 commented Jan 16, 2019

It doesn't depend on elasticsearch version, but on minimum-stability value.

Here is infinity loop in \yii\elasticsearch\ActiveDataProvider:

getModels()
prepare()
prepareModels()
getPagination()
prepareTotalCount()
getQueryResults()
prepare()

We have to rethink yiisoft/yii2#16951! Specifically:

if (($this->_pagination !== false) && ($this->_pagination->totalCount === null)) {
            if ($this->_totalCount === null) {
                $this->setTotalCount($this->prepareTotalCount());
            }
            $this->_pagination->totalCount = $this->_totalCount;
        }

@taobig
Copy link

taobig commented Jan 17, 2019

Change yii\data\BaseDataProvider, make sure that prepareTotalCount() can only be called for the first time in getPagination()
or
Change yii\elasticsearch\ActiveDataProvider, make sure that prepareTotalCount() shouldn't call $this->getPagination()
Do you have any other ideas?

@np25071984
Copy link
Contributor

np25071984 commented Jan 17, 2019

Let`s look at

protected function prepareModels()
{
if (!$this->query instanceof Query) {
throw new InvalidConfigException('The "query" property must be an instance "' . Query::className() . '" or its subclasses.');
}
$query = clone $this->query;
if (($pagination = $this->getPagination()) !== false) {
// pagination fails to validate page number, because total count is unknown at this stage
$pagination->validatePage = false;
$query->limit($pagination->getLimit())->offset($pagination->getOffset());
}
if (($sort = $this->getSort()) !== false) {
$query->addOrderBy($sort->getOrders());
}
if (is_array(($results = $query->search($this->db)))) {
$this->setQueryResults($results);
if ($pagination !== false) {
$pagination->totalCount = $this->getTotalCount();
}
return $results['hits']['hits'];
}
$this->setQueryResults([]);
return [];
}

First we call getPagination(), getSort() and only after that $query->search(). Here is my offer:

    protected function prepareModels1()
    {
        if (!$this->query instanceof Query) {
            throw new InvalidConfigException('The "query" property must be an instance "' . Query::className() . '" or its subclasses.');
        }

        $query = clone $this->query;
        $results = $query->search($this->db);
        $this->setQueryResults($results);

        if (($pagination = $this->getPagination()) !== false) {
            // pagination fails to validate page number, because total count is unknown at this stage
            $pagination->validatePage = false;
            $query->limit($pagination->getLimit())->offset($pagination->getOffset());
        }
        if (($sort = $this->getSort()) !== false) {
            $query->addOrderBy($sort->getOrders());
        }

        if ($pagination !== false) {
            $pagination->totalCount = $this->getTotalCount();
        }

        return $results['hits']['hits'];
    }

It works, but I didn't run tests yet.

@taobig
Copy link

taobig commented Jan 17, 2019

$this->setQueryResults(is_array($result)?$result:[]);//changed, the param must be an array

@taobig
Copy link

taobig commented Jan 17, 2019

I don't think this is an efficient solution, query will be executed twice. And the method will fetch "all records" at the first query.

@np25071984
Copy link
Contributor

np25071984 commented Jan 17, 2019

I don't think this is an efficient solution, query will be executed twice. And the method will fetch "all records" at the first query.

The solution doesn't add any queries, it just rearrange them.

But in general I agree with you. There are many places have to be refactored.

@taobig
Copy link

taobig commented Jan 17, 2019

I' sorry, I overlooked your code. There is no second query.

samdark pushed a commit that referenced this issue Jan 17, 2019
@samdark samdark closed this as completed Jan 17, 2019
@ghost
Copy link
Author

ghost commented Jan 17, 2019

Please apply this fix for 2.1.x-dev

@samdark samdark reopened this Jan 17, 2019
samdark pushed a commit that referenced this issue Jan 17, 2019
@samdark samdark closed this as completed Jan 17, 2019
@samdark samdark added this to the 2.0.6 milestone Jan 17, 2019
@yuniorsk
Copy link
Contributor

yuniorsk commented Jan 25, 2019

Hi guys, PR #204 introduced regression that's breaking sorting and pagination in data provider. It is because query is now executed before pagination and sort methods are applied, so they have no effect at all.

@samdark samdark reopened this Jan 25, 2019
@samdark samdark closed this as completed Jan 25, 2019
@samdark samdark reopened this Jan 25, 2019
@samdark
Copy link
Member

samdark commented Jan 25, 2019

@GHopperMSK any other idea about fixing it?

@np25071984
Copy link
Contributor

@GHopperMSK any other idea about fixing it?

I have a few ideas, but it will take some time.

@samdark
Copy link
Member

samdark commented Jan 25, 2019

Would these require changes in framework itself? Asking because we'd like to tag 2.0.16 before end of the month.

@np25071984
Copy link
Contributor

I don't think so. But as you know, it is quite unpredictable )
I think I will get some essential data during this weekend.

This was referenced Jan 25, 2019
@np25071984
Copy link
Contributor

np25071984 commented Jan 26, 2019

The current solution isn't optimal due to it makes redundant request for obtain totalCount. For solving this situation I need BaseDataProvider::_pagination to be protected members, but it is in kernel.
@samdark What do you think?

@np25071984
Copy link
Contributor

In general, it solved. You are welcome for checking.

@machour
Copy link
Member

machour commented Feb 18, 2019

Seems to be affecting yii2-sphinx as well: https://travis-ci.org/yiisoft/yii2-sphinx/jobs/495100578

@samdark
Copy link
Member

samdark commented Feb 22, 2019

Yii 2 change reverted. I'll clean up commits to ES as well.

@samdark samdark closed this as completed Feb 22, 2019
samdark added a commit that referenced this issue Feb 22, 2019
samdark added a commit that referenced this issue Feb 22, 2019
samdark added a commit that referenced this issue May 1, 2019
* Update Query.php (#131)

* PHPUnit compatibility, provide token for github auth

* token update

* yet another update

* Fix #134 infinite query loop when the index does not exist

* improved error message for cluster auto detection

issue #137

* updated docs about cluster autodetection

fixes #137

* Elasticsearch logo added to README.md

* Composer json change

* Added alias actions to Command

* Fix for desirialization of plain response from elasticsearch

* Bring back old name for package

* Fix for php 5.4

* Added phpdocs, fixes and tests

* CHANGELOG.md

* clarify version compatibility

* Fixing broken elasticsearch doc links (#144) [skip ci]

* Fixes #149: Changed `yii\base\Object` to `yii\base\BaseObject`

* since elasticsearch 6, content type is mandatory (#150)

since elasticsearch 6, content type is mandatory. Otherwise, requests will all fail.

https://www.elastic.co/blog/strict-content-type-checking-for-elasticsearch-rest-requests

* added CHANGELOG for #150

* Reset query results after calling refresh() method

close #125

* Fixes case (#153) [skip ci]

* Fix spelling (#165) [skip ci]

* fix findAll and findOne to filter input to avoid passing manipulated condition

* release version 2.0.5

* prepare for next release

* Update README.md

* docs/guide-ja updated [ci skip] (#178)

* Removed redundant line from license [skip ci]

* guide-ja revised [ci skip]

* Fixed `count()` compatibility with PHP 7.2

do not call it on scalar values.

fixes #180

* Translate into Russian (#194) [skip ci]

* Improve Russian docs (#200) [skip ci]

* Bug #201: Fixed infinite loop

* Updated issue template [skip ci]

* Made tests running again (#210)

* Revert "Bug #201: Fixed infinite loop"

This reverts commit aa22ffd.

* Fixes #218: Comment out invalid syntax in sample [skip ci] (#219)

* docs/guide-ja/mapping-indexing.md updated [ci skip] (#221)

* Fixes #227: Fixed `Bad Request (#400): Unable to verify your data submission.` in debug details panel 'run query'

* Travis adjustments (#228)

* Use assertCount() in tests

* Add missing relations to Order test model

* Add note delete-by-query plugin (#158)

* Fixes #117: Add support for `QueryInterface::emulateExecution()`

* Specified versions that work
tunecino added a commit to tunecino/yii2-elasticsearch that referenced this issue Jun 23, 2019
related to yiisoft#134 and yiisoft#201.

There is currently an infinite loop that happens when following method is called before getting any results from ElasticSearch:

```
prepareModels()
->getPagination()
->prepareTotalCount()
->getQueryResults()
->prepare()
->prepareModels()
```

It seems the infinite loop issue has already been fixed at some point, but then reverted by yiisoft@58466a8.

This PR tries a different fix to the same issue by simply ensuring `prepareTotalCount()` to request the number from the database in case we have no available query response, the exact same way it was done with core `ActiveDataProvider::prepareTotalCount()` for SQL databases, Mongo and Sphinx extensions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status:need more info status:to be verified Needs to be reproduced and validated. type:bug Bug
Projects
None yet
Development

No branches or pull requests

5 participants