Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add getViews and categorize table types #3327

Open
wants to merge 12 commits into
base: 5.x
Choose a base branch
from

Conversation

masterbater
Copy link
Contributor

@masterbater masterbater commented Mar 24, 2025

Fix #3320
This fixes db:show and migrate:fresh for database that has virtual view type collections

Feat Suggestion
add getViews()
php artisan db:show --views

Checklist

  • Add tests and ensure they pass

Sorry, something went wrong.

@masterbater masterbater requested a review from a team as a code owner March 24, 2025 12:28
@masterbater masterbater requested a review from jmikola March 24, 2025 12:28
@masterbater
Copy link
Contributor Author

Laravel has

php artisan db:show  --views
$views = Schema::getViews();

To follow laravel convention Ill adjust and add getViews

@masterbater masterbater changed the title feat: exclude system.* collections feat: exclude system.* collections, add getViews Mar 24, 2025
Copy link
Member

@GromNaN GromNaN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds perfect to me. A few minor comments.

$name = $collection->getName();

// Skip system collections
if (str_starts_with($name, 'system.')) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will preserve system collection from being dropped 👍🏻

Even for the gridfs bucket metadata, I think that's good to not remove this collection when dropAllTables is called.
https://www.mongodb.com/docs/manual/reference/system-collections/#database-specific-collections

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

system.buckets stores data for time series collections. It's unrelated to GridFS, which stores its data in <bucketName>.fs (metadata) and <bucketName.files> (chunk data) collections.

I'm not sure how dropAllTables() is used, but I don't see why you'd need system collections to persist if everything else is being removed. Moreover, if Blueprint::drop() (called by dropAllTables()) is going to invoke Collection::drop() for each collection, you could conceivably just drop the entire database to be more efficient.

@masterbater masterbater requested a review from GromNaN March 24, 2025 21:08
Copy link
Member

@GromNaN GromNaN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the corrections.
@jmikola could you also review please.

$name = $collection->getName();

// Skip system collections
if (str_starts_with($name, 'system.')) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

system.buckets stores data for time series collections. It's unrelated to GridFS, which stores its data in <bucketName>.fs (metadata) and <bucketName.files> (chunk data) collections.

I'm not sure how dropAllTables() is used, but I don't see why you'd need system collections to persist if everything else is being removed. Moreover, if Blueprint::drop() (called by dropAllTables()) is going to invoke Collection::drop() for each collection, you could conceivably just drop the entire database to be more efficient.

foreach ($db->listCollections() as $collectionInfo) {
$collectionName = $collectionInfo->getName();

// Skip system collections
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The system collections are actual collections. In the case of system.profile, that might actually be something an application would want to query.

I'm not certain it makes sense to exclude these from all output through the integration.

'schema_qualified_name' => $db->getDatabaseName() . '.' . $collectionName,
'size' => null,
'comment' => null,
'collation' => null,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is actually relevant, the information should be available via the options field for each collection info result. See: listCollections

'name' => $collectionName,
'schema' => $db->getDatabaseName(),
'schema_qualified_name' => $db->getDatabaseName() . '.' . $collectionName,
'size' => null,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is relevant, the information can be provided via $collStats (MongoDB 6.2+). I'm not sure that'd be worth the overhead to obtain this for all collections/views, though.

@masterbater masterbater changed the title feat: exclude system.* collections, add getViews feat: add getViews and categorize table types Mar 25, 2025
@masterbater
Copy link
Contributor Author

masterbater commented Mar 26, 2025

This is now a simple PR, separating tables(collections in mongodb) and views(standard views). No more segregating SYSTEM collections.

For dropAllTables, I followed the recommendation to drop the entire database instead of deleting collections individually. However, this approach may lead to errors, such as "database is in the process of being dropped." While I’m unsure if tthere any production workloads that could affect this, it does cause issues in parallel testing when using the same database—an incorrect setup that should instead use separate databases for each test.

@jmikola
Copy link
Member

jmikola commented Mar 26, 2025

For dropAllTables, I followed the recommendation to drop the entire database instead of deleting collections individually. However, this approach may lead to errors, such as "database is in the process of being dropped." While I’m unsure if tthere any production workloads that could affect this, it does cause issues in parallel testing when using the same database—an incorrect setup that should instead use separate databases for each test.

If the library needs to work around this, it might be sufficient to catch a ServerException and no-op for DatabaseDropPending code 215.

I reckon that the original approach to delete collections individually would still be subject to a race condition if a collection was created by another connection between dropAllTables() listing the collections and dropping them individually.

Copy link
Member

@jmikola jmikola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GromNaN: Let me know if there's anything else for me to review, but I think you can take it from here.

public function dropAllTables()
{
foreach ($this->getAllCollections() as $collection) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GromNaN: This is now outside the scope of this PR, but I noted that getAllCollections() could more easily use Database::listCollectionNames().

That said, I'm not even sure that method is still necessary unless you're overriding a base method. It was protected, but this seemed like the only reference to it (in this file at least).

* In MongoDB, dropping the whole database is much faster than dropping collections
* one by one. The database will be automatically recreated when a new connection
* writes to it.
*/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doc block and its method have inconsistent indentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

exclude System Collections in listCollectionNames()
3 participants