Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Avoid creating unnecessary thread pools on non-remote store node #16028

Open
Bukhtawar opened this issue Sep 21, 2024 · 0 comments
Open
Labels
bug Something isn't working Storage:Performance Storage Issues and PRs relating to data and metadata storage

Comments

@Bukhtawar
Copy link
Collaborator

Describe the bug

Noticed this thread stack on a non-remote store node. Each thread has a cost(memory) associated with it

"opensearch[9a03514d5df099aadc9e4bef14225c16][low_transfer_queue_consumer][T#3]" #85 daemon prio=5 os_prio=0 cpu=0.10ms elapsed=2140342.61s tid=0x0000fffe82d70de0 nid=0x556d waiting on condition  [0x0000fffe66003000]
   java.lang.Thread.State: WAITING (parking)
        at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
        - parking to wait for  <0x0000001009156530> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:341)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block([email protected]/AbstractQueuedSynchronizer.java:506)
        at java.util.concurrent.ForkJoinPool.unmanagedBlock([email protected]/ForkJoinPool.java:3465)
        at java.util.concurrent.ForkJoinPool.managedBlock([email protected]/ForkJoinPool.java:3436)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:1625)
        at org.opensearch.repositories.s3.async.SizeBasedBlockingQ$Consumer.consume(SizeBasedBlockingQ.java:180)
        at org.opensearch.repositories.s3.async.SizeBasedBlockingQ$Consumer.run(SizeBasedBlockingQ.java:161)
        at java.util.concurrent.Executors$RunnableAdapter.call([email protected]/Executors.java:539)
        at java.util.concurrent.FutureTask.run([email protected]/FutureTask.java:264)
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:863)
        at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1136)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635)
        at java.lang.Thread.run([email protected]/Thread.java:840)

Related component

Storage:Performance

To Reproduce

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior

Avoid creating unused threadpools

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@Bukhtawar Bukhtawar added bug Something isn't working untriaged and removed untriaged labels Sep 21, 2024
@Bukhtawar Bukhtawar added Storage Issues and PRs relating to data and metadata storage and removed untriaged labels Sep 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Storage:Performance Storage Issues and PRs relating to data and metadata storage
Projects
Status: 🆕 New
Development

No branches or pull requests

1 participant