Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don’t try to connect to blacklisted resources #202

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

earlonrails
Copy link

Hello! Here is the PR for not trying to connect to a blacklisted host.
Here is some background:
When using makara with a master/slave setup and the master goes down, we would like to have the connection timeout and the master blacklisted for a period of time. Just handle reads from the slave and writes will fail. This is useful when AWS or other db providers decide to cycle machines; for instance applying meltdown patch. Another use case for this is to be able to resize your master db at any point without significant down time, assuming you are read heavy. We are 90% reads in this case and cannot go down for a db resize. We would like to continue reading from the slave and serving 90% of our requests.

Our database.yml file looks like this with this patch:

development: &defaults
  adapter: postgresql_makara
  makara:
    blacklist_duration: 5
    master_ttl: 5
    master_strategy: round_robin
    sticky: true
    connections:
      - role: master
        url: postgresql://localhost:5432/app_dev?pool=15&timeout=500
      - role: slave
        url: postgresql://localhost:5433/app_dev?pool=15&timeout=500
    connection_error_matchers:
      - !ruby/regexp /invalid encoding name/
  encoding: unicode
  reconnect: true
  connect_timeout: 1

The first request to hit the DB will take 1 second, for client connection timeout from postgres, plus the normal request time to return. The problem seems to occur when calling https://github.com/taskrabbit/makara/blob/master/lib/makara/pool.rb#L131 connection_made? That calls _makara_connected? which will call _makara_connection which will try to connect to the db even if it is blacklisted already.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant