Credential issues when using custom source provider for bitbucket server

[noorul]

I made changes to dependabot-core to support the bitbucket server source.

Everything is working fine. I started using cli verify certain things a few days back and everything was working fine even without specifying the credentials, for example input

input:
    job:
      package-manager: maven
      allowed-updates:
        - update-type: all
      existing-pull-requests:
        - - dependency-name: com.arangodb:arangodb-java-driver
            dependency-version: 7.1.0
      source:
        provider: bitbucket_server
        repo: proj/test-repo
        directory: /
        commit: 0103c642c39289b0e0bece5494a485e5d859d5c8
      ignore-conditions:
        - dependency-name: com.arangodb:arangodb-java-driver
          version-requirement: "7.0.0"
    credentials:
      - type: maven_repository
        url: https://xxxx.jfrog.io/xxxx/libs-release-local
        username: $JFROG_USERNAME
        password: $JFROG_PASSWORD
      - type: maven_repository
        url: https://xxxx.jfrog.io/xxxx/libs-snapshot-local
        username: $JFROG_USERNAME
        password: $JFROG_PASSWORD

But all of a sudden it stopped working. Now I get the following error:

    cli | 2023/08/03 05:52:23 Inserting $LOCAL_GITHUB_ACCESS_TOKEN into credentials
    cli | 2023/08/03 05:52:23 Adding missing credentials-metadata into job definition
    cli | 2023/08/03 05:52:23 using image ghcr.io/github/dependabot-update-job-proxy/dependabot-update-job-proxy:latest at sha256:64a9250977fc206582758ae46861428e144abf6daf74448bd2b195706bc301a0
    cli | 2023/08/03 05:52:23 using image ghcr.io/dependabot/dependabot-updater-maven at sha256:ba5ede6cfda51f3b2c06875644bf990d461c42e4204266066f8ea119b4fa370b
  proxy | 2023/08/03 05:52:24 proxy starting, commit: 7a5d8c20c9a94f571abb6857bf47b26103757412
  proxy | 2023/08/03 05:52:24 initializing metrics client: No address passed and autodetection from environment failed
  proxy | 2023/08/03 05:52:24 Listening (:1080)
updater | Updating certificates in /etc/ssl/certs...
updater | rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
updater | 1 added, 0 removed; done.
updater | Running hooks in /etc/ca-certificates/update.d...
updater | done.
updater | 2023/08/03 05:52:26 INFO Raven 3.1.2 configured not to capture errors: DSN not set
updater | 2023/08/03 05:52:27 INFO Starting job processing
  proxy | 2023/08/03 05:52:27 [002] GET https://example.com:443/rest/api/1.0/projects/proj/repos/test-repo/raw/pom.xml?at=0103c642c39289b0e0bece5494a485e5d859d5c8
  proxy | 2023/08/03 05:52:28 [002] 401 https://example.com:443/rest/api/1.0/projects/proj/repos/test-repo/raw/pom.xml?at=0103c642c39289b0e0bece5494a485e5d859d5c8
updater | 2023/08/03 05:52:28 ERROR Error during file fetching; aborting
updater | 2023/08/03 05:52:28 ERROR Dependabot::Clients::BitbucketServer::Unauthorized
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/common/lib/dependabot/clients/bitbucket_server.rb:261:in `get'
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/common/lib/dependabot/clients/bitbucket_server.rb:73:in `fetch_file_contents'
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/common/lib/dependabot/file_fetchers/base.rb:550:in `_fetch_file_content_fully_specified'
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/common/lib/dependabot/file_fetchers/base.rb:525:in `_fetch_file_content'
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/common/lib/dependabot/file_fetchers/base.rb:163:in `fetch_file_from_host'
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/maven/lib/dependabot/maven/file_fetcher.rb:33:in `pom'
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/maven/lib/dependabot/maven/file_fetcher.rb:25:in `fetch_files'
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/common/lib/dependabot/file_fetchers/base.rb:77:in `files'
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_command.rb:67:in `dependency_files'
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_command.rb:30:in `perform_job'
updater | 2023/08/03 05:52:28 ERROR /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:52:in `run'
updater | 2023/08/03 05:52:28 ERROR bin/fetch_files.rb:23:in `<main>'
  proxy | 2023/08/03 05:52:28 [003] POST http://host.docker.internal:53131/update_jobs/cli/record_update_job_error
    cli | 2023/08/03 05:52:28 type was unexpected: expected create_pull_request got record_update_job_error
  proxy | 2023/08/03 05:52:28 [003] 200 http://host.docker.internal:53131/update_jobs/cli/record_update_job_error
  proxy | 2023/08/03 05:52:28 [004] PATCH http://host.docker.internal:53131/update_jobs/cli/mark_as_processed
    cli | 2023/08/03 05:52:28 missing expectation
  proxy | 2023/08/03 05:52:28 [004] 200 http://host.docker.internal:53131/update_jobs/cli/mark_as_processed
updater | 2023/08/03 05:52:28 INFO Finished job processing
updater | 2023/08/03 05:52:28 INFO Results:
updater | Dependabot encountered '1' error(s) during execution, please check the logs for more details.
updater | +---------------+
updater | |    Errors     |
updater | +---------------+
updater | | unknown_error |
updater | +---------------+
  proxy | 2023/08/03 05:52:29 0/1 calls cached (0%)

I tried several combination of setting credentials for the type git_source but not helping, for example

input:
    job:
      package-manager: maven
      allowed-updates:
        - update-type: all
      existing-pull-requests:
        - - dependency-name: com.arangodb:arangodb-java-driver
            dependency-version: 7.1.0
      source:
        provider: bitbucket_server
        repo: proj/test-repo
        directory: /
        commit: 0103c642c39289b0e0bece5494a485e5d859d5c8
      ignore-conditions:
        - dependency-name: com.arangodb:arangodb-java-driver
          version-requirement: "7.0.0"
    credentials:
      - type: git_source
         host: example.com
         token: $BITBUCKET_TOKEN
      - type: maven_repository
        url: https://xxxx.jfrog.io/xxxx/libs-release-local
        username: $JFROG_USERNAME
        password: $JFROG_PASSWORD
      - type: maven_repository
        url: https://xxxx.jfrog.io/xxxx/libs-snapshot-local
        username: $JFROG_USERNAME
        password: $JFROG_PASSWORD

I think the proxy is not passing credentials as bearer tokens.

Is the code available in public for ghcr.io/github/dependabot-update-job-proxy/dependabot-update-job-proxy:latest ?

[noorul]

I intercepted the request from the proxy service and found that only the Basic auth header is added. It ignores the token settings in the credentials

[jeffwidman]

@noorul thanks for the report.

The proxy isn't currently open source. I'm personally interested in changing that, but I can't speak for the company and that's a much bigger discussion for us internally, so don't hold your breath anytime soon.

As far as the token settings though, can you document here what you're seeing from the proxy vs what you'd like to see sent? I think I know what you're asking for, but it'd be helpful if it was super clearly stated what you're looking for.