Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add generic metric support for Florence-2 #53

Merged
merged 32 commits into from
Sep 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
e8b971d
add generic metric support for Florence-2
SkalskiP Sep 18, 2024
3619135
fix(pre_commit): 🎨 auto format pre-commit hooks
pre-commit-ci[bot] Sep 18, 2024
dda35c2
convert images to `RGB` after loading
SkalskiP Sep 18, 2024
41d711d
Merge remote-tracking branch 'origin/feature/florence_2_generic_metri…
SkalskiP Sep 18, 2024
50aa290
fix(pre_commit): 🎨 auto format pre-commit hooks
pre-commit-ci[bot] Sep 18, 2024
3ffe234
plug in generic text metrics into Florence-2 `evaluate` method
SkalskiP Sep 18, 2024
9dbe7b8
data loaders refactor
SkalskiP Sep 19, 2024
b6bceba
fix(pre_commit): 🎨 auto format pre-commit hooks
pre-commit-ci[bot] Sep 19, 2024
44a1fb6
ready for tests
SkalskiP Sep 19, 2024
0d23420
Merge remote-tracking branch 'origin/feature/florence_2_generic_metri…
SkalskiP Sep 19, 2024
bc6abd7
fix(pre_commit): 🎨 auto format pre-commit hooks
pre-commit-ci[bot] Sep 19, 2024
bb3752b
early version of `maestro` docs
SkalskiP Sep 20, 2024
b5818e9
Merge remote-tracking branch 'origin/feature/florence_2_generic_metri…
SkalskiP Sep 20, 2024
75e6450
fix(pre_commit): 🎨 auto format pre-commit hooks
pre-commit-ci[bot] Sep 20, 2024
be135ce
Merge pull request #55 from roboflow/feature/florence_2_generic_metri…
SkalskiP Sep 20, 2024
c497f05
basic docs without TODOs
SkalskiP Sep 20, 2024
375a5c4
fix(pre_commit): 🎨 auto format pre-commit hooks
pre-commit-ci[bot] Sep 20, 2024
e46efa4
docs: 🔧 initial python based configs are added
onuralpszr Sep 20, 2024
9797ef3
fix(pre_commit): 🎨 auto format pre-commit hooks
pre-commit-ci[bot] Sep 20, 2024
b626277
chore: 🧹 formatting typing fixes added for ruff checks
onuralpszr Sep 20, 2024
7324690
Merge pull request #58 from roboflow/doc/mkdoc-configs
SkalskiP Sep 20, 2024
07ae467
Florence-2 video + MKDocs tabs configuration
SkalskiP Sep 20, 2024
1e980f4
Merge branch 'feature/florence_2_generic_metric_support' into feature…
SkalskiP Sep 20, 2024
2250e93
fix(pre_commit): 🎨 auto format pre-commit hooks
pre-commit-ci[bot] Sep 20, 2024
4f108c3
docs: 📝 multi doc version configs are added
onuralpszr Sep 20, 2024
a3af69c
ci: 👷 pypi user/password to token method
onuralpszr Sep 20, 2024
1ab22e7
ci: cosmetic title changes
onuralpszr Sep 20, 2024
3272ee3
ci: 👷 removal of old documentation ci
onuralpszr Sep 20, 2024
10c2aad
docs improvements
SkalskiP Sep 23, 2024
f2c5f2c
fix(pre_commit): 🎨 auto format pre-commit hooks
pre-commit-ci[bot] Sep 23, 2024
aeef672
Merge pull request #59 from roboflow/docs/config-multi-versions
SkalskiP Sep 23, 2024
f7fa5f9
Merge pull request #57 from roboflow/feature/florence_2_generic_metri…
SkalskiP Sep 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 0 additions & 23 deletions .github/workflows/docs.yml

This file was deleted.

4 changes: 2 additions & 2 deletions .github/workflows/maestro-tests.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Test WorkFlow
name: Maestro PyTest WorkFlow 🧪

on:
pull_request:
Expand Down Expand Up @@ -40,5 +40,5 @@ jobs:
pip install .
pip install pytest

- name: 🧪 Test
- name: 🧪 Run Tests
run: "python -m pytest ./test"
39 changes: 39 additions & 0 deletions .github/workflows/publish-dev-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Maestro Develop Documentation Workflow 📚

on:
push:
branches:
- develop
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.event_name == 'push' && github.ref}}
cancel-in-progress: true

permissions:
contents: write
pages: write
pull-requests: write


jobs:
maestro-dev-docs:
runs-on: ubuntu-latest
steps:
- name: 🔄 Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: 🐍 Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: 📦 Install Packages
run: pip install "mkdocs-material" "mkdocstrings[python]" "mkdocs-material[imaging]" mike
- name: ⚙️ Configure git for github-actions
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
- name: 🚀 Deploy MkDoc-Material with mike
run: |
MKDOCS_GIT_COMMITTERS_APIKEY=${{ secrets.GITHUB_TOKEN }} mike deploy --push develop
41 changes: 41 additions & 0 deletions .github/workflows/publish-release-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Maestro Release Documentation Workflow 📚
on:
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.event_name == 'push' && github.ref}}
cancel-in-progress: true

permissions:
contents: write
pages: write
pull-requests: write


jobs:
maestro-release-docs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10"]
steps:
- name: 🛎️ Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ github.head_ref }}

- name: 🐍 Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: 📦 Install Packages
run: pip install "mkdocs-material" "mkdocstrings[python]" "mkdocs-material[imaging]" mike
- name: ⚙️ Configure git for github-actions 👷
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
- name: 🚀 Deploy MkDoc-Material 📚
run: |
latest_tag=$(git describe --tags `git rev-list --tags --max-count=1`)
MKDOCS_GIT_COMMITTERS_APIKEY=${{ secrets.GITHUB_TOKEN }} mike deploy --push --update-aliases $latest_tag latest
17 changes: 8 additions & 9 deletions .github/workflows/pypi-publish.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
name: Multimodal Maestro Releases to PyPi
name: Maestro Releases to PyPi
on:
push:
tags:
- '[0-9]+.[0-9]+[0-9]+.[0-9]'
- '[0-9]+.[0-9]+.[0-9]'

workflow_dispatch:

Expand All @@ -27,14 +28,12 @@ jobs:
python -m pip install --upgrade build twine
python -m build
twine check --strict dist/*
- name: 🚀 Publish to PyPi
uses: pypa/gh-action-pypi-publish@release/v1
- name: 🚀 Publish to PyPi - Release
uses: pypa/gh-action-pypi-publish@release/v1.10
with:
user: ${{ secrets.PYPI_USERNAME }}
password: ${{ secrets.PYPI_PASSWORD }}
- name: 🚀 Publish to Test-PyPi
uses: pypa/gh-action-pypi-publish@release/v1
password: ${{ secrets.PYPI_API_TOKEN }}
- name: 🚀 Publish to Test-PyPi - Release
uses: pypa/gh-action-pypi-publish@release/v1.10
with:
repository-url: https://test.pypi.org/legacy/
user: ${{ secrets.PYPI_TEST_USERNAME }}
password: ${{ secrets.PYPI_TEST_PASSWORD }}
password: ${{ secrets.TEST_PYPI_API_TOKEN }}
19 changes: 10 additions & 9 deletions .github/workflows/pypi-test-publish.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
name: Multimodal Maestro Test Releases to PyPi
name: Maestro Test Releases to PyPi
on:
push:
tags:
- '[0-9]+.[0-9]+[0-9]+.[0-9]+a[0-9]'
- '[0-9]+.[0-9]+[0-9]+.[0-9]+b[0-9]'
- '[0-9]+.[0-9]+[0-9]+.[0-9]+rc[0-9]'
- '[0-9]+.[0-9]+.[0-9]+a[0-9]'
- '[0-9]+.[0-9]+.[0-9]+b[0-9]'
- '[0-9]+.[0-9]+.[0-9]+rc[0-9]'

workflow_dispatch:

Expand All @@ -30,14 +33,12 @@ jobs:
python -m pip install --upgrade build twine
python -m build
twine check --strict dist/*
- name: 🚀 Publish distribution to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
- name: 🚀 Publish to PyPi - Prelease
uses: pypa/gh-action-pypi-publish@release/v1.10
with:
user: ${{ secrets.PYPI_USERNAME }}
password: ${{ secrets.PYPI_PASSWORD }}
- name: 🚀 Publish to Test-PyPi
uses: pypa/gh-action-pypi-publish@release/v1
password: ${{ secrets.PYPI_API_TOKEN }}
- name: 🚀 Publish to Test-PyPi - Prelease
uses: pypa/gh-action-pypi-publish@release/v1.10
with:
repository-url: https://test.pypi.org/legacy/
user: ${{ secrets.PYPI_TEST_USERNAME }}
password: ${{ secrets.PYPI_TEST_PASSWORD }}
password: ${{ secrets.TEST_PYPI_API_TOKEN }}
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
**maestro** is a tool designed to streamline and accelerate the fine-tuning process for
multimodal models. It provides ready-to-use recipes for fine-tuning popular
vision-language models (VLMs) such as **Florence-2**, **PaliGemma**, and
**Phi-3.5 Vision** on downstream vision-language tasks.
**Qwen2-VL** on downstream vision-language tasks.

## 💻 install

Expand Down Expand Up @@ -40,9 +40,9 @@ arguments as the CLI example above:

```python
from maestro.trainer.common import MeanAveragePrecisionMetric
from maestro.trainer.models.florence_2 import train, TrainingConfiguration
from maestro.trainer.models.florence_2 import train, Configuration

config = TrainingConfiguration(
config = Configuration(
dataset='<DATASET_PATH>',
epochs=10,
batch_size=8,
Expand Down
Loading