Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple deps #9

Merged
merged 38 commits into from
Jan 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
32cc267
First, preliminary decoupling
FrancescAlted Jan 8, 2024
c6b03ca
New /api/fetch REST. Tests are passing now.
FrancescAlted Jan 9, 2024
c5518f6
[WIP] Decoupling utils.py (eventually removed)
FrancescAlted Jan 9, 2024
09c787f
Avoid unneeded second slicing operation
ivilata Jan 9, 2024
d00e505
Fix a bug in start, stop calculation
FrancescAlted Jan 9, 2024
5331b90
Add undeclared dependency on pydantic
ivilata Jan 10, 2024
9716aca
Avoid client dependency on models/pydantic
ivilata Jan 10, 2024
79d2fe6
Move read_metadata function to service-specific utilities
ivilata Jan 10, 2024
a9cc2fe
Move pending Blosc2 functions to service-specific utilities
ivilata Jan 10, 2024
51a37b1
Move FastAPI server-related functions to service-specific utilities
ivilata Jan 10, 2024
f63278d
Move FastAPI client-related functions to service-specific utilities
ivilata Jan 10, 2024
6c5f6c6
Move some dependencies into the services extra
ivilata Jan 10, 2024
91c6370
Move Blosc2-related utilities into their own module
ivilata Jan 10, 2024
27967e4
Move download function into client utilities
ivilata Jan 10, 2024
ef528aa
Have download function caller pass progress report function in
ivilata Jan 10, 2024
df70fa0
Move dependency on tqdm to clients extra
ivilata Jan 10, 2024
b2ad877
Fix invocation of download function from file API call
ivilata Jan 10, 2024
4bd9c7b
Notes on possible outcome of moving dependencies to services/clients
ivilata Jan 10, 2024
8e1343d
No more dependency of blosc2 (and other) in clients
FrancescAlted Jan 12, 2024
5db802b
Mount sub cache files in /files
FrancescAlted Jan 13, 2024
1cb6517
Introduced a 2-step download. 1) get url, 2) download from url
FrancescAlted Jan 14, 2024
2ad5bfe
Make sure intermediate dirs are created
FrancescAlted Jan 14, 2024
22d0b59
b2_utils.py merged into srv_utils.py
FrancescAlted Jan 14, 2024
31576a7
Add docstrings to API
FrancescAlted Jan 14, 2024
f1e2be8
Refine downloads *and* slicing
FrancescAlted Jan 14, 2024
fe86dd5
__getitem__() goes to File, and more tests
FrancescAlted Jan 15, 2024
5e8d649
cli show is using the API now
FrancescAlted Jan 15, 2024
778561e
Add new API functions and use them in cli
FrancescAlted Jan 15, 2024
0fe8b00
Use names that don't collide with Python names
FrancescAlted Jan 15, 2024
4edab93
Documented HTTP API for sub; code beautification.
FrancescAlted Jan 15, 2024
45763fd
Fixes for some PEP8 style suggestions
FrancescAlted Jan 15, 2024
8f687ee
Remove duplicated code
FrancescAlted Jan 15, 2024
1bb0a4c
Fixes for some PEP8 style suggestions
FrancescAlted Jan 15, 2024
1647a5a
Fix subscription with ND datasets consisting of strings
ivilata Jan 15, 2024
b3a6c3e
More PEP8 style fixes
FrancescAlted Jan 16, 2024
8f5df8a
Small improvements
FrancescAlted Jan 16, 2024
b0b66c7
More PEP8 style fixes
FrancescAlted Jan 16, 2024
ce32899
Fix for function call in api_utils
FrancescAlted Jan 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions SPECS.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,11 @@ You can find an example of a data root in the `root-example` folder. It contain
a = np.arange(1000, dtype="int64"))
blosc2.asarray(a, chunks=(100,), blocks=(10,), urlpath="ds-1d.b2nd", mode="w")

- `ds-1d-b.b2nd`: A 1D array (6-byte strings). Constructed as:

a = np.array([b'foobar'] * 1000)
blosc2.asarray(a, chunks=(100,), blocks=(10,), urlpath="ds-1d-b.b2nd", mode="w")

- `dir1/ds-2d.b2nd`: A 2D array (uint16). Constructed as:

a = np.arange(200, dtype="uint16").reshape(10, 20)
Expand Down
10 changes: 9 additions & 1 deletion caterva2/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,13 @@
__version__ = "0.1"

from .api import bro_host_default, pub_host_default, sub_host_default
from .api import get_roots, Root, File, Dataset
from .api import get_roots, subscribe, get_list, get_info, fetch, download
from .api import Root, File, Dataset

import pytest
import pathlib


def test(verbose=False):
"""Run the test suite.

Expand All @@ -34,11 +36,17 @@ def test(verbose=False):
verb = "-v" if verbose else ""
return pytest.main([verb, test_dir])


__all__ = [
'bro_host_default',
'pub_host_default',
'sub_host_default',
'get_roots',
'subscribe',
'get_list',
'get_info',
'fetch',
'download',
'Root',
'File',
'Dataset',
Expand Down
Loading