Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

119 joss example data generator #121

Merged
merged 11 commits into from
May 21, 2024
Merged

Conversation

rogerkuou
Copy link
Contributor

@rogerkuou rogerkuou commented May 10, 2024

Fix #119. Meanwhile also solve some popped up issues.

  • Updated notebooks with data description.
  • Removed redundant model saving in motrainer/util.py::performance since the model graph issue has been solved
  • Changed tf.keras.losses.mean_squared_error to tf.keras.losses.MeanSquaredError() to adapt to Tensorflow change
  • Temporally disabled MacOS Github Action since the tests take toolong to run. See MacOS Github Action takes too long to run #124.

@rogerkuou rogerkuou linked an issue May 10, 2024 that may be closed by this pull request
@rogerkuou rogerkuou force-pushed the 119_joss_example_data_generator branch from 553b41c to 5b4b02b Compare May 10, 2024 14:00
@rogerkuou rogerkuou force-pushed the 119_joss_example_data_generator branch from 3d4d97a to 8b737e5 Compare May 14, 2024 14:32
@rogerkuou rogerkuou force-pushed the 119_joss_example_data_generator branch from 3b7afb2 to c9ff6c0 Compare May 16, 2024 12:02
@rogerkuou rogerkuou force-pushed the 119_joss_example_data_generator branch from c9ff6c0 to f6dc236 Compare May 16, 2024 12:50
@rogerkuou rogerkuou requested a review from SarahAlidoost May 16, 2024 13:11
@rogerkuou
Copy link
Contributor Author

Hi @SarahAlidoost, when you have time, could you please review this PR? I this regarding a comment we got from JOSS review. When working on this, several other issues poped up so I fixed them in one go.

@@ -7,7 +7,8 @@
"source": [
"This notebooks demonstrate how to split data to train-test execute parallel DNN trainings.\n",
"\n",
"The example dataset `./example1_data.zarr/` can be generated using this [Jupyter Notebook](https://vegewaterdynamics.github.io/motrainer/notebooks/example_daskml/)."
"The example dataset `./example1_data.zarr/` can be generated using the following Jupyter Notebook:\n",
"- [Covert a nested DataFrame to a Dataset](../example_read_from_one_df)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"- [Covert a nested DataFrame to a Dataset](../example_read_from_one_df)"
"- [Covert a nested DataFrame to a Dataset](./example_read_from_one_df.ipynb)"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @SarahAlidoost, here I am linking to the rendered webpage rather than the Notebook itself. I find the original link works. Maybe let's keep it as it was?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussion, we used the absolute links in the docs site.

Comment on lines 14 to 15
"- [Prallely training sklearn models with dask-ml](../example_daskml)\n",
"- [Prallely training DNN with Tensorflow](../example_dnn)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"- [Prallely training sklearn models with dask-ml](../example_daskml)\n",
"- [Prallely training DNN with Tensorflow](../example_dnn)"
"- [Prallely training sklearn models with dask-ml](./example_daskml.ipynb)\n",
"- [Prallely training DNN with Tensorflow](./example_dnn.ipynb)"

@@ -6,7 +6,8 @@
"source": [
"This notebooks demonstrate how to execute parallel machine learning training using [`dask-ml`](https://ml.dask.org/) and motrainer.\n",
"\n",
"The example dataset `./example1_data.zarr/` can be generated using this [Jupyter Notebook](https://vegewaterdynamics.github.io/motrainer/notebooks/example_daskml/)."
"The example dataset `./example1_data.zarr/` can be generated using the following Jupyter Notebook:\n",
"- [Covert a nested DataFrame to a Dataset](../example_read_from_one_df)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"- [Covert a nested DataFrame to a Dataset](../example_read_from_one_df)"
"- [Covert a nested DataFrame to a Dataset](./example_read_from_one_df.ipynb)"

Copy link
Member

@SarahAlidoost SarahAlidoost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rogerkuou thanks, the relative links were not working. I suggested fixes. The rest of the changes look good. 👍

@rogerkuou rogerkuou merged commit baa9693 into main May 21, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[JOSS] Example dataset generator Jupyter Notebook
2 participants