Update & add examples to work with Release 0.7.1

[aufdenkampe]

The existing examples were created by @sreeder about 1-2 years ago to work with releases 0.5 and 0.6.

It seems that none of the examples work with Release v0.7.1 and probably release 0.7.0 because of the major changes with those releases.

@emiliom, what would it take to update, replace or point to other working examples (via ReadMe) that function with the latest release?

@horsburgh or @miguelcleon, do you have any examples that work with the latest release?

cc: @roelofversteeg

Connected to #146 (comment)

[emiliom]

@aufdenkampe I can do this next week. Shouldn't be a problem -- I will test and tweak this notebook from the BiGCZ workshop.

I'm on leave today and tomorrow.

[aufdenkampe]

@emiliom, thanks for that offer! And thanks for the link to
https://github.com/BiG-CZ/wshp2017_tutorial_content/blob/master/notebooks/ODM2_Example3.ipynb

Is this also worth updating?
https://big-cz.github.io/notebook_data_demo/notebooks/2017-06-24-odm2api_sample_fromsqlite/

@miguelcleon, might you be able to update any of these four Jupyter Notebook examples to work with ODM2api v>0.7, so that we could point to them from the Examples ReadMe file?

• Connect to an ODM2 database and create a variable.
• Using ODM2API and SQLAlchemy to load time series data from a file.
• Using ODM2API, ipython widgets, and matplotlib to display time series data in an ipython notebook.
• Using ODM2API and Bokeh to create result value annotations. Bokeh allows a user to interactively select values from a graph.

[emiliom]

@aufdenkampe A question about this:

It seems that none of the examples work with Release v0.7.1 and probably release 0.7.0 because of the major changes with those releases

It sounds like you're saying this not from direct testing of every example (including Miguel's notebooks), but from interpreting the impact of changes in releases 0.7.0 and 0.7.1, specially the simplification of the module hierarchy? If so, keep in mind that those major changes were implemented in such a way that they did not break old code. We created a "transition" when the old module reference hierarchy still works but is considered deprecated (it gives warnings), and the intent to remove it altogether in some future release.

So, most of the old code should still work fine, unless it's really old and fails for reasons involving changes in even older releases! It's simply not presenting the most up-to-date and future-proof module reference style.

Regardless, I agree that we should ensure the "official" examples presented in https://github.com/ODM2/ODM2PythonAPI/tree/master/Examples should always be tested to ensure they work with the latest release; and that's probably not the case with what's currently there. I'll work on that next week, and will probably remove some of those old examples.

Is this also worth updating?
https://big-cz.github.io/notebook_data_demo/notebooks/2017-06-24-odm2api_sample_fromsqlite/

Maybe. ODM2_Example3.ipynb largely superseded it, I think.

[aufdenkampe]

cc: @erekalper & @dougvj, who work with @roelofversteeg

[aufdenkampe]

@emiliom, it might also be worth adding to the top of the examples that ODM2PythonAPI is compatible with Python 3 since since Jan. 19, 2018 (Release 0.7, https://github.com/ODM2/ODM2PythonAPI/releases/tag/v0.7.0). Ideally we would ensure that the examples are also compatible with Python 3.

[aufdenkampe]

@emiliom, in response to your #157 (comment), I think @erekalper & @dougvj & @roelofversteeg did do some direct testing, but that it didn't work. However, I don't know the details.

@erekalper & @dougvj & @roelofversteeg, could you let us know what worked and what didn't?

[emiliom]

Pasting my comments from the email, and responding to this point:

@emiliom, it might also be worth adding to the top of the examples that ODM2PythonAPI is compatible with Python 3 since since Jan. 19, 2018 (Release 0.7, https://github.com/ODM2/ODM2PythonAPI/releases/tag/v0.7.0). Ideally we would ensure that the examples are also compatible with Python 3.

I'm tied up today and tomorrow and have only scanned these emails. But before a misunderstanding sets in, I want to clarify the status of Python 3 compatibility and ODM2PythonAPI. As far as I know it's not guaranteed to work with Python 3, nor has there been much testing against it. The text Anthony is quoting on https://github.com/ODM2/ODM2PythonAPI/releases/tag/v0.7.0 ("Python 3 Compatibility and Flake 8 Improvements", which I probably wrote myself) does not mean that it works with Python 3. Instead, it just says that improvements were made to the code to move towards Python 3 compatibility.

I don't recall anyone doing significant and sustained testing for Python 3 compatibility -- or any testing at all. You should assume that it's not compatible, and Python 2.7 is required.

[emiliom]

@aufdenkampe Regarding the BiGCZ workshop odm2api notebook ODM2_Example3.ipynb: I've rerun (confirmed) and made small tweaks.

I first reran it with the "odm2client" conda env created for the workshop, and everything was fine. Then I created a new conda env using the same environment file, except for an important update: I removed the reference to the retired odm2 conda channel. I reran the notebook as is with this updated environment, and all its odm2api components ran w/o problem or updates, but the Folium map statement failed b/c of an update in a recent version. So, I fixed that. Everything ran.

But I went ahead and updated the odm2api code use to reflect recommended practices, specially to get rid of things that throw a warning b/c we've slated them to be deprecated (there were a bunch, involving mainly the import module hierarchy and a couple of parameter name changes). For good measure I also made a couple of cosmetic improvements for readability. I haven't pushed my updated notebook to any github repo yet.

We're slated to chat on Monday. Let's talk about this then, to decide what path to follow in updating the odm2api examples at https://github.com/ODM2/ODM2PythonAPI/tree/master/Examples. eg, we might want to:

• 10/7: Decided not to do this. merge that folder into https://github.com/ODM2/ODM2PythonAPI/tree/master/docs
• 10/7: All done. remove ancient and now-pointless examples
• decide which examples to target for inclusion
  • 10/7: Done. Added updated version of ODM2_Example3.ipynb notebook to Examples folder
  • 10/7: Done. https://big-cz.github.io/notebook_data_demo/notebooks/2017-06-24-odm2api_sample_fromsqlite/ should be updated and included. I thought ODM2_Example3.ipynb was very similar, but there are important differences: the latter reads a measurement-results database, while the former reads a timeseries-results database
• 10/17: Done. make the example code/notebooks also readily accessible from the odm2api Sphinx documentation
• 10/17: Done. Add a section to the Sphinx docs for "tips and tricks" or useful code snippets. It can include:
  • statements used to open an ODM2 DB connection and read some entities (sampling features, variables, etc).
  • code snippet presented in Editing individual entries from the API #155.

[aufdenkampe]

@emiliom, thanks for doing this and laying out possible next steps! Talk to you Monday!

[emiliom]

I've cleaned up the Examples folder, via PR #160 (now merged into master). Removed old sample code, and added a new notebook, WaterQualityMeasurements_RetrieveVisualize.ipynb, that's an updated and polished version of https://github.com/BiG-CZ/wshp2017_tutorial_content/blob/master/notebooks/ODM2_Example3.ipynb (works with odm2api 0.7.1, uses current best practices, added more background info). Also added auxiliary files for that notebook: the SQLite ODM2 database and conda environment files used by the notebook.

I haven't made any changes to the Sphinx documentation yet. I'll get to that in a few days.

[emiliom]

Added new notebook to Examples folder, via PR #162 (now merged into master). New notebook TimeSeries_RetrieveVisualize.ipynb complements the previously added one by focusing on a SQLite database with time series results. It's based on https://big-cz.github.io/notebook_data_demo/notebooks/2017-06-24-odm2api_sample_fromsqlite/ but with A LOT of cleanups, updates and polish. Made its organization and background info similar and consistent with the other notebook. Also added corresponding SQLite ODM2 database file.

[emiliom]

@aufdenkampe I'm done with all the updates and changes identified in this issue. The Sphinx docs are now updated. Check them out, http://odm2.github.io/ODM2PythonAPI/. If you have suggestions for further changes, let's handle them via new issues.

[aufdenkampe]

@emiliom, wow! Thank you for substantially enhancing the documentation!
The Sphinx docs are much, much more helpful and the Example Jupyter notebooks are awesome! Thanks!