Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to NE 5.1.2 - Add Config for new POVs #2078

Merged
merged 17 commits into from
May 17, 2022

Conversation

tgrigsby-sc
Copy link
Contributor

@tgrigsby-sc tgrigsby-sc commented Apr 28, 2022

For #2082, I added the functionality to drop it into the output, but we aren't doing anything with it yet, so I haven't written any tests for it. Once we decide to adopt a label_x and label_y if present, for example, there will be compelling function to test.

@tgrigsby-sc tgrigsby-sc changed the title updating to newest date in assets.yaml NE Config for new POVs Apr 28, 2022
@tgrigsby-sc tgrigsby-sc changed the title NE Config for new POVs Move to NE 5.1.0 - Add Config for new POVs Apr 28, 2022
Copy link
Member

@nvkelso nvkelso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the versioned asset URLs to match the new pattern. #2074

And also add the new NE table to the data assets list and wire it into the NE tables. #2075

@tgrigsby-sc
Copy link
Contributor Author

Updated for #2074

@tgrigsby-sc tgrigsby-sc requested a review from nvkelso May 4, 2022 23:52
@tgrigsby-sc tgrigsby-sc force-pushed the travisg/20220428-add-NE-support-for-new-POVs branch from eddd5d4 to e0fa3de Compare May 4, 2022 23:53
data/assets.yaml Outdated
@@ -1,5 +1,5 @@
bucket: nextzen-tile-assets
datestamp: 20220426
datestamp: 20220429
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIt: but 5.1.0 was only officially released on May 4th, so bring this forward?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated to most recent (which I think @jeffdefacto found an issue in for the iso countries.zip)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That zero length char definition is really odd. Opens fine in QGIS but PostGIS barfs on it (saying it doesn't support 0 length varchar fields).

Filed mbloch/mapshaper#541 upstream.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With mbloch/mapshaper#541 fixed upstream, I'll update Natural Earth to v5.1.1 with the fix.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Natural Earth 5.1.1 is now out, please update here and see if the problem persists.

data/assets.yaml Outdated
@@ -55,98 +55,104 @@ shapefiles:
url: http://s3.amazonaws.com/tilezen-assets/curated/admin_areas_20180409.zip

- name: ne_110m_lakes
url: http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/physical/ne_110m_lakes.zip
url: https://naciscdn.org/naturalearth/5.1.0/110m/physical/ne_110m_lakes.zip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for switching these to versioned CDN links... much more obvious which version we're on and makes updating NE a deliberate process.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. Much clearer

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These will all need to switch to 5.1.1 to pick up the other change.


-- There is no label_x and label_y for the non-countries
SELECT
fclass_iso, fclass_tlc, NULL, NULL INTO fclass_iso, fclass_tlc, label_x, label_y
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is confusingly called: "latitude" (label_y) and "longitude" (label_x) in the NE admin-1 table. Please update to pull them out.

Copy link
Member

@nvkelso nvkelso May 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(so keep the output as label_{x,y} but the input would be longitude, latitude)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in 31a5b4a

-- finally, try localities
IF NOT FOUND THEN
SELECT
fclass_iso, fclass_tlc, NULL, NULL INTO fclass_iso, fclass_tlc, label_x, label_y
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same deal: This is confusingly called: "latitude" (label_y) and "longitude" (label_x) in the NE populated places table. Please update to pull them out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in 31a5b4a

Copy link
Member

@nvkelso nvkelso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add NE and OSM unit tests for TLC, please?

data/assets.yaml Outdated
@@ -141,12 +141,15 @@ shapefiles:

- name: ne_10m_admin_0_countries
url: https://naciscdn.org/naturalearth/5.1.0/10m/cultural/ne_10m_admin_0_countries.zip
prj: 3857
Copy link
Member

@nvkelso nvkelso May 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These files aren't web mercator (3857). Instead they need to be re-projected and clipped into web mercator from WGS84 lat/lng.

That's usually a later step in the (mostly?) automated build assets scripting. Doesn't look like that part ran in the ZIP you posted.

(Like the other NE files here, the prj: 3857 line is both not needed and in this case wrong and should be removed.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated here: 2635448

@nvkelso nvkelso changed the title Move to NE 5.1.0 - Add Config for new POVs Move to NE 5.1.1 - Add Config for new POVs May 9, 2022
@tgrigsby-sc
Copy link
Contributor Author

@nvkelso added tests in 761ae3c

@tgrigsby-sc tgrigsby-sc requested a review from nvkelso May 9, 2022 20:32
@tgrigsby-sc tgrigsby-sc changed the title Move to NE 5.1.1 - Add Config for new POVs Move to NE 5.1.2 - Add Config for new POVs May 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants