Memorystore migration cleanup

wescpy · wescpy · commit d55e7f4e6c02 · 2021-12-16T18:32:41.000-08:00
diff --git a/README.md b/README.md
@@ -83,7 +83,7 @@ Module | Topic | Video | Codelab | START here | FINISH here
 3|Migrate to Cloud Datastore| [link](http://twitter.com/googledevs/status/1422966928910393347?utm_source=twitter&utm_medium=unpaidsoc&utm_campaign=CDR_wes_aap-serverless_mgrcloudds_201003&utm_content=-) | [link](http://g.co/codelabs/pae-migrate-datastore) | Module 2 [code](/mod2a-cloudndb) (2.x) &amp; [code](/mod2b-cloudndb) (3.x) | Module 3 [code](/mod3a-datastore) (2.x) &amp; [code](/mod3b-datastore) (3.x)
 4|Migrate to Cloud Run with Docker| [link](https://twitter.com/googledevs/status/1428041270702735362?utm_source=twitter&utm_medium=unpaidsoc&utm_campaign=CDR_wes_aap-serverless_mgrcrdckr_sms_201017&utm_content=-)| [link](http://g.co/codelabs/pae-migrate-rundocker) | Module 2 [code](/mod2a-cloudndb) (2.x) &amp; Module 3 [code](/mod3b-datastore) (3.x) | Module 4 [code](/mod4a-rundocker) (2.x) &amp; [code](/mod4b-rundocker) (3.x)
 5|Migrate to Cloud Run with Buildpacks| [link](https://twitter.com/googledevs/status/1433113274984271875?utm_source=twitter&utm_medium=unpaidsoc&utm_campaign=CDR_wes_aap-serverless_mgrcrbdpk_sms_201031&utm_content=-) | [link](http://g.co/codelabs/pae-migrate-runbldpks) | Module 2 [code](/mod2b-cloudndb) (3.x) | Module 5 [code](/mod5-runbldpks) (3.x)
-6|Migrate to Cloud Firestore| _N/A_ | _N/A_ | Module 3 [code](/mod3b-datastore) (3.x) | _no work required; Datastore upgrade automatic_
+6|Migrate to Cloud Firestore| _N/A_ | _N/A_ | Module 3 [code](/mod3b-datastore) (3.x) | _no work required; [Datastore upgrade automatic](https://cloud.google.com/datastore/docs/upgrade-to-firestore)_
 7|Add App Engine `taskqueue` push tasks| [link](https://twitter.com/googledevs/status/1443410302113099778?utm_source=twitter&utm_medium=unpaidsoc&utm_campaign=CDR_wes_aap-serverless_mgrgaetasks_sms_201028&utm_content=-) | [link](http://g.co/codelabs/pae-migrate-gaetasks) | Module 1 [code](/mod1-flask) (2.x) | Module 7 [code](/mod7-gaetasks) (2.x) &amp; [code](/mod7b-gaetasks) (3.x)
 8|Migrate to Cloud Tasks| [link](https://twitter.com/googledevs/status/1450960021018267656?utm_source=twitter&utm_medium=unpaidsoc&utm_campaign=CDR_wes_aap-serverless_mgrcloudtasks_sms_201112&utm_content=-) | [link](http://g.co/codelabs/pae-migrate-cloudtasks) | Module 7 [code](/mod7-gaetasks) (2.x) | Module 8 [code](/mod8-cloudtasks) (2.x)
 9|Migrate to Python 3, Cloud Datastore &amp; Cloud Tasks v2| _TBD_ | _TBD_ | Module 8 [code](/mod8-cloudtasks) (2.x) | _TBD_
diff --git a/mod13a-memorystore/README.md b/mod13a-memorystore/README.md
@@ -1,3 +1,3 @@
-# Module 12 - Add usage of App Engine `memcache` to Flask `ndb` sample app
+# Module 13 - Migrate from App Engine `memcache` to Cloud Memorystore
 
-This repo folder is the corresponding Python 2 code to the Module 12 codelab (TBD). The tutorial STARTs with the Python 2 code in the [Module 1 repo folder](/mod1-flask) and leads developers through adding usage of App Engine's `memcache`, culminating in the code in this folder.
+This repo folder is the corresponding Python 2 code to the Module 13 codelab (TBD). The tutorial STARTs with the Python 2 code in the [Module 12 repo folder](/mod12-memcache) and leads developers through a migration to Cloud Memorystore, culminating in the code in this (`mod13a-memorystore`) folder. Also included is a migration from App Engine `ndb` to Google Cloud NDB, mirroring the content covered in [Module 2](http://g.co/codelabs/pae-migrate-cloudndb).
diff --git a/mod13a-memorystore/main.py b/mod13a-memorystore/main.py
@@ -15,10 +15,11 @@
 import os
 import pickle
 from flask import Flask, render_template, request
-from google.appengine.ext import ndb
+from google.cloud import ndb
 import redis
 
 app = Flask(__name__)
+ds_client = ndb.Client()
 HOUR = 3600
 REDIS_HOST = os.environ.get('REDIS_HOST', 'localhost')
 REDIS_PORT = os.environ.get('REDIS_PORT', '6379')
@@ -31,12 +32,14 @@ class Visit(ndb.Model):
 
 def store_visit(remote_addr, user_agent):
     'create new Visit entity in Datastore'
-    Visit(visitor='{}: {}'.format(remote_addr, user_agent)).put()
+    with ds_client.context():
+        Visit(visitor='{}: {}'.format(remote_addr, user_agent)).put()
 
 def fetch_visits(limit):
     'get most recent visits'
-    return (v.to_dict() for v in Visit.query().order(
-            -Visit.timestamp).fetch(limit))
+    with ds_client.context():
+        return (v.to_dict() for v in Visit.query().order(
+                -Visit.timestamp).fetch(limit))
 
 @app.route('/')
 def root():
@@ -51,6 +54,6 @@ def root():
     if not visits or visits[0]['visitor'] != visitor:
         store_visit(ip_addr, usr_agt)
         visits = list(fetch_visits(10))
-        REDIS.set('visits', pickle.dumps(visits), ex=HOUR)  # set() not add()
+        REDIS.set('visits', pickle.dumps(visits), ex=HOUR)
 
     return render_template('index.html', visits=visits)
diff --git a/mod13a-memorystore/requirements.txt b/mod13a-memorystore/requirements.txt
@@ -1,2 +1,3 @@
 flask
 redis
+google-cloud-ndb
diff --git a/mod13b-memorystore/README.md b/mod13b-memorystore/README.md
@@ -1,5 +1,8 @@
-# Module 12 - Add usage of App Engine `memcache` to Flask `ndb` sample app
+# Module 13 - Migrate from App Engine `memcache` to Cloud Memorystore
 
-This repo folder is the corresponding Python 3 code to the Module 12 codelab (TBD). The tutorial STARTs with the Python 2 code in the [Module 1 repo folder](/mod1-flask) and leads developers through adding usage of App Engine's `memcache`, followed by a bonus migration to Python 3, culminating in the code in this folder.
+This repo folder is the corresponding Python 3 code to a pair of different migrations related to the Module 12 and 13 codelabs. The tutorial either...
 
-> **LEGACY SERVICES PUBLIC PREVIEW**: Accessing legacy services such as App Engine `ndb` and `memcache` from Python 3 (and next generation App Engine in general) is available in a public preview. See the [Sep 2021 announcement](https://twitter.com/googledevs/status/1445916786755571712) for more information.
+- STARTs with the _Python 3_ code in the [Module 12b repo folder](/mod12b-memcache) and leads developers through migrations from App Engine `memcache` to Cloud Memorystore and App Engine `ndb` to Cloud Datastore (skipping Cloud NDB); **or**,
+- STARTs with the _Python 2_ code in the [Module 13a repo folder](/mod13a-memorystore) and leads developers through migrations from Cloud NDB to Cloud Datastore, mirroring the content covered in [Module 3](http://g.co/codelabs/pae-migrate-datastore), and also a BONUS migration from Python 2 to 3.
+
+Either way, both set of migrations culminate in the code in this (`mod13b-memorystore`) folder.
diff --git a/mod13b-memorystore/main.py b/mod13b-memorystore/main.py
@@ -12,34 +12,34 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from datetime import datetime
 import os
 import pickle
 from flask import Flask, render_template, request
-from google.cloud import ndb
+from google.cloud import datastore
 import redis
 
 app = Flask(__name__)
-ds_client = ndb.Client()
+ds_client = datastore.Client()
 HOUR = 3600
 REDIS_HOST = os.environ.get('REDIS_HOST', 'localhost')
 REDIS_PORT = os.environ.get('REDIS_PORT', '6379')
 REDIS = redis.Redis(host=REDIS_HOST, port=REDIS_PORT)
 
-class Visit(ndb.Model):
-    'Visit entity registers visitor IP address & timestamp'
-    visitor   = ndb.StringProperty()
-    timestamp = ndb.DateTimeProperty(auto_now_add=True)
-
 def store_visit(remote_addr, user_agent):
     'create new Visit entity in Datastore'
-    with ds_client.context():
-        Visit(visitor='{}: {}'.format(remote_addr, user_agent)).put()
+    entity = datastore.Entity(key=ds_client.key('Visit'))
+    entity.update({
+        'timestamp': datetime.now(),
+        'visitor': '{}: {}'.format(remote_addr, user_agent),
+    })
+    ds_client.put(entity)
 
 def fetch_visits(limit):
     'get most recent visits'
-    with ds_client.context():
-        return (v.to_dict() for v in Visit.query().order(
-                -Visit.timestamp).fetch(limit))
+    query = ds_client.query(kind='Visit')
+    query.order = ['-timestamp']
+    return query.fetch(limit=limit)
 
 @app.route('/')
 def root():
@@ -54,6 +54,6 @@ def root():
     if not visits or visits[0]['visitor'] != visitor:
         store_visit(ip_addr, usr_agt)
         visits = list(fetch_visits(10))
-        REDIS.set('visits', pickle.dumps(visits), ex=HOUR)  # set() not add()
+        REDIS.set('visits', pickle.dumps(visits), ex=HOUR)
 
     return render_template('index.html', visits=visits)
diff --git a/mod13b-memorystore/requirements.txt b/mod13b-memorystore/requirements.txt
@@ -1,3 +1,3 @@
 flask
 redis
-google-cloud-ndb
+google-cloud-datastore
diff --git a/mod2a-cloudndb/README.md b/mod2a-cloudndb/README.md
@@ -1,3 +1,3 @@
-# Module 2 - Migrate from App Engine `ndb` to Google Cloud NDB
+# Module 2 - Migrate from App Engine `ndb` to Cloud NDB
 
-This repo folder is the corresponding Python 2 code to the [Module 2 codelab](http://g.co/codelabs/pae-migrate-cloudndb). The tutorial STARTs with the Python 2 code in the [Module 1 repo folder](/mod1-flask) and leads developers through migrating away from App Engine's `ndb` to Cloud NDB to access Datastore, culminating in the code in this folder.
+This repo folder is the corresponding Python 2 code to the [Module 2 codelab](http://g.co/codelabs/pae-migrate-cloudndb). The tutorial STARTs with the Python 2 code in the [Module 1 repo folder](/mod1-flask) and leads developers through migrating away from App Engine's `ndb` to Cloud NDB to access Datastore, culminating in the code in this (`mod2a-cloudndb`) folder.
diff --git a/mod2b-cloudndb/README.md b/mod2b-cloudndb/README.md
@@ -1,3 +1,3 @@
-# Module 2 - Migrate from App Engine `ndb` to Google Cloud NDB
+# Module 2 - Migrate from App Engine `ndb` to Cloud NDB
 
-This repo folder is the corresponding Python 3 code to the [Module 2 codelab](http://g.co/codelabs/pae-migrate-cloudndb). The tutorial STARTs with the Python 2 code in the [Module 1 repo folder](/mod1-flask) and leads developers through migrating away from App Engine's `ndb` to Cloud NDB to access Datastore culminating in the code in the [mod2a-cloudndb](/mod2a-cloudndb) folder. That is followed by a BONUS migration to Python 3, thus the code in *this* (`mod2b-cloudndb`) folder.
+This repo folder is the corresponding Python 3 code to the [Module 2 codelab](http://g.co/codelabs/pae-migrate-cloudndb). The tutorial STARTs with the Python 2 code in the [Module 1 repo folder](/mod1-flask) and leads developers through migrating away from App Engine's `ndb` to Cloud NDB to access Datastore culminating in the code in the [mod2a-cloudndb](/mod2a-cloudndb) folder. That is followed by a BONUS migration to Python 3, culminating in the code in *this* (`mod2b-cloudndb`) folder.
diff --git a/mod8-cloudtasks/README.md b/mod8-cloudtasks/README.md
@@ -1,3 +1,3 @@
-# Module 8 - Migrate from App Engine `taskqueue` to Google Cloud Tasks
+# Module 8 - Migrate from App Engine `taskqueue` to Cloud Tasks
 
 This repo folder is the corresponding Python 2 code to the [Module 8 codelab](http://g.co/codelabs/pae-migrate-cloudtasks). The tutorial STARTs with the Python 2 code in the [Module 7 repo folder](/mod7-gaetasks) and leads developers through migrating from `taskqueue` to Cloud Tasks, culminating in the code in this folder.

Original file line number	Diff line number	Diff line change
`@@ -1,2 +1,3 @@`
`1`	`1`	`flask`
`2`	`2`	`redis`
	`3`	`+google-cloud-ndb`
Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,3 @@`
`1`		-# Module 8 - Migrate from App Engine `taskqueue` to Google Cloud Tasks
	`1`	+# Module 8 - Migrate from App Engine `taskqueue` to Cloud Tasks
`2`	`2`
`3`	`3`	This repo folder is the corresponding Python 2 code to the [Module 8 codelab](http://g.co/codelabs/pae-migrate-cloudtasks). The tutorial STARTs with the Python 2 code in the [Module 7 repo folder](/mod7-gaetasks) and leads developers through migrating from `taskqueue` to Cloud Tasks, culminating in the code in this folder.