|
| 1 | +# 🎨 Image processing as a service 🐍 |
| 2 | + |
| 3 | + |
| 4 | + |
| 5 | +> _This is not an official Google product. This is an article aiming at giving you ideas…_ |
| 6 | +
|
| 7 | +## 👋 Hello |
| 8 | + |
| 9 | +Have you ever written a script to transform an image? Did you share the script with others or did you run it on multiple computers? How many times did you need to update the script or the setup instructions? Did you end up making it a service or an online app? If your script is useful, you’ll likely want to make it available to others. Deploying processing services is a recurring need – one that comes with its own set of challenges. Serverless technologies let you solve these challenges easily and efficiently. |
| 10 | + |
| 11 | +In this post, you’ll see how to… |
| 12 | + |
| 13 | +- Create an image processing service that generates coloring pages |
| 14 | +- Make it available online using minimal resources |
| 15 | + |
| 16 | +…and do it all in less than 200 lines of Python and JavaScript! |
| 17 | + |
| 18 | +## 🛠️ Tools |
| 19 | + |
| 20 | +To build and deploy a coloring page generator, you’ll need a few tools: |
| 21 | + |
| 22 | +- A library to process images |
| 23 | +- A web application framework |
| 24 | +- A web server |
| 25 | +- A serverless solution to make the demo available 24/7 |
| 26 | + |
| 27 | +## 🧱 Architecture |
| 28 | + |
| 29 | +Here is one possible architecture for a coloring page generator using Cloud Run: |
| 30 | + |
| 31 | + |
| 32 | + |
| 33 | +And here is the workflow: |
| 34 | + |
| 35 | +- 1 - The user opens the web app: the browser requests the main page. |
| 36 | +- 2 - Cloud Run serves the app HTML code. |
| 37 | +- 3 - The browser requests the additional needed resources. |
| 38 | +- 4 - Cloud Run serves the CSS, JavaScript, and other resources. |
| 39 | +- A - The user selects an image and the frontend sends the image to the `/api/coloring-page` endpoint. |
| 40 | +- B - The backend processes the input image and returns an output image, which the user can then visualize, download, or print via the browser. |
| 41 | + |
| 42 | + |
| 43 | +## 🐍 Software stack |
| 44 | + |
| 45 | +Of course, there are many different software stacks that you could use to implement such an architecture. |
| 46 | + |
| 47 | +Here is a good one based on Python: |
| 48 | + |
| 49 | + |
| 50 | + |
| 51 | +It includes: |
| 52 | + |
| 53 | +- [Gunicorn](https://pypi.org/project/gunicorn): A production-grade WSGI HTTP server |
| 54 | +- [Flask](https://pypi.org/project/Flask): A popular web app framework |
| 55 | +- [scikit-image](https://pypi.org/project/scikit-image): An extensive image processing library |
| 56 | + |
| 57 | +Define these app dependencies in a file named `requirements.txt`: |
| 58 | + |
| 59 | +```txt |
| 60 | +# https://pypi.org/project/gunicorn |
| 61 | +gunicorn==20.1.0 |
| 62 | +
|
| 63 | +# https://pypi.org/project/flask |
| 64 | +Flask==2.1.1 |
| 65 | +
|
| 66 | +# https://pypi.org/project/scikit-image |
| 67 | +# scikit-image dependencies include NumPy and Pillow |
| 68 | +scikit-image==0.19.2 |
| 69 | +``` |
| 70 | + |
| 71 | +## 🎨 Image processing |
| 72 | + |
| 73 | +How do you remove colors from an image? One way is by detecting the object edges and removing everything but the edges in the result image. This can be done with a [Sobel](https://wikipedia.org/wiki/Sobel_operator) filter, a convolution filter that detects the regions in which the image intensity changes the most. |
| 74 | + |
| 75 | +Create a Python file named `main.py`, define an image processing function, and within it use the Sobel filter and other functions from scikit-image: |
| 76 | + |
| 77 | +```py |
| 78 | +import numpy as np |
| 79 | +import skimage |
| 80 | +from PIL import Image |
| 81 | +from PIL.Image import Image as PilImage |
| 82 | + |
| 83 | + |
| 84 | +def generate_coloring_page(input: PilImage) -> PilImage: |
| 85 | + # Convert to grayscale if needed |
| 86 | + if input.mode != "L": |
| 87 | + input = input.convert("L") |
| 88 | + np_image = np.asarray(input) |
| 89 | + |
| 90 | + # Detect the edges |
| 91 | + np_image = skimage.filters.sobel(np_image) |
| 92 | + # Convert to 8 bpp |
| 93 | + np_image = skimage.util.img_as_ubyte(np_image) |
| 94 | + # Invert to get dark edges on a light background |
| 95 | + np_image = 255 - np_image |
| 96 | + # Improve the contrast |
| 97 | + np_image = skimage.exposure.rescale_intensity(np_image) |
| 98 | + |
| 99 | + return Image.fromarray(np_image) |
| 100 | +``` |
| 101 | + |
| 102 | +> Note: The NumPy and Pillow libraries are automatically installed as dependencies of scikit-image. |
| 103 | +
|
| 104 | +As an example, here is how the Cloud Run logo is processed at each step: |
| 105 | + |
| 106 | + |
| 107 | + |
| 108 | +## ✨ Web app |
| 109 | + |
| 110 | +### Backend |
| 111 | + |
| 112 | +To expose both endpoints (`GET /` and `POST /api/coloring-page`), add Flask routes in `main.py`: |
| 113 | + |
| 114 | +```py |
| 115 | +import io |
| 116 | + |
| 117 | +import flask |
| 118 | +from PIL import Image |
| 119 | + |
| 120 | +app = flask.Flask(__name__, static_url_path="") |
| 121 | + |
| 122 | + |
| 123 | +@app.get("/") |
| 124 | +def index(): |
| 125 | + return app.send_static_file("index.html") |
| 126 | + |
| 127 | + |
| 128 | +@app.post("/api/coloring-page") |
| 129 | +def coloring_page(): |
| 130 | + file = flask.request.files.get("input-image") |
| 131 | + if file is None: |
| 132 | + return "Missing input-image parameter", 400 |
| 133 | + |
| 134 | + input_image = Image.open(file.stream) |
| 135 | + output_image = generate_coloring_page(input_image) |
| 136 | + |
| 137 | + image_io = io.BytesIO() |
| 138 | + output_format = "png" |
| 139 | + output_image.save(image_io, format=output_format) |
| 140 | + image_io.seek(0) |
| 141 | + |
| 142 | + return flask.send_file(image_io, mimetype=f"image/{output_format}") |
| 143 | +``` |
| 144 | + |
| 145 | +### Frontend |
| 146 | + |
| 147 | +On the browser side, write a JavaScript function that calls the `/api/coloring-page` endpoint and receives the processed image: |
| 148 | + |
| 149 | +```js |
| 150 | +async function fetchColoringPage(inputFile) { |
| 151 | + const formData = new FormData() |
| 152 | + formData.append('input-image', inputFile) |
| 153 | + |
| 154 | + const url = '/api/coloring-page' |
| 155 | + const init = { method: 'POST', body: formData } |
| 156 | + try { |
| 157 | + const response = await fetch(url, init) |
| 158 | + return response.ok ? response.blob() : null |
| 159 | + } catch (error) { |
| 160 | + console.error(error) |
| 161 | + return null |
| 162 | + } |
| 163 | +} |
| 164 | +``` |
| 165 | + |
| 166 | +The base of your app is there. Now you just need to add a mix of HTML + CSS + JS to complete the desired user experience. |
| 167 | + |
| 168 | +### Local development |
| 169 | + |
| 170 | +To develop and test the app on your computer, once your environment is set up, make sure you have the needed dependencies: |
| 171 | + |
| 172 | +```sh |
| 173 | +pip install --upgrade -r requirements.txt |
| 174 | +``` |
| 175 | + |
| 176 | +Add the following block to `main.py`. It will only execute when you run your app manually: |
| 177 | + |
| 178 | +```py |
| 179 | +import os |
| 180 | + |
| 181 | +# ... |
| 182 | + |
| 183 | +if __name__ == "__main__": |
| 184 | + os.environ["FLASK_ENV"] = "development" |
| 185 | + app.run(host="localhost", port=8080, debug=True) |
| 186 | +``` |
| 187 | + |
| 188 | +Run your app: |
| 189 | + |
| 190 | +```sh |
| 191 | +python main.py |
| 192 | +``` |
| 193 | + |
| 194 | +Flask starts a local web server: |
| 195 | + |
| 196 | +```txt |
| 197 | + * Serving Flask app 'main' (lazy loading) |
| 198 | + * Environment: development |
| 199 | + * Debug mode: on |
| 200 | + * Restarting with stat |
| 201 | + * Debugger is active! |
| 202 | + * Debugger PIN: 718-408-327 |
| 203 | + * Running on http://localhost:8080/ (Press CTRL+C to quit) |
| 204 | +``` |
| 205 | + |
| 206 | +> Note: In this mode, you’re using a development web server (one that is not suited for production). You’ll next set up the deployment to serve your app with Gunicorn, a production-grade server. |
| 207 | +
|
| 208 | +You're all set. Open `localhost:8080` in your browser, test, refine, and iterate. |
| 209 | + |
| 210 | +## 🚀 Deployment |
| 211 | + |
| 212 | +Once your app is ready for prime time, you can define how it will be served with this single line in a file named `Procfile`: |
| 213 | + |
| 214 | +```sh |
| 215 | +web: gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app |
| 216 | +``` |
| 217 | + |
| 218 | +At this stage, here are the files found in a typical project: |
| 219 | + |
| 220 | +```txt |
| 221 | +. |
| 222 | +├── main.py |
| 223 | +├── Procfile |
| 224 | +├── requirements.txt |
| 225 | +└── static |
| 226 | + ├── favicon.ico |
| 227 | + ├── index.html |
| 228 | + ├── scripts.js |
| 229 | + └── styles.css |
| 230 | +``` |
| 231 | + |
| 232 | +That's it, you can now deploy your app from the source folder: |
| 233 | + |
| 234 | +```sh |
| 235 | +SERVICE="coloring-page-generator" |
| 236 | +SOURCE="." |
| 237 | + |
| 238 | +gcloud run deploy $SERVICE --source $SOURCE --allow-unauthenticated |
| 239 | +``` |
| 240 | + |
| 241 | +## ⚙️ Under the hood |
| 242 | + |
| 243 | +The command line output details all the different steps: |
| 244 | + |
| 245 | +```txt |
| 246 | +This command is equivalent to running `gcloud builds submit --pack image=[IMAGE] SOURCE` and `gcloud run deploy SERVICE --image [IMAGE]` |
| 247 | +
|
| 248 | +Building using Buildpacks and deploying container to Cloud Run service [SERVICE] in project [PROJECT_ID] region [REGION] |
| 249 | +OK Building and deploying... Done. |
| 250 | + OK Creating Container Repository... |
| 251 | + OK Uploading sources... |
| 252 | + OK Building Container... Logs are available at […]. |
| 253 | + OK Creating Revision... |
| 254 | + OK Routing traffic... |
| 255 | + OK Setting IAM Policy... |
| 256 | +Done. |
| 257 | +Service [SERVICE] revision [SERVICE-REVISION] has been deployed and is serving 100 percent of traffic. |
| 258 | +Service URL: https://SERVICE-PROJECTHASH-REGIONID.a.run.app |
| 259 | +``` |
| 260 | + |
| 261 | +Cloud Build is indirectly called to containerize your app. One of its core components is Google Cloud [Buildpacks](https://github.com/GoogleCloudPlatform/buildpacks), which automatically builds a production-ready container image from your source code. Here are the main steps: |
| 262 | + |
| 263 | +- Cloud Build fetches the source code. |
| 264 | +- Buildpacks autodetects the app language (Python, in this case) and uses the corresponding secure base image. |
| 265 | +- Buildpacks installs the app dependencies (defined in `requirements.txt` for Python). |
| 266 | +- Buildpacks configures the service entrypoint (defined in `Procfile` for Python). |
| 267 | +- Cloud Build pushes the container image to [Artifact Registry](https://cloud.google.com/artifact-registry). |
| 268 | +- Cloud Run creates a new revision of the service based on this container image. |
| 269 | +- Cloud Run routes production traffic to it. |
| 270 | + |
| 271 | +Notes: |
| 272 | + |
| 273 | +- Buildpacks currently supports the following runtimes: Go, Java, .NET, Node.js, and Python. |
| 274 | +- The base image is actively maintained by Google, scanned for security vulnerabilities, and patched against known issues. This means that, when you deploy an update, your service is based on an image that is as secure as possible. |
| 275 | +- If you need to build your own container image, for example with a custom runtime, you can add your own `Dockerfile` and Buildpacks will use it instead. |
| 276 | + |
| 277 | +## 💫 Updates |
| 278 | + |
| 279 | +More testing from real-life users shows some issues. |
| 280 | + |
| 281 | +First, the app does not handle pictures taken with digital cameras in non-native orientations. You can fix this using the EXIF orientation data: |
| 282 | + |
| 283 | +```diff |
| 284 | +-from PIL import Image |
| 285 | ++from PIL import Image, ImageOps |
| 286 | +... |
| 287 | +def generate_coloring_page(input: PilImage) -> PilImage: |
| 288 | + # Convert to grayscale if needed |
| 289 | + if input.mode != "L": |
| 290 | + input = input.convert("L") |
| 291 | ++ # Transpose if taken in non-native orientation (rotated digital camera) |
| 292 | ++ NATIVE_ORIENTATION = 1 |
| 293 | ++ if input.getexif().get(0x0112, NATIVE_ORIENTATION) != NATIVE_ORIENTATION: |
| 294 | ++ input = ImageOps.exif_transpose(input) |
| 295 | + np_image = np.asarray(input) |
| 296 | +... |
| 297 | +``` |
| 298 | + |
| 299 | +In addition, the app is too sensitive to details in the input image. Textures in paintings, or noise in pictures, can generate many edges in the processed image. You can improve the processing algorithm by adding a denoising step upfront: |
| 300 | + |
| 301 | +```diff |
| 302 | +... |
| 303 | +def generate_coloring_page(input: PilImage) -> PilImage: |
| 304 | +... |
| 305 | + np_image = np.asarray(input) |
| 306 | + |
| 307 | ++ # Remove some noise to keep the most visible edges |
| 308 | ++ np_image = skimage.restoration.denoise_tv_chambolle(np_image, weight=0.05) |
| 309 | + # Detect the edges |
| 310 | + np_image = skimage.filters.sobel(np_image) |
| 311 | +... |
| 312 | +``` |
| 313 | + |
| 314 | +This additional step makes the coloring page cleaner and reduces the quantity of ink used if you print it: |
| 315 | + |
| 316 | + |
| 317 | + |
| 318 | +Redeploy, and the app is automatically updated: |
| 319 | + |
| 320 | +```sh |
| 321 | +gcloud run deploy $SERVICE --source $SOURCE |
| 322 | +``` |
| 323 | + |
| 324 | +## 🎉 It's alive |
| 325 | + |
| 326 | +The app is visible as a service in Cloud Run: |
| 327 | + |
| 328 | + |
| 329 | + |
| 330 | +The service dashboard gives you an overview of app usage: |
| 331 | + |
| 332 | + |
| 333 | + |
| 334 | +That's it; your image processing app is in production! |
| 335 | + |
| 336 | + |
| 337 | + |
| 338 | +## 🤯 It's serverless |
| 339 | + |
| 340 | +There are many benefits to using Cloud Run in this architecture: |
| 341 | + |
| 342 | +- Your app is available 24/7. |
| 343 | +- The environment is fully managed: you can focus on your code and not worry about the infrastructure. |
| 344 | +- Your app is automatically available through HTTPS. |
| 345 | +- You can map your app to a custom domain. |
| 346 | +- Cloud Run scales the number of instances automatically and the billing includes only the resources used when your code runs. |
| 347 | +- If your app is not used, Cloud Run scales down to zero. |
| 348 | +- If your app gets more traffic (imagine it makes the news), Cloud Run scales up to the number of instances needed. |
| 349 | +- You can control performance and cost by fine-tuning many settings: CPU, memory, concurrency, minimum instances, maximum instances, and more. |
| 350 | +- Every month, the [free tier](https://cloud.google.com/run/pricing) offers the first 50 vCPU-hours, 100 GiB-hours, and 2 million requests for no cost. |
| 351 | + |
| 352 | +## 💾 Source code |
| 353 | + |
| 354 | +The project includes just seven files and less than 200 lines of Python + JavaScript code. |
| 355 | + |
| 356 | +You can reuse this demo as a base to build your own image processing app: |
| 357 | + |
| 358 | +- Check out the source code on [GitHub](https://github.com/PicardParis/cherry-on-py/tree/main/cr_image_processing). |
| 359 | +- For step-by-step instructions on deploying the app yourself in a few minutes, see [“Deploying from scratch”](https://github.com/PicardParis/cherry-on-py/blob/main/cr_image_processing/DEPLOY.md). |
| 360 | + |
| 361 | +## 🖖 More |
| 362 | + |
| 363 | +- [Try the demo](https://coloring-page.lolo.dev) and generate your own coloring pages. |
| 364 | +- [Learn more](https://cloud.google.com/run/docs) about Cloud Run. |
| 365 | +- For more cloud content, follow me on Twitter ([@PicardParis](https://twitter.com/PicardParis)) or LinkedIn ([in/PicardParis](https://linkedin.com/in/PicardParis)), and feel free to get in touch with any feedback or questions. |
0 commit comments