Digits Recognizer using Python and React. Build backend with Flask.

It is the second part of my long journey to creating the web application for recognition handwritten digits. Here I’ll explain how to integrate our trained model into the Flask application.

Set up environment

As we’ll build the application from scratch we need to create a virtual environment in order to incapsulate all our dependencies. Let’s create python3 virtual environment.

1
2
python3 -m venv venv
source venv/bin/activate

Now we have the environment and we can install all needed dependencies.

1
pip install flask flask-script scipy scikit-image numpy pillow

That’s all about setting up the environment.

Configure application

Create manage.py

We need the ability of running our application without setting env variables everytime. This goal can be achieved using flask-script package. At first we should create a file called manage.py

1
touch manage.py

Next step is getting this manage.py ready.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
from flask_script import Manager
from app import create_app

app = create_app()
manager = Manager(app)


@manager.command
def runserver():
    app.run(debug=True, host='0.0.0.0', port=5000)


if __name__ == '__main__':
    manager.run()

As you see there is an unknown function create_app inside the app package. Now we’ll make this function known.
I start from creating the package.

1
2
mkdir app
touch app/__init__.py

And then make the target function

1
2
3
4
5
6
from flask import Flask


def create_app():
    app = Flask(__name__)
    return app

It’s now our app ready and can be run by calling.

1
python3 manage.py runserver

Add storage

As we’ll deal with the kNN classifier we need to remember the clusters after the training process. The simplies way is to store serialized version of the classifier in a file. Let’s create this file.

1
2
mkdir storage
touch storage/classifier.txt

Create settings file

Finally we need to create settings file where we’ll have the paths of the application’s base directory and the classifier storage.

1
touch settings.py

And file’s content

1
2
3
4
import os

BASE_DIR = os.getcwd()
CLASSIFIER_STORAGE = os.path.join(BASE_DIR, 'storage/classifier.txt')

The app itself

Write views

The first step is to specify handlers for incoming requests. We’ll develop a SPA so the first handler will always return index.html page.

1
2
3
4
5
def create_root_view(app):
    @app.route('/', defaults={'path': ''})
    @app.route('/<path:path>')
    def root(path):
        return render_template("index.html")

Besides this view we also must create an api endpoint for predictions.

1
2
3
class PredictDigitView(MethodView):
    def post(self):
        pass

Construct prediction logic

For our prediction view we need 2 things: kNN classifier and image processor.
Then let’s make a repo for getting and updating our classifier.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import pickle


class ClassifierRepo:
    def __init__(self, storage):
        self.storage = storage

    def get(self):
        with open(self.storage, 'rb') as out:
            try:
                classifier_str = out.read()
                if classifier_str != '':
                    return pickle.loads(classifier_str)
                else:
                    return None
            except Exception:
                return None

    def update(self, classifier):
        with open(self.storage, 'wb') as in_:
            pickle.dump(classifier, in_)

Also we need a factory to create and fit our kNN model.

1
2
3
4
5
6
7
8
9
10
11
12
13
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier


class ClassifierFactory:
    @staticmethod
    def create_with_fit(data, target):
        (X_train, X_test, y_train, y_test) = train_test_split(
            data, target, test_size=0.25, random_state=42
        )
        model = KNeighborsClassifier(n_neighbors=3)
        model.fit(X_train, y_train)
        return model

Image processor will be represented by the lots of functions to convert our image from data uri to the flat numpy array of the 8x8 greyscaled image with intensity form 0 to 16(like the original ones from digits dataset).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
import numpy as np
from skimage import exposure
import base64
from PIL import Image
from io import BytesIO


def data_uri_to_image(uri):
    encoded_data = uri.split(',')[1]
    image = base64.b64decode(encoded_data)
    return Image.open(BytesIO(image))


def replace_transparent_background(image):
    image_arr = np.array(image)
    alpha1 = 0
    r2, g2, b2, alpha2 = 255, 255, 255, 255

    red, green, blue, alpha = image_arr[:, :, 0], image_arr[:, :, 1], image_arr[:, :, 2], image_arr[:, :, 3]
    mask = (alpha == alpha1)
    image_arr[:, :, :4][mask] = [r2, g2, b2, alpha2]

    return Image.fromarray(image_arr)


def resize_image(image):
    return image.resize((8, 8), Image.ANTIALIAS)


def white_to_black(image):
    image_arr = np.array(image)
    image_arr[image_arr > 230] = 0
    return Image.fromarray(image_arr)


def reduce_intensity(image):
    image_arr = np.array(image)
    image_arr = exposure.rescale_intensity(image_arr, out_range=(0, 16))
    return Image.fromarray(image_arr)


def to_classifier_input_format(data_uri):
    raw_image = data_uri_to_image(data_uri)
    image_with_background = replace_transparent_background(raw_image).convert('L')
    resized_image = resize_image(image_with_background)
    inverted_image = white_to_black(resized_image)
    low_intensed_image = reduce_intensity(inverted_image)
    flat_image = np.array(low_intensed_image).flatten()
    return np.array([flat_image])

It’s almost done. The only thing to finish our view is to combine all the things above into a service.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
from sklearn.datasets import load_digits

from app.classifier import ClassifierFactory
from app.utils import to_classifier_input_format


class PredictDigitService:
    def __init__(self, repo):
        self.repo = repo

    def handle(self, image_data_uri):
        classifier = self.repo.get()
        if classifier is None:
            digits = load_digits()
            classifier = ClassifierFactory.create(digits.data, digits.target)
            self.repo.update(classifier)
        x = to_classifier_input_format(image_data_uri)
        prediction = classifier.predict(x)[0]
        return prediction

Use this service in our view.

1
2
3
4
5
6
7
class PredictDigitView(MethodView):
    def post(self):
        repo = ClassifierRepo(CLASSIFIER_STORAGE)
        service = PredictDigitService(repo)
        image_data_uri = request.json['image']
        prediction = service.handle(image_data_uri)
        return Response(str(prediction).encode(), status=200)

And initialize handlers by calling this function inside the create_app.

1
2
3
4
5
6
7
def init_urls(app):
    app.add_url_rule(
        '/api/predict',
        view_func=PredictDigitView.as_view('predict_digit'),
        methods=['POST']
    )
    create_root_view(app)

So the final version of __init__.py inside the app folder

1
2
3
4
5
6
7
8
from flask import Flask
from .urls import init_urls


def create_app():
    app = Flask(__name__)
    init_urls(app)
    return app

Testing

Now you can test our app. Let’s run the application

1
python3 manage.py runserver

And send an image in base64 format. You can do it by downloading this image, then convert it to base64 using this resource, copy the code and save it in the file called test_request.json. Now we can send this file to get a prediction.

1
curl 'http://localhost:5000/api/predict' -X "POST" -H "Content-Type: application/json" -d @test_request.json -i && echo -e '\n\n'

You should see the following output.

1
2
3
4
5
6
7
8
9
10
(venv) Teimurs-MacBook-Pro:digits-recognizer teimurgasanov$ curl 'http://localhost:5000/api/predict' -X "POST" -H "Content-Type: application/json" -d @test_request.json -i && echo -e '\n\n'
HTTP/1.1 100 Continue

HTTP/1.0 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 1
Server: Werkzeug/0.14.1 Python/3.6.3
Date: Tue, 27 Mar 2018 07:02:08 GMT

4

As you see our web app correctly detected that it is 4.

Final result

You can find the code from this article in my Github repository.

To be continued

In the third and also the last part of building our digit recognizer, we’ll create a React application to draw digits and send them to our classifier.