How to export and load automodel

[psaks]

Hi,

I am trying to save and load an automodel using the test script below.

import tensorflow as tf
from tensorflow.keras.models import load_model

import autokeras as ak


import numpy as np
import pandas as pd


TRAIN_DATA_URL = "https://storage.googleapis.com/tf-datasets/titanic/train.csv"
TEST_DATA_URL = "https://storage.googleapis.com/tf-datasets/titanic/eval.csv"

train_file_path = tf.keras.utils.get_file("train.csv", TRAIN_DATA_URL)
test_file_path = tf.keras.utils.get_file("eval.csv", TEST_DATA_URL)

x_train = pd.read_csv(train_file_path)
y_train = x_train.pop("survived")

# Preparing testing data.
x_test = pd.read_csv(test_file_path)
y_test = x_test.pop("survived")

clf = ak.StructuredDataClassifier(overwrite=True, max_trials=3)
clf.fit(x_train, y_train, epochs=10)

model = clf.export_model()

model.save('model_autokeras', save_format='tf')

loaded_model = load_model('model_autokeras', custom_objects=ak.CUSTOM_OBJECTS)

predicted_y = loaded_model.predict(x_test)
print(predicted_y)

and get the following error;

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float).

I understand that the loaded model is just the underlying keras model and when looking at automodel.predict method, the pipeline is being used before passing the processed data to the underlying keras model predict method.

How do you export/import the entire automodel?

[haifeng-jin]

Maybe you can first convert the type of the input to string type instead of float type?

[stancsz]

I took a look at the data, it seems like for now, you can resolve this by doing

• select columns as features/target
• encode your data if is a categorical field

afaik loaded_model.predict is a keras function. we can dig into this a bit deeper, perhaps replicating the pre-processing logic in a ak.transformx(x_test) wrapper if it's not implemented already? @mklasby @haifeng-jin

predicted_y = loaded_model.predict(x_test.astype('float32'))
>> ValueError: could not convert string to float: 'male'

[mklasby]

Here's one solution that will work to fix your encoding issues, but requires a label encoder from sklearn. However, predicted_y consists of a list of floats rather than binary labels, so that will require some adjustment, perhaps by endoing y_train and y_test before training?

import tensorflow as tf
from tensorflow.keras.models import load_model

import autokeras as ak


import numpy as np
import pandas as pd
from sklearn.preprocessing import LabelEncoder


TRAIN_DATA_URL = "https://storage.googleapis.com/tf-datasets/titanic/train.csv"
TEST_DATA_URL = "https://storage.googleapis.com/tf-datasets/titanic/eval.csv"

train_file_path = tf.keras.utils.get_file("train.csv", TRAIN_DATA_URL)
test_file_path = tf.keras.utils.get_file("eval.csv", TEST_DATA_URL)

x_train = pd.read_csv(train_file_path)
y_train = x_train.pop("survived")

# Preparing testing data.
x_test = pd.read_csv(test_file_path)
y_test = x_test.pop("survived")

for df in [x_train, x_test]: 
    for col in df.columns: 
        if df[col].dtypes == 'object': 
            le = preprocessing.LabelEncoder()
            df[col] = le.fit_transform(df[col])


clf = ak.StructuredDataClassifier(overwrite=True, max_trials=3)
clf.fit(x_train, y_train, epochs=10)

model = clf.export_model()

model.save('model_autokeras', save_format='tf')

loaded_model = load_model('model_autokeras', custom_objects=ak.CUSTOM_OBJECTS)

predicted_y = loaded_model.predict(x_test)
print(predicted_y)

[psaks]

Thanks all,
Yes, I did manage to work around the problem with OneHot encoding, but the deeper issue is not related to this particular dataset and work-around. It is about saving artefacts that can be loaded again in a different process to make predictions, e.g., deployment in production.
What are the challenges associated with saving and loading an automodel instead of just the underlying keras model? if anyone has insights regarding this, please let me know before i chase down the rabbit hole :)

[alankrantas]

I had the same problem. I used Tensorflow's plot_model(loaded_model, show_dtype=True) to inspect the input layer of the loaded Keras model, and it turned out to be in string type.

So simply convert the test data to string (no need to encode beforehand) and it should work:

predicted_y = loaded_model.predict(x_test.astype('str'))

[LUS24]

Hi, I had the same issue, I solved it by serializing the entire StructuredDataClassifier instance using the pickle module like this:

import pickle

with open("autokeras_model.pkl", "wb") as f:
    pickle.dump(clf, f)

with open("autokeras_model.pkl", "rb") as f:
    model = pickle.load(f)

model.predict(holdout_data)

The output was:

24/24 [==============================] - 0s 3ms/step
array([['no'],
       ['no'],
       ['no'],
       ['no'],
       ['no'],
       ['no'],
       ['no'],
       ['no'],
       ['no'],
       ['yes'],
       ['yes'],
       ['no'],
       ['no'],
...
...

[psaks]

I think you have to be careful with pickle and TF in general - it might work with a simple model, but there are no guarantees for complicated ones, which it is why it is recommended to use model.save.

https://www.tensorflow.org/guide/keras/save_and_serialize

[LUS24]

Hey, thanks for the heads up. The link does not seem to be working.

I tried to use model.save and the cited error popped up:

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float).

Even when following the approach cited in the docs.