Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TenorRT] Predict return bad request 400 #172

Closed
peacealex63 opened this issue Jul 22, 2021 · 0 comments · Fixed by #174
Closed

[TenorRT] Predict return bad request 400 #172

peacealex63 opened this issue Jul 22, 2021 · 0 comments · Fixed by #174

Comments

@peacealex63
Copy link

Hi sir,
I deployed TensorRT on k8s by the following:
trt_model = Model(
name="testtensorrt",
platform=ModelFramework.TensorRT,
local_folder="",
uri="gs://seldon-models/triton/tf_cifar10",
description="An TensorRT model",
protocol=KFServingV2Protocol(),
)
from tempo.serve.metadata import KubernetesOptions
from tempo.seldon.k8s import SeldonCoreOptions
runtime_options = SeldonCoreOptions(
k8s_options=KubernetesOptions(
namespace="production",
authSecretName="minio-secret"
)
)
remote_model = deploy(trt_model, options=runtime_options)

And it deploy Tensorrt image successful, but when I want to do predict by the input:
https://github.com/SeldonIO/seldon-core/blob/master/notebooks/triton_examples.ipynb

payload = (
'{"inputs":[{"name":"input_1","datatype":"FP32","shape":[1, 32, 32, 3],"data":'
+ f"{test_example}"
+ "}]}"
)
print(remote_model.predict(payload))
And it returns error:
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://172.18.255.1/seldon/production/testtensorrt/v2/models/testtensorrt/infer

After this I set my payload to only numpy array type, like this:
np_test_example = np.array(test_example)
print(remote_model.predict(np_test_example))

But got the same error, Is my input wrong?
Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant