We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hi sir, I deployed TensorRT on k8s by the following: trt_model = Model( name="testtensorrt", platform=ModelFramework.TensorRT, local_folder="", uri="gs://seldon-models/triton/tf_cifar10", description="An TensorRT model", protocol=KFServingV2Protocol(), ) from tempo.serve.metadata import KubernetesOptions from tempo.seldon.k8s import SeldonCoreOptions runtime_options = SeldonCoreOptions( k8s_options=KubernetesOptions( namespace="production", authSecretName="minio-secret" ) ) remote_model = deploy(trt_model, options=runtime_options)
And it deploy Tensorrt image successful, but when I want to do predict by the input: https://github.com/SeldonIO/seldon-core/blob/master/notebooks/triton_examples.ipynb
payload = ( '{"inputs":[{"name":"input_1","datatype":"FP32","shape":[1, 32, 32, 3],"data":' + f"{test_example}" + "}]}" ) print(remote_model.predict(payload)) And it returns error: raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://172.18.255.1/seldon/production/testtensorrt/v2/models/testtensorrt/infer
After this I set my payload to only numpy array type, like this: np_test_example = np.array(test_example) print(remote_model.predict(np_test_example))
But got the same error, Is my input wrong? Thanks.
The text was updated successfully, but these errors were encountered:
Successfully merging a pull request may close this issue.
Hi sir,
I deployed TensorRT on k8s by the following:
trt_model = Model(
name="testtensorrt",
platform=ModelFramework.TensorRT,
local_folder="",
uri="gs://seldon-models/triton/tf_cifar10",
description="An TensorRT model",
protocol=KFServingV2Protocol(),
)
from tempo.serve.metadata import KubernetesOptions
from tempo.seldon.k8s import SeldonCoreOptions
runtime_options = SeldonCoreOptions(
k8s_options=KubernetesOptions(
namespace="production",
authSecretName="minio-secret"
)
)
remote_model = deploy(trt_model, options=runtime_options)
And it deploy Tensorrt image successful, but when I want to do predict by the input:
https://github.com/SeldonIO/seldon-core/blob/master/notebooks/triton_examples.ipynb
payload = (
'{"inputs":[{"name":"input_1","datatype":"FP32","shape":[1, 32, 32, 3],"data":'
+ f"{test_example}"
+ "}]}"
)
print(remote_model.predict(payload))
And it returns error:
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://172.18.255.1/seldon/production/testtensorrt/v2/models/testtensorrt/infer
After this I set my payload to only numpy array type, like this:
np_test_example = np.array(test_example)
print(remote_model.predict(np_test_example))
But got the same error, Is my input wrong?
Thanks.
The text was updated successfully, but these errors were encountered: