unexpected explicit tensor data for input tensor 'attention_mask' for model 'pipeline-poc-inference__isvc-211152d1e7' of type 'INT32', expected datatype 'INT64'

[MLHafizur]

Hi deployed an ONNX model on ModelMesh serving on Triton Runtime. Now trying to create python grpc client. But getting the following issues:

Got the model metadata:
platform: "onnxruntime_onnx"

inputs {
  name: "attention_mask"
  datatype: "INT64"
  shape: 32
  shape: 64
}
inputs {
  name: "input.1"
  datatype: "INT64"
  shape: 32
  shape: 64
}
outputs {
  name: "1643"
  datatype: "FP32"
  shape: 32
  shape: 16
}

But now struggling to figure out the input data type issue, although tried in many different ways:

Traceback (most recent call last):
  File "predict.py", line 97, in <module>
    response = grpc_stub.ModelInfer(request)
  File "/home/hafizur/miniconda3/envs/onnx/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/home/hafizur/miniconda3/envs/onnx/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.INVALID_ARGUMENT
        details = "inference.GRPCInferenceService/ModelInfer: INVALID_ARGUMENT: unexpected explicit tensor data for input tensor 'attention_mask' for model 'pipeline-poc-inference__isvc-211152d1e7' of type 'INT32', expected datatype 'INT64'"
        debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:8033 {created_time:"2023-02-22T15:50:59.6900669-05:00", grpc_status:3, grpc_message:"inference.GRPCInferenceService/ModelInfer: INVALID_ARGUMENT: unexpected explicit tensor data for input tensor \'attention_mask\' for model \'pipeline-poc-inference__isvc-211152d1e7\' of type \'INT32\', expected datatype \'INT64\'"}"

The input datas are like:
[[  101  2424  2041 ...  1997  1037   102]
 [  101  1996  2343 ...  9228  2003   102]
 [  101  9710 22002 ...  1010  2256   102]
 ...
 [  101 26624  2139 ...  2055  4825   102]
 [  101  2508 22889 ...     0     0     0]
 [  101 10352 10958 ...  2053  2386   102]]
[[1 1 1 ... 1 1 1]
 [1 1 1 ... 1 1 1]
 [1 1 1 ... 1 1 1]
 ...
 [1 1 1 ... 1 1 1]
 [1 1 1 ... 0 0 0]
 [1 1 1 ... 1 1 1]]
b_input_ids data type: int64
b_input_ids shape: (32, 64)
b_input_mask data type: int64
b_input_mask shape: (32, 64)

Here are the final codes I tried to run:

def to_numpy(tensor):
    return tensor.detach().cpu().numpy() if tensor.requires_grad else tensor.cpu().numpy()

input_path = "."

# load the needed input (as needed by ONNX)
dataloader = joblib.load(os.path.join(input_path, "eval_dataloader.pkl"))

# pick a sample batch as sample input
batches = [batch for batch in dataloader]
_batch = batches[0]
# inputs needed for the model
#b_input_ids = _batch[0].to('cpu').long().to(torch.int64)
b_input_ids = _batch[0].to('cpu')
b_input_ids = to_numpy(b_input_ids)
#b_input_mask = _batch[1].to('cpu').long().to(torch.int64)
b_input_mask = _batch[1].to('cpu')
b_input_mask = to_numpy(b_input_mask)
print(b_input_ids)
print(b_input_mask)

print("b_input_ids data type:", b_input_ids.dtype)
print("b_input_ids shape:", b_input_ids.shape)
print("b_input_mask data type:", b_input_mask.dtype)
print("b_input_mask shape:", b_input_mask.shape)

# Send request to the server
grpc_channel = grpc.insecure_channel("localhost:8033")
grpc_stub = service_pb2_grpc.GRPCInferenceServiceStub(grpc_channel)


model_name = "pipeline-poc-inference"
model_version = ""


request = service_pb2.ModelMetadataRequest(name=model_name,
                                               version=model_version)
response = grpc_stub.ModelMetadata(request)
print("model metadata:\n{}".format(response))

#b_input_mask = b_input_mask.astype('int64')


#b_input_mask = b_input_mask.astype(np.int64)

b_input_mask = np.array(b_input_mask, dtype=np.int64)




# Infer
request = service_pb2.ModelInferRequest()
request.model_name = model_name
request.model_version = model_version
request.id = "my request id"

input0 = service_pb2.ModelInferRequest().InferInputTensor()
input0.name = "attention_mask"
# input0.datatype = "INT64"
# input0.shape.extend([32, 64])
# #input0.contents.int_contents[:] = b_input_mask
# #input0.contents.int_contents[:] = b_input_mask.tolist()
# #input0.contents.int_contents[:] = list(map(int, b_input_mask.tolist()))
# input0.contents.int_contents[:] = list(map(int, b_input_mask.ravel().tolist()))


input1 = service_pb2.ModelInferRequest().InferInputTensor()
input1.name = "input.1"
# input1.datatype = "INT64"
# input1.shape.extend([32, 64])
# #input0.contents.int_contents[: : ] = b_input_ids
# #input1.contents.int_contents[:] = b_input_ids.tolist()
# #input1.contents.int_contents[:] = list(map(int, b_input_ids.tolist()))
# input1.contents.int_contents[:] = list(map(int, b_input_ids.ravel().tolist()))

input0.datatype = "INT64"
input0.shape.extend([b_input_mask.shape[0], b_input_mask.shape[1]])
input0.contents.int_contents.extend(b_input_mask.ravel().tolist())

input1.datatype = "INT64"
input1.shape.extend([b_input_ids.shape[0], b_input_ids.shape[1]])
input1.contents.int_contents.extend(b_input_ids.ravel().tolist())


request.inputs.extend([input0, input1])

output0 = service_pb2.ModelInferRequest().InferRequestedOutputTensor()
output0.name = "1643"
request.outputs.extend([output0])

response = grpc_stub.ModelInfer(request)

print("response:\n{}".format(response))

What could be the issue?