Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PluggableDevice] Always link kernels_experimental c api #60786

Conversation

Zantares
Copy link

@Zantares Zantares commented Jun 6, 2023

PluggableDevice architecture relies on C APIs to communicate with the TensorFlow binary. To support pluggable device for third party like tensorflow serving, we need add always_link=1 for kernels_experimental.

Signed-off-by: Lu Teng [email protected]

@google-ml-butler google-ml-butler bot added the size:XS CL Change Size: Extra Small label Jun 6, 2023
@github-actions github-actions bot added the kokoro:force-run Tests on submitted change label Jun 6, 2023
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Jun 6, 2023
Copy link
Member

@penpornk penpornk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR!

@google-ml-butler google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Jun 6, 2023
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Jun 6, 2023
@Zantares
Copy link
Author

Hi @gbaned can you help to check the PR process? It's approved but the states is still "Assigned Reviewer".

@gbaned
Copy link
Contributor

gbaned commented Jun 12, 2023

Hi @Zantares Sorry for the delay, can you please take a look on below internal errors? Thank you!

Traceback (most recent call last):
File "/tensorflow/lite/python/analyzer_test.py", line 224, in testTxtWithEinsum
self.assertIn('Op#0 RESHAPE(T#1, T#4[512, 512]) -> [T#5]', txt)
AssertionError: 'Op#0 RESHAPE(T#1, T#4[512, 512]) -> [T#5]' not found in "=== TFLite ModelAnalyzer ===\n\nYour TFLite model has '1' subgraph(s). In the subgraph description below,\nT# represents the Tensor numbers. For example, in Subgraph#0, the RESHAPE op takes\ntensor #1 and tensor #3 as input and produces tensor #4 as output.\n\nSubgraph#0 main(T#0, T#1) -> [T#6]\n Op#0 RESHAPE(T#1, T#3[512, 512]) -> [T#4]\n Op#1 BATCH_MATMUL(T#0, T#4) -> [T#5]\n Op#2 RESHAPE(T#5, T#2[1, 100, 8, 64]) -> [T#6]\n\nTensors of Subgraph#0\n T#0(lhs) shape:[1, 100, 512], type:FLOAT32\n T#1(rhs) shape:[512, 8, 64], type:FLOAT32\n T#2(einsum/Einsum) shape:[4], type:INT32 RO 16 bytes, buffer: 3, data:[1, 100, 8, 64]\n T#3(einsum/Einsum1) shape:[2], type:INT32 RO 8 bytes, buffer: 4, data:[512, 512]\n T#4(einsum/Einsum2) shape:[512, 512], type:FLOAT32\n T#5(einsum/Einsum3) shape:[1, 100, 512], type:FLOAT32\n T#6(Identity) shape:[1, 100, 8, 64], type:FLOAT32\n\n---------------------------------------------------------------\n Model size: 1204 bytes\n Non-data buffer size: 1080 bytes (89.70 %)\n Total data buffer size: 124 bytes (10.30 %)\n (Zero value buffers): 0 bytes (00.00 %)\n\n* Buffers of TFLite model are mostly used for constant tensors.\n And zero value buffers are buffers filled with zeros.\n Non-data buffers area are used to store operators, subgraphs and etc.\n You can find more details from https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/schema/schema.fbs\n\n"

@google-ml-butler google-ml-butler bot removed the ready to pull PR ready for merge process label Jun 14, 2023
@github-actions github-actions bot added the kokoro:force-run Tests on submitted change label Jun 14, 2023
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Jun 14, 2023
@Zantares
Copy link
Author

Hi @Zantares Sorry for the delay, can you please take a look on below internal errors? Thank you!

Traceback (most recent call last): File "/tensorflow/lite/python/analyzer_test.py", line 224, in testTxtWithEinsum self.assertIn('Op#0 RESHAPE(T#1, T#4[512, 512]) -> [T#5]', txt) AssertionError: 'Op#0 RESHAPE(T#1, T#4[512, 512]) -> [T#5]' not found in "=== TFLite ModelAnalyzer ===\n\nYour TFLite model has '1' subgraph(s). In the subgraph description below,\nT# represents the Tensor numbers. For example, in Subgraph#0, the RESHAPE op takes\ntensor #1 and tensor #3 as input and produces tensor #4 as output.\n\nSubgraph#0 main(T#0, T#1) -> [T#6]\n Op#0 RESHAPE(T#1, T#3[512, 512]) -> [T#4]\n Op#1 BATCH_MATMUL(T#0, T#4) -> [T#5]\n Op#2 RESHAPE(T#5, T#2[1, 100, 8, 64]) -> [T#6]\n\nTensors of Subgraph#0\n T#0(lhs) shape:[1, 100, 512], type:FLOAT32\n T#1(rhs) shape:[512, 8, 64], type:FLOAT32\n T#2(einsum/Einsum) shape:[4], type:INT32 RO 16 bytes, buffer: 3, data:[1, 100, 8, 64]\n T#3(einsum/Einsum1) shape:[2], type:INT32 RO 8 bytes, buffer: 4, data:[512, 512]\n T#4(einsum/Einsum2) shape:[512, 512], type:FLOAT32\n T#5(einsum/Einsum3) shape:[1, 100, 512], type:FLOAT32\n T#6(Identity) shape:[1, 100, 8, 64], type:FLOAT32\n\n---------------------------------------------------------------\n Model size: 1204 bytes\n Non-data buffer size: 1080 bytes (89.70 %)\n Total data buffer size: 124 bytes (10.30 %)\n (Zero value buffers): 0 bytes (00.00 %)\n\n* Buffers of TFLite model are mostly used for constant tensors.\n And zero value buffers are buffers filled with zeros.\n Non-data buffers area are used to store operators, subgraphs and etc.\n You can find more details from https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/schema/schema.fbs\n\n"

Seems this internal error is not related to this PR, I have merged the newest master to see if it can be fixed, thanks!

@gbaned gbaned added the awaiting review Pull request awaiting review label Jul 12, 2023
@gbaned gbaned requested a review from penpornk July 12, 2023 05:05
@Zantares
Copy link
Author

Hi @penpornk may you reapprove this PR?

@gbaned gbaned requested review from penpornk and removed request for penpornk July 21, 2023 08:30
@gbaned
Copy link
Contributor

gbaned commented Aug 3, 2023

Hi @penpornk Can you please review this PR ? Thank you!

Copy link
Member

@penpornk penpornk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay! Reapproving.

@google-ml-butler google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Aug 3, 2023
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Aug 3, 2023
@copybara-service copybara-service bot merged commit 8c80c69 into tensorflow:master Aug 3, 2023
@Zantares Zantares deleted the tenglu/build_pluggable_device branch November 27, 2023 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting review Pull request awaiting review ready to pull PR ready for merge process size:XS CL Change Size: Extra Small
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants