Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Unity][TVMScript] Update call_packed semantics to support empty sinfo_args #16379

Merged
merged 1 commit into from
Jan 24, 2024

Conversation

Hzfengsy
Copy link
Member

In low-level Relax (after the pass CallTIRewrite), the call_packed nodes do not always have explicit sinfo_args. This PR extends the parser to support this case.

@Hzfengsy
Copy link
Member Author

cc @junrushao @cyx-6

@slyubomirsky
Copy link
Contributor

What is the resulting StructInfo if sinfo_args is empty?

@Hzfengsy
Copy link
Member Author

It should be the R.Object, which only happens in the low-level stage I think

…nfo_args

In low-level Relax (after pass `CallTIRewrite`), the `call_packed` nodes
do not always have explicit `sinfo_args`. This PR extents the parser to
support this case.
@Hzfengsy Hzfengsy force-pushed the tvmscript_call_packed branch from ad13de0 to 26d9a0b Compare January 15, 2024 06:51
@Lunderberg
Copy link
Contributor

It should be the R.Object, which only happens in the low-level stage I think

At the low-level relax, does this correspond to a void return type?

@tqchen tqchen changed the base branch from unity to main January 21, 2024 20:50
@Hzfengsy
Copy link
Member Author

At the low-level relax, does this correspond to a void return type?

You are right, it's void. But for now it is printed as an Object. Here is a real-world example:

_5_1: R.Object = R.call_packed("vm.builtin.paged_attention_kv_cache_attention", paged_kv_cache, R.prim_value(0), alloc15, alloc16, alloc17, alloc18)

@Lunderberg
Copy link
Contributor

Ah, thank you. I had interpreted not having "explicit sinfo_args" as having a null pointer for the Array<StructInfo> sinfo_args, and wanted to check if that would instead indicate a bug in CallTIRRewrite. Instead, CallTIRRewrite produces a well-defined, but empty, sinfo_args. The mismatch is in the TVMScript printer, which suppresses printing of empty sinfo_args, and the TVMScript parser, which currently requires sinfo_args even if empty.

Thank you for clearing up my confusion.

@tqchen tqchen merged commit a86e41b into apache:main Jan 24, 2024
16 checks passed
@Hzfengsy Hzfengsy deleted the tvmscript_call_packed branch January 25, 2024 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants