Sync from apache/tvm 12/2/2020, Replace TRT integration with JSONRuntime implementation #181
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
trevor-m
force-pushed
the
dev-upstream-trt
branch
from
590caa0 to
cbb0430
Compare
zhiics
approved these changes
Dec 2, 2020
This was referenced Dec 2, 2020
apache#6690) * Turn on Rust docs and MxNet based ResNet * Add deps needed for Rust examples and docs * Setup Rust path * Bump Jenkinsfile * Fix broken version setting, which instead redirects stdout and stderr * Update Jenkinsfile * Format * Disable Rust change for now * Completely disable ResNet * Reset test changes * Remove temporary labels * Remove patch needed for docs
…pache#6602) * add support for 1.6 quantized models * fix lint * move version check function to a common utils * fix lint Co-authored-by: masa <[email protected]>
…nNLP (apache#6699) * update Update type_relations.cc Update transform.cc Update transform.cc Update transform.cc Update transform.cc Update transform.cc Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py Update mxnet.py update Update mxnet.py debug Update generic.py Update topi_integration.py fix bug update Update test_forward.py Update test_forward.py fix test case Update mxnet.py update Update mxnet.py Update mxnet.py Update test_forward.py Update mxnet.py Update mxnet.py Update test_forward.py Update mxnet.py Update mxnet.py Update mxnet.py debug Update mxnet.py Update mxnet.py Update test_forward.py Update mxnet.py * address comments * Update mxnet.py * Update mxnet.py * fix * improve where test * Update test_forward.py * Update test_forward.py * Update test_forward.py * update * Update mxnet.py * Update mxnet.py * Update mxnet.py debug Update common.py update Update mxnet.py update Update test_forward.py Update test_forward.py * update * fix lint * Update mxnet.py * Update test_op_level1.py * fix lint
…pache#6707) The dll search directories need to be manually added by os.add_dll_directory after python3.8.
* Add task scheduler * fix lint * fix tests * fix tests * fix tests * fix test cases * fix test cases * fix tests * address comments
* Add shape to DAG print * avoid useless cross-thread reduction * Fix stage order * support multiple cache_read * lint * fix * fix * address comment * fix ci * Trigger CI & Update doc strings Co-authored-by: Lianmin Zheng <[email protected]>
Co-authored-by: YushengMa <[email protected]>
* TensorRT integration using JSONRuntime Support input nodes with multiple data entries Fix failing tests Support layout transform, add engine caching Add comment Add PruneSubgraph pass Use prune_subgraph pass, make params member of trt runtime class Hide deprecation warnings coming from TRT headers Remove general prune subgraph Save/load use_implicit_batch and workspace size Clean up Fix cpp lint Addressing review comments Refactor tests Use relay.bind instead of VarReplacer. Improve some annotation functions Add TRT docs Use DLOG, formatting Use logging.info instead of print also refactor integ tests also refactor integ tests Formatting Formatting Format python fix python format Fix pylint Fix sphinx precheck Add tensorrt.rst to toctree Allow codegen to be tested when TRT runtime is not available. Enable TRT codegen in CI linty Address more comments Formatting Formatting * Documentation changes * Address comments * Rename USE_TENSORRT->USE_TENSORRT_CODEGEN and USE_TENSORRT_GRAPH_RUNTIME->USE_TENSORRT_RUNTIME * Fix comment typo * Test CI without TRT codegen enabled * formatting * Enable USE_TENSORRT_CODEGEN in CI * Change file_util.h -> file_utils.h
* Onnx loop almost working, checkpointing for safety. * Very close to working. * Last piece is fixing scan initialization. * snapshotting for debug. * Fix Josh's issue * Use subgraph proto class. * Loop with scan. * Simple loop test now working. * Scan outputs now working. * Added second loop test. * Removed unneeded helper functions. * Remove bad merge artifact. * Cleaned up scan output creation. * Cleaned up some style mistakes. * Add pylint skip for unused-argument. * Remove onnx dependency. * Remove now obsolete checks for 0 shaped tensors. Co-authored-by: Jared Roesch <[email protected]>
…st (apache#6712) * add a regression test * fix dyn upsampling infer layout * fix lint
* [TVMScript] refactor * [TVMScript] pylint * [TVMScript] pylint
* Use tempfile in tutorials * address comment * Update tutorials/auto_scheduler/tune_matmul_x86.py * Update tutorials/auto_scheduler/tune_conv2d_layer_cuda.py Co-authored-by: Lianmin Zheng <[email protected]>
* Update batch_matmul.py Update batch_matmul.py * fix
set_target_properties. set_target_properties does not append to existing properties. There were a couple place where previously set properties were overridden with different properties. For example, the debug flags for relay were not set correctly because set_target_properties was called twice in a row with different options.
* add_annotate_fn * Reshape_ann_fn * Prune Subgraph * Dynamic Shape * Make PT Mask RCNN Work * Cleanup * Remove comments * Remove COmments * GetBatchSizeFix * Fix Remove Droupout * Fix Remove Droupout * TRT Runtime * Add MaskrCNN R50 * New Testing code * Fix black * Test Maskrcnn r50 done * Test MR50 * Space typo * Change Log to Dlog * Move test to tensorrt.py * Remove imports * Remove function * Add it to trt * import error * Imports * Add torch to CI * trt_test * Check test * Revert Pytorch install * Fix * test dynamic batch * TRT * Resolve PR comments * Zero batch size add Co-authored-by: Ubuntu <[email protected]>
* [RELAY,TOPI] Add scatter_nd op Scatter_nd is the inverse of gather_nd and also happens to be its gradient. The implementation here is not optimized. There are no cpu or gpu specific implementations. * formatting * Fix tests * formatting * specify types on test * Fix grad test * scatter_nd cuda impl * cuda impl * x86 impl * formatting * fix shape rel * fix tests * formatting
* [TOPI] deformable_conv2d in NHWC * Update python/tvm/topi/generic/nn.py Co-authored-by: Cody Yu <[email protected]> * Update python/tvm/topi/testing/deformable_conv2d_python.py Co-authored-by: Cody Yu <[email protected]> * style * fix * style Co-authored-by: Cody Yu <[email protected]>
Co-authored-by: zhangfucheng <[email protected]>
* Add recommendation to install vbguest plugin. * Update directories to match checked-in.
Remove duplicate shape function for 'copy'
* add files * update tests * test this * test this case * update jenkins file * fix offload * update * update variables * rollback ci files
* [AutoScheduler] Add layout rewrite pass in relay * fix * fix lint * fix attrs * trigger CI * Apply suggestions from code review * trigger CI * Update python/tvm/auto_scheduler/relay_integration.py * Update python/tvm/auto_scheduler/relay_integration.py * Update python/tvm/auto_scheduler/compute_dag.py * Trigger CI * Apply suggestions from code review
* Fix trt Test * Fixed stuff * Done * fix 0 * Trigger Build Co-authored-by: Ubuntu <[email protected]>
…PU (apache#7019) * [AutoScheduler] Add tutorial on auto-scheduling a network for CPU * update * update * update * improve * improve * address comments * add help on layout conversion * add help for layout conversion * update target string * update cuda logs
…ting and fix issue with pretty printing/parsing roundtrip. (apache#6874) * Update Parser in order to handle the NMS code * Add support for displaying traces optionally * WIP * Fix * Fix error reporting in parser and clean up __init__.py due to CR * Format * Quick fix for If * Fix format * Fix lint
This change fix problem with version of IP protocol on MacOS. Previous the `rpc_tracker` and `query_rpc_tracker` were not able connect to each other with default hostnames. The root cause was in method `socket.getaddrinfo`. In `rpc_tracker` the default hostname is "0.0.0.0" and `getaddrinfo` returns IPv4 type. In `query_rpc_tracker` the default hastname is "localhost" and `getaddrinfo` on MacOS returns IPv6 type. Note: on Linux both have IPv4 type. These tools worked by different protocols and this is why `query_rpc_tracker` wasn't able connect to `rpc_tracker`. Now we will prefer IPv4 type. And both `rpc_tracker` and `query_rpc_tracker` will use the same version of protocol.
…pache#7020) * [AutoScheduler] Mics update to hardware parameter and task scheduler * update * update * update * update * fix * fix * update * improve warning message * update * lint * update * update * fix * Apply suggestions from code review * trigger CI
apache#7018) * Fix GPU dynamic Topk * Fix style * Minor fix * Simplfy dynamic checking * Fix lint * More improvements * Disable test any topk
…che#7029) * Clean up DCE tests * Format * Fix * Fix
…pache#7028) * [AutoScheduler] Refactor task interface * updae tutorials and tests * update * fix lint * fix lint * update * fix test
While the functional approach is pretty neat, we ended up having global state (default frontend, dtype) and it'll be more soon (caching of inferred types, see apache#6900). To not have to pass around the state, this moves the op conversion into a class with instances having the state.
…ache#7031) * Add support for Size op in Onnx frontend. * Simplify target testing.
trevor-m
force-pushed
the
dev-upstream-trt
branch
from
6654404 to
a8bad30
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We are switching to the version of the TRT integration which is in apache/tvm. There are quite a few differences, so please see the PR for more info apache#6395