Skip to content

Commit

Permalink
Add diagrams to ml-frameworks website pages. (#14145)
Browse files Browse the repository at this point in the history
Diagrams are a nice way to break up the flow of a webpage and
communicate relationships visually. This "ml-frameworks" pages
(https://openxla.github.io/iree/guides/ml-frameworks/) already uses
sections, icons, and lists, but I want to keep making the website easier
to scan through, as we bridge between so many different technologies.


![image](https://github.com/openxla/iree/assets/4010439/7e31fe15-ce7d-46b1-84e9-53885ded20e2)

![image](https://github.com/openxla/iree/assets/4010439/b2d690c5-ff35-47f5-a8b6-6b6daa156a25)

![image](https://github.com/openxla/iree/assets/4010439/aee56bfa-d15a-4085-88fc-85f0410662ef)

![image](https://github.com/openxla/iree/assets/4010439/279ea730-89ac-4527-9700-b0241b724020)

---

I also tried a version of this diagram with more detail, but the layout
was tricky to tune:

![image](https://github.com/openxla/iree/assets/4010439/bbd58a8d-4c52-421c-b21d-032bcd4043f1)
  • Loading branch information
ScottTodd authored Jun 21, 2023
1 parent 39d45a1 commit e6d6c39
Show file tree
Hide file tree
Showing 5 changed files with 136 additions and 12 deletions.
46 changes: 36 additions & 10 deletions docs/website/docs/guides/ml-frameworks/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,26 @@
IREE supports popular machine learning frameworks using the same underlying
technology.

``` mermaid
graph LR
accTitle: ML framework to runtime deployment workflow overview
accDescr {
Programs start in some ML framework.
Programs are imported into MLIR.
The IREE compiler uses the imported MLIR.
Compiled programs are used by the runtime.
}
A[ML frameworks]
B[Imported MLIR]
C[IREE compiler]
D[Runtime deployment]
A --> B
B --> C
C --> D
```

## :octicons-list-unordered-16: Supported frameworks

See end-to-end examples of how to use each framework with IREE:
Expand All @@ -25,8 +45,10 @@ as well as the
Each machine learning framework has some "export" mechanism that snapshots the
structure and data in your program. These exported programs can then be
"imported" into IREE's compiler by using either a stable import format or one of
IREE's importer tools. This export/import process is specific to each frontend
and typically involves a number of stages:
IREE's importer tools.

This export/import process is specific to each frontend and typically involves a
number of stages:

1. Capture/trace/freeze the ML model into a graph
2. Write that graph to an interchange format (e.g. SavedModel, TorchScript)
Expand All @@ -39,14 +61,18 @@ language and framework.

## :octicons-gear-16: Compilation

During compilation we load an MLIR file and compile for the specified set of
backends (CPU, GPU, etc). Each of these backends creates custom native code to
execute on the target device. Once compiled, the resulting artifact can be
executed on the specified devices using IREE's runtime.
IREE compiles MLIR files for specified sets of backends (CPU, GPU, etc). Each
backend generates optimized native code custom to the input program and
intended target platform. Once compiled, modules can be executed using IREE's
runtime.

See the [deployment configuration guides](../deployment-configurations/index.md)
for details on selecting a compiler backend and tuning options for your choice
of target platform(s) or device(s).

## :octicons-rocket-16: Execution

The final stage is executing the now compiled module. This involves selecting
what compute devices should be used, loading the module, and executing the
module with the intended inputs. IREE provides several
[language bindings](../../reference/bindings/index.md) for its runtime API.
Compiled modules can be executed by selecting what compute devices to use,
loading the module, and then executing it with the intended inputs. IREE
provides several [language bindings](../../reference/bindings/index.md) for its
runtime API.
36 changes: 36 additions & 0 deletions docs/website/docs/guides/ml-frameworks/pytorch.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,42 @@ IREE supports compiling and running PyTorch programs represented as
`nn.Module` [classes](https://pytorch.org/docs/stable/generated/torch.nn.Module.html)
as well as models defined using [`functorch`](https://pytorch.org/functorch/).

``` mermaid
graph LR
accTitle: PyTorch to runtime deployment workflow overview
accDescr {
Programs start as either PyTorch nn.Module or functorch programs.
Programs are imported into MLIR as either StableHLO, TOSA, or Linalg.
The IREE compiler uses the imported MLIR.
Compiled programs are used by the runtime.
}
subgraph A[PyTorch]
direction TB
A1[nn.Module]
A2[functorch]
A1 --- A2
end
subgraph B[MLIR]
direction TB
B1[StableHLO]
B2[TOSA]
B3[Linalg]
B1 --- B2
B2 --- B3
end
C[IREE compiler]
D[Runtime deployment]
A -- torch_mlir --> B
B --> C
C --> D
```

## Prerequisites

Install IREE pip packages, either from pip or by
Expand Down
30 changes: 29 additions & 1 deletion docs/website/docs/guides/ml-frameworks/tensorflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,35 @@ IREE supports compiling and running TensorFlow programs represented as
or stored in the `SavedModel`
[format](https://www.tensorflow.org/guide/saved_model).

<!-- TODO(??): notes about TensorFlow 2.0, supported features? -->
``` mermaid
graph LR
accTitle: TensorFlow to runtime deployment workflow overview
accDescr {
Programs start as either TensorFlow SavedModel or tf.Module programs.
Programs are imported into MLIR as StableHLO.
The IREE compiler uses the imported MLIR.
Compiled programs are used by the runtime.
}
subgraph A[TensorFlow]
direction TB
A1[SavedModel]
A2[tf.Module]
A1 --- A2
end
subgraph B[MLIR]
B1[StableHLO]
end
C[IREE compiler]
D[Runtime deployment]
A -- iree-import-tf --> B
B --> C
C --> D
```

## Prerequisites

Expand Down
26 changes: 26 additions & 0 deletions docs/website/docs/guides/ml-frameworks/tflite.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,32 @@ IREE supports compiling and running TensorFlow Lite programs stored as [TFLite
FlatBuffers](https://www.tensorflow.org/lite/guide). These files can be
imported into an IREE-compatible format then compiled to a series of backends.

``` mermaid
graph LR
accTitle: TFLite to runtime deployment workflow overview
accDescr {
Programs start as TensorFlow Lite FlatBuffers.
Programs are imported into MLIR's TOSA dialect using iree-import-tflite.
The IREE compiler uses the imported MLIR.
Compiled programs are used by the runtime.
}
subgraph A[TFLite]
A1[FlatBuffer]
end
subgraph B[MLIR]
B1[TOSA]
end
C[IREE compiler]
D[Runtime deployment]
A -- iree-import-tflite --> B
B --> C
C --> D
```

## Prerequisites

Install TensorFlow by following the
Expand Down
10 changes: 9 additions & 1 deletion docs/website/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,15 @@ markdown_extensions:
options:
custom_icons:
- overrides/.icons
- pymdownx.superfences
# Diagram support, see
# https://squidfunk.github.io/mkdocs-material/reference/diagrams/
# Docs : https://mermaid.js.org/
# Editor: https://mermaid.live/edit
- pymdownx.superfences:
custom_fences:
- name: mermaid
class: mermaid
format: !!python/name:pymdownx.superfences.fence_code_format
- pymdownx.tabbed:
alternate_style: true
- pymdownx.tasklist:
Expand Down

0 comments on commit e6d6c39

Please sign in to comment.