• Room TR313
  • Room TR309
  • Room TR312
  • Room TR510
  • Room 310-1
  • Room 409-1
  • Room TR411
  • Room 412-1
  • Room 412-2
  • Room 409-2
  • Room TR511 & TR512
  • Room 413-1
  • Room 413-2
  • Room TR509
  • Room IB101
  • Room IB302
×
Let’s read the source code
Skilled

TFLite GPU Delegate and New NNAPI Delegate

TR510 14:40 ~ 15:10 English
Slide

TensorFlow is the most popular machine learning framework nowadays. TensorFlow Lite (TFLite), open sourced in late 2017, is TensorFlow’s runtime designed for mobile devices, esp. Android cell phones. TFLite is getting more and more mature. One the most interesting new components introduced recently are its GPU delegate and new NNAPI delegate. The GPU delegate uses Open GL ES compute shader on Android platforms and Metal shade on iOS devices. The original NNAPI delegate is an all-or-nothing design (if one of the ops in the compute graph is not supported by NNAPI, the whole graph is not delegated). The new one is a per-op design. When an op in a graph is not supported by NNAPI, the op is automatically fell back to the CPU runtime. I’ll have a quick review TFLite and its interpreter, then walk the audience through example usage of the two delegates and important source code of them.

I am contributor of TensorFlow, quite familiar with TensorFlow on edge devices. See my previous slide decks [1-6].

Collaborative note: https://hackmd.io/@coscup/HJbsxXg4H

Koan-Sin Tan

“freedom” Koan-Sin Tan is an old programmer, learned to use “open source” stuff on VAX-11/780 running 4.3BSD before the term “open source” was coined. He is interested in running neural networks on edge devices the past three years. When learning TensorFlow, he sent some PRs.