TensorFlow is the most popular machine learning framework nowadays. TensorFlow Lite (TFLite), open sourced in late 2017, is TensorFlow’s runtime designed for mobile devices, esp. Android cell phones. TFLite is getting more and more mature. One the most interesting new components introduced recently are its GPU delegate and new NNAPI delegate. The GPU delegate uses Open GL ES compute shader on Android platforms and Metal shade on iOS devices. The original NNAPI delegate is an all-or-nothing design (if one of the ops in the compute graph is not supported by NNAPI, the whole graph is not delegated). The new one is a per-op design. When an op in a graph is not supported by NNAPI, the op is automatically fell back to the CPU runtime. I’ll have a quick review TFLite and its interpreter, then walk the audience through example usage of the two delegates and important source code of them.

I am contributor of TensorFlow, quite familiar with TensorFlow on edge devices. See my previous slide decks [1-6].

• [1] https://www.slideshare.net/kstan2/tensorflow-on-android
• [2] https://www.slideshare.net/kstan2/introduction-to-tensorflow-lite
• [3] https://www.slideshare.net/kstan2/caffe2-on-android
• [4] https://www.slideshare.net/kstan2/open-source-nn-frameworks-on-cellphones
• [5] https://www.slideshare.net/kstan2/why-you-cannot-use-neural-engine-to-run-your-nn-models-on-a11-devices
• [6] https://www.slideshare.net/kstan2/a-peek-into-googles-edge-tpu

共筆: https://hackmd.io/@coscup/HJbsxXg4H