動態形狀卷積類神經網路

Abstract

儘管卷積神經網絡 (CNN) 架構和機器學習框架取得了重大進展，但大多數深度學習框架在進行計算機視覺任務時，只能定義一個 CNN 模型來處理相同形狀的圖像批次，這在很大程度上限制了 AI 應用的設計。在執行訓練或推理階段，圖像會在進入卷積層之前被轉換成特定的大小，作為批次輸入。雖然批次推理由於高 GPU 利用率而具有良好的吞吐量，但在輸入圖像高而瘦的情況下，可能會導致低準確度。我們觀察到一些應用需要進行批次推理，而不需調整圖像的大小，例如 ResizeNet 模型和物體檢測模型。在本文中，我們提出了一種深度學習框架，該框架可以定義一個 CNN 模型，同時處理批次中形狀不同的圖像。與最先進的深度學習框架不同，我們實現的 CNN 模型中的神經網絡層不需要固定維度的四維輸入。我們通過將 ResizeNet 模型的推理部分替換為我們實現的模型，實現了最高達4.35倍的加速，同時僅略微降低了準確度。

Despite the advances in convolutional neural network (CNN) architectures and machine learning frameworks, most deep learning frameworks can only define a CNN model to process batches of images with the same shape when doing computer vision tasks, which limits the design of AI applications significantly. Images are transformed to a specific size before being fed into the convolutional layer as a batch in either the training or inference phase. Although the batch inference could have good throughput due to high GPU utilization, it can lead to low accuracy when the input images are tall and skinny. We observe that some applications need batch inference without resizing the images, for example, ResizeNet model and object detection models. In this paper, We present a deep learning framework with which a CNN model can be defined to process a batch of images with different shapes at the same time. Unlike state of the art deep learning frameworks, neural network layers in a CNN model defined with our implementation do not require the 4-dimensional input to have fixed dimensions. We modify the ResizeNet model by replacing the inference part with a model we implemented and achieve a speedup of up to 4.35 with little accuracy lost.