分散式牛頓法在規則化羅吉斯回歸上之應用

Abstract

規則化羅吉斯回歸在分類問題上， 是一個十分有用的方法。 但是對於大規模數據而言， 分散式訓練並沒有被深入的研究。 在這份工作中， 我們提出了分散式牛頓法來訓練羅吉斯回歸。 許多用來降低通訊開銷和加速計算的方法也在本文中進行了討論。 實驗結果表明， 我們提出的和現有最好的方法相比， 是十分有競爭力的。 甚至說是快於現有的方法， 如交替方向乘子法(ADMM)和Vowpal Wabbit (VW)。 我們發佈了一個基於MPI的實現以供大眾使用。

Regularized logistic regression is a very useful classification method, but for large- scale data, its distributed training has not been investigated much. In this work, we propose a distributed Newton method for training logistic regression. Many interesting techniques are discussed for reducing the communication cost and speeding up the computation. Experiments show that the proposed method is competitive with or even faster than state-of-the-art approaches such as Alternating Direction Method of Multipliers (ADMM) and Vowpal Wabbit (VW). We have released an MPI-based implementation for public use.