Advanced Interdisciplinary Projects Lab(AIPLab) 討論區

Please login or register.

請輸入帳號, 密碼以及預計登入時間

新聞:

[慶賀]恭喜張清貿醫師升任北榮傳醫科主治醫師-20170201

作者 主題: SyntaxNet-Neural Models of Syntax  (閱讀 58 次)

admin

  • 管理員
  • Hero Member
  • *****
  • 文章: 1702
    • 檢視個人資料
SyntaxNet-Neural Models of Syntax
« 於: 八月 04, 2017, 03:13:09 am »

The reason for that is a little bit subtle. SyntaxNet, like other TensorFlow models, has a lot of knobs to turn, which affect accuracy and speed. These knobs are called hyperparameters, and control things like the learning rate and its decay, momentum, and random initialization. Because neural networks are more sensitive to the choice of these hyperparameters than many other machine learning algorithms, picking the right hyperparameter setting is very important. Unfortunately there is no tested and proven way of doing this and picking good hyperparameters is mostly an empirical science -- we try a bunch of settings and see what works best.

An additional challenge is that training these models can take a long time, several days on very fast hardware. Our solution is to train many models in parallel via MapReduce, and when one looks promising, train a bunch more models with similar settings to fine-tune the results. This can really add up -- on average, we train more than 70 models per language. The plot below shows how the accuracy varies depending on the hyperparameters as training progresses. The best models are up to 4% absolute more accurate than ones trained without hyperparameter tuning.
已記錄
 

SimplePortal Classic 2.0.5