Advanced Interdisciplinary Projects Lab(AIPLab) 討論區

Please login or register.

請輸入帳號, 密碼以及預計登入時間


[開學]106學年第1學期的課程看版開張了 歡迎同學問問題-20170917

作者 主題: SyntaxNet-Neural Models of Syntax  (閱讀 33 次)


  • 管理員
  • Hero Member
  • *****
  • 文章: 1687
    • 檢視個人資料
SyntaxNet-Neural Models of Syntax
« 於: 八月 04, 2017, 03:13:09 am »

The reason for that is a little bit subtle. SyntaxNet, like other TensorFlow models, has a lot of knobs to turn, which affect accuracy and speed. These knobs are called hyperparameters, and control things like the learning rate and its decay, momentum, and random initialization. Because neural networks are more sensitive to the choice of these hyperparameters than many other machine learning algorithms, picking the right hyperparameter setting is very important. Unfortunately there is no tested and proven way of doing this and picking good hyperparameters is mostly an empirical science -- we try a bunch of settings and see what works best.

An additional challenge is that training these models can take a long time, several days on very fast hardware. Our solution is to train many models in parallel via MapReduce, and when one looks promising, train a bunch more models with similar settings to fine-tune the results. This can really add up -- on average, we train more than 70 models per language. The plot below shows how the accuracy varies depending on the hyperparameters as training progresses. The best models are up to 4% absolute more accurate than ones trained without hyperparameter tuning.

SimplePortal Classic 2.0.5