Classical Structured Prediction Losses for Sequence to Sequence Learning
By: Researcher
Disclaimer: The provided code links for this paper are external links. Science Nest has no responsibility for the accuracy, legality or content of these links. Also, by downloading this code(s), you agree to comply with the terms of use as set out by the author(s) of the code(s).
Please contact us in case of a broken link from here
| Authors | Myle Ott, Michael Auli, Sergey Edunov, David Grangier, Marc'Aurelio Ranzato |
| Journal/Conference Name | NAACL 2018 6 |
| Paper Category | Artificial Intelligence |
| Paper Abstract | There has been much recent work on training neural attention models at the sequence-level using either reinforcement learning-style methods or by optimizing the beam. In this paper, we survey a range of classical objective functions that have been widely used to train linear models for structured prediction and apply them to neural sequence to sequence models. Our experiments show that these losses can perform surprisingly well by slightly outperforming beam search optimization in a like for like setup. We also report new state of the art results on both IWSLT'14 German-English translation as well as Gigaword abstractive summarization. On the larger WMT'14 English-French translation task, sequence-level training achieves 41.5 BLEU which is on par with the state of the art. |
| Date of publication | 2017 |
| Code Programming Language | Python |
| Comment |
Copyright Researcher 2021