swift-models
swift-models copied to clipboard
Improve BERT model
After importing the initial BERT model into the repository, there are a few remaining to-do items to expand upon this work (extracted from Dan's comment):
-
[ ] Rewrite utilities for downloading/extracting data in this PR using unified ModelSupport APIs. Consider generalizing DatasetUtilities.downloadResource to support URLs that don't end in the zipped file name. The CoLA dataset URL is one such example. Alternatively, we could host the dataset elsewhere.
-
[ ] Improve code organization. Currently, most BERT code lives at the top-level in Models/Text. Models/Text could be better organized, like Models/ImageClassification.
-
[ ] Verify that BERT training converges, and that results match a reference implementation.
-
[ ] Verify that other BERT variants work. BERT variants like RoBERTa and albert were added in this PR but are untested.