SynapseML
SynapseML copied to clipboard
microsoft
feat: Remove CNTK functionality and replace with ONNX
What changes are proposed in this pull request?
Currently the ImageFeaturizer uses CNTK models. This PR replaces this underlying dependency with ONNX models, effectively removing usage of CNTK from the library.
Associated changes: Create ONNXHub for referring to predefined SynapseML models in the cloud Directly use ONNX-ML protobuf classes for splitting models at internal nodes
How is this patch tested?
TODO
- [ ] I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.
Does this PR change any dependencies?
TODO
- [ ] No. You can skip this section.
- [ ] Yes. Make sure the dependencies are resolved correctly, and list changes here.
Does this PR add a new feature? If so, have you added samples on website?
TODO
- [ ] No. You can skip this section.
- [ ] Yes. Make sure you have added samples following below steps.
TODO
- Find the corresponding markdown file for your new feature in
website/docs/documentationfolder. Make sure you choose the correct classestimators/transformersand namespace. - Follow the pattern in markdown file and add another section for your new API, including pyspark, scala (and .NET potentially) samples.
- Make sure the
DocTablepoints to correct API link. - Navigate to website folder, and run
yarn run startto make sure the website renders correctly. - Don't forget to add
<!--pytest-codeblocks:cont-->before each python code blocks to enable auto-tests for python samples. - Make sure the
WebsiteSamplesTestsjob pass in the pipeline.
AB#1910261
Hey @svotaw :wave:! Thank you so much for contributing to our repository :raised_hands:. Someone from SynapseML Team will be reviewing this pull request soon. We appreciate your patience and contributions :100:!
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
![azure-pipelines[bot] avatar](https://avatars.githubusercontent.com/in/9426?v=4 "Published by a pub.dev verified publisher"){kind=small}
Codecov Report
Merging #1593 (f82069e) into master (abdfe19) will increase coverage by
1.03%. The diff coverage is77.55%.
@@ Coverage Diff @@
## master #1593 +/- ##
==========================================
+ Coverage 83.99% 85.02% +1.03%
==========================================
Files 272 277 +5
Lines 14224 14495 +271
Branches 732 777 +45
==========================================
+ Hits 11948 12325 +377
+ Misses 2276 2170 -106
| Impacted Files | Coverage Δ | |
|---|---|---|
| ...om/microsoft/azure/synapse/ml/onnx/ONNXUtils.scala | 54.22% <54.22%> (ø) |
|
| .../microsoft/azure/synapse/ml/onnx/ONNXRuntime.scala | 73.52% <73.52%> (ø) |
|
| .../com/microsoft/azure/synapse/ml/onnx/ONNXHub.scala | 83.01% <83.01%> (ø) |
|
| ...om/microsoft/azure/synapse/ml/onnx/ONNXModel.scala | 72.48% <83.33%> (-6.53%) |
:arrow_down: |
| ...oft/azure/synapse/ml/opencv/ImageTransformer.scala | 96.45% <93.93%> (-0.05%) |
:arrow_down: |
| ...rosoft/azure/synapse/ml/cntk/ImageFeaturizer.scala | 95.37% <94.94%> (+6.90%) |
:arrow_up: |
| ...oft/azure/synapse/ml/codegen/GenerationUtils.scala | 79.16% <100.00%> (ø) |
|
| .../microsoft/azure/synapse/ml/codegen/RCodegen.scala | 90.32% <100.00%> (ø) |
|
| ...soft/azure/synapse/ml/core/env/FileUtilities.scala | 79.41% <100.00%> (+0.62%) |
:arrow_up: |
| ...osoft/azure/synapse/ml/core/env/PackageUtils.scala | 100.00% <100.00%> (ø) |
|
| ... and 19 more |
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
@mhamilton723
Scott thanks so much for iterating on this, I think the big things still left are
- Deprecation of CNTK Stuff
- Replacing CNTK in Demos where possible (its okay if we need to drop the RNN notebook)
- Moving the spruced up notebooks to the deep-learning category instead of other!
Can we do this in a separate PR? I think it will be easier to separate doc/deprecation/namespace moves into a PR right after this.