bugbug
bugbug copied to clipboard
Published
20 hours ago •
mozilla
mozilla
Evaluate multiple output component model
#224
We are calculating metrics for each output(i.e., product, component and conflated_component) separately.
Codecov Report
Merging #906 into master will decrease coverage by
0.17%. The diff coverage is33.33%.
@@ Coverage Diff @@
## master #906 +/- ##
==========================================
- Coverage 53.03% 52.85% -0.18%
==========================================
Files 81 81
Lines 5461 5487 +26
==========================================
+ Hits 2896 2900 +4
- Misses 2565 2587 +22
| Impacted Files | Coverage Δ | |
|---|---|---|
| bugbug/models/component.py | 18.48% <20%> (+1.24%) |
:arrow_up: |
| bugbug/model.py | 63.86% <37.14%> (-4.26%) |
:arrow_down: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update e2810c2...9c2cee0. Read the comment docs.
No confidence threshold - 12463 classified for product model
pre rec spe f1 geo iba sup
Core 0.71 0.97 0.46 0.82 0.66 0.46 7257
DevTools 0.87 0.57 0.99 0.69 0.75 0.54 1074
DevTools Graveyard 0.83 0.26 1.00 0.40 0.51 0.24 19
External Software Affecting Firefox 0.62 0.22 1.00 0.32 0.47 0.20 23
Firefox 0.80 0.38 0.98 0.52 0.61 0.35 1852
Firefox Build System 0.85 0.37 1.00 0.51 0.60 0.34 620
Firefox for Android 0.81 0.36 1.00 0.50 0.60 0.34 267
Toolkit 0.82 0.25 1.00 0.38 0.50 0.23 884
WebExtensions 0.81 0.57 0.99 0.67 0.76 0.55 467
avg / total 0.76 0.74 0.68 0.70 0.65 0.43 12463
╒══════════════════════════════════════════════╤════════════════════╤════════════════════════╤══════════════════════════════════╤═══════════════════════════════════════════════════╤═══════════════════════╤════════════════════════════════════╤═══════════════════════════════════╤═══════════════════════╤═════════════════════════════╕
│ │ Core (Predicted) │ DevTools (Predicted) │ DevTools Graveyard (Predicted) │ External Software Affecting Firefox (Predicted) │ Firefox (Predicted) │ Firefox Build System (Predicted) │ Firefox for Android (Predicted) │ Toolkit (Predicted) │ WebExtensions (Predicted) │
╞══════════════════════════════════════════════╪════════════════════╪════════════════════════╪══════════════════════════════════╪═══════════════════════════════════════════════════╪═══════════════════════╪════════════════════════════════════╪═══════════════════════════════════╪═══════════════════════╪═════════════════════════════╡
│ Core (Actual) │ 7042 │ 63 │ 0 │ 3 │ 72 │ 33 │ 9 │ 18 │ 17 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ DevTools (Actual) │ 436 │ 609 │ 1 │ 0 │ 21 │ 1 │ 0 │ 1 │ 5 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ DevTools Graveyard (Actual) │ 12 │ 2 │ 5 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ External Software Affecting Firefox (Actual) │ 18 │ 0 │ 0 │ 5 │ 0 │ 0 │ 0 │ 0 │ 0 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ Firefox (Actual) │ 1094 │ 8 │ 0 │ 0 │ 707 │ 2 │ 7 │ 18 │ 16 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ Firefox Build System (Actual) │ 380 │ 5 │ 0 │ 0 │ 1 │ 227 │ 4 │ 0 │ 3 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ Firefox for Android (Actual) │ 145 │ 0 │ 0 │ 0 │ 19 │ 3 │ 96 │ 3 │ 1 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ Toolkit (Actual) │ 600 │ 3 │ 0 │ 0 │ 39 │ 1 │ 0 │ 222 │ 19 │
├──────────────────────────────────────────────┼────────────────────┼────────────────────────┼──────────────────────────────────┼───────────────────────────────────────────────────┼───────────────────────┼────────────────────────────────────┼───────────────────────────────────┼───────────────────────┼─────────────────────────────┤
│ WebExtensions (Actual) │ 150 │ 13 │ 0 │ 0 │ 24 │ 0 │ 3 │ 9 │ 268 │
╘══════════════════════════════════════════════╧════════════════════╧════════════════════════╧══════════════════════════════════╧═══════════════════════════════════════════════════╧═══════════════════════╧════════════════════════════════════╧═══════════════════════════════════╧═══════════════════════╧═════════════════════════════╛
No confidence threshold - 12463 classified for component model
pre rec spe f1 geo iba sup
Accessibility Tools 0.61 0.65 1.00 0.63 0.80 0.62 17
Activity Stream 0.62 0.29 1.00 0.40 0.54 0.27 17
Add-on Manager 0.50 0.10 1.00 0.17 0.32 0.09 10
Add-ons Manager 0.60 0.62 1.00 0.61 0.79 0.59 126
Address Bar 0.60 0.62 0.99 0.61 0.79 0.60 157
Android 0.50 0.23 1.00 0.32 0.48 0.21 13
Android Studio and Gradle Integration 0.57 0.34 1.00 0.43 0.58 0.32 47
Android Sync 0.30 0.15 1.00 0.20 0.39 0.14 20
Application Panel 0.86 0.86 1.00 0.86 0.93 0.84 7
Application Update 0.45 0.53 1.00 0.48 0.72 0.50 57
Audio/Video 0.46 0.25 1.00 0.32 0.50 0.23 64
Audio/Video: GMP 0.30 0.25 1.00 0.27 0.50 0.23 12
Audio/Video: MediaStreamGraph 0.50 0.11 1.00 0.18 0.33 0.10 18
Audio/Video: Playback 0.61 0.64 0.99 0.63 0.80 0.61 356
Audio/Video: Recording 0.82 0.64 1.00 0.72 0.80 0.62 14
Audio/Video: cubeb 0.47 0.30 1.00 0.36 0.54 0.28 27
Awesomescreen 0.23 0.18 1.00 0.20 0.42 0.16 17
Blocklist Policy Requests 0.94 0.89 1.00 0.91 0.94 0.88 54
Bookmarks & History 0.51 0.67 1.00 0.58 0.82 0.65 83
Bootstrap Configuration 0.50 0.57 1.00 0.53 0.76 0.55 14
CSS Parsing and Computation 0.71 0.71 0.99 0.71 0.84 0.68 576
CSS Transitions and Animations 0.67 0.22 1.00 0.33 0.47 0.20 9
CSS and Themes 0.00 0.00 1.00 0.00 0.00 0.00 2
Canvas Debugger 1.00 0.50 1.00 0.67 0.71 0.48 4
Canvas: 2D 0.80 0.60 1.00 0.69 0.77 0.58 40
Canvas: WebGL 0.90 0.91 1.00 0.90 0.95 0.90 170
Compatibility 0.00 0.00 1.00 0.00 0.00 0.00 8
Console 0.69 0.79 1.00 0.74 0.88 0.77 168
Crash Reporting 0.42 0.55 1.00 0.48 0.74 0.52 31
DOM 0.00 0.00 1.00 0.00 0.00 0.00 1
DOM: Animation 0.59 0.42 1.00 0.49 0.65 0.40 52
DOM: Bindings (WebIDL) 0.00 0.00 1.00 0.00 0.00 0.00 5
DOM: CSS Object Model 0.78 0.45 1.00 0.57 0.67 0.43 40
DOM: Content Processes 0.20 0.13 1.00 0.16 0.36 0.12 39
DOM: Core & HTML 0.30 0.62 0.92 0.40 0.76 0.56 656
DOM: Device Interfaces 0.81 0.49 1.00 0.61 0.70 0.46 45
DOM: Events 0.62 0.47 1.00 0.53 0.69 0.44 85
DOM: File 0.00 0.00 1.00 0.00 0.00 0.00 11
DOM: Geolocation 1.00 0.75 1.00 0.86 0.87 0.73 4
DOM: IndexedDB 0.73 0.70 1.00 0.71 0.84 0.68 50
DOM: Networking 0.50 0.20 1.00 0.29 0.45 0.18 10
DOM: Push Notifications 0.67 0.50 1.00 0.57 0.71 0.47 8
DOM: Quota Manager 0.50 0.27 1.00 0.35 0.52 0.25 11
DOM: Security 0.78 0.55 1.00 0.65 0.74 0.52 220
DOM: Service Workers 0.84 0.59 1.00 0.69 0.76 0.56 82
DOM: Web Authentication 0.33 0.20 1.00 0.25 0.45 0.18 5
DOM: Web Crypto 1.00 0.33 1.00 0.50 0.58 0.31 3
DOM: Web Payments 0.54 0.28 1.00 0.37 0.53 0.26 25
DOM: Web Storage 0.40 0.36 1.00 0.38 0.60 0.34 11
DOM: Workers 0.62 0.56 1.00 0.59 0.75 0.54 41
Debugger 0.71 0.77 1.00 0.74 0.88 0.75 152
Developer Environment Integration 0.00 0.00 1.00 0.00 0.00 0.00 1
Developer Outreach 0.00 0.00 1.00 0.00 0.00 0.00 6
Developer Tools 0.67 0.36 1.00 0.47 0.60 0.34 11
Disability Access APIs 0.82 0.77 1.00 0.79 0.88 0.75 118
Document Navigation 0.48 0.24 1.00 0.32 0.49 0.22 42
Documentation 0.00 0.00 1.00 0.00 0.00 0.00 1
Downloads API 0.29 0.33 1.00 0.31 0.58 0.31 12
Downloads Panel 0.39 0.68 1.00 0.50 0.83 0.66 19
Drag and Drop 0.55 0.38 1.00 0.44 0.61 0.35 16
Editor 0.67 0.53 1.00 0.59 0.72 0.50 78
Enterprise Policies 0.72 0.75 1.00 0.74 0.87 0.73 28
Experiments 1.00 0.33 1.00 0.50 0.58 0.31 3
Extension Compatibility 0.40 0.10 1.00 0.16 0.32 0.09 20
File Handling 0.36 0.21 1.00 0.26 0.46 0.19 24
Find Toolbar 0.67 0.43 1.00 0.53 0.66 0.41 23
Firefox Accounts 0.38 0.29 1.00 0.32 0.53 0.27 21
Flash (Adobe) 0.00 0.00 1.00 0.00 0.00 0.00 2
Form Autofill 0.55 0.72 1.00 0.63 0.85 0.70 29
Form Manager 0.69 0.46 1.00 0.55 0.68 0.44 39
Framework 0.50 0.35 1.00 0.41 0.59 0.33 43
Frontend 0.45 0.23 1.00 0.31 0.48 0.21 78
Gecko Profiler 0.56 0.54 1.00 0.55 0.74 0.52 70
General 0.32 0.57 0.93 0.41 0.73 0.51 689
General: Unsupported Platforms 0.43 0.33 1.00 0.38 0.58 0.31 9
Generated Documentation 0.00 0.00 1.00 0.00 0.00 0.00 1
Graphic Commandline and Toolbar 0.67 0.25 1.00 0.36 0.50 0.23 8
Graphics 0.54 0.37 0.99 0.44 0.61 0.35 245
Graphics: Layers 0.38 0.29 1.00 0.33 0.53 0.26 77
Graphics: Text 0.38 0.28 1.00 0.32 0.52 0.26 58
Graphics: WebRender 0.81 0.71 0.99 0.76 0.84 0.68 425
HTML: Parser 0.29 0.14 1.00 0.19 0.38 0.13 14
Headless 0.92 0.69 1.00 0.79 0.83 0.67 16
IPC 0.43 0.40 1.00 0.41 0.63 0.38 57
IPC: MSCOM 0.25 0.20 1.00 0.22 0.45 0.18 5
ImageLib 0.51 0.52 1.00 0.52 0.72 0.50 44
Inspector 0.69 0.70 1.00 0.70 0.83 0.68 163
Inspector: Animations 0.69 0.73 1.00 0.71 0.86 0.71 15
Inspector: Changes 0.29 0.50 1.00 0.36 0.71 0.47 4
Inspector: Computed 0.00 0.00 1.00 0.00 0.00 0.00 2
Inspector: Fonts 0.73 0.62 1.00 0.67 0.78 0.59 13
Inspector: Layout 0.00 0.00 1.00 0.00 0.00 0.00 1
Inspector: Rules 0.71 0.45 1.00 0.56 0.67 0.43 44
Installer 0.67 0.64 1.00 0.65 0.80 0.62 28
Internationalization 0.76 0.61 1.00 0.67 0.78 0.58 56
JSON Viewer 0.61 0.78 1.00 0.68 0.88 0.76 18
JavaScript Engine 0.51 0.60 0.98 0.55 0.77 0.56 392
JavaScript Engine: JIT 0.75 0.48 1.00 0.59 0.70 0.46 159
JavaScript: GC 0.65 0.48 1.00 0.55 0.69 0.45 119
JavaScript: Internationalization API 0.80 0.35 1.00 0.48 0.59 0.33 23
JavaScript: Standard Library 0.00 0.00 1.00 0.00 0.00 0.00 10
Javascript: WebAssembly 0.46 0.43 1.00 0.45 0.66 0.41 44
Keyboard Navigation 0.00 0.00 1.00 0.00 0.00 0.00 7
Keyboards and IME 0.57 0.50 1.00 0.53 0.71 0.47 8
Layout 0.50 0.56 0.98 0.53 0.74 0.52 405
Layout: Block and Inline 0.58 0.32 1.00 0.42 0.57 0.30 34
Layout: Columns 0.50 0.57 1.00 0.53 0.76 0.55 7
Layout: Flexbox 0.00 0.00 1.00 0.00 0.00 0.00 7
Layout: Floats 0.50 0.62 1.00 0.56 0.79 0.60 8
Layout: Form Controls 0.61 0.55 1.00 0.58 0.74 0.52 102
Layout: Generated Content, Lists, and Counters 0.00 0.00 1.00 0.00 0.00 0.00 0
Layout: Grid 0.00 0.00 1.00 0.00 0.00 0.00 4
Layout: Images, Video, and HTML Frames 0.14 0.07 1.00 0.10 0.27 0.06 14
Layout: Positioned 0.62 0.71 1.00 0.67 0.85 0.69 7
Layout: Ruby 0.00 0.00 1.00 0.00 0.00 0.00 2
Layout: Scrolling and Overflow 0.42 0.28 1.00 0.33 0.53 0.26 18
Layout: Tables 0.38 0.36 1.00 0.37 0.60 0.34 22
Layout: Text and Fonts 0.31 0.25 1.00 0.28 0.50 0.23 76
Lint and Formatting 0.59 0.52 1.00 0.55 0.72 0.50 46
MFBT 0.33 0.04 1.00 0.07 0.19 0.03 27
Mach Core 0.38 0.27 1.00 0.32 0.52 0.25 11
MathML 1.00 0.76 1.00 0.87 0.87 0.75 17
Measuring Tool 0.00 0.00 1.00 0.00 0.00 0.00 2
Memory 0.60 0.43 1.00 0.50 0.65 0.40 7
Memory Allocator 0.83 0.26 1.00 0.40 0.51 0.24 19
Menus 0.34 0.40 1.00 0.37 0.63 0.37 55
Messaging System 0.89 0.48 1.00 0.62 0.69 0.45 50
Metrics 0.50 0.57 1.00 0.53 0.76 0.55 7
Migration 0.71 0.50 1.00 0.59 0.71 0.47 24
Netmonitor 0.76 0.74 1.00 0.75 0.86 0.72 101
Networking 0.47 0.30 1.00 0.37 0.54 0.28 128
Networking: Cache 0.58 0.35 1.00 0.44 0.60 0.33 31
Networking: Cookies 0.40 0.56 1.00 0.47 0.74 0.53 18
Networking: DNS 0.63 0.67 1.00 0.65 0.82 0.64 18
Networking: Domain Lists 0.00 0.00 1.00 0.00 0.00 0.00 0
Networking: FTP 1.00 0.50 1.00 0.67 0.71 0.48 2
Networking: File 0.00 0.00 1.00 0.00 0.00 0.00 3
Networking: HTTP 0.52 0.32 1.00 0.40 0.56 0.30 72
Networking: JAR 0.00 0.00 1.00 0.00 0.00 0.00 3
Networking: WebSockets 0.60 0.55 1.00 0.57 0.74 0.52 11
New Tab Page 0.70 0.59 0.99 0.64 0.77 0.57 246
Normandy Client 0.83 0.92 1.00 0.87 0.96 0.92 26
Notifications and Alerts 0.10 0.07 1.00 0.08 0.27 0.06 14
Object Inspector 0.00 0.00 1.00 0.00 0.00 0.00 1
OpenH264 0.00 0.00 1.00 0.00 0.00 0.00 0
Other 0.80 0.42 1.00 0.55 0.65 0.40 19
PDF Viewer 0.74 0.78 1.00 0.76 0.88 0.76 40
Page Info Window 0.62 0.62 1.00 0.62 0.79 0.60 8
Panning and Zooming 0.49 0.52 1.00 0.51 0.72 0.49 73
Password Manager 0.65 0.75 1.00 0.70 0.86 0.73 63
Password Manager: Site Compatibility 0.00 0.00 1.00 0.00 0.00 0.00 3
Performance 0.67 0.29 1.00 0.40 0.53 0.27 7
Performance Monitoring 0.91 0.50 1.00 0.65 0.71 0.47 20
Performance Tools (Profiler/Timeline) 0.78 0.58 1.00 0.67 0.76 0.56 24
Places 0.58 0.47 1.00 0.52 0.68 0.44 62
Plug-ins 0.52 0.67 1.00 0.59 0.82 0.64 48
Pocket 0.73 0.89 1.00 0.80 0.94 0.88 18
Preferences 0.58 0.68 0.99 0.63 0.82 0.65 150
Preferences: Backend 0.78 0.37 1.00 0.50 0.61 0.35 19
Print Preview 0.43 0.50 1.00 0.46 0.71 0.47 6
Printing: Output 0.33 0.41 1.00 0.37 0.64 0.39 17
Printing: Setup 0.33 0.25 1.00 0.29 0.50 0.23 4
Privacy: Anti-Tracking 0.33 0.23 1.00 0.27 0.48 0.21 13
Private Browsing 0.45 0.70 1.00 0.55 0.84 0.68 20
Reader Mode 0.92 0.85 1.00 0.88 0.92 0.83 26
Remote Settings Client 0.60 0.19 1.00 0.29 0.43 0.17 16
Request Handling 0.75 0.47 1.00 0.58 0.69 0.45 38
Responsive Design Mode 0.76 0.79 1.00 0.77 0.89 0.78 39
SVG 0.57 0.75 1.00 0.64 0.86 0.73 103
Safe Browsing 0.78 0.63 1.00 0.70 0.79 0.61 46
Scratchpad 0.00 0.00 1.00 0.00 0.00 0.00 1
Screenshots 0.59 0.67 1.00 0.62 0.82 0.64 15
Search 0.45 0.56 1.00 0.50 0.74 0.53 54
Security 0.55 0.32 1.00 0.41 0.57 0.30 74
Security: PSM 0.72 0.57 1.00 0.64 0.76 0.55 108
Security: Process Sandboxing 0.76 0.55 1.00 0.64 0.74 0.52 62
Selection 0.26 0.31 1.00 0.29 0.56 0.29 16
Session Restore 0.61 0.60 1.00 0.60 0.77 0.57 57
Settings and Preferences 0.67 0.25 1.00 0.36 0.50 0.23 8
Shared Components 1.00 0.15 1.00 0.26 0.39 0.14 20
Shell Integration 0.29 0.29 1.00 0.29 0.53 0.27 7
Site Identity and Permission Panels 0.71 0.50 1.00 0.59 0.71 0.47 44
Source Code Analysis 0.44 0.42 1.00 0.43 0.65 0.39 36
Source Editor 1.00 0.67 1.00 0.80 0.82 0.64 6
Spelling checker 0.80 0.71 1.00 0.75 0.84 0.68 17
Startup and Profile System 0.35 0.32 1.00 0.33 0.57 0.30 25
Storage 0.28 0.29 1.00 0.29 0.54 0.27 17
Storage Inspector 0.80 0.57 1.00 0.67 0.76 0.55 14
String 0.60 0.25 1.00 0.35 0.50 0.23 12
Style Editor 0.67 0.67 1.00 0.67 0.82 0.64 6
Sync 0.59 0.70 1.00 0.64 0.84 0.68 108
Tabbed Browser 0.41 0.54 0.99 0.47 0.73 0.51 133
Task Configuration 0.55 0.45 1.00 0.50 0.67 0.43 84
Telemetry 0.61 0.68 0.99 0.64 0.82 0.66 152
Testing 0.56 0.63 1.00 0.59 0.79 0.60 70
Theme 0.46 0.45 0.99 0.45 0.67 0.42 137
Theme and Visual Design 0.36 0.24 1.00 0.29 0.49 0.22 21
Themes 0.52 0.31 1.00 0.39 0.56 0.29 45
Toolbar 0.33 0.20 1.00 0.25 0.45 0.18 10
Toolbars and Customization 0.69 0.51 1.00 0.58 0.71 0.48 99
Toolchains 0.44 0.21 1.00 0.29 0.46 0.19 19
Tours 0.67 0.75 1.00 0.71 0.87 0.73 8
Tracking Protection 0.67 0.55 1.00 0.61 0.74 0.52 60
Try 0.50 0.33 1.00 0.40 0.58 0.31 9
Untriaged 0.39 0.18 1.00 0.25 0.42 0.16 78
User events and focus handling 0.40 0.14 1.00 0.21 0.38 0.13 56
Video/Audio Controls 1.00 0.42 1.00 0.59 0.65 0.40 19
Web Apps 0.93 0.67 1.00 0.78 0.82 0.64 21
Web Audio 0.67 0.44 1.00 0.53 0.66 0.41 32
Web Audio Editor 1.00 0.75 1.00 0.86 0.87 0.73 4
Web Painting 0.60 0.37 1.00 0.46 0.61 0.35 84
Web Replay 0.94 0.76 1.00 0.84 0.87 0.74 38
WebGL Shader Editor 0.00 0.00 1.00 0.00 0.00 0.00 3
WebIDE 0.80 0.67 1.00 0.73 0.82 0.64 6
WebPayments UI 0.81 0.82 1.00 0.82 0.91 0.81 51
WebRTC 0.69 0.65 1.00 0.67 0.80 0.62 161
WebRTC: Audio/Video 0.53 0.31 1.00 0.39 0.56 0.29 86
WebRTC: Networking 0.42 0.32 1.00 0.36 0.57 0.30 25
WebRTC: Signaling 0.43 0.18 1.00 0.26 0.43 0.17 33
WebVR 0.90 0.73 1.00 0.81 0.85 0.71 52
Widget 0.00 0.00 1.00 0.00 0.00 0.00 16
Widget: Cocoa 0.44 0.38 1.00 0.41 0.62 0.36 58
Widget: Gtk 0.42 0.54 0.99 0.47 0.73 0.51 84
Widget: Win32 0.31 0.31 1.00 0.31 0.56 0.29 35
XML 0.25 0.20 1.00 0.22 0.45 0.18 5
XPCOM 0.42 0.32 1.00 0.36 0.57 0.30 127
XPConnect 0.71 0.40 1.00 0.52 0.64 0.38 42
XUL 0.30 0.22 1.00 0.26 0.47 0.20 27
XUL Widgets 0.62 0.41 1.00 0.49 0.64 0.38 32
about:debugging 0.86 0.76 1.00 0.81 0.87 0.74 63
mozglue 0.71 0.42 1.00 0.53 0.65 0.39 12
avg / total 0.58 0.55 0.99 0.54 0.72 0.52 12463
@marco-c I have a trained model saved in case you need to see full results.
These lines should not be there, as these components should be ignored:
Untriaged 0.39 0.18 1.00 0.25 0.42 0.16 78
General 0.32 0.57 0.93 0.41 0.73 0.51 689
The results seem worse than the current model (can you post the results of the current model?), so perhaps this is not a good idea. The classifier chain might work better in this case.
@marco-c we cannot use classifier chain here , as it is used to combine binary classifiers which is not our case i think as written here https://scikit-learn.org/stable/modules/multiclass.html#classifier-chain
@marco-c The results of the current model for no threshold case..
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
'precision', 'predicted', average, warn_for)
pre rec spe f1 geo iba sup
Core::Audio/Video 0.00 0.00 1.00 0.00 0.00 0.00 22
Core::Audio/Video: GMP 1.00 0.19 1.00 0.32 0.43 0.17 16
Core::Audio/Video: MediaStreamGraph 0.43 0.20 1.00 0.27 0.45 0.18 15
Core::Audio/Video: Playback 0.61 0.69 0.99 0.65 0.83 0.66 339
Core::Audio/Video: Recording 0.70 0.39 1.00 0.50 0.62 0.37 18
Core::Audio/Video: cubeb 0.71 0.52 1.00 0.60 0.72 0.50 42
Core::CSS Parsing and Computation 0.72 0.69 0.99 0.70 0.83 0.66 557
Core::CSS Transitions and Animations 1.00 0.25 1.00 0.40 0.50 0.23 4
Core::Canvas: 2D 0.61 0.67 1.00 0.64 0.82 0.64 33
Core::Canvas: WebGL 0.86 0.81 1.00 0.84 0.90 0.79 180
Core::DOM 0.25 0.70 0.87 0.37 0.78 0.60 711
Core::DOM: Animation 0.59 0.52 1.00 0.55 0.72 0.49 52
Core::DOM: Bindings (WebIDL) 0.00 0.00 1.00 0.00 0.00 0.00 12
Core::DOM: CSS Object Model 0.88 0.35 1.00 0.50 0.59 0.33 40
Core::DOM: Content Processes 0.31 0.17 1.00 0.22 0.42 0.16 52
Core::DOM: Device Interfaces 0.63 0.49 1.00 0.55 0.70 0.46 39
Core::DOM: Events 0.57 0.57 1.00 0.57 0.75 0.54 81
Core::DOM: File 0.00 0.00 1.00 0.00 0.00 0.00 11
Core::DOM: IndexedDB 0.82 0.74 1.00 0.78 0.86 0.72 43
Core::DOM: Networking 0.60 0.25 1.00 0.35 0.50 0.23 12
Core::DOM: Push Notifications 0.50 0.33 1.00 0.40 0.58 0.31 6
Core::DOM: Quota Manager 0.67 0.75 1.00 0.71 0.87 0.73 8
Core::DOM: Security 0.73 0.60 1.00 0.66 0.77 0.57 188
Core::DOM: Service Workers 0.95 0.70 1.00 0.81 0.84 0.68 80
Core::DOM: Web Authentication 0.33 0.08 1.00 0.13 0.29 0.08 12
Core::DOM: Web Payments 0.89 0.59 1.00 0.71 0.77 0.57 27
Core::DOM: Web Storage 0.50 0.27 1.00 0.35 0.52 0.25 11
Core::DOM: Workers 0.76 0.61 1.00 0.68 0.78 0.59 41
Core::Disability Access APIs 0.71 0.65 1.00 0.68 0.81 0.63 104
Core::Document Navigation 0.42 0.23 1.00 0.30 0.48 0.21 43
Core::Drag and Drop 0.17 0.22 1.00 0.19 0.47 0.20 9
Core::Editor 0.75 0.63 1.00 0.68 0.79 0.61 84
Core::Gecko Profiler 0.66 0.54 1.00 0.59 0.73 0.51 80
Core::Graphics 0.44 0.34 0.99 0.38 0.58 0.32 210
Core::Graphics: Layers 0.39 0.23 1.00 0.29 0.48 0.21 78
Core::Graphics: Text 0.61 0.34 1.00 0.43 0.58 0.32 68
Core::Graphics: WebRender 0.84 0.70 1.00 0.77 0.84 0.68 435
Core::HTML: Parser 0.64 0.44 1.00 0.52 0.66 0.41 16
Core::IPC 0.41 0.42 1.00 0.41 0.65 0.39 67
Core::ImageLib 0.44 0.47 1.00 0.45 0.68 0.44 43
Core::Internationalization 0.62 0.42 1.00 0.50 0.65 0.40 57
Core::JavaScript 0.56 0.59 0.98 0.58 0.76 0.56 396
Core::JavaScript Engine: JIT 0.63 0.53 1.00 0.58 0.73 0.51 144
Core::JavaScript: GC 0.53 0.52 1.00 0.53 0.72 0.50 82
Core::JavaScript: Internationalization API 0.46 0.35 1.00 0.40 0.59 0.33 17
Core::JavaScript: Standard Library 0.67 0.15 1.00 0.25 0.39 0.14 13
Core::Javascript: WebAssembly 0.53 0.63 1.00 0.58 0.80 0.61 41
Core::Layout 0.56 0.53 0.98 0.55 0.72 0.50 445
Core::Layout: Block and Inline 0.80 0.32 1.00 0.45 0.56 0.29 38
Core::Layout: Columns 0.67 0.57 1.00 0.62 0.76 0.55 7
Core::Layout: Floats 0.40 0.67 1.00 0.50 0.82 0.64 3
Core::Layout: Form Controls 0.58 0.61 1.00 0.59 0.78 0.58 76
Core::Layout: Images, Video, and HTML Frames 0.27 0.16 1.00 0.20 0.40 0.14 19
Core::Layout: Positioned 0.75 0.33 1.00 0.46 0.58 0.31 18
Core::Layout: Scrolling and Overflow 0.38 0.50 1.00 0.43 0.71 0.47 6
Core::Layout: Tables 0.62 0.55 1.00 0.58 0.74 0.53 29
Core::Layout: Text and Fonts 0.39 0.46 1.00 0.42 0.68 0.43 85
Core::MFBT 0.67 0.31 1.00 0.42 0.55 0.29 26
Core::MathML 0.88 0.75 1.00 0.81 0.87 0.73 20
Core::Memory Allocator 0.33 0.17 1.00 0.22 0.41 0.15 18
Core::Networking 0.59 0.36 1.00 0.45 0.60 0.34 141
Core::Networking: Cache 0.54 0.37 1.00 0.44 0.61 0.35 19
Core::Networking: Cookies 0.67 0.57 1.00 0.62 0.76 0.55 21
Core::Networking: DNS 0.61 0.55 1.00 0.58 0.74 0.52 20
Core::Networking: HTTP 0.51 0.27 1.00 0.35 0.52 0.25 81
Core::Networking: WebSockets 0.75 0.43 1.00 0.55 0.65 0.40 7
Core::Panning and Zooming 0.58 0.56 1.00 0.57 0.75 0.54 94
Core::Performance 0.33 0.17 1.00 0.22 0.41 0.15 6
Core::Plug-ins 0.72 0.57 1.00 0.63 0.75 0.54 58
Core::Preferences: Backend 0.64 0.38 1.00 0.47 0.61 0.35 24
Core::Print 0.48 0.46 1.00 0.47 0.68 0.44 28
Core::Print Preview 0.67 0.43 1.00 0.52 0.65 0.40 14
Core::Printing: Setup 0.67 0.57 1.00 0.62 0.76 0.55 7
Core::Privacy: Anti-Tracking 0.00 0.00 1.00 0.00 0.00 0.00 8
Core::SVG 0.62 0.71 1.00 0.66 0.84 0.69 107
Core::Security 0.56 0.17 1.00 0.26 0.41 0.15 30
Core::Security: PSM 0.61 0.64 1.00 0.62 0.80 0.62 78
Core::Security: Process Sandboxing 0.61 0.58 1.00 0.59 0.76 0.55 59
Core::Selection 0.28 0.29 1.00 0.29 0.54 0.27 17
Core::Spelling checker 0.93 0.54 1.00 0.68 0.74 0.52 24
Core::String 0.25 0.06 1.00 0.10 0.24 0.05 17
Core::User events and focus handling 0.21 0.15 1.00 0.17 0.38 0.13 34
Core::Web Audio 0.88 0.61 1.00 0.72 0.78 0.59 49
Core::Web Painting 0.48 0.26 1.00 0.34 0.51 0.24 76
Core::Web Replay 0.77 0.62 1.00 0.69 0.79 0.60 32
Core::WebRTC 0.67 0.68 1.00 0.67 0.82 0.65 143
Core::WebRTC: Audio/Video 0.56 0.27 1.00 0.36 0.52 0.25 82
Core::WebRTC: Networking 0.60 0.17 1.00 0.26 0.41 0.15 36
Core::WebRTC: Signaling 0.52 0.28 1.00 0.36 0.52 0.25 40
Core::WebVR 0.88 0.73 1.00 0.80 0.86 0.71 49
Core::Widget 0.25 0.06 1.00 0.10 0.25 0.06 16
Core::Widget: Cocoa 0.38 0.31 1.00 0.34 0.56 0.29 54
Core::Widget: Gtk 0.51 0.48 1.00 0.50 0.69 0.45 109
Core::Widget: Win32 0.54 0.33 1.00 0.41 0.58 0.31 57
Core::XPCOM 0.38 0.24 1.00 0.30 0.49 0.22 123
Core::XPConnect 0.69 0.23 1.00 0.35 0.48 0.22 47
Core::XUL 0.17 0.08 1.00 0.11 0.28 0.07 26
Core::mozglue 0.25 0.11 1.00 0.15 0.33 0.10 9
DevTools 0.59 0.52 0.99 0.55 0.72 0.49 193
DevTools::Accessibility Tools 0.44 0.40 1.00 0.42 0.63 0.38 20
DevTools::Application Panel 1.00 0.50 1.00 0.67 0.71 0.48 10
DevTools::Console 0.67 0.74 0.99 0.70 0.86 0.72 176
DevTools::Debugger 0.73 0.78 1.00 0.75 0.88 0.76 169
DevTools::Framework 0.57 0.33 1.00 0.42 0.58 0.31 39
DevTools::Inspector 0.70 0.61 1.00 0.66 0.78 0.59 161
DevTools::Inspector: Animations 0.83 0.79 1.00 0.81 0.89 0.77 24
DevTools::Inspector: Fonts 0.89 0.89 1.00 0.89 0.94 0.88 18
DevTools::Inspector: Rules 0.58 0.42 1.00 0.49 0.65 0.39 43
DevTools::JSON Viewer 0.65 0.79 1.00 0.71 0.89 0.77 19
DevTools::Netmonitor 0.72 0.74 1.00 0.73 0.86 0.72 94
DevTools::Performance Tools (Profiler/Timeline) 0.50 0.36 1.00 0.42 0.60 0.33 14
DevTools::Responsive Design Mode 0.88 0.76 1.00 0.82 0.87 0.74 38
DevTools::Shared Components 0.40 0.13 1.00 0.20 0.37 0.12 15
DevTools::Storage Inspector 1.00 0.71 1.00 0.83 0.84 0.69 17
DevTools::Style Editor 0.80 0.67 1.00 0.73 0.82 0.64 6
DevTools::about:debugging 0.81 0.81 1.00 0.81 0.90 0.80 64
External Software Affecting Firefox 0.62 0.59 1.00 0.60 0.77 0.57 27
Firefox Build System 0.58 0.64 0.99 0.61 0.80 0.61 348
Firefox Build System::Android Studio and Gradle Integration 0.60 0.60 1.00 0.60 0.77 0.58 30
Firefox Build System::Bootstrap Configuration 0.52 0.58 1.00 0.55 0.76 0.55 19
Firefox Build System::General: Unsupported Platforms 0.62 0.28 1.00 0.38 0.53 0.26 18
Firefox Build System::Lint and Formatting 0.73 0.68 1.00 0.70 0.82 0.66 47
Firefox Build System::Mach Core 0.33 0.08 1.00 0.13 0.29 0.08 12
Firefox Build System::Source Code Analysis 0.58 0.51 1.00 0.54 0.72 0.49 37
Firefox Build System::Task Configuration 0.67 0.58 1.00 0.62 0.76 0.55 95
Firefox Build System::Toolchains 0.00 0.00 1.00 0.00 0.00 0.00 15
Firefox Build System::Try 0.55 0.60 1.00 0.57 0.77 0.58 10
Firefox for Android::Activity Stream 0.71 0.31 1.00 0.43 0.56 0.29 16
Firefox for Android::Add-on Manager 0.67 0.25 1.00 0.36 0.50 0.23 8
Firefox for Android::Android Sync 0.71 0.38 1.00 0.50 0.62 0.36 13
Firefox for Android::Audio/Video 0.50 0.27 1.00 0.35 0.52 0.25 26
Firefox for Android::Awesomescreen 0.60 0.20 1.00 0.30 0.45 0.18 15
Firefox for Android::Firefox Accounts 0.50 0.17 1.00 0.25 0.41 0.15 6
Firefox for Android::Keyboards and IME 0.67 0.25 1.00 0.36 0.50 0.23 8
Firefox for Android::Metrics 0.50 0.40 1.00 0.44 0.63 0.38 10
Firefox for Android::Settings and Preferences 0.00 0.00 1.00 0.00 0.00 0.00 12
Firefox for Android::Testing 0.58 0.71 1.00 0.64 0.84 0.69 90
Firefox for Android::Theme and Visual Design 0.56 0.20 1.00 0.29 0.45 0.18 25
Firefox for Android::Toolbar 0.50 0.22 1.00 0.31 0.47 0.20 9
Firefox for Android::Web Apps 0.65 0.61 1.00 0.63 0.78 0.59 18
Firefox::Address Bar 0.62 0.63 0.99 0.63 0.79 0.61 160
Firefox::Bookmarks & History 0.62 0.68 1.00 0.65 0.83 0.66 101
Firefox::Downloads Panel 0.37 0.43 1.00 0.40 0.66 0.41 23
Firefox::Enterprise Policies 0.81 0.83 1.00 0.82 0.91 0.82 36
Firefox::Extension Compatibility 0.20 0.08 1.00 0.11 0.28 0.07 13
Firefox::File Handling 0.48 0.30 1.00 0.37 0.55 0.28 33
Firefox::Firefox Accounts 0.50 0.33 1.00 0.40 0.58 0.31 9
Firefox::Headless 1.00 0.73 1.00 0.84 0.85 0.71 11
Firefox::Installer 0.74 0.62 1.00 0.68 0.79 0.60 32
Firefox::Keyboard Navigation 0.00 0.00 1.00 0.00 0.00 0.00 9
Firefox::Menus 0.27 0.32 1.00 0.29 0.56 0.30 44
Firefox::Messaging System 0.71 0.49 1.00 0.58 0.70 0.46 41
Firefox::Migration 0.50 0.60 1.00 0.55 0.77 0.58 10
Firefox::New Tab Page 0.72 0.58 1.00 0.64 0.76 0.55 229
Firefox::Normandy Client 0.75 0.80 1.00 0.77 0.89 0.78 15
Firefox::PDF Viewer 0.76 0.90 1.00 0.82 0.95 0.89 49
Firefox::Page Info Window 0.83 0.50 1.00 0.62 0.71 0.47 10
Firefox::Pocket 0.57 0.80 1.00 0.67 0.89 0.78 15
Firefox::Preferences 0.63 0.69 1.00 0.66 0.83 0.67 143
Firefox::Private Browsing 0.52 0.67 1.00 0.59 0.82 0.64 18
Firefox::Remote Settings Client 0.80 0.67 1.00 0.73 0.82 0.64 6
Firefox::Screenshots 0.50 0.75 1.00 0.60 0.87 0.73 16
Firefox::Search 0.55 0.53 1.00 0.54 0.73 0.50 72
Firefox::Security 0.62 0.24 1.00 0.34 0.49 0.22 42
Firefox::Session Restore 0.62 0.63 1.00 0.63 0.80 0.61 52
Firefox::Shell Integration 0.57 0.40 1.00 0.47 0.63 0.38 10
Firefox::Site Identity and Permission Panels 0.66 0.43 1.00 0.52 0.66 0.41 44
Firefox::Sync 0.58 0.74 1.00 0.65 0.86 0.72 92
Firefox::Tabbed Browser 0.41 0.63 0.99 0.50 0.79 0.60 123
Firefox::Theme 0.46 0.46 0.99 0.46 0.68 0.44 155
Firefox::Toolbars and Customization 0.46 0.48 1.00 0.47 0.69 0.45 86
Firefox::Tours 0.56 0.62 1.00 0.59 0.79 0.60 8
Firefox::Tracking Protection 0.62 0.67 1.00 0.64 0.82 0.65 55
Firefox::WebPayments UI 0.83 0.80 1.00 0.81 0.89 0.78 44
Toolkit::Add-ons Manager 0.62 0.64 1.00 0.63 0.80 0.62 128
Toolkit::Application Update 0.58 0.66 1.00 0.62 0.81 0.63 70
Toolkit::Blocklist Policy Requests 0.91 0.95 1.00 0.93 0.98 0.95 43
Toolkit::Crash Reporting 0.52 0.53 1.00 0.52 0.73 0.51 32
Toolkit::Downloads API 0.40 0.32 1.00 0.35 0.56 0.29 19
Toolkit::Find Toolbar 0.89 0.42 1.00 0.57 0.65 0.40 19
Toolkit::Form Autofill 0.82 0.60 1.00 0.69 0.77 0.58 30
Toolkit::Form Manager 0.62 0.64 1.00 0.63 0.80 0.62 28
Toolkit::Notifications and Alerts 0.40 0.36 1.00 0.38 0.60 0.34 11
Toolkit::Password Manager 0.77 0.79 1.00 0.78 0.89 0.77 62
Toolkit::Performance Monitoring 0.88 0.50 1.00 0.64 0.71 0.47 14
Toolkit::Places 0.48 0.36 1.00 0.41 0.60 0.33 56
Toolkit::Preferences 0.75 0.46 1.00 0.57 0.68 0.44 13
Toolkit::Reader Mode 0.75 0.75 1.00 0.75 0.87 0.73 20
Toolkit::Safe Browsing 0.88 0.64 1.00 0.74 0.80 0.62 45
Toolkit::Startup and Profile System 0.56 0.41 1.00 0.47 0.64 0.38 37
Toolkit::Storage 0.43 0.25 1.00 0.32 0.50 0.23 12
Toolkit::Telemetry 0.54 0.68 0.99 0.61 0.82 0.66 135
Toolkit::Themes 0.50 0.17 1.00 0.25 0.41 0.15 30
Toolkit::Video/Audio Controls 0.94 0.60 1.00 0.73 0.77 0.58 25
Toolkit::XUL Widgets 0.34 0.32 1.00 0.33 0.57 0.30 31
WebExtensions 0.54 0.61 0.99 0.57 0.78 0.58 299
WebExtensions::Android 0.50 0.40 1.00 0.44 0.63 0.38 15
WebExtensions::Compatibility 0.00 0.00 1.00 0.00 0.00 0.00 8
WebExtensions::Developer Tools 1.00 0.18 1.00 0.31 0.43 0.17 11
WebExtensions::Frontend 0.30 0.11 1.00 0.16 0.33 0.10 65
WebExtensions::Request Handling 0.78 0.44 1.00 0.56 0.66 0.41 41
WebExtensions::Storage 0.50 0.11 1.00 0.18 0.33 0.10 9
WebExtensions::Themes 0.67 0.50 1.00 0.57 0.71 0.47 12
avg / total 0.59 0.56 0.99 0.56 0.73 0.53 12431
These lines should not be there, as these components should be ignored:
Untriaged 0.39 0.18 1.00 0.25 0.42 0.16 78General 0.32 0.57 0.93 0.41 0.73 0.51 689
I think these aren't ignored because these inverse mapping contains both General and Untriaged. https://github.com/mozilla/bugbug/blob/8b59321c5f694788606d7a07008238057292a38f/bugbug/models/component.py#L51-L54