signoz icon indicating copy to clipboard operation
signoz copied to clipboard

Published 20 hours ago • verified publisher icon SigNoz

Introduce exportman

Open webpro opened this issue 2 years ago • 8 comments

Hi! Since your repo came up high in the GitHub Explore area when I was looking for a good TypeScript workload for exportman, I could just as well open a pull request with the results. It's a tool to help with repo maintenance by reporting unused files and exports. Here's what it reports in the /frontend folder with the configuration added in this pull request:

❯ exportman --config exportman.json --onlyFiles
--- UNUSED FILES (26)
src/container/EditRules/styles.ts
src/container/FormAlertChannels/Settings/LabelFilter.tsx
src/container/ListOfDashboard/dashboardSearchAndFilter.ts
src/container/NewDashboard/DashboardSettings/General/Description/index.tsx
src/container/NewDashboard/DashboardSettings/General/Description/styles.ts
src/container/NewDashboard/DescriptionOfDashboard/NameOfTheDashboard/index.tsx
src/container/Trace/Filters/Panel/PanelBody/styles.ts
src/container/Trace/Filters/styles.ts
src/hooks/useMountedState.ts
src/lib/JSXtoHTML.ts
src/lib/convertIntoHr.ts
src/lib/getGlobalDropDownFormatedDate.ts
src/lib/getSettingsPeroid.ts
src/pages/CreateAlert/styles.ts
src/store/actions/logs/addToSelectedField.ts
src/store/actions/trace/selectTraceFilter.ts
src/store/actions/trace/updateTagsSelected.ts
src/types/api/alerts/queryType.ts
src/types/api/dashboard/shared.ts
src/types/api/logs/operator.ts
src/types/api/logs/removeSelectedField.ts
src/types/api/user/setUserPreference.ts
src/typings/environment.ts
src/typings/window.ts
src/utils/fixtures/TraceData.ts
src/utils/token.ts

The results seem pretty accurate, but I'm not 100% sure about each file separately. For instance, I see some styles.ts files that are not imported anywhere, but they may be automatically included by a framework or build? If so, we can ignore such files and add them to the filePatterns with !**/styles.ts, and so on.

Running the tool without the --onlyFiles flag will also report unused exports, unused types, and duplicate exports (didn't add them here for brevity).

I believe this tool can help significantly when maintaining (and refactoring) repositories like this.

Also see https://github.com/webpro/exportman if you're interested. Have a great day and best of luck with your project.

webpro avatar Oct 05 '22 06:10 webpro

Welcome to the SigNoz community! Thank you for your first pull request and making this project better. 🤗

welcome[bot] avatar Oct 05 '22 06:10 welcome[bot]

Thanks @webpro for raising will check this one :-D

palashgdev avatar Oct 06 '22 06:10 palashgdev

There have been a few updates, so I updated the PR. One of the updates is an exit code for unused files, so exportman can become part of CI.

❯ yarn exportman --only files
yarn run v1.22.19
$ /Users/lars/p/signoz/frontend/node_modules/.bin/exportman --only files
src/container/EditRules/styles.ts
src/container/FormAlertChannels/Settings/LabelFilter.tsx
src/container/ListOfDashboard/dashboardSearchAndFilter.ts
src/container/NewDashboard/DashboardSettings/General/Description/index.tsx
src/container/NewDashboard/DashboardSettings/General/Description/styles.ts
src/container/NewDashboard/DescriptionOfDashboard/NameOfTheDashboard/index.tsx
src/container/Trace/Filters/Panel/PanelBody/styles.ts
src/container/Trace/Filters/styles.ts
src/hooks/useMountedState.ts
src/lib/JSXtoHTML.ts
src/lib/convertIntoHr.ts
src/lib/getGlobalDropDownFormatedDate.ts
src/lib/getSettingsPeroid.ts
src/pages/CreateAlert/styles.ts
src/store/actions/logs/addToSelectedField.ts
src/store/actions/trace/selectTraceFilter.ts
src/store/actions/trace/updateTagsSelected.ts
src/types/api/alerts/queryType.ts
src/types/api/dashboard/shared.ts
src/types/api/logs/operator.ts
src/types/api/logs/removeSelectedField.ts
src/types/api/user/setUserPreference.ts
src/typings/environment.ts
src/typings/window.ts
src/utils/fixtures/TraceData.ts
src/utils/token.ts
error Command failed with exit code 26.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I've also updated the docs considerably, feel free to read @ https://github.com/webpro/exportman

Happy to hear any feedback or questions you may have!

webpro avatar Oct 07 '22 16:10 webpro

Hello @webpro Thank you for sharing the PR. Everything seems good to me, but I am concerned about the use of the typing "*.d.ts" even though it is listed in this one.

palashgdev avatar Oct 09 '22 14:10 palashgdev

Hello @webpro Thank you for sharing the PR. Everything seems good to me, but I am concerned about the use of the typing "*.d.ts" even though it is listed in this one.

From a "production files" perspective, the *.d.ts files are not used. Basically much like the test files (*.test.{ts,tsx}). Is this wat you mean?

I think detecting unused DTS files would require a new category of issues in exportman, this is not supported (yet).

webpro avatar Oct 09 '22 15:10 webpro

FYI, I'm working on some new features. Might be interesting for this project as well. It seems to contain some unused dependencies, such as the following:

--- UNUSED DEPENDENCIES (97)
...
d3-flame-graph
d3-tip
...
react-graph-vis
react-vis

Am I right? Also a lot of false positives, because it's not trivial to detect stuff being referenced like Babel, ESLint & Webpack plugins, etc. Work in progress.

Another feature I'm working on is unlisted dependencies in package.json. It reports this:

--- UNLISTED DEPENDENCIES (14)
AppRoutes
api
assets
components
container
hooks
lib
modules
pages
rc-picker
reportWebVitals
store
types
utils

Most of them are imports that I think should be relative (e.g. ./AppRoutes), I guess this is maybe some Webpack configuration that allows this? I think it's more common to use either relative paths or TypeScript paths prefixed such as @lib (because of unambiguity, e.g. store could be an external package in node_modules as well). Not trying to make you change it, just an observation from my end.

However, at least one of them is actually missing in the package.json: rc-picker (apparently it's a transitive dependency from another package).

webpro avatar Oct 09 '22 15:10 webpro

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

sonarqubecloud[bot] avatar Oct 10 '22 15:10 sonarqubecloud[bot]

Apologies to bother you again. Over the weekend I had an idea and it worked out pretty well. It even made me change the name of the project to "knip" (Dutch for "cut"), because the old name just didn't... cut it.

The tool now also finds (1) unused dependencies and (2) dependencies not listed in package.json.

The results are currently a bit rough:

❯ yarn knip -c knip.dev.json --include files,dependencies
yarn run v1.22.19
$ /Users/lars/p/signoz/frontend/node_modules/.bin/knip -c knip.dev.json --include files,dependencies
--- UNUSED FILES (25)
src/container/EditRules/styles.ts
src/container/FormAlertChannels/Settings/LabelFilter.tsx
src/container/ListOfDashboard/dashboardSearchAndFilter.ts
...
--- UNUSED DEPENDENCIES (30)
@testing-library/jest-dom
@testing-library/user-event
babel-eslint
babel-jest
babel-loader
babel-plugin-named-asset-import
babel-preset-minify
babel-preset-react-app
cross-env
css-loader
css-minimizer-webpack-plugin
d3-flame-graph
d3-tip
dotenv
file-loader
html-webpack-plugin
jest
less
less-loader
mini-css-extract-plugin
react-graph-vis
react-vis
stream
style-loader
terser-webpack-plugin
ts-node
tsconfig-paths-webpack-plugin
typescript
webpack
webpack-dev-server

You might want to consider and move some of these dependencies to devDependencies (such as Babel and Webpack plugins). Afterwards, it will become more clear what dependencies are actually not used (such as the ones I pointed out in the previous comment).

Cheers!

webpro avatar Oct 10 '22 15:10 webpro

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
5.1% 5.1% Duplication

sonarqubecloud[bot] avatar Nov 13 '22 01:11 sonarqubecloud[bot]

Apologies to bother you again. Over the weekend I had an idea and it worked out pretty well. It even made me change the name of the project to "knip" (Dutch for "cut"), because the old name just didn't... cut it.

The tool now also finds (1) unused dependencies and (2) dependencies not listed in package.json.

The results are currently a bit rough:

❯ yarn knip -c knip.dev.json --include files,dependencies
yarn run v1.22.19
$ /Users/lars/p/signoz/frontend/node_modules/.bin/knip -c knip.dev.json --include files,dependencies
--- UNUSED FILES (25)
src/container/EditRules/styles.ts
src/container/FormAlertChannels/Settings/LabelFilter.tsx
src/container/ListOfDashboard/dashboardSearchAndFilter.ts
...
--- UNUSED DEPENDENCIES (30)
@testing-library/jest-dom
@testing-library/user-event
babel-eslint
babel-jest
babel-loader
babel-plugin-named-asset-import
babel-preset-minify
babel-preset-react-app
cross-env
css-loader
css-minimizer-webpack-plugin
d3-flame-graph
d3-tip
dotenv
file-loader
html-webpack-plugin
jest
less
less-loader
mini-css-extract-plugin
react-graph-vis
react-vis
stream
style-loader
terser-webpack-plugin
ts-node
tsconfig-paths-webpack-plugin
typescript
webpack
webpack-dev-server

You might want to consider and move some of these dependencies to devDependencies (such as Babel and Webpack plugins). Afterwards, it will become more clear what dependencies are actually not used (such as the ones I pointed out in the previous comment).

Cheers!

Hi @webpro thanks for pointing out sure will reflect your suggestions :-D

palashgdev avatar Nov 21 '22 08:11 palashgdev

It was very useful for knip to use this repo for some test drives! Have a great day.

webpro avatar Jan 09 '23 20:01 webpro