iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Published 20 hours ago • verified publisher icon apache

Python: Add license checker to the source distribution

Open Fokko opened this issue 3 years ago • 1 comments

This adds a light version of the check-license script (without the exclusions), and a command in the makefile to easily run it

Fokko avatar Sep 23 '22 10:09 Fokko

Why remove the exclusions? It seems like that's valuable and we'll probably want them in the future.

rdblue avatar Sep 23 '22 16:09 rdblue

@rdblue It would be an empty list for now, so I figured it doesn't make any sense to add it. I can add it, but I think it would be best to avoid exclusions in general.

This would also mean that we would need to maintain two lists; one in the root dev/ and one in the python folder. Also, the source distribution does not contain all the files that are in the repo (for example, the poetry.lock is compiled into a setup.py).

Another option is to copy the script and exclusions from the root at release time. This would include filtering down the exclusion list to only include python/. This is a bit more involved but would remove the duplicates.

Fokko avatar Sep 23 '22 18:09 Fokko 

➜  python git:(fd-license-checker) poetry build
Building pyiceberg (0.1.0.dev0)
  - Building sdist
  - Built pyiceberg-0.1.0.dev0.tar.gz
  - Building wheel
  - Built pyiceberg-0.1.0.dev0-py3-none-any.whl
➜  python git:(fd-license-checker) cd dist 
➜  dist git:(fd-license-checker) tar -xf pyiceberg-0.1.0.dev0.tar.gz
➜  dist git:(fd-license-checker) cd pyiceberg-0.1.0.dev0 
➜  pyiceberg-0.1.0.dev0 git:(fd-license-checker) make check-license
./dev/check-license
Attempting to fetch rat

*****************************************************
Summary
-------
Generated at: 2022-09-23T23:13:12+02:00

Notes: 2
Binaries: 0
Archives: 1
Standards: 103

Apache Licensed: 103
Generated Documents: 0

JavaDocs are generated, thus a license header is optional.
Generated files do not require license headers.

0 Unknown Licenses

Archives:

 + /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/lib/apache-rat-0.12.jar
 
*****************************************************
  Files with Apache License headers will be marked AL
  Binary files (which do not require any license headers) will be marked B
  Compressed archives will be marked A
  Notices, licenses etc. will be marked N
  N     /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/LICENSE
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/Makefile
  N     /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/NOTICE
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/PKG-INFO
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/README.md
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyproject.toml
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/setup.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/dev/check-license
  A     /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/lib/apache-rat-0.12.jar
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/conversions.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/exceptions.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/files.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/manifest.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/schema.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/serializers.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/transforms.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/typedef.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/types.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/decoder.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/file.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/reader.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/resolver.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/codecs/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/codecs/bzip2.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/codecs/codec.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/codecs/deflate.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/codecs/snappy_codec.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/avro/codecs/zstandard_codec.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/catalog/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/catalog/hive.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/catalog/rest.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/cli/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/cli/console.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/cli/output.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/expressions/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/expressions/base.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/expressions/literals.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/io/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/io/fsspec.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/io/memory.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/io/pyarrow.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/table/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/table/metadata.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/table/partitioning.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/table/refs.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/table/snapshots.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/table/sorting.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/utils/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/utils/bin_packing.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/utils/config.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/utils/datetime.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/utils/decimal.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/utils/iceberg_base_model.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/utils/parsing.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/utils/schema_conversion.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/pyiceberg/utils/singleton.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/conftest.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/test_conversions.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/test_schema.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/test_transforms.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/test_typedef.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/test_types.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/test_version.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/avro/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/avro/test_decoder.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/avro/test_file.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/avro/test_reader.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/avro/test_resolver.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/catalog/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/catalog/test_base.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/catalog/test_hive.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/catalog/test_rest.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/cli/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/cli/test_console.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/cli/test_output.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/expressions/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/expressions/test_expressions_base.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/expressions/test_literals.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/io/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/io/test_fsspec.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/io/test_io.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/io/test_pyarrow.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/table/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/table/test_init.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/table/test_metadata.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/table/test_partitioning.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/table/test_refs.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/table/test_snapshots.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/table/test_sorting.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/utils/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/utils/test_bin_packing.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/utils/test_config.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/utils/test_manifest.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/utils/test_schema_conversion.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/tests/utils/test_singleton.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/vendor/fb303/FacebookService.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/vendor/fb303/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/vendor/fb303/constants.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/vendor/fb303/ttypes.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/vendor/hive_metastore/ThriftHiveMetastore.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/vendor/hive_metastore/__init__.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/vendor/hive_metastore/constants.py
  AL    /Users/fokkodriesprong/Desktop/iceberg/python/dist/pyiceberg-0.1.0.dev0/vendor/hive_metastore/ttypes.py
 
*****************************************************

Fokko avatar Sep 23 '22 21:09 Fokko