TileDB
TileDB copied to clipboard
TileDB-Inc
Use CPack in release workflow
This PR changes the default behavior of release workflow to use cpack commands to generate source and binary releases of TileDB library. The produced releases can be seen here:
https://github.com/dudoslav/TileDB/actions/runs/7757551437
How release look: https://github.com/dudoslav/TileDB/releases/tag/t03
TYPE: BUILD DESC: CPack in Release Workflow
This pull request has been linked to Shortcut Story #40359: Use CPack targets in deployment workflow.
![shortcut-integration[bot] avatar](https://avatars.githubusercontent.com/in/96?v=4 "Published by a pub.dev verified publisher"){kind=small}
I believe that backwards_compatibility tests not passing is not an issue of this PR
- please run the workflow here against a prior release (eg 2.20.0-rc1 is fine) and compare the bundles with existing workflow to this one, in order to make sure the directory structure etc. is what we expect.
So I executed the workflow on my fork and produced release 2.21.0 (you can no specify release version, because it just bundles the current repo).
The pipeline looks like this: https://github.com/dudoslav/TileDB/actions/runs/7798144811
The automatically produced release is here: https://github.com/dudoslav/TileDB/releases/tag/t05
I downloaded the zip source export and this is what is inside:
.rw-rw-rw- 3.0k dudoslav 6 Feb 09:32 .clang-format
.rw-rw-rw- 66 dudoslav 6 Feb 09:32 .dockerignore
.rw-rw-rw- 431 dudoslav 6 Feb 09:32 .readthedocs.yml
.rw-rw-rw- 2.0k dudoslav 6 Feb 09:32 _quarto.yml
.rw-rw-rw- 11k dudoslav 6 Feb 09:32 bootstrap
.rw-rw-rw- 10k dudoslav 6 Feb 09:32 bootstrap.ps1
drwxrwxrwx - dudoslav 6 Feb 09:37 cmake
.rw-rw-rw- 25k dudoslav 6 Feb 09:32 CMakeLists.txt
.rw-rw-rw- 2.4k dudoslav 6 Feb 09:32 CODE_OF_CONDUCT.md
.rw-rw-rw- 7.0k dudoslav 6 Feb 09:32 CONTRIBUTING.md
drwxrwxrwx - dudoslav 6 Feb 09:37 doc
drwxrwxrwx - dudoslav 6 Feb 09:37 examples
drwxrwxrwx - dudoslav 6 Feb 09:37 experimental
drwxrwxrwx - dudoslav 6 Feb 09:37 external
drwxrwxrwx - dudoslav 6 Feb 09:37 format_spec
.rw-rw-rw- 231k dudoslav 6 Feb 09:32 HISTORY.md
.rw-rw-rw- 103 dudoslav 6 Feb 09:32 index.md
.rw-rw-rw- 1.2k dudoslav 6 Feb 09:32 LICENSE
drwxrwxrwx - dudoslav 6 Feb 09:37 ports
drwxrwxrwx - dudoslav 6 Feb 09:37 quarto-materials
.rw-rw-rw- 6.3k dudoslav 6 Feb 09:32 README.md
drwxrwxrwx - dudoslav 6 Feb 09:37 scripts
drwxrwxrwx - dudoslav 6 Feb 09:37 test
drwxrwxrwx - dudoslav 6 Feb 09:37 tiledb
drwxrwxrwx - dudoslav 6 Feb 09:37 tools
.rw-rw-rw- 79 dudoslav 6 Feb 09:32 vcpkg-configuration.json
.rw-rw-rw- 2.5k dudoslav 6 Feb 09:32 vcpkg.json
I was also able to compile it. Next I downloaded release build and tried to build (linux-x86_64) Vector-Search against it and it worked as well (build finished for all tests). This is what release folder tree looks like:
(base) taco:tiledb-release dudoslav$ tree .
.
├── include
│ ├── tiledb
│ │ ├── api
│ │ │ └── c_api
│ │ │ ├── api_external_common.h
│ │ │ ├── attribute
│ │ │ │ ├── attribute_api_external_experimental.h
│ │ │ │ └── attribute_api_external.h
│ │ │ ├── buffer
│ │ │ │ └── buffer_api_external.h
│ │ │ ├── buffer_list
│ │ │ │ └── buffer_list_api_external.h
│ │ │ ├── config
│ │ │ │ └── config_api_external.h
│ │ │ ├── context
│ │ │ │ └── context_api_external.h
│ │ │ ├── data_order
│ │ │ │ ├── data_order_api_enum.h
│ │ │ │ └── data_order_api_external.h
│ │ │ ├── datatype
│ │ │ │ ├── datatype_api_enum.h
│ │ │ │ └── datatype_api_external.h
│ │ │ ├── dimension
│ │ │ │ └── dimension_api_external.h
│ │ │ ├── dimension_label
│ │ │ │ └── dimension_label_api_external.h
│ │ │ ├── domain
│ │ │ │ └── domain_api_external.h
│ │ │ ├── enumeration
│ │ │ │ └── enumeration_api_experimental.h
│ │ │ ├── error
│ │ │ │ └── error_api_external.h
│ │ │ ├── filesystem
│ │ │ │ ├── filesystem_api_enum.h
│ │ │ │ └── filesystem_api_external.h
│ │ │ ├── filter
│ │ │ │ ├── filter_api_enum.h
│ │ │ │ └── filter_api_external.h
│ │ │ ├── filter_list
│ │ │ │ └── filter_list_api_external.h
│ │ │ ├── group
│ │ │ │ ├── group_api_external_experimental.h
│ │ │ │ └── group_api_external.h
│ │ │ ├── object
│ │ │ │ ├── object_api_enum.h
│ │ │ │ └── object_api_external.h
│ │ │ ├── query
│ │ │ │ ├── query_api_enum.h
│ │ │ │ └── query_api_external.h
│ │ │ ├── query_aggregate
│ │ │ │ └── query_aggregate_api_external_experimental.h
│ │ │ ├── query_field
│ │ │ │ └── query_field_api_external_experimental.h
│ │ │ ├── query_plan
│ │ │ │ └── query_plan_api_external_experimental.h
│ │ │ ├── string
│ │ │ │ └── string_api_external.h
│ │ │ └── vfs
│ │ │ ├── vfs_api_enum.h
│ │ │ ├── vfs_api_experimental.h
│ │ │ └── vfs_api_external.h
│ │ ├── array_deprecated.h
│ │ ├── array_experimental.h
│ │ ├── array.h
│ │ ├── array_schema_evolution.h
│ │ ├── array_schema_experimental.h
│ │ ├── array_schema.h
│ │ ├── arrowio
│ │ ├── arrow_io_impl.h
│ │ ├── as_built_experimental.h
│ │ ├── attribute_experimental.h
│ │ ├── attribute.h
│ │ ├── capi_string.h
│ │ ├── channel_operation.h
│ │ ├── channel_operator.h
│ │ ├── config.h
│ │ ├── consolidation_plan_experimental.h
│ │ ├── context.h
│ │ ├── core_interface.h
│ │ ├── deleter.h
│ │ ├── dimension.h
│ │ ├── dimension_label_experimental.h
│ │ ├── domain.h
│ │ ├── enumeration_experimental.h
│ │ ├── error.h
│ │ ├── exception.h
│ │ ├── filter.h
│ │ ├── filter_list.h
│ │ ├── fragment_info.h
│ │ ├── group_experimental.h
│ │ ├── group.h
│ │ ├── log.h
│ │ ├── object.h
│ │ ├── object_iter.h
│ │ ├── query_channel.h
│ │ ├── query_condition_experimental.h
│ │ ├── query_condition.h
│ │ ├── query_experimental.h
│ │ ├── query.h
│ │ ├── schema_base.h
│ │ ├── stats.h
│ │ ├── subarray_experimental.h
│ │ ├── subarray.h
│ │ ├── tiledb
│ │ ├── tiledb_dimension_label_experimental.h
│ │ ├── tiledb_enum.h
│ │ ├── tiledb_experimental
│ │ ├── tiledb_experimental.h
│ │ ├── tiledb_export.h
│ │ ├── tiledb.h
│ │ ├── tiledb_serialization.h
│ │ ├── tiledb_version.h
│ │ ├── type.h
│ │ ├── utils.h
│ │ ├── version.h
│ │ └── vfs.h
│ └── tiledb_export.h
└── lib
├── cmake
│ └── TileDB
│ ├── TileDBConfig.cmake
│ ├── TileDBConfigVersion.cmake
│ ├── TileDBTargets.cmake
│ └── TileDBTargets-release.cmake
├── libtiledb.so -> libtiledb.so.2.21
├── libtiledb.so.2.21
└── pkgconfig
└── tiledb.pc
32 directories, 97 files
Building tiledb-py against the release bundle gives an unexpected error which I don't see when I build against local libtiledb - I can't tell if this is due to a change specifically here or some other change on dev that doesn't show up in other test builds.
tiledb/array_schema.py:8: in <module>
import tiledb.cc as lt
E ImportError: dlopen(/Users/inorton/work/git/TileDB-Py/tiledb/cc.cpython-310-darwin.so, 0x0002): Symbol not found: _tiledb_group_get_member_by_index_v2
E Referenced from: <CC612D20-5C29-3547-86E3-6C35A786FF27> /Users/inorton/work/git/TileDB-Py/tiledb/cc.cpython-310-darwin.so
E Expected in: <9443745E-A479-3FF1-B3EB-2CBD162668A7> /Users/inorton/work/git/TileDB-Py/tiledb/native/libtiledb.dylib
==================================================================== short test summary info ====================================================================
ERROR - ImportError: dlopen(/Users/inorton/work/git/TileDB-Py/tiledb/cc.cpython-310-darwin.so, 0x0002): Symbol not found: _tiledb_group_get_member_by_index_v2
ERROR - ImportError: dlopen(/Users/inorton/work/git/TileDB-Py/tiledb/cc.cpython-310-darwin.so, 0x0002): Symbol not found: _tiledb_group_get_member_by_index_v2
ERROR - ImportError: dlopen(/Users/inorton/work/git/TileDB-Py/tiledb/cc.cpython-310-darwin.so, 0x0002): Symbol not found: _tiledb_group_get_member_by_index_v2
ERROR - ImportError: dlopen(/Users/inorton/work/git/TileDB-Py/tiledb/cc.cpython-310-darwin.so, 0x0002): Symbol not found: _tiledb_group_get_member_by_index_v2
- I don't see that error when building against the most recent libtiledb nightly on conda-forge
- I do see the error building against the osx-arm64 release artifact from https://github.com/TileDB-Inc/TileDB/actions/runs/7892923968 -- so I think it is unrelated to this PR.
Story: https://app.shortcut.com/tiledb-inc/story/40980