eodag icon indicating copy to clipboard operation
eodag copied to clipboard

feat: server mode collections search

Open dalpasso opened this issue 2 years ago • 6 comments

Implements free text search of collections https://github.com/stac-api-extensions/collection-search#free-text-search

dalpasso avatar Nov 01 '23 08:11 dalpasso

Test Results

    4 files  ±0      4 suites  ±0   5m 55s :stopwatch: +58s   503 tests +2    500 :white_check_mark: +2   3 :zzz: ±0  0 :x: ±0  2 012 runs  +8  1 926 :white_check_mark: +6  86 :zzz: +2  0 :x: ±0 

Results for commit 55a9a62a. ± Comparison against base commit d1df25ec.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Nov 01 '23 08:11 github-actions[bot]

Code Coverage (Ubuntu)

File Coverage
All files 81% :white_check_mark:

Minimum allowed coverage is 70%

Generated by :monkey: cobertura-action against c056162ad5d8bda0a751cae2b8605e8173b6d6da

github-actions[bot] avatar Nov 01 '23 08:11 github-actions[bot]

Code Coverage (Windows)

File Coverage
All files 76% :white_check_mark:

Minimum allowed coverage is 70%

Generated by :monkey: cobertura-action against c056162ad5d8bda0a751cae2b8605e8173b6d6da

github-actions[bot] avatar Nov 01 '23 08:11 github-actions[bot]

@dalpasso the following STAC extension should be added to the server conformance classes:

https://api.stacspec.org/v1.0.0-rc.1/collection-search#free-text

alambare avatar Nov 16 '23 20:11 alambare

Feature reimplemented in EODataAccessGateway.list_product_types() using whoosh

dalpasso avatar Nov 22 '23 11:11 dalpasso

@dalpasso the new parameter is missing from the OpenAPI definition.

It should be added at https://github.com/CS-SI/eodag/blob/852-server-mode-collections-search/eodag/resources/stac_api.yml#L165

You can check how it is done for /search: https://github.com/CS-SI/eodag/blob/852-server-mode-collections-search/eodag/resources/stac_api.yml#L305

alambare avatar Nov 23 '23 17:11 alambare

badge

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     11       0  100.00%
cli.py                                         300      47  84.33%   61, 647-686, 788-839, 843
config.py                                      312      27  91.35%   80-82, 91, 99, 103-105, 178, 190, 385-387, 451-454, 501-502, 511-512, 591, 660-665, 667
crunch.py                                        6       6  0.00%    18-24
api/__init__.py                                  1       0  100.00%
api/core.py                                    750      79  89.47%   101-110, 383, 594, 638-641, 679, 784, 788-793, 819, 889, 959, 1076-1081, 1167-1179, 1219, 1221, 1225, 1246-1248, 1252-1263, 1276-1282, 1372-1375, 1404-1424, 1472, 1478-1481, 1490, 1841, 1874-1880, 2143, 2147-2150, 2164-2166, 2231, 2241, 2273, 2276
api/search_result.py                            44       6  86.36%   33-35, 70, 79, 86, 100
api/product/__init__.py                          5       0  100.00%
api/product/_assets.py                          44       5  88.64%   27-29, 79, 129
api/product/_product.py                        208      32  84.62%   48-55, 59-61, 165-172, 256-257, 347, 383, 444, 458-461, 474, 498-501, 544-550
api/product/metadata_mapping.py                648      87  86.57%   66-68, 129-131, 232, 264-265, 317-329, 331, 342, 346-362, 401-404, 441, 462-465, 479, 487-488, 561-562, 586-587, 593-596, 611-612, 776, 822, 879-880, 923-928, 1059, 1073-1093, 1113, 1118, 1228, 1250, 1264, 1277-1296, 1335, 1387, 1425-1429, 1448
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              1       0  100.00%
plugins/base.py                                 23       3  86.96%   25, 48, 55
plugins/manager.py                             130      11  91.54%   49-51, 93-98, 142, 271-272
plugins/apis/__init__.py                         1       0  100.00%
plugins/apis/base.py                            38      10  73.68%   36-43, 103, 107, 174, 219
plugins/apis/cds.py                            204      49  75.98%   72-80, 126-129, 176-188, 244, 309, 317-322, 335-342, 360-362, 383-385, 395-397, 411-412, 457, 460, 470, 482, 515, 517
plugins/apis/ecmwf.py                           98      15  84.69%   48-56, 157-159, 206-207, 233-235
plugins/apis/usgs.py                           168      36  78.57%   59-64, 125, 198, 232, 267-269, 274, 300-301, 306, 334-341, 350-355, 377-383, 385-391, 414
plugins/authentication/__init__.py               7       1  85.71%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       2  89.47%   26, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                17       1  94.12%   27
plugins/authentication/keycloak.py              88      17  80.68%   32-34, 159-160, 190-212, 238-243
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       103      64  37.86%   39-41, 138-150, 154-172, 180-222, 228-237, 246-286, 291-299, 304-305
plugins/authentication/qsauth.py                35       2  94.29%   32, 80
plugins/authentication/sas_auth.py              47       2  95.74%   32, 71
plugins/authentication/token.py                 75       8  89.33%   35-37, 102, 127, 154-157
plugins/crunch/__init__.py                       1       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      10  80.00%   32-34, 51-52, 71, 80-83, 85, 92-95
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      17  75.00%   28-30, 33, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     1       0  100.00%
plugins/download/aws.py                        490     165  66.33%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 453-454, 460-464, 497, 532, 536, 543, 573-581, 585, 623-631, 638-640, 681-755, 773-834, 845-850, 866-879, 908, 923-925, 928, 938-946, 954-967, 977-996, 1003-1015, 1056, 1082, 1127-1129, 1349
plugins/download/base.py                       261      57  78.16%   58-64, 145, 180, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 689, 711, 719
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       418      97  76.79%   81-88, 122, 170, 191-198, 208-221, 258-263, 313, 345, 372-374, 384-392, 414, 453-457, 514, 587-643, 657, 681-683, 713-714, 722, 725, 741, 765, 788-789, 796, 855-861, 916-917, 923, 933, 999, 1017-1033
plugins/download/s3rest.py                     116      27  76.72%   55-58, 124, 165, 192, 222-229, 232-234, 238, 251-257, 265-266, 269-273, 296, 316-319
plugins/search/__init__.py                       1       0  100.00%
plugins/search/base.py                         101      13  87.13%   43-47, 101, 105, 131-135, 264, 284
plugins/search/build_search_result.py           56       6  89.29%   67, 112-113, 121, 132, 176
plugins/search/creodias_s3.py                   55       3  94.55%   56, 74, 108
plugins/search/csw.py                          107      83  22.43%   43-45, 57-58, 62-63, 74-122, 128-141, 149-181, 199-240
plugins/search/data_request_search.py          195      60  69.23%   52, 89-92, 108, 120, 139, 144, 149, 156, 169-172, 226-227, 231, 241-247, 252, 276, 286-297, 309, 311, 318-319, 321-322, 337-341, 374, 381, 392, 405, 411-423, 428
plugins/search/qssearch.py                     517      55  89.36%   86, 353-354, 448-454, 481-484, 557-558, 599, 617, 632, 685, 706, 709-710, 719, 730, 739, 750-751, 762, 822-827, 831-832, 860, 923, 942-959, 994, 1128-1129, 1150, 1192-1193, 1203-1209, 1249, 1262, 1282, 1370
plugins/search/static_stac_search.py            46       3  93.48%   39-40, 81
rest/__init__.py                                 1       0  100.00%
rest/core.py                                   168      13  92.26%   238, 506-507, 510, 512, 515, 518-520, 572, 579-583
rest/server.py                                 306      54  82.35%   81-82, 106, 129-130, 241-243, 259, 299-300, 312-328, 410-415, 445, 607-614, 645, 688-689, 782-784, 801-806, 836, 838, 842-843, 847-848
rest/stac.py                                   435      95  78.16%   59-61, 229-231, 273, 286-295, 314-320, 365, 402-404, 427, 462-463, 549-594, 639, 659-660, 839, 904-906, 1125, 1135-1147, 1160-1182, 1196-1241, 1400-1401
rest/types/__init__.py                           1       0  100.00%
rest/types/eodag_search.py                     185       9  95.14%   52-55, 231-235, 288, 291, 359
rest/types/stac_queryables.py                   38       7  81.58%   28, 120-126, 129
rest/types/stac_search.py                      122      10  91.80%   48-51, 167, 182-184, 192, 196
rest/utils/__init__.py                         116      14  87.93%   53, 108-109, 128-130, 180, 190-204, 236
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       3  90.32%   78, 90, 92
types/__init__.py                               76       6  92.11%   53, 87, 152, 172, 177, 185
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
utils/__init__.py                              486      43  91.15%   82, 87, 108-110, 188-189, 198-225, 228, 242, 324-328, 404-408, 429-431, 513, 518, 528, 566-567, 963-966, 974-975, 1016-1017, 1097, 1181, 1199, 1372
utils/constraints.py                           123      42  65.85%   31, 84-93, 134, 139, 143, 154, 177-178, 189-197, 206, 220-236, 245-256
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/stac_reader.py                            89      27  69.66%   54-55, 62-83, 90-92, 96, 135, 149-152
TOTAL                                         8654    1568  81.88%

Diff against develop

Filename        Stmts    Miss  Cover
------------  -------  ------  -------
api/core.py        +8       0  +0.12%
rest/stac.py       +7      +2  -0.11%
TOTAL             +15      +2  +0.01%

Results for commit: 55a9a62a9b9c2d0ef3d8281c9f7a67e7b7287874

Minimum allowed coverage is 70%

:recycle: This comment has been updated with latest results

github-actions[bot] avatar Feb 29 '24 15:02 github-actions[bot]

badge

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     11       0  100.00%
cli.py                                         300      47  84.33%   61, 647-686, 788-839, 843
config.py                                      312      28  91.03%   80-82, 91, 99, 103-105, 178, 190, 385-387, 451-454, 501-502, 511-512, 591, 625, 660-665, 667
crunch.py                                        6       6  0.00%    18-24
api/__init__.py                                  1       0  100.00%
api/core.py                                    750      83  88.93%   101-110, 383, 594, 638-641, 679, 784, 788-793, 819, 889, 959, 1076-1081, 1167-1179, 1219, 1221, 1225, 1246-1248, 1252-1263, 1276-1282, 1372-1375, 1404-1424, 1472, 1478-1481, 1490, 1841, 1874-1880, 2143, 2147-2150, 2164-2166, 2208-2215, 2231, 2241, 2273, 2276
api/search_result.py                            44       6  86.36%   33-35, 70, 79, 86, 100
api/product/__init__.py                          5       0  100.00%
api/product/_assets.py                          44       5  88.64%   27-29, 79, 129
api/product/_product.py                        208      32  84.62%   48-55, 59-61, 165-172, 256-257, 347, 383, 444, 458-461, 474, 498-501, 544-550
api/product/metadata_mapping.py                648      88  86.42%   66-68, 129-131, 232, 264-265, 317-329, 331, 342, 346-362, 401-404, 441, 462-465, 479, 487-488, 561-562, 586-587, 593-596, 611-612, 776, 822, 879-880, 923-928, 1059, 1073-1093, 1113, 1118, 1228, 1250, 1264, 1277-1296, 1335, 1387, 1410, 1425-1429, 1448
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              1       0  100.00%
plugins/base.py                                 23       4  82.61%   25, 48, 55, 68
plugins/manager.py                             130      11  91.54%   49-51, 93-98, 142, 271-272
plugins/apis/__init__.py                         1       0  100.00%
plugins/apis/base.py                            38      11  71.05%   36-43, 103, 107, 120, 174, 219
plugins/apis/cds.py                            204      56  72.55%   72-80, 126-129, 176-188, 244, 309, 317-322, 335-342, 360-362, 383-385, 395-397, 411-412, 457, 460, 470, 482, 507-522
plugins/apis/ecmwf.py                           98      15  84.69%   48-56, 157-159, 206-207, 233-235
plugins/apis/usgs.py                           168      36  78.57%   59-64, 125, 198, 232, 267-269, 274, 300-301, 306, 334-341, 350-355, 377-383, 385-391, 414
plugins/authentication/__init__.py               7       1  85.71%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                17       1  94.12%   27
plugins/authentication/keycloak.py              88      17  80.68%   32-34, 159-160, 190-212, 238-243
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       103      64  37.86%   39-41, 138-150, 154-172, 180-222, 228-237, 246-286, 291-299, 304-305
plugins/authentication/qsauth.py                35       2  94.29%   32, 80
plugins/authentication/sas_auth.py              47       2  95.74%   32, 71
plugins/authentication/token.py                 75       8  89.33%   35-37, 102, 127, 154-157
plugins/crunch/__init__.py                       1       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      35  30.00%   32-34, 48-53, 69-114
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      17  75.00%   28-30, 33, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     1       0  100.00%
plugins/download/aws.py                        490     165  66.33%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 453-454, 460-464, 497, 532, 536, 543, 573-581, 585, 623-631, 638-640, 681-755, 773-834, 845-850, 866-879, 908, 923-925, 928, 938-946, 954-967, 977-996, 1003-1015, 1056, 1082, 1127-1129, 1349
plugins/download/base.py                       261      59  77.39%   58-64, 145, 180, 250-252, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 689, 711, 719
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       418      97  76.79%   81-88, 122, 170, 191-198, 208-221, 258-263, 313, 345, 372-374, 384-392, 414, 453-457, 514, 587-643, 657, 681-683, 713-714, 722, 725, 741, 765, 788-789, 796, 855-861, 916-917, 923, 933, 999, 1017-1033
plugins/download/s3rest.py                     116      27  76.72%   55-58, 124, 165, 192, 222-229, 232-234, 238, 251-257, 265-266, 269-273, 296, 316-319
plugins/search/__init__.py                       1       0  100.00%
plugins/search/base.py                         101      13  87.13%   43-47, 101, 105, 131-135, 264, 284
plugins/search/build_search_result.py           56       6  89.29%   67, 112-113, 121, 132, 176
plugins/search/creodias_s3.py                   55       3  94.55%   56, 74, 108
plugins/search/csw.py                          107      83  22.43%   43-45, 57-58, 62-63, 74-122, 128-141, 149-181, 199-240
plugins/search/data_request_search.py          195      60  69.23%   52, 89-92, 108, 120, 139, 144, 149, 156, 169-172, 226-227, 231, 241-247, 252, 276, 286-297, 309, 311, 318-319, 321-322, 337-341, 374, 381, 392, 405, 411-423, 428
plugins/search/qssearch.py                     517      79  84.72%   86, 353-354, 448-454, 481-484, 557-558, 599, 617, 632, 685, 706, 709-710, 719, 730, 739, 750-751, 762, 822-827, 831-832, 860, 923, 942-959, 994, 1128-1129, 1150, 1192-1193, 1203-1209, 1249, 1262, 1282, 1330-1399
plugins/search/static_stac_search.py            46       3  93.48%   39-40, 81
rest/__init__.py                                 1       0  100.00%
rest/core.py                                   168      60  64.29%   135, 137, 139, 145-146, 163-171, 178-184, 230-259, 413-440, 458, 505-535, 572, 579-583
rest/server.py                                 306     306  0.00%    18-861
rest/stac.py                                   435     154  64.60%   59-61, 214, 229-231, 273, 286-295, 314-320, 365, 402-404, 427, 462-463, 549-594, 639, 647-648, 652-660, 781, 839, 904-906, 923-925, 933-935, 948-950, 964-981, 991-1012, 1022-1044, 1052-1069, 1092-1115, 1125, 1135-1147, 1160-1182, 1196-1241, 1394-1420
rest/types/__init__.py                           1       0  100.00%
rest/types/eodag_search.py                     185      18  90.27%   52-55, 231-235, 268-270, 288, 291, 297, 301, 359, 371-374
rest/types/stac_queryables.py                   38      13  65.79%   28, 53-58, 120-126, 129, 133-134
rest/types/stac_search.py                      122      12  90.16%   48-51, 167, 182-184, 192, 196, 240, 243
rest/utils/__init__.py                         116      31  73.28%   53, 79-85, 105, 108-109, 128-130, 147, 173-181, 188-209, 236
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       5  83.87%   73-74, 78, 90, 92
types/__init__.py                               76      11  85.53%   53, 87, 125, 152, 162-164, 172, 177, 185, 195
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
utils/__init__.py                              486      43  91.15%   82, 87, 108-110, 188-189, 198-225, 228, 242, 324-328, 404-408, 429-431, 513, 518, 528, 566-567, 963-966, 974-975, 1016-1017, 1097, 1181, 1199, 1372
utils/constraints.py                           123      42  65.85%   31, 84-93, 134, 139, 143, 154, 177-178, 189-197, 206, 220-236, 245-256
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/stac_reader.py                            89      27  69.66%   54-55, 62-83, 90-92, 96, 135, 149-152
TOTAL                                         8654    2034  76.50%

Diff against develop

Filename        Stmts    Miss  Cover
------------  -------  ------  -------
api/core.py        +8       0  +0.12%
rest/stac.py       +7      +7  -1.05%
TOTAL             +15      +7  -0.04%

Results for commit: 55a9a62a9b9c2d0ef3d8281c9f7a67e7b7287874

Minimum allowed coverage is 70%

:recycle: This comment has been updated with latest results

github-actions[bot] avatar Feb 29 '24 15:02 github-actions[bot]