ckanext-validation
ckanext-validation copied to clipboard
frictionlessdata
TypeError when editing ressources
I am Using Ckan 2.10.1 and just installed ckanext-validation. I have linked the required presets and the required presets
scheming.dataset_schemas = ckanext.validation.examples:ckan_default_schema.json scheming.presets = ckanext.scheming:presets.json Ckanext.validation:presets.json
Now when I click on an existing resource and even when I am not changing anything but clicking on "Update resource" I get this error:
TypeError: str cannot be used as validator because it is not a user-defined function. Why is that and how can I fix this? I am quite new to the ckan world and it's also the first time working for me with ckanext-validation so maybe I am just missing something I don't know. Any help is much appreciated, thanks in advance!
What is your ckanext-scheming version? Older versions may cause this error.
I'm encountering the same error even after a clean installation. I've attempted this with CKAN version 2.10.1 and Python versions 3.10, 3.8.13, and 3.9.18, and in all cases, I encountered the same error when attempting to upload a file. Unfortunately, I don't have any additional information to provide about the error at this time. Here's the content of my ckan.ini file:
#
# CKAN configuration
#
# These are some of the configuration options available for your CKAN
# instance. Check the documentation in 'doc/configuration.rst' or at the
# following URL for a description of what they do and the full list of
# available options:
#
# http://docs.ckan.org/en/latest/maintaining/configuration.html
#
# The %(here)s variable will be replaced with the parent directory of this file
#
[DEFAULT]
## Default settings ############################################################
## This enables the `Flask-DebugToolbar
## <https://flask-debugtoolbar.readthedocs.io/>`_ in the web interface, makes
## Webassets serve unminified JS and CSS files, and enables CKAN templates'
## debugging features.
##
## You will need to ensure the ``Flask-DebugToolbar`` python package is installed,
## by activating your ckan virtual environment and then running::
##
## pip install -r /usr/lib/ckan/default/src/ckan/dev-requirements.txt
##
## If you are running CKAN on Apache, you must change the WSGI
## configuration to run a single process of CKAN. Otherwise
## the execution will fail with: ``AssertionError: The EvalException
## middleware is not usable in a multi-process environment``. Eg. change::
##
## WSGIDaemonProcess ckan_default display-name=ckan_default processes=2 threads=15
## to
## WSGIDaemonProcess ckan_default display-name=ckan_default threads=15
##
## .. warning:: This option should be set to ``False`` for a public site.
## With debug mode enabled, a visitor to your site could execute malicious
## commands.
debug = false
[app:main]
## General settings ############################################################
use = egg:ckan
ckan.legacy_route_mappings = {}
config.mode = default
## Development settings ########################################################
ckan.devserver.host = localhost
ckan.devserver.port = 5000
ckan.devserver.threaded = false
ckan.devserver.multiprocess = 1
ckan.devserver.watch_patterns =
ckan.devserver.ssl_cert =
ckan.devserver.ssl_key =
## Session settings ############################################################
ckan.user.last_active_interval = 600
cache_dir = /tmp/%(ckan.site_id)s
beaker.session.key = ckan
beaker.session.secret = OcRYZS4yk-N2rknfMu7-CdipiI8
beaker.session.auto = false
beaker.session.cookie_expires = false
# beaker.session.cookie_domain = .example.com
beaker.session.save_accessed_time = true
beaker.session.secure = false
beaker.session.timeout = 600
## Database settings ###########################################################
sqlalchemy.url = postgresql://ckan_default:12345@localhost/ckan_default
sqlalchemy.pool_pre_ping = true
# sqlalchemy.<OPTION> =
## Site Settings ###############################################################
ckan.site_url = http://127.0.0.1:5000
apikey_header_name = X-CKAN-API-Key
ckan.cache_expires = 0
ckan.cache_enabled = false
ckan.mimetype_guess = file_ext
ckan.static_max_age = 3600
ckan.tracking_enabled = false
ckan.valid_url_schemes = http https ftp
ckan.requests.timeout = 5
ckan.hide_version = false
ckan.redirect_to_login_if_not_authorized = true
## Authorization Settings ######################################################
ckan.auth.anon_create_dataset = false
ckan.auth.create_unowned_dataset = false
ckan.auth.create_dataset_if_not_in_organization = true
ckan.auth.user_create_groups = true
ckan.auth.user_create_organizations = true
ckan.auth.user_delete_groups = true
ckan.auth.user_delete_organizations = true
ckan.auth.create_user_via_api = false
ckan.auth.create_user_via_web = true
ckan.auth.roles_that_cascade_to_sub_groups = admin
ckan.auth.public_user_details = true
ckan.auth.public_activity_stream_detail = false
ckan.auth.allow_dataset_collaborators = false
ckan.auth.allow_admin_collaborators = false
ckan.auth.allow_collaborators_to_change_owner_org = false
ckan.auth.create_default_api_keys = false
ckan.auth.login_view = user.login
ckan.auth.reveal_private_datasets = false
ckan.auth.enable_cookie_auth_in_api = true
ckan.auth.route_after_login = dashboard.datasets
## CSRF Protection #############################################################
WTF_CSRF_ENABLED = true
WTF_CSRF_CHECK_DEFAULT = true
WTF_CSRF_SECRET_KEY = AM3LDx_t1OKaABxkFbh_4mEPKxrAeRg3Ou_LxquSBX0
WTF_CSRF_METHODS = POST PUT PATCH DELETE
WTF_CSRF_FIELD_NAME = _csrf_token
WTF_CSRF_HEADERS = X-CSRFToken X-CSRF-Token
WTF_CSRF_TIME_LIMIT = 3600
WTF_CSRF_SSL_STRICT = true
WTF_I18N_ENABLED = true
ckan.csrf_protection.ignore_extensions = true
## Flask-Login Remember me cookie settings #####################################
REMEMBER_COOKIE_NAME = remember_token
REMEMBER_COOKIE_DURATION = 31536000
# REMEMBER_COOKIE_DOMAIN = .example.com
REMEMBER_COOKIE_PATH = /
REMEMBER_COOKIE_SECURE = false
REMEMBER_COOKIE_HTTPONLY = true
REMEMBER_COOKIE_REFRESH_EACH_REQUEST = false
REMEMBER_COOKIE_SAMESITE = None
## API Token Settings ##########################################################
api_token.nbytes = 32
api_token.jwt.encode.secret = string:%(beaker.session.secret)s
api_token.jwt.decode.secret = string:%(beaker.session.secret)s
api_token.jwt.algorithm = HS256
## Search Settings #############################################################
ckan.site_id = default
solr_url = http://127.0.0.1:8983/solr/ckan
solr_user =
solr_password =
ckan.search.remove_deleted_packages = true
ckan.search.solr_commit = true
ckan.search.show_all_types = dataset
ckan.search.default_include_private = true
ckan.search.default_package_sort = score desc, metadata_modified desc
search.facets = organization groups tags res_format license_id
search.facets.limit = 50
search.facets.default = 10
ckan.extra_resource_fields =
ckan.search.rows_max = 1000
ckan.group_and_organization_list_max = 1000
ckan.group_and_organization_list_all_fields_max = 25
solr_timeout = 60
## Redis Settings ##############################################################
ckan.redis.url = redis://localhost:6379/0
## CORS Settings ###############################################################
ckan.cors.origin_allow_all = false
ckan.cors.origin_whitelist =
################ ______DATASTORE SETTINGS_____#############
ckan.datastore.write_url = postgresql://ckan_default:12345@localhost/datastore_default
ckan.datastore.read_url = postgresql://datastore_default:12345@localhost/datastore_default
###############______XLOADER SETTINGS______##############
ckanext.xloader.api_token = eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJqdGkiOiI4TFdWQjF6cFFnOGxvMmdxU19mU2czUGZfbF85ZUMwSWRGX2JBRjlaREE0IiwiaWF0IjoxNjk2OTcxNTYxfQ.F-myzVPpq0TMrZe-Scd1HbElSbt_D6OKoCdha1LWCqc
ckanext.xloader.jobs_db.uri = postgresql://ckan_default:12345@localhost/ckan_default
#############_______VALIDATION SETTINGS______##############
scheming.dataset_schemas = ckanext.validation.examples:ckan_default_schema.json
scheming.presets = ckanext.scheming:presets.json
ckanext.validation:presets.json
ckanext.validation.run_on_create_async = True
ckanext.validation.run_on_update_async = True
ckanext.validation.formats = csv xlsx
ckanext.validation.show_badges_in_listings = True
ckanext.validation.default_validation_options={
"skip_errors": ["blank-row", "duplicate-label"],
}
## Plugins Settings ############################################################
ckan.plugins = activity datastore xloader validation scheming_datasets
ckan.resource_proxy.timeout = 5
## Front-End Settings ##########################################################
ckan.site_title = CKAN
ckan.site_description =
ckan.site_intro_text =
ckan.site_logo = /base/images/ckan-logo.png
ckan.site_about =
ckan.theme = css/main
ckan.favicon = /base/images/ckan.ico
ckan.datasets_per_page = 20
package_hide_extras =
ckan.dumps_url =
ckan.dumps_format =
ckan.recaptcha.publickey =
ckan.recaptcha.privatekey =
ckan.featured_groups =
ckan.featured_orgs =
ckan.default_group_sort = title
ckan.gravatar_default = identicon
ckan.debug_supress_header = false
ckan.homepage_style = 1
ckan.site_custom_css =
## Resource Views Settings #####################################################
ckan.views.default_views = image_view datatables_view
## Theming Settings ############################################################
ckan.template_head_end =
ckan.template_footer_end =
ckan.template_title_delimiter = -
extra_template_paths =
extra_public_paths =
ckan.base_public_folder = public
ckan.base_templates_folder = templates
ckan.default.package_type = dataset
ckan.default.group_type = group
ckan.default.organization_type = organization
ckan.admin_tabs = {}
## Storage Settings ############################################################
ckan.storage_path = /var/lib/ckan/default
ckan.max_resource_size = 10
ckan.max_image_size = 2
## Uploader Settings ###########################################################
ckan.upload.user.types =
ckan.upload.user.mimetypes =
ckan.upload.group.types =
ckan.upload.group.mimetypes =
## Webassets Settings ##########################################################
ckan.webassets.path =
ckan.webassets.use_x_sendfile = false
## User Settings ###############################################################
ckan.user_list_limit = 20
ckan.user_reset_landing_page = home.index
## Activity Streams Settings ###################################################
ckan.activity_streams_enabled = true
ckan.activity_streams_email_notifications = false
ckan.activity_list_limit = 31
ckan.activity_list_limit_max = 100
ckan.email_notifications_since = 2 days
ckan.hide_activity_from_users = %(ckan.site_id)s
## Feeds Settings ##############################################################
ckan.feeds.author_name =
ckan.feeds.author_link =
ckan.feeds.authority_name =
ckan.feeds.date =
ckan.feeds.limit = 20
## Internationalisation Settings ###############################################
ckan.locale_default = en
ckan.locales_offered =
ckan.locales_filtered_out =
ckan.locale_order =
ckan.i18n_directory =
ckan.i18n.extra_directory =
ckan.i18n.extra_gettext_domain =
ckan.i18n.extra_locales =
ckan.i18n.rtl_languages = he ar fa_IR
ckan.i18n.rtl_theme = css/main-rtl
ckan.display_timezone = UTC
ckan.root_path =
ckan.resource_formats = /usr/lib/ckan/default/src/ckan/ckan/config/resource_formats.json
## Form Settings ###############################################################
ckan.dataset.create_on_ui_requires_resources = true
package_new_return_url =
package_edit_return_url =
licenses_group_url =
## Email settings ##############################################################
smtp.server = localhost
smtp.starttls = false
smtp.user =
smtp.password =
smtp.mail_from =
smtp.reply_to =
email_to =
error_email_from =
## Background Job Settings #####################################################
ckan.jobs.timeout = 180
## Logging configuration
[loggers]
keys = root, ckan, ckanext, werkzeug
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARNING
handlers = console
[logger_werkzeug]
level = WARNING
handlers = console
qualname = werkzeug
propagate = 0
[logger_ckan]
level = INFO
handlers = console
qualname = ckan
propagate = 0
[logger_ckanext]
level = DEBUG
handlers = console
qualname = ckanext
propagate = 0
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s] %(message)s
I've double-checked the version of ckanext-scheming, and it appears to be up to date at version 3.0.0. Here is a list of the libraries installed in the environment:
alembic==1.8.1
attrs==23.1.0
Babel==2.10.3
Beaker==1.11.0
bleach==5.0.1
blinker==1.5
boto3==1.28.62
botocore==1.31.62
cached-property==1.5.2
certifi==2021.10.8
chardet==5.2.0
charset-normalizer==2.0.12
-e git+https://github.com/ckan/ckan.git@9915ba0022b9a74a65e61c097b2fee584b044087#egg=ckan
ckanapi==4.7
-e git+https://github.com/ckan/ckanext-scheming.git@8646a9dce79aa0b5a46274271ca6330dc0870b92#egg=ckanext_scheming
-e git+https://github.com/frictionlessdata/ckanext-validation.git@1073c80dace453a404df3d5f1ac1ae86a88b5029#egg=ckanext_validation
ckanext-xloader==0.12.2
ckantoolkit==0.0.7
click==8.1.3
colorama==0.4.6
Deprecated==1.2.13
docopt==0.6.2
dominate==2.7.0
et-xmlfile==1.1.0
feedgen==0.9.0
Flask==2.0.3
Flask-Babel==1.0.0
Flask-Login==0.6.1
flask-multistatic==1.0
Flask-WTF==1.0.1
frictionless==5.0.0b9
greenlet==1.1.2
humanize==4.8.0
idna==3.3
ijson==3.2.3
importlib-metadata==4.11.3
isodate==0.6.1
itsdangerous==2.1.1
Jinja2==3.1.2
jmespath==1.0.1
jsonlines==4.0.0
jsonschema==4.19.1
jsonschema-specifications==2023.7.1
linear-tsv==1.1.0
lxml==4.9.1
Mako==1.2.2
Markdown==3.4.1
markdown-it-py==3.0.0
marko==2.0.0
MarkupSafe==2.0.1
mdurl==0.1.2
mypy==0.971
mypy-extensions==0.4.3
nose==1.3.7
openpyxl==3.1.2
packaging==21.3
passlib==1.7.4
petl==1.7.14
polib==1.1.1
psycopg2==2.9.3
Pygments==2.16.1
PyJWT==2.4.0
pyparsing==3.0.7
pysolr==3.9.0
python-dateutil==2.8.2
python-magic==0.4.27
python-slugify==8.0.1
pytz==2021.3
pytz-deprecation-shim==0.1.0.post0
PyUtilib==6.0.0
PyYAML==6.0
redis==4.1.4
referencing==0.30.2
requests==2.31.0
rfc3986==2.0.0
rich==13.6.0
rpds-py==0.10.4
rq==1.11.0
s3transfer==0.7.0
shellingham==1.5.3
simpleeval==0.9.13
simplejson==3.17.6
six==1.16.0
SQLAlchemy==1.4.41
sqlalchemy2-stubs==0.0.2a27
sqlparse==0.4.2
stringcase==1.2.0
tableschema==1.20.2
tabulate==0.9.0
tabulator==1.53.5
text-unidecode==1.3
tomli==2.0.1
typer==0.9.0
typing_extensions==4.3.0
tzdata==2022.1
tzlocal==4.2
unicodecsv==0.14.1
Unidecode==1.0.22
urllib3==1.26.9
validators==0.22.0
watchdog==2.1.6
webassets==2.0
webencodings==0.5.1
Werkzeug==2.0.3
wrapt==1.14.0
WTForms==3.0.1
xlrd==2.0.1
zipp==3.7.0
zope.interface==5.4.0
and for information, here is the error mensage when i try to upload something:
**2023-10-10 17:11:54,461 INFO [ckan.config.middleware.flask_app] 304 /base/vendor/select2/select2.png render time 0.005 seconds
2023-10-10 17:12:03,877 ERROR [ckan.config.middleware.flask_app] str cannot be used as validator because it is not a user-defined function
Traceback (most recent call last):
File "/usr/lib/ckan/default/src/ckan/ckan/lib/navl/dictization_functions.py", line 246, in convert
nargs = converter._code_.co_argcount
AttributeError: type object 'str' has no attribute '_code_'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/ckan/default/lib/python3.10/site-packages/flask/app.py", line 1516, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib/ckan/default/lib/python3.10/site-packages/flask/app.py", line 1502, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File "/usr/lib/ckan/default/lib/python3.10/site-packages/flask/views.py", line 84, in view
return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs)
File "/usr/lib/ckan/default/lib/python3.10/site-packages/flask/views.py", line 158, in dispatch_request
return current_app.ensure_sync(meth)(*args, **kwargs)
File "/usr/lib/ckan/default/src/ckan/ckan/config/middleware/../../views/resource.py", line 377, in post
get_action(u'resource_update')(context, data)
File "/usr/lib/ckan/default/src/ckan/ckan/logic/_init_.py", line 551, in wrapped
result = _action(context, data_dict, **kw)
File "/usr/lib/ckan/default/src/ckanext-validation/ckanext/validation/logic.py", line 544, in resource_update
return up_func(context, data_dict)
File "/usr/lib/ckan/default/src/ckan/ckan/logic/action/update.py", line 111, in resource_update
updated_pkg_dict = _get_action('package_update')(context, pkg_dict)
File "/usr/lib/ckan/default/src/ckan/ckan/logic/_init_.py", line 551, in wrapped
result = _action(context, data_dict, **kw)
File "/usr/lib/ckan/default/src/ckan/ckan/logic/action/update.py", line 311, in package_update
data, errors = lib_plugins.plugin_validate(
File "/usr/lib/ckan/default/src/ckan/ckan/lib/plugins.py", line 327, in plugin_validate
result = plugin.validate(context, data_dict, schema, action)
File "/usr/lib/ckan/default/src/ckanext-scheming/ckanext/scheming/plugins.py", line 312, in validate
return navl_validate(data_dict, schema, context)
File "/usr/lib/ckan/default/src/ckan/ckan/lib/navl/dictization_functions.py", line 305, in validate
flat_data, errors = _validate(flattened, schema, validators_context)
File "/usr/lib/ckan/default/src/ckan/ckan/lib/navl/dictization_functions.py", line 356, in _validate
convert(converter, key, converted_data, errors, context)
File "/usr/lib/ckan/default/src/ckan/ckan/lib/navl/dictization_functions.py", line 248, in convert
raise TypeError(
TypeError: str cannot be used as validator because it is not a user-defined function**
Given this situation, I would greatly appreciate any assistance in resolving this issue. Thank you very much.
The example schema references the unicode validator when it should be unicode_safe.