faust icon indicating copy to clipboard operation
faust copied to clipboard
verified publisher icon faust-streaming

Error to migrate from old faust to new fork (Internal version)

Open antonioIrizar opened this issue 3 years ago • 1 comments

Checklist

  • [x] I have included information about relevant versions
  • [ ] I have verified that the issue persists when using the master branch of Faust.

Steps to reproduce

I'm already using the old version in production system (RobinHood version) and I want to start to use this fork. Anyways thanks of the fork and for doing the maintenance of the project.

When I change the dependency, the project cannot start and it gives an exception.

Code to start faust:

import ssl
import faust
import settings

ssl_context = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH)
urls_kafka = ['kafka://' + url for url in settings.KAFKA_URL.split(',')]

app = faust.App(
    settings.APP_CREDIT_FAUST,
    serializer='json',
    broker=urls_kafka,
    broker_credentials=ssl_context,
)

events_topic = app.topic(
    settings.TOPIC_CREDITS_EVENTS,
    key_type=str,
    value_serializer='json',
)

Faust fail when try to start topic.

I saw it is taken a strange version from docker container. I tried to add to faust.App a specific version but It sill failing with the same error.

Like this:

import ssl
import faust

ssl_context = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH)
urls_kafka = ['kafka://' + url for url in settings.KAFKA_URL.split(',')]

app = faust.App(
    settings.APP_CREDIT_FAUST,
    serializer='json',
    broker=urls_kafka,
    broker_credentials=ssl_context,
   version=123,
)

events_topic = app.topic(
    settings.TOPIC_CREDITS_EVENTS,
    key_type=str,
    value_serializer='json',
)

Expected behavior

Works like old version of faust.

Actual behavior

I get this error

ValueError

Cannot import 'app': invalid literal for int() with base 10: 'dirty_ce71a5b6a9e224a23679abe613f69e26c3d96ad7_b7ec154d0c8b2a5e158e122f77fe9987b28e05ac'

Full traceback

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 543, in to_python
    return self.number_aliases[value]
KeyError: 'dirty_ce71a5b6a9e224a23679abe613f69e26c3d96ad7_00692776f8f0ac549574eb0aa1bb5c06f71baeea'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/mode/utils/imports.py", line 262, in symbol_by_name
    module = imp(  # type: ignore
  File "/usr/local/lib/python3.8/site-packages/mode/utils/imports.py", line 376, in import_from_cwd
    return imp(module, package=package)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/app/app.py", line 68, in <module>
    events_topic = app.topic(
  File "/usr/local/lib/python3.8/site-packages/faust/app/base.py", line 797, in topic
    self.conf.Topic(  # type: ignore
  File "/usr/local/lib/python3.8/site-packages/faust/app/base.py", line 1922, in conf
    self._configure()
  File "/usr/local/lib/python3.8/site-packages/faust/app/base.py", line 1862, in _configure
    conf = self._load_settings(silent=silent)
  File "/usr/local/lib/python3.8/site-packages/faust/app/base.py", line 1875, in _load_settings
    return self.Settings(appid, **self._prepare_compat_settings(conf))
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/base.py", line 100, in _new_init
    self._init_entrypoint(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/base.py", line 145, in _init_entrypoint
    self._init_settings(kwargs)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/base.py", line 168, in _init_settings
    param.on_init_set_value(conf, provided_value)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 416, in on_init_set_value
    self.__set__(conf, provided_value)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 381, in __set__
    self.on_set(obj, self.prepare_set(obj, value))
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 462, in prepare_set
    new_value = self.to_python(conf, value)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 545, in to_python
    return self.convert(conf, value)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 572, in convert
    return cast(OT, int(cast(int, value)))
ValueError: invalid literal for int() with base 10: 'dirty_ce71a5b6a9e224a23679abe613f69e26c3d96ad7_00692776f8f0ac549574eb0aa1bb5c06f71baeea'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/faust", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 781, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/usr/local/lib/python3.8/site-packages/faust/cli/base.py", line 423, in make_context
    self._maybe_import_app()
  File "/usr/local/lib/python3.8/site-packages/faust/cli/base.py", line 388, in _maybe_import_app
    find_app(appstr)
  File "/usr/local/lib/python3.8/site-packages/faust/cli/base.py", line 312, in find_app
    val = symbol_by_name(app, imp=imp)
  File "/usr/local/lib/python3.8/site-packages/mode/utils/imports.py", line 271, in symbol_by_name
    raise ValueError(
  File "/usr/local/lib/python3.8/site-packages/mode/utils/imports.py", line 262, in symbol_by_name
    module = imp(  # type: ignore
  File "/usr/local/lib/python3.8/site-packages/mode/utils/imports.py", line 376, in import_from_cwd
    return imp(module, package=package)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/app/app.py", line 68, in <module>
    events_topic = app.topic(
  File "/usr/local/lib/python3.8/site-packages/faust/app/base.py", line 797, in topic
    self.conf.Topic(  # type: ignore
  File "/usr/local/lib/python3.8/site-packages/faust/app/base.py", line 1922, in conf
    self._configure()
  File "/usr/local/lib/python3.8/site-packages/faust/app/base.py", line 1862, in _configure
    conf = self._load_settings(silent=silent)
  File "/usr/local/lib/python3.8/site-packages/faust/app/base.py", line 1875, in _load_settings
    return self.Settings(appid, **self._prepare_compat_settings(conf))
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/base.py", line 100, in _new_init
    self._init_entrypoint(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/base.py", line 145, in _init_entrypoint
    self._init_settings(kwargs)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/base.py", line 168, in _init_settings
    param.on_init_set_value(conf, provided_value)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 416, in on_init_set_value
    self.__set__(conf, provided_value)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 381, in __set__
    self.on_set(obj, self.prepare_set(obj, value))
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 462, in prepare_set
    new_value = self.to_python(conf, value)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 545, in to_python
    return self.convert(conf, value)
  File "/usr/local/lib/python3.8/site-packages/faust/types/settings/params.py", line 572, in convert
    return cast(OT, int(cast(int, value)))
ValueError: Cannot import 'app': invalid literal for int() with base 10: 'dirty_ce71a5b6a9e224a23679abe613f69e26c3d96ad7_00692776f8f0ac549574eb0aa1bb5c06f71baeea'

Versions

  • Python version 3.8
  • Faust version v8.0
  • Operating system Docker python slim 3.8.12 (Inside of ECS AWS)
  • Kafka version 2.41

Update issue

I have an environment variable inside docker with name APP_VERSION. I think the new fork is ussing it for something or it in the old project support a string and not integer.

antonioIrizar avatar Jan 14 '22 15:01 antonioIrizar

I ran a Faust app with version=123 specified using v0.8.8 and I did not see this bug. I did not run it with an identical configuration as yours, so maybe that could be why. Are you still having this issue?

wbarnha avatar Sep 01 '22 03:09 wbarnha