docker-mailman Creating a new domain in postorius gives "A server error occurred. Please contact the administrator." error due to some constraint error in the MySQL backend

Creating a new domain in postorius gives me the error: "A server error occurred. Please contact the administrator."

However, when that happens nothing new appears in mailman-web's /opt/mailman-web-data/logs/mailmanweb.log or mailman-core's docker log. There is no javascript console error either. In fact, everything looks perfectly fine except that there's a red text in the site stating "A server error occurred. Please contact the administrator." and no new domain is listed.

Sep 30 '17 07:09 ghost

Did you look into Mailman Core's logs? /opt/mailman/core/var/logs/mailman.log.

Sep 30 '17 19:09 maxking

Ah, that log in mailman-core at /opt/mailman/var/logs/mailman.log indeed has an error:

Sep 30 19:32:20 2017 (45) REST request handler error:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 250, in execute
    self.errorhandler(self, exc, value)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
    raise errorvalue
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 247, in execute
    res = self._query(query)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 411, in _query
    rowcount = self._do_query(q)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 374, in _do_query
    db.query(q)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 277, in query
    _mysql.connection.query(self, query)
_mysql_exceptions.OperationalError: (4025, 'CONSTRAINT `CONSTRAINT_1` failed for `mailman3`.`preferences`')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/wsgiref/handlers.py", line 137, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 50, in wrapper
    rtn = function(*args, **kws)
  File "/usr/local/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", line 214, in __call__
    return super().__call__(environ, start_response)
  File "/usr/local/lib/python3.6/site-packages/falcon/api.py", line 242, in __call__
    responder(req, resp, **params)
  File "/usr/local/lib/python3.6/site-packages/mailman/rest/domains.py", line 159, in on_post
    domain = domain_manager.add(**values)
  File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper
    return function(args[0], config.db.store, *args[1:], **kws)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/domain.py", line 126, in add
    domain = Domain(mail_host, description, owners)
  File "<string>", line 4, in __init__
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 414, in _initialize_instance
    manager.dispatch.init_failure(self, args, kwargs)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 411, in _initialize_instance
    return manager.original_init(*mixed[1:], **kwargs)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/domain.py", line 67, in __init__
    self.add_owners(owners)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/domain.py", line 101, in add_owners
    self.add_owner(owner)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/domain.py", line 93, in add_owner
    user = user_manager.create_user(owner)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/usermanager.py", line 42, in create_user
    user = User(display_name, Preferences())
  File "<string>", line 4, in __init__
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 414, in _initialize_instance
    manager.dispatch.init_failure(self, args, kwargs)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 411, in _initialize_instance
    return manager.original_init(*mixed[1:], **kwargs)
  File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper
    return function(args[0], config.db.store, *args[1:], **kws)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/user.py", line 80, in __init__
    user_id = uid_factory.new()
  File "/usr/local/lib/python3.6/site-packages/mailman/utilities/uid.py", line 79, in new
    return self._next_unpredictable_id()
  File "/usr/local/lib/python3.6/site-packages/mailman/utilities/uid.py", line 130, in _next_unpredictable_id
    UID.record(uid)
  File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper
    return function(args[0], config.db.store, *args[1:], **kws)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/uid.py", line 69, in record
    if existing.count() != 0:
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3089, in count
    return self.from_self(col).scalar()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2843, in scalar
    ret = self.one()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2814, in one
    ret = self.one_or_none()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2784, in one_or_none
    ret = list(self)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2854, in __iter__
    self.session._autoflush()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1407, in _autoflush
    util.raise_from_cause(e)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1397, in _autoflush
    self.flush()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2171, in flush
    self._flush(objects)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2291, in _flush
    transaction.rollback(_capture_exception=True)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2255, in _flush
    flush_context.execute()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
    rec.execute(self)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
    uow
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
    mapper, table, insert)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
    execute(statement, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 250, in execute
    self.errorhandler(self, exc, value)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
    raise errorvalue
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 247, in execute
    res = self._query(query)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 411, in _query
    rowcount = self._do_query(q)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 374, in _do_query
    db.query(q)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 277, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (_mysql_exceptions.OperationalError) (4025, 'CONSTRAINT `CONSTRAINT_1` failed for `mailman3`.`preferences`') [SQL: 'INSERT INTO preferences (acknowledge_posts, hide_address, preferred_language, receive_list_copy, receive_own_postings, delivery_mode, delivery_status) VALUES (%s, %s, %s, %s, %s, %s, %s)'] [parameters: (None, None, None, None, None, None, None)]
Sep 30 19:32:20 2017 (45) 172.16.0.18 - - "POST /3.0/domains HTTP/1.1" 500 59

In case that's related, I added those two lines to mailman-core to be able to use the MySQL backend:

RUN apk add --no-cache mariadb-dev gcc python3-dev linux-headers musl-dev
RUN pip3 install mysqlclient

Sep 30 '17 19:09 ghost

Yeah, this would be because of missing mysqlclient library in Core.

Duplicate of #80 #71 #136 #123

Sep 30 '17 22:09 maxking

For what it's worth, before I manually added the mysqlclient I got a completely different backtrace out of the Django core with a missing module, while this seems to actually execute a MySQL statement but fail. So to me it looks at least potentially like a different issue.

Sep 30 '17 22:09 ghost

It would be great if you can try again with the latest images (after an hour or so ;-) to see if this problem still persists.

I have fixed the MySQL related problems in the images so it should work.

Oct 01 '17 03:10 maxking

I'd be happy to test it, but the docker hub images seem to be stuck in a 4 month old state: https://hub.docker.com/r/maxking/mailman-core/builds/ https://hub.docker.com/r/maxking/mailman-web/builds/

Oct 02 '17 14:10 ghost

I have triggered the builds, it should be updated in next hour or so.

Oct 02 '17 19:10 maxking

For what it's worth, there still seems to be some MySQL issue in the latest mailman-core image that I just pulled:

homeofjones.net-PRODUCTION-hetzner01 # sudo docker-compose logs mailman-core | tail -n 20
mailman-core_1    | Traceback (most recent call last):
mailman-core_1    |   File "/usr/local/bin/mailman", line 11, in <module>
mailman-core_1    |     sys.exit(main())
mailman-core_1    |   File "/usr/local/lib/python3.6/site-packages/mailman/bin/mailman.py", line 97, in main
mailman-core_1    |     initialize(config_path)
mailman-core_1    |   File "/usr/local/lib/python3.6/site-packages/mailman/core/initialize.py", line 185, in initialize
mailman-core_1    |     initialize_2(propagate_logs=propagate_logs)
mailman-core_1    |   File "/usr/local/lib/python3.6/site-packages/mailman/core/initialize.py", line 156, in initialize_2
mailman-core_1    |     config.db = getUtility(IDatabaseFactory, utility_name).create()
mailman-core_1    |   File "/usr/local/lib/python3.6/site-packages/mailman/database/factory.py", line 54, in create
mailman-core_1    |     database.initialize()
mailman-core_1    |   File "/usr/local/lib/python3.6/site-packages/mailman/database/base.py", line 106, in initialize
mailman-core_1    |     self.engine = create_engine(url, isolation_level='READ UNCOMMITTED')
mailman-core_1    |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/__init__.py", line 391, in create_engine
mailman-core_1    |     return strategy.create(*args, **kwargs)
mailman-core_1    |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 80, in create
mailman-core_1    |     dbapi = dialect_cls.dbapi(**dbapi_args)
mailman-core_1    |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 110, in dbapi
mailman-core_1    |     return __import__('MySQLdb')
mailman-core_1    | ModuleNotFoundError: No module named 'MySQLdb'
homeofjones.net-PRODUCTION-hetzner01 # sudo docker images | grep mailman-core
mail_mailman-core            latest              d230a4212449        32 minutes ago      174MB
maxking/mailman-core         latest              d230a4212449        32 minutes ago      174MB
maxking/mailman-core         <none>              40e1b751b84e        7 weeks ago         157MB
homeofjones.net-PRODUCTION-hetzner01 # sudo docker inspect mail_mailman-core | grep "Id"
        "Id": "sha256:d230a4212449b1840373dff0ea76083f81b5affa94aa411fae85470bb7bd2dc0",
homeofjones.net-PRODUCTION-hetzner01 # sudo docker pull maxking/mailman-core
Using default tag: latest
latest: Pulling from maxking/mailman-core
Digest: sha256:2c53aa8208891da2f844e35cadc6fa4e3181b601c829f3bcbd8ad4db394f981f
Status: Image is up to date for maxking/mailman-core:latest
homeofjones.net-PRODUCTION-hetzner01 #

mailman-web on the other hand gave me a proper database connection error, so the MySQL backend seems to be possibly fixed for that one.

Oct 02 '17 19:10 ghost

Can you show me the DATABASE_URL for core?

It should be something like mysql+pymysql://username:password@database/mailmandb

I am assuming the above error is because of missing +pymysql in above URL. The reason for urls being different is because Core uses pymysql as the preferred python library for MySQL.

Oct 02 '17 19:10 maxking

Indeed the URL is only mysql://... - since again this is somewhat redundant as far as the user is concerned (after all, the user intention is clear and the core only has one working way of using MySQL so it could figure this out itself) is there a way to get a smarter default behavior here? Maybe by evaluating the URL manually and replacing this sensefully

Oct 02 '17 20:10 ghost

Core might work with mysqlclient which provides MySQLdb and doesn't need the +pymysql suffix.

SQLAlchemy (ORM that Core uses) docs say that it works but without running the entire test suite with it, I am not sure about it. SQLAlchemy is what needs that URL and above mentioned URL is recommended style it needed.

Adding +pymysql un-conditionally might not be the best thing to do since it is not the only way, it is the only way it is tested by CI to work. I am inclined to test mysqlclient soon with Core and see if it works.

Oct 02 '17 20:10 maxking

Although, turns out +mysqldb is required for using mysqlclient so it doesn't go away completely :(

Oct 02 '17 20:10 maxking

Well but you could simply check for a mysql:// start and replace that with mysql+pymysql:// in some container startup script, couldn't you? For the container that seems like a reasonable default behavior, and it wouldn't break any of the more specific variants.

It would also allow you to change the backend later without breaking everyone's specific db URLs that still refer to the other backend that one day might be removed. So I'd actually suggest it is harmful that the exact backend needs to be specified (for the maxking/mailman-core container that is, not mailman 3 itself when manually installed)

Oct 02 '17 20:10 ghost

Indeed it can be done for mailman-core container images I suppose.

Can you help me with a bash function to do same? core/docker-entrypoint.sh is where it would live.

Oct 02 '17 21:10 maxking

Sure, I'll implement it real quick gimme a few minutes

Oct 02 '17 21:10 ghost

Here is a pull request: https://github.com/maxking/docker-mailman/pull/149

Independently of this, I'm back to this error again:

bash-4.3# cat /opt/mailman/var/logs/mailman.log | tail -n 155
Oct 02 21:38:50 2017 (28) command runner started.
Oct 02 21:39:12 2017 (34) 172.16.0.9 - - "GET /3.0/lists?count=0&page=1 HTTP/1.1" 200 90
Oct 02 21:39:12 2017 (34) 172.16.0.9 - - "GET /3.0/lists?count=10&page=1 HTTP/1.1" 200 90
Oct 02 21:39:12 2017 (34) 172.16.0.9 - - "GET /3.0/domains HTTP/1.1" 200 90
Oct 02 21:39:48 2017 (34) REST request handler error:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.6/site-packages/pymysql/cursors.py", line 166, in execute
    result = self._query(query)
  File "/usr/local/lib/python3.6/site-packages/pymysql/cursors.py", line 322, in _query
    conn.query(q)
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 856, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 1057, in _read_query_result
    result.read()
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 1340, in read
    first_packet = self.connection._read_packet()
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 1014, in _read_packet
    packet.check_error()
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 393, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/local/lib/python3.6/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (4025, 'CONSTRAINT `CONSTRAINT_1` failed for `mailman3`.`preferences`')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/wsgiref/handlers.py", line 137, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 50, in wrapper
    rtn = function(*args, **kws)
  File "/usr/local/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", line 214, in __call__
    return super().__call__(environ, start_response)
  File "/usr/local/lib/python3.6/site-packages/falcon/api.py", line 242, in __call__
    responder(req, resp, **params)
  File "/usr/local/lib/python3.6/site-packages/mailman/rest/domains.py", line 159, in on_post
    domain = domain_manager.add(**values)
  File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper
    return function(args[0], config.db.store, *args[1:], **kws)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/domain.py", line 126, in add
    domain = Domain(mail_host, description, owners)
  File "<string>", line 4, in __init__
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 414, in _initialize_instance
    manager.dispatch.init_failure(self, args, kwargs)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 411, in _initialize_instance
    return manager.original_init(*mixed[1:], **kwargs)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/domain.py", line 67, in __init__
    self.add_owners(owners)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/domain.py", line 101, in add_owners
    self.add_owner(owner)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/domain.py", line 93, in add_owner
    user = user_manager.create_user(owner)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/usermanager.py", line 42, in create_user
    user = User(display_name, Preferences())
  File "<string>", line 4, in __init__
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 414, in _initialize_instance
    manager.dispatch.init_failure(self, args, kwargs)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 411, in _initialize_instance
    return manager.original_init(*mixed[1:], **kwargs)
  File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper
    return function(args[0], config.db.store, *args[1:], **kws)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/user.py", line 80, in __init__
    user_id = uid_factory.new()
  File "/usr/local/lib/python3.6/site-packages/mailman/utilities/uid.py", line 79, in new
    return self._next_unpredictable_id()
  File "/usr/local/lib/python3.6/site-packages/mailman/utilities/uid.py", line 130, in _next_unpredictable_id
    UID.record(uid)
  File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper
    return function(args[0], config.db.store, *args[1:], **kws)
  File "/usr/local/lib/python3.6/site-packages/mailman/model/uid.py", line 69, in record
    if existing.count() != 0:
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3089, in count
    return self.from_self(col).scalar()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2843, in scalar
    ret = self.one()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2814, in one
    ret = self.one_or_none()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2784, in one_or_none
    ret = list(self)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2854, in __iter__
    self.session._autoflush()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1407, in _autoflush
    util.raise_from_cause(e)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1397, in _autoflush
    self.flush()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2177, in flush
    self._flush(objects)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2297, in _flush
    transaction.rollback(_capture_exception=True)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2261, in _flush
    flush_context.execute()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
    rec.execute(self)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
    uow
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
    mapper, table, insert)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
    execute(statement, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.6/site-packages/pymysql/cursors.py", line 166, in execute
    result = self._query(query)
  File "/usr/local/lib/python3.6/site-packages/pymysql/cursors.py", line 322, in _query
    conn.query(q)
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 856, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 1057, in _read_query_result
    result.read()
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 1340, in read
    first_packet = self.connection._read_packet()
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 1014, in _read_packet
    packet.check_error()
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 393, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/local/lib/python3.6/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
sqlalchemy.exc.InternalError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (pymysql.err.InternalError) (4025, 'CONSTRAINT `CONSTRAINT_1` failed for `mailman3`.`preferences`') [SQL: 'INSERT INTO preferences (acknowledge_posts, hide_address, preferred_language, receive_list_copy, receive_own_postings, delivery_mode, delivery_status) VALUES (%(acknowledge_posts)s, %(hide_address)s, %(preferred_language)s, %(receive_list_copy)s, %(receive_own_postings)s, %(delivery_mode)s, %(delivery_status)s)'] [parameters: {'acknowledge_posts': None, 'hide_address': None, 'preferred_language': None, 'receive_list_copy': None, 'receive_own_postings': None, 'delivery_mode': None, 'delivery_status': None}]
Oct 02 21:39:48 2017 (34) 172.16.0.9 - - "POST /3.0/domains HTTP/1.1" 500 59
bash-4.3#

I can reproduce this reliably by doing the following steps:

Remove all containers, remove all volumes including the database
Create all containers and start them
Use password forgotten to get admin login
Login
Confirm e-mail
Go to domains in postorius
Enter the domain already used for the django instance ("Web Host") as a new "Mail Host" and click "Create Domain" button

Is the problem maybe that I'm trying to have the mail host domain be the same as the web host domain?

Oct 02 '17 21:10 ghost

When I tested, I went up to the Index page and because all migrations were working, I didn't think database connection would be a problem anymore. :(

I can reproduce this issue on my test instance and will look into it. It should not matter if you have same Mail Host and Web Host.

Oct 02 '17 21:10 maxking

OK, this is an issue with Core itself and it's compatibility with Mariadb/MySQL.

Can you please report this issue over at https://gitlab.com/mailman/mailman/issues

I tried out with mysqlclient library and it still errors out with same traceback so it looks like a bug in Core.

Oct 02 '17 22:10 maxking

Ok, issue is filed here: https://gitlab.com/mailman/mailman/issues/412

Oct 02 '17 22:10 ghost

This issue has not been updated for more than 1year

Nov 15 '22 22:11 github-actions[bot]