Uncaught exception `ValueError: 245 is not a valid MySQLType` when using `dbsake frmdump`

[golflima]

I'm using your excellent tool to extract the schema data from the .frm files of a database to recreate it on another server (the plan is to have a disaster recovery procedure in case every other backup options failed, and to copy prod data to a BI server with Percona XtraDB Backup). It works very well for most tables, on few I have the following error and I cant't get the schema for them:

# dbsake frmdump company_user_csv_import_configuration.frm
Uncaught exception!Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/local/bin/dbsake/dbsake/cli/__init__.py", line 106, in handle_uncaught_exception
UnicodeDecodeError: 'ascii' codec can't decode byte 0xb0 in position 8: ordinal not in range(128)

Original exception was:
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/dbsake/__main__.py", line 22, in <module>
  File "/usr/local/bin/dbsake/__main__.py", line 18, in main
  File "/usr/local/bin/dbsake/dbsake/cli/__init__.py", line 123, in main
  File "/usr/local/bin/dbsake/click/core.py", line 722, in __call__
  File "/usr/local/bin/dbsake/click/core.py", line 697, in main
  File "/usr/local/bin/dbsake/click/core.py", line 1066, in invoke
  File "/usr/local/bin/dbsake/click/core.py", line 895, in invoke
  File "/usr/local/bin/dbsake/click/core.py", line 535, in invoke
  File "/usr/local/bin/dbsake/click/decorators.py", line 17, in new_func
  File "/usr/local/bin/dbsake/dbsake/cli/cmd/frm.py", line 71, in frmdump
  File "/usr/local/bin/dbsake/dbsake/cli/cmd/frm.py", line 26, in parse_and_print
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/__init__.py", line 41, in parse
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/binaryfrm.py", line 402, in parse
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/binaryfrm.py", line 287, in unpack_columns
  File "/usr/local/bin/dbsake/dbsake/util/enum.py", line 295, in __call__
  File "/usr/local/bin/dbsake/dbsake/util/enum.py", line 575, in __new__
ValueError: 245 is not a valid MySQLType

By default, we are (still) using Python 2.7.16, so I tried with Python 3.7.3 too:

# python3.7 /usr/local/bin/dbsake frmdump company_user_csv_import_configuration.frm
Uncaught exception! (╯°□°)╯ ︵ ┻━┻
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/bin/dbsake/__main__.py", line 22, in <module>
    sys.exit(main())
  File "/usr/local/bin/dbsake/__main__.py", line 18, in main
    sys.exit(dbsake.cli.main())
  File "/usr/local/bin/dbsake/dbsake/cli/__init__.py", line 123, in main
    dbsake(args=argv, auto_envvar_prefix='DBSAKE', obj={})
  File "/usr/local/bin/dbsake/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/bin/dbsake/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/bin/dbsake/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/bin/dbsake/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/bin/dbsake/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/bin/dbsake/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/bin/dbsake/dbsake/cli/cmd/frm.py", line 71, in frmdump
    failures += parse_and_print(name, type_codes, replace)
  File "/usr/local/bin/dbsake/dbsake/cli/cmd/frm.py", line 26, in parse_and_print
    table = frm.parse(frm_path)
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/__init__.py", line 41, in parse
    return dispatch(path)
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/binaryfrm.py", line 402, in parse
    columns = list(unpack_columns(packed_frm_data.columns, table))
  File "/usr/local/bin/dbsake/dbsake/core/mysql/frm/binaryfrm.py", line 287, in unpack_columns
    type_code=constants.MySQLType(metadata.uint8_at(13, os.SEEK_CUR))
  File "/usr/local/bin/dbsake/dbsake/util/enum.py", line 295, in __call__
    return cls.__new__(cls, value)
  File "/usr/local/bin/dbsake/dbsake/util/enum.py", line 575, in __new__
    raise ValueError("%s is not a valid %s" % (value, cls.__name__))
ValueError: 245 is not a valid MySQLType
It's okay. ┬─┬ノ( º_ ºノ)
Consider filing a bug report at https://github.com/abg/dbsake/issues

OS is: Linux 4.19.0-8-cloud-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux. dbsake version is: dbsake, version 2.1.2 (git: 50de953)

For information, DBeaver generate this query to recreate the table:

CREATE TABLE `company_user_csv_import_configuration` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `company_id` int(11) DEFAULT NULL,
  `directory` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
  `file_pattern` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
  `report_emails` json DEFAULT NULL,
  `error_emails` json DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `UNIQ_27AB5994979B1AD6` (`company_id`),
  CONSTRAINT `FK_27AB5994979B1AD6` FOREIGN KEY (`company_id`) REFERENCES `companies` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

And DESCRIBE returns this:

DESCRIBE company_user_csv_import_configuration;

Field        |Type        |Null|Key|Default|Extra         |
-------------|------------|----|---|-------|--------------|
id           |int(11)     |NO  |PRI|       |auto_increment|
company_id   |int(11)     |YES |UNI|       |              |
directory    |varchar(255)|NO  |   |       |              |
file_pattern |varchar(255)|NO  |   |       |              |
report_emails|json        |YES |   |       |              |
error_emails |json        |YES |   |       |              |

[golflima]

Ok, I just had a look into the other 2 tables having the same issue, they all use the MySQL type json, which is not used on the other tables. So maybe dbsake doesn't support yet this json type?

[golflima]

Indeed, just checked your sources: https://github.com/abg/dbsake/blob/master/dbsake/core/mysql/frm/constants.py#L152-L182 Perhaps you should add something like: JSON = 245 ?