machine-learning
machine-learning copied to clipboard
Published
20 hours ago •
jeff1evesque
jeff1evesque
Update puppet5 + python3
We need to take advantage of the ubuntu 16.04 base container, by implementing puppet5x, and python3x.
Currently, our install_rancher properly deploys our containers:
However, both the webserver-api, and webserver-web have python syntax errors, likely not compatible with the updated python3 implementation:
10/4/2018 5:04:15 PM File "/var/machine-learning/brain/load_data.py", line 12, in <module>
10/4/2018 5:04:15 PM from brain.session.data_new import DataNew
10/4/2018 5:04:15 PM File "/var/machine-learning/brain/session/data_new.py", line 14, in <module>
10/4/2018 5:04:15 PM from brain.session.base_data import BaseData
10/4/2018 5:04:15 PM File "/var/machine-learning/brain/session/base_data.py", line 74
10/4/2018 5:04:15 PM except Exception, error:
10/4/2018 5:04:15 PM ^
10/4/2018 5:04:15 PMSyntaxError: invalid syntax
10/4/2018 5:04:15 PM[2018-10-04 17:04:15 -0400] [15] [INFO] Worker exiting (pid: 15)
10/4/2018 5:04:15 PM[2018-10-04 17:04:15 -0400] [16] [ERROR] Exception in worker process
10/4/2018 5:04:15 PMTraceback (most recent call last):
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 583, in spawn_worker
10/4/2018 5:04:15 PM worker.init_process()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py", line 129, in init_process
10/4/2018 5:04:15 PM self.load_wsgi()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py", line 138, in load_wsgi
10/4/2018 5:04:15 PM self.wsgi = self.app.wsgi()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/base.py", line 67, in wsgi
10/4/2018 5:04:15 PM self.callable = self.load()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load
10/4/2018 5:04:15 PM return self.load_wsgiapp()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
10/4/2018 5:04:15 PM return util.import_app(self.app_uri)
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/util.py", line 350, in import_app
10/4/2018 5:04:15 PM __import__(module)
10/4/2018 5:04:15 PM File "/var/machine-learning/factory.py", line 20, in <module>
10/4/2018 5:04:15 PM from interface.views_api import blueprint_api
10/4/2018 5:04:15 PM File "/var/machine-learning/interface/views_api.py", line 22, in <module>
10/4/2018 5:04:15 PM from brain.load_data import Load_Data
10/4/2018 5:04:15 PM File "/var/machine-learning/brain/load_data.py", line 12, in <module>
10/4/2018 5:04:15 PM from brain.session.data_new import DataNew
10/4/2018 5:04:15 PM File "/var/machine-learning/brain/session/data_new.py", line 14, in <module>
10/4/2018 5:04:15 PM from brain.session.base_data import BaseData
10/4/2018 5:04:15 PM File "/var/machine-learning/brain/session/base_data.py", line 74
10/4/2018 5:04:15 PM except Exception, error:
10/4/2018 5:04:15 PM ^
10/4/2018 5:04:15 PMSyntaxError: invalid syntax
10/4/2018 5:04:15 PM[2018-10-04 17:04:15 -0400] [16] [INFO] Worker exiting (pid: 16)
10/4/2018 5:04:15 PM[2018-10-04 17:04:15 -0400] [17] [ERROR] Exception in worker process
10/4/2018 5:04:15 PMTraceback (most recent call last):
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 583, in spawn_worker
10/4/2018 5:04:15 PM worker.init_process()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py", line 129, in init_process
10/4/2018 5:04:15 PM self.load_wsgi()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py", line 138, in load_wsgi
10/4/2018 5:04:15 PM self.wsgi = self.app.wsgi()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/base.py", line 67, in wsgi
10/4/2018 5:04:15 PM self.callable = self.load()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load
10/4/2018 5:04:15 PM return self.load_wsgiapp()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
10/4/2018 5:04:15 PM return util.import_app(self.app_uri)
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/util.py", line 350, in import_app
10/4/2018 5:04:15 PM __import__(module)
10/4/2018 5:04:15 PM File "/var/machine-learning/factory.py", line 20, in <module>
10/4/2018 5:04:15 PM from interface.views_api import blueprint_api
10/4/2018 5:04:15 PM File "/var/machine-learning/interface/views_api.py", line 22, in <module>
10/4/2018 5:04:15 PM from brain.load_data import Load_Data
10/4/2018 5:04:15 PM File "/var/machine-learning/brain/load_data.py", line 12, in <module>
10/4/2018 5:04:15 PM from brain.session.data_new import DataNew
10/4/2018 5:04:15 PM File "/var/machine-learning/brain/session/data_new.py", line 14, in <module>
10/4/2018 5:04:15 PM from brain.session.base_data import BaseData
10/4/2018 5:04:15 PM File "/var/machine-learning/brain/session/base_data.py", line 74
10/4/2018 5:04:15 PM except Exception, error:
10/4/2018 5:04:15 PM ^
10/4/2018 5:04:15 PMSyntaxError: invalid syntax
10/4/2018 5:04:15 PM[2018-10-04 17:04:15 -0400] [17] [INFO] Worker exiting (pid: 17)
10/4/2018 5:04:15 PMTraceback (most recent call last):
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 210, in run
10/4/2018 5:04:15 PM self.sleep()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 360, in sleep
10/4/2018 5:04:15 PM ready = select.select([self.PIPE[0]], [], [], 1.0)
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 245, in handle_chld
10/4/2018 5:04:15 PM self.reap_workers()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 525, in reap_workers
10/4/2018 5:04:15 PM raise HaltServer(reason, self.WORKER_BOOT_ERROR)
10/4/2018 5:04:15 PMgunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
10/4/2018 5:04:15 PM
10/4/2018 5:04:15 PMDuring handling of the above exception, another exception occurred:
10/4/2018 5:04:15 PM
10/4/2018 5:04:15 PMTraceback (most recent call last):
10/4/2018 5:04:15 PM File "/usr/local/bin/gunicorn", line 11, in <module>
10/4/2018 5:04:15 PM sys.exit(run())
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 61, in run
10/4/2018 5:04:15 PM WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/base.py", line 223, in run
10/4/2018 5:04:15 PM super(Application, self).run()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/base.py", line 72, in run
10/4/2018 5:04:15 PM Arbiter(self).run()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 232, in run
10/4/2018 5:04:15 PM self.halt(reason=inst.reason, exit_status=inst.exit_status)
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 345, in halt
10/4/2018 5:04:15 PM self.stop()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 393, in stop
10/4/2018 5:04:15 PM time.sleep(0.1)
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 245, in handle_chld
10/4/2018 5:04:15 PM self.reap_workers()
10/4/2018 5:04:15 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 525, in reap_workers
10/4/2018 5:04:15 PM raise HaltServer(reason, self.WORKER_BOOT_ERROR)
10/4/2018 5:04:15 PMgunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
Our webserver containers have the following errors:
10/6/2018 1:04:37 PM from brain.session.base_data import BaseData
10/6/2018 1:04:37 PM File "/var/machine-learning/brain/session/base_data.py", line 16, in <module>
10/6/2018 1:04:37 PM from brain.session.data.dataset import dataset2dict
10/6/2018 1:04:37 PM File "/var/machine-learning/brain/session/data/dataset.py", line 12, in <module>
10/6/2018 1:04:37 PM import requests
10/6/2018 1:04:37 PMImportError: No module named 'requests'
10/6/2018 1:04:37 PM[2018-10-06 13:04:37 -0400] [15] [ERROR] Exception in worker process
10/6/2018 1:04:37 PMTraceback (most recent call last):
10/6/2018 1:04:37 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 583, in spawn_worker
10/6/2018 1:04:37 PM worker.init_process()
10/6/2018 1:04:37 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py", line 129, in init_process
10/6/2018 1:04:37 PM self.load_wsgi()
10/6/2018 1:04:37 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py", line 138, in load_wsgi
10/6/2018 1:04:37 PM self.wsgi = self.app.wsgi()
10/6/2018 1:04:37 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/base.py", line 67, in wsgi
10/6/2018 1:04:37 PM self.callable = self.load()
10/6/2018 1:04:37 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load
10/6/2018 1:04:37 PM return self.load_wsgiapp()
10/6/2018 1:04:37 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
10/6/2018 1:04:37 PM return util.import_app(self.app_uri)
10/6/2018 1:04:37 PM File "/usr/local/lib/python3.5/dist-packages/gunicorn/util.py", line 350, in import_app
10/6/2018 1:04:37 PM __import__(module)
10/6/2018 1:04:37 PM File "/var/machine-learning/factory.py", line 20, in <module>
10/6/2018 1:04:37 PM from interface.views_api import blueprint_api
10/6/2018 1:04:37 PM File "/var/machine-learning/interface/views_api.py", line 22, in <module>
10/6/2018 1:04:37 PM from brain.load_data import Load_Data
10/6/2018 1:04:37 PM File "/var/machine-learning/brain/load_data.py", line 12, in <module>
10/6/2018 1:04:37 PM from brain.session.data_new import DataNew
10/6/2018 1:04:37 PM File "/var/machine-learning/brain/session/data_new.py", line 14, in <module>
Currently, our webserver-web, and webesrver-api containers need the MySQLdb package:
[2018-10-06 23:53:25 -0400] [12] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.5/dist-packages/gunicorn/util.py", line 350, in import_app
__import__(module)
File "/var/machine-learning/factory.py", line 20, in <module>
from interface.views_api import blueprint_api
File "/var/machine-learning/interface/views_api.py", line 22, in <module>
from brain.load_data import Load_Data
File "/var/machine-learning/brain/load_data.py", line 12, in <module>
from brain.session.data_new import DataNew
File "/var/machine-learning/brain/session/data_new.py", line 14, in <module>
from brain.session.base_data import BaseData
File "/var/machine-learning/brain/session/base_data.py", line 17, in <module>
from brain.database.dataset import Collection
File "/var/machine-learning/brain/database/dataset.py", line 10, in <module>
from brain.database.query import NoSQL
File "/var/machine-learning/brain/database/query.py", line 10, in <module>
import MySQLdb as MariaClient
ImportError: No module named 'MySQLdb'
[2018-10-06 23:53:25 -0400] [12] [INFO] Worker exiting (pid: 12)
Additionally, the mariadb container needs the mysqld package:
10/6/2018 11:50:05 PM * Starting MariaDB database server mysqld
10/6/2018 11:50:08 PM ...done.
10/6/2018 11:50:08 PMTraceback (most recent call last):
10/6/2018 11:50:08 PM File "setup_tables.py", line 34, in <module>
10/6/2018 11:50:08 PM import MySQLdb as DB
10/6/2018 11:50:08 PMImportError: No module named 'MySQLdb'
10/6/2018 11:50:08 PM * Stopping MariaDB database server mysqld
10/6/2018 11:50:10 PM/usr/bin/mysqladmin: connect to server at 'localhost' failed
10/6/2018 11:50:10 PMerror: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2 "No such file or directory")'
10/6/2018 11:50:10 PMCheck that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
10/6/2018 11:50:12 PM ...done.
10/6/2018 11:50:12 PM181006 23:50:12 [Note] mysqld (mysqld 10.0.36-MariaDB-0ubuntu0.16.04.1) starting as process 343 ...
Ubuntu does not natively support MySQLdb for python3. Therefore, we must install the mysqlclient as a workaround using pip3:
root@mariadb:/# apt-get update
root@mariadb:/# apt-get install python3-dev libmysqlclient-dev
root@mariadb:/# pip3 install mysqlclient
root@mariadb:/# python3.5
Python 3.5.2 (default, Dec 15 2017, 15:32:37)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
The above 500 error can be further debugged from our webserver-web container:
root@webserver-web:/var/machine-learning# cat /var/log/webserver/flask.log
[2018-10-08 21:00:43,794] {/usr/local/lib/python3.5/dist-packages/flask/app.py:1761} ERROR - Exception on /retrieve-prediction-titles [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/var/machine-learning/interface/views_web.py", line 408, in retrieve_prediction_titles
response = prediction.get_all_titles(model_type)
File "/var/machine-learning/brain/database/prediction.py", line 143, in get_all_titles
response = self.sql.execute('select', sql_statement, args)
File "/var/machine-learning/brain/database/query.py", line 281, in execute
self.cursor.execute(statement, sql_args)
File "/usr/local/lib/python3.5/dist-packages/MySQLdb/cursors.py", line 234, in execute
args = tuple(map(db.literal, args))
TypeError: 'int' object is not iterable
[2018-10-08 21:02:39,331] {/usr/local/lib/python3.5/dist-packages/flask/app.py:1761} ERROR - Exception on /load-data [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/var/machine-learning/interface/views_web.py", line 94, in load_data
response = loader.load_data_new()
File "/var/machine-learning/brain/load_data.py", line 74, in load_data_new
session.validate_premodel_settings()
File "/var/machine-learning/brain/session/base.py", line 69, in validate_premodel_settings
self.session_type
File "/var/machine-learning/brain/validator/settings.py", line 53, in validate_settings
Required('collection'): All(unicode, Length(min=1)),
NameError: name 'unicode' is not defined
Note: we supplied the svm-1.json sample dataset during the /session/data-new submission.
We now have the following database error in our webserver-web container:
root@webserver-web:/var/machine-learning# cat /var/log/webserver/flask.log
[2018-10-09 19:33:54,674] {/usr/local/lib/python3.5/dist-packages/flask/app.py:1761} ERROR - Exception on /retrieve-prediction-titles [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/var/machine-learning/interface/views_web.py", line 408, in retrieve_prediction_titles
response = prediction.get_all_titles(model_type)
File "/var/machine-learning/brain/database/prediction.py", line 143, in get_all_titles
response = self.sql.execute('select', sql_statement, args)
File "/var/machine-learning/brain/database/query.py", line 281, in execute
self.cursor.execute(statement, sql_args)
File "/usr/local/lib/python3.5/dist-packages/MySQLdb/cursors.py", line 234, in execute
args = tuple(map(db.literal, args))
TypeError: 'int' object is not iterable
[2018-10-09 19:34:08,008] {/usr/local/lib/python3.5/dist-packages/flask/app.py:1761} ERROR - Exception on /load-data [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/var/machine-learning/interface/views_web.py", line 94, in load_data
response = loader.load_data_new()
File "/var/machine-learning/brain/load_data.py", line 76, in load_data_new
session.save_premodel_dataset()
File "/var/machine-learning/brain/session/base_data.py", line 91, in save_premodel_dataset
collection_count = entity.get_collection_count(self.uid)
File "/var/machine-learning/brain/database/entity.py", line 170, in get_collection_count
response = self.sql.execute('select', sql_statement, args)
File "/var/machine-learning/brain/database/query.py", line 281, in execute
self.cursor.execute(statement, sql_args)
File "/usr/local/lib/python3.5/dist-packages/MySQLdb/cursors.py", line 234, in execute
args = tuple(map(db.literal, args))
TypeError: 'int' object is not iterable
Our webserver-web container is not able to serialize the corresponding response upon data-new:
root@webserver-web:/var/machine-learning# cat /var/log/webserver/flask.log
[2018-10-10 20:05:08,356] {/usr/local/lib/python3.5/dist-packages/flask/app.py:1761} ERROR - Exception on /load-data [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/var/machine-learning/interface/views_web.py", line 94, in load_data
response = loader.load_data_new()
File "/var/machine-learning/brain/load_data.py", line 94, in load_data_new
return json.dumps(response)
File "/usr/lib/python3.5/json/__init__.py", line 230, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python3.5/json/encoder.py", line 198, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.5/json/encoder.py", line 256, in iterencode
return _iterencode(o, 0)
File "/usr/lib/python3.5/json/encoder.py", line 179, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: ProgrammingError(1146, "Table 'db_machine_learning.tbl_dataset_entity' doesn't exist") is not JSON serializable
Upon /session/data-new submission, we don't have any logs in our /var/logs/webserver/flask.log from our webserver-web container. However, the frontend is returned the following json string, indicating that the backend validation needs some adjustments:
[
{
"error": {
"validation": [
{
"location": "data_new",
"message": [
"expected a tuple for dictionary value @ data['collection']. Got 'collection-100'",
"expected a tuple for dictionary value @ data['session_name']. Got 'session-100'"
]
}
]
}
},
{
"validation": [
{
"location": "svm-1.xml",
"message": [
"expected a tuple for dictionary value @ data['dependent-variable']. Got 'dep-variable-1'",
"extra keys not allowed @ data['error']. Got None",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-1']. Got '23.45'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-2']. Got '98.01'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-3']. Got '0.432'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-4']. Got '325'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-5']. Got '56.64'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-6']. Got '0.002'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-7']. Got '23'",
"required key not provided @ data['independent-variables'][0][All((<class 'str'>,), Length(min=1, max=None), msg=None)]. Got None",
"expected a tuple for dictionary value @ data['dependent-variable']. Got 'dep-variable-4'",
"extra keys not allowed @ data['error']. Got None",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-1']. Got '22.1'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-2']. Got '95.96'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-3']. Got '0.743'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-4']. Got '342'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-5']. Got '66.67'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-6']. Got '0.001'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-7']. Got '32'",
"required key not provided @ data['independent-variables'][0][All((<class 'str'>,), Length(min=1, max=None), msg=None)]. Got None",
"expected a tuple for dictionary value @ data['dependent-variable']. Got 'dep-variable-5'",
"extra keys not allowed @ data['error']. Got None",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-1']. Got '23.27'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-2']. Got '95.03'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-3']. Got '0.488'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-4']. Got '295'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-5']. Got '55.83'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-6']. Got '0.001'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-7']. Got '27'",
"required key not provided @ data['independent-variables'][0][All((<class 'str'>,), Length(min=1, max=None), msg=None)]. Got None",
"expected a tuple for dictionary value @ data['dependent-variable']. Got 'dep-variable-3'",
"extra keys not allowed @ data['error']. Got None",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-1']. Got '22.67'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-2']. Got '101.21'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-3']. Got '0.832'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-4']. Got '427'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-5']. Got '75.45'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-6']. Got '0.002'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-7']. Got '26'",
"required key not provided @ data['independent-variables'][0][All((<class 'str'>,), Length(min=1, max=None), msg=None)]. Got None",
"expected a tuple for dictionary value @ data['dependent-variable']. Got 'dep-variable-5'",
"extra keys not allowed @ data['error']. Got None",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-1']. Got '19.99'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-2']. Got '97.78'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-3']. Got '0.638'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-4']. Got '303'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-5']. Got '58.88'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-6']. Got '0.001'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-7']. Got '29'",
"required key not provided @ data['independent-variables'][0][All((<class 'str'>,), Length(min=1, max=None), msg=None)]. Got None",
"expected a tuple for dictionary value @ data['dependent-variable']. Got 'dep-variable-1'",
"extra keys not allowed @ data['error']. Got None",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-1']. Got '23.27'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-2']. Got '95.03'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-3']. Got '0.488'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-4']. Got '295'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-5']. Got '55.83'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-6']. Got '0.001'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-7']. Got '27'",
"required key not provided @ data['independent-variables'][0][All((<class 'str'>,), Length(min=1, max=None), msg=None)]. Got None",
"expected a tuple for dictionary value @ data['dependent-variable']. Got 'dep-variable-1'",
"extra keys not allowed @ data['error']. Got None",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-1']. Got '20.71'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-2']. Got '99.33'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-3']. Got '0.648'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-4']. Got '342'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-5']. Got '75.67'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-6']. Got '0.001'",
"extra keys not allowed @ data['independent-variables'][0]['indep-variable-7']. Got '30'",
"required key not provided @ data['independent-variables'][0][All((<class 'str'>,), Length(min=1, max=None), msg=None)]. Got None"
]
}
]
}
]
Note: there are some reactjs issues, when the browser is fullscreen (or significantly wide).
Upon /session/data-new submission, we don't have any logs in our /var/logs/webserver/flask.log from our webserver-web container. However, the frontend is returned the following json string, indicating that the backend validation needs some adjustments:
[
{
"validation": [
{
"location": "svm-1.xml",
"message": [
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-1']. Got '23.45'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-2']. Got '98.01'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-3']. Got '0.432'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-4']. Got '325'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-5']. Got '56.64'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-6']. Got '0.002'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-7']. Got '23'",
"extra keys not allowed @ data['error']. Got None",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-1']. Got '22.1'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-2']. Got '95.96'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-3']. Got '0.743'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-4']. Got '342'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-5']. Got '66.67'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-6']. Got '0.001'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-7']. Got '32'",
"extra keys not allowed @ data['error']. Got None",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-1']. Got '23.27'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-2']. Got '95.03'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-3']. Got '0.488'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-4']. Got '295'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-5']. Got '55.83'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-6']. Got '0.001'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-7']. Got '27'",
"extra keys not allowed @ data['error']. Got None",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-1']. Got '22.67'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-2']. Got '101.21'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-3']. Got '0.832'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-4']. Got '427'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-5']. Got '75.45'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-6']. Got '0.002'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-7']. Got '26'",
"extra keys not allowed @ data['error']. Got None",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-1']. Got '19.99'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-2']. Got '97.78'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-3']. Got '0.638'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-4']. Got '303'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-5']. Got '58.88'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-6']. Got '0.001'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-7']. Got '29'",
"extra keys not allowed @ data['error']. Got None",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-1']. Got '23.27'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-2']. Got '95.03'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-3']. Got '0.488'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-4']. Got '295'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-5']. Got '55.83'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-6']. Got '0.001'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-7']. Got '27'",
"extra keys not allowed @ data['error']. Got None",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-1']. Got '20.71'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-2']. Got '99.33'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-3']. Got '0.648'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-4']. Got '342'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-5']. Got '75.67'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-6']. Got '0.001'",
"expected int for dictionary value @ data['independent-variables'][0]['indep-variable-7']. Got '30'",
"extra keys not allowed @ data['error']. Got None"
]
}
]
}
]
Our /session/data-new submission completes without error. However, the corresponding /session/model-generate does not supply any dataset to work on. Furthermore, inspection of the mariadb container, indicates that no tables have been created in the db_machine_learning database:
root@mariadb:/# mysql -u root -p db_machine_learning
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 14
Server version: 10.0.36-MariaDB-0ubuntu0.16.04.1 Ubuntu 16.04
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [db_machine_learning]> show tables;
Empty set (0.00 sec)
This means our corresponding mariadb puppet module did not create necessary database tables. So, we'll need to make some puppet adjustments, before recreating the respective docker image.
The following errors need to be mitigated during build of the mariadb image:
10/11/2018 10:20:46 PMTraceback (most recent call last):
10/11/2018 10:20:46 PM File "setup_tables.py", line 34, in <module>
10/11/2018 10:20:46 PM import MySQLdb as DB
10/11/2018 10:20:46 PMImportError: No module named 'MySQLdb'
10/11/2018 10:20:50 PM181011 22:20:50 [Note] mysqld (mysqld 10.0.36-MariaDB-0ubuntu0.16.04.1) starting as process 328 ...
Note: the mariadb container does not need the MySQLdb package, after the tables have been created. Therefore, we can conditionally install, then remove the package after the tables have been populated.
Our /session/model-generate receives the following error:
10/12/2018 11:28:26 PM[2018-10-12 23:28:26,386] ERROR in app: Exception on /load-data [POST]
10/12/2018 11:28:26 PMTraceback (most recent call last):
10/12/2018 11:28:26 PM File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2292, in wsgi_app
10/12/2018 11:28:26 PM response = self.full_dispatch_request()
10/12/2018 11:28:26 PM File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1815, in full_dispatch_request
10/12/2018 11:28:26 PM rv = self.handle_user_exception(e)
10/12/2018 11:28:26 PM File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1718, in handle_user_exception
10/12/2018 11:28:26 PM reraise(exc_type, exc_value, tb)
10/12/2018 11:28:26 PM File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 35, in reraise
10/12/2018 11:28:26 PM raise value
10/12/2018 11:28:26 PM File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1813, in full_dispatch_request
10/12/2018 11:28:26 PM rv = self.dispatch_request()
10/12/2018 11:28:26 PM File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1799, in dispatch_request
10/12/2018 11:28:26 PM return self.view_functions[rule.endpoint](**req.view_args)
10/12/2018 11:28:26 PM File "/var/machine-learning/interface/views_web.py", line 98, in load_data
10/12/2018 11:28:26 PM response = loader.load_model_generate()
10/12/2018 11:28:26 PM File "/var/machine-learning/brain/load_data.py", line 151, in load_model_generate
10/12/2018 11:28:26 PM session.generate_model()
10/12/2018 11:28:26 PM File "/var/machine-learning/brain/session/model_generate.py", line 81, in generate_model
10/12/2018 11:28:26 PM self.list_error
10/12/2018 11:28:26 PM File "/var/machine-learning/brain/session/model/sv.py", line 93, in generate
10/12/2018 11:28:26 PM clf.fit(grouped_features, encoded_labels)
10/12/2018 11:28:26 PM File "/usr/local/lib/python3.5/dist-packages/sklearn/svm/base.py", line 149, in fit
10/12/2018 11:28:26 PM accept_large_sparse=False)
10/12/2018 11:28:26 PM File "/usr/local/lib/python3.5/dist-packages/sklearn/utils/validation.py", line 747, in check_X_y
10/12/2018 11:28:26 PM estimator=estimator)
10/12/2018 11:28:26 PM File "/usr/local/lib/python3.5/dist-packages/sklearn/utils/validation.py", line 547, in check_array
10/12/2018 11:28:26 PM "if it contains a single sample.".format(array))
10/12/2018 11:28:26 PMValueError: Expected 2D array, got 1D array instead:
10/12/2018 11:28:26 PMarray=[].
10/12/2018 11:28:26 PMReshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.
We added the following temporary raise Exception to our brain/session/data/dataset.py:
...
raise Exception('instance: {}, converted: {}, error: {}'.format(
repr(instance),
repr(converted),
repr(error)
))
# return results
if list_error:
return {
'dataset': converted,
'settings': settings,
'error': {
'validation': list_error
}
}
else:
return {
'dataset': converted,
'settings': settings,
'error': None,
}
/session/data-new submission on the webserver-web container indicates the dataset was not converted:
10/14/2018 8:44:24 PM[2018-10-14 20:44:24 -0400] [8] [INFO] Starting gunicorn 19.9.0
10/14/2018 8:44:24 PM[2018-10-14 20:44:24 -0400] [8] [INFO] Listening at: http://0.0.0.0:5001 (8)
10/14/2018 8:44:24 PM[2018-10-14 20:44:24 -0400] [8] [INFO] Using worker: sync
10/14/2018 8:44:24 PM[2018-10-14 20:44:24 -0400] [11] [INFO] Booting worker with pid: 11
10/14/2018 8:44:25 PM[2018-10-14 20:44:25 -0400] [12] [INFO] Booting worker with pid: 12
10/14/2018 8:44:25 PM[2018-10-14 20:44:25 -0400] [13] [INFO] Booting worker with pid: 13
10/14/2018 8:44:25 PM[2018-10-14 20:44:25 -0400] [14] [INFO] Booting worker with pid: 14
10/14/2018 8:44:25 PM[2018-10-14 20:44:25 -0400] [15] [INFO] Booting worker with pid: 15
10/14/2018 8:44:25 PM[2018-10-14 20:44:25 -0400] [16] [INFO] Booting worker with pid: 16
10/14/2018 8:51:38 PM[2018-10-14 20:51:38,660] ERROR in app: Exception on /load-data [POST]
10/14/2018 8:51:38 PMTraceback (most recent call last):
10/14/2018 8:51:38 PM File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2292, in wsgi_app
10/14/2018 8:51:38 PM response = self.full_dispatch_request()
10/14/2018 8:51:38 PM File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1815, in full_dispatch_request
10/14/2018 8:51:38 PM rv = self.handle_user_exception(e)
10/14/2018 8:51:38 PM File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1718, in handle_user_exception
10/14/2018 8:51:38 PM reraise(exc_type, exc_value, tb)
10/14/2018 8:51:38 PM File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 35, in reraise
10/14/2018 8:51:38 PM raise value
10/14/2018 8:51:38 PM File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1813, in full_dispatch_request
10/14/2018 8:51:38 PM rv = self.dispatch_request()
10/14/2018 8:51:38 PM File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1799, in dispatch_request
10/14/2018 8:51:38 PM return self.view_functions[rule.endpoint](**req.view_args)
10/14/2018 8:51:38 PM File "/var/machine-learning/interface/views_web.py", line 94, in load_data
10/14/2018 8:51:38 PM response = loader.load_data_new()
10/14/2018 8:51:38 PM File "/var/machine-learning/brain/load_data.py", line 75, in load_data_new
10/14/2018 8:51:38 PM session.convert_dataset()
10/14/2018 8:51:38 PM File "/var/machine-learning/brain/session/base_data.py", line 171, in convert_dataset
10/14/2018 8:51:38 PM response = dataset2dict(self.model_type, self.premodel_data)
10/14/2018 8:51:38 PM File "/var/machine-learning/brain/session/data/dataset.py", line 124, in dataset2dict
10/14/2018 8:51:38 PM repr(error)
10/14/2018 8:51:38 PMException: instance: <FileStorage: 'svm-1.json' ('application/json')>, converted: [], error: False
We can add debug=True in our app factory, to allow flask to reload on any python code changes. This will allow us to debug without the need to rebuild containers.
We'll remove the rancher implementation to speed up local builds, before resuming debugging this issue.