Arrange stress test on main and submission interface

[fpietrosanti]

In order to be sure that, following the public launch of partner-x website, globaleaks will support many requests all together, a stress test must be arranged.

This has to be implemented and executed in a timebox of less than 4 hours.

That's not the full stress test suite as defined on #426 .

This has to be done before public launch on production infrastructure.

[fpietrosanti]

A quick python or shell script of automation with curl or ab maybe enough

[hellais]

I wrote a python script to be able to stress test the submission interface: https://github.com/globaleaks/GlobaLeaks/blob/master/scripts/stresser.py

When running it I end up getting this error:

2013-08-25 15:07:31+0200 Unhandled Error
    Traceback (most recent call last):
      File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 524, in __bootstrap
        self.__bootstrap_inner()
      File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 551, in __bootstrap_inner
        self.run()
      File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 504, in run
        self.__target(*self.__args, **self.__kwargs)
    --- <exception caught here> ---
      File "/Users/y/.virtualenvs/glbackend/lib/python2.7/site-packages/twisted/python/threadpool.py", line 191, in _worker
        result = context.call(ctx, function, *args, **kwargs)
      File "/Users/y/.virtualenvs/glbackend/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext
        return self.currentContext().callWithContext(ctx, func, *args, **kw)
      File "/Users/y/.virtualenvs/glbackend/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext
        return func(*args,**kw)
      File "/Users/y/code/GL/GLBackend/globaleaks/settings.py", line 550, in _wrap
        self.store = self.get_store()
      File "/Users/y/code/GL/GLBackend/globaleaks/settings.py", line 543, in get_store
        return zstorm.get(GLSetting.store_name)
      File "/Users/y/.virtualenvs/glbackend/lib/python2.7/site-packages/storm/zope/zstorm.py", line 177, in get
        return self.create(name, default_uri)
      File "/Users/y/.virtualenvs/glbackend/lib/python2.7/site-packages/storm/zope/zstorm.py", line 153, in create
        store = Store(database)
      File "/Users/y/.virtualenvs/glbackend/lib/python2.7/site-packages/storm/store.py", line 73, in __init__
        self._connection = database.connect(self._event)
      File "/Users/y/.virtualenvs/glbackend/lib/python2.7/site-packages/storm/database.py", line 493, in connect
        return self.connection_factory(self, event)
      File "/Users/y/.virtualenvs/glbackend/lib/python2.7/site-packages/storm/database.py", line 188, in __init__
        self._raw_connection = self._database.raw_connect()
      File "/Users/y/.virtualenvs/glbackend/lib/python2.7/site-packages/storm/databases/sqlite.py", line 192, in raw_connect
        isolation_level=None)
    pysqlite2.dbapi2.OperationalError: unable to open database file

I will continue doing a bit more testing and see what else comes up.

[fpietrosanti]

What the stress test script test? Does it also make concurrency (like 100/200 parallel connection) testing?

[hellais]

@fpietrosanti

https://github.com/globaleaks/GlobaLeaks/blob/master/scripts/stresser.py#L183

https://github.com/globaleaks/GlobaLeaks/blob/master/scripts/stresser.py#L189

[fpietrosanti]

@vecna did you managed to test/stress test fixing the exceptions?

[vecna]

No, its something that require a deep twisted debug skill, I've start with few result in the few moment given. we'll talk about assignment

[fpietrosanti]

@evilalive , there is vecna that can reproduce this on linux

[evilaliv3]

@vecna can you give additional infos?

how can i get the same "pysqlite2.dbapi2.OperationalError: unable to open database file" ?

[evilaliv3]

this is related to: https://github.com/globaleaks/GlobaLeaks/issues/768